cdxgen fails to find all dependencies

[emcfins]

When viewing the dependency graph for Connected Mobility Solution on AWS, it shows a total of 7,484.
But when I clone the repo and run CDXGEN_DEBUG_MODE=debug FETCH_LICENSE=true cdxgen -t universal --spec-version 1.4 -o bom.json, the components section of the bom.json file, the components section only has 4289.

For example, I find some dependencies missing from the sbom that are defined in the lock file here

What am I doing wrong?

Thank you

[prabhu]

@emcfins Have you tried running with -t python --deep, since it appears like a Python project. We have some logic in place, where -t universal would try to go broader rather than deeper into each language. You may have to run cdxgen from within a python virtual environment with all the packages installed for best results.

[emcfins]

We are using the same tooling for multiple projects - we have a mix of projects of different languages and some projects are a mix of languages so we use -t universal to cover all our requirements.

[prabhu]

@emcfins have you tried without specifying any type?

[emcfins]

Oh - no I haven't. I'll give that a shot. Thank you!

[emcfins]

Not specifying didn't work. Unfortunately, that returned 4287 components

[prabhu]

@emcfins, this needs a bit more triaging. Could you create a folder with just this file and test cdxgen?

The logic to parse a pipfile.lock is here. Is there any bug in the logic?

You can add a unit test similar to this, once you have identified some fixes.

[prabhu]

@aryan-rajoria could you kindly work with @emcfins, since this is false negatives?

[emcfins]

So I did some digging.
When I run cdxgen locally, I noticed the output:

Parsing /Users/erinmc/projects/aws_solutions/connected-mobility/Connected-mobility-solution-on-aws/source/modules/backstage/yarn.lock
About to fetch license information for 3237 packages in parseYarnLock
Found 4226 npm packages at .
Found 61 python packages at .
Found 0 ruby packages at .
Found 0 cpp packages at .
BOM includes 4287 components and 3979 dependencies after dedupe
BOM includes 4289 unfiltered components 3979 dependencies so far
BOM includes 4289 components and 3979 dependencies after dedupe

Is it possible that cdxgen is just looking for a single Pipfile rather than all Pipfiles?

[prabhu]

@emcfins Nice find. I think it only looks for a single file and performs a single pipenv install.

cdxgen/index.js

Line 2473 in 4e6f8da

const pipenvMode = existsSync(join(path, "Pipfile"));

cdxgen/index.js

Line 2625 in 4e6f8da

if (pipenvMode) {

[prabhu]

@emcfins Let me know if you would like to sponsor or contribute this feature.

[emcfins]

I'm happy to sponsor - what does it take?

[prabhu]

@emcfins, Thank you so much! Our donation link is here

One of us will find time and implement this.