Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] multi-declare (optional) dependencies are treated as root dependencies #840

Closed
rocketrose99 opened this issue Jan 14, 2025 · 6 comments · Fixed by #842
Closed

[BUG] multi-declare (optional) dependencies are treated as root dependencies #840

rocketrose99 opened this issue Jan 14, 2025 · 6 comments · Fixed by #842
Assignees
Labels
bug Something isn't working source: poetry

Comments

@rocketrose99
Copy link

Describe the bug

Optional dependencies are treated as root dependencies when surrounded by brackets.

In other words, there is no way to ignore them when generating the SBOM.

For example, defining a dependency like this:

numpy = [
  { version = "*", optional = true },
]

vs like this:

numpy = { version = "*", optional = true }

yields different results in the sbom generated via cyclonedx-py poetry -o sbom.json --of json --sv 1.4. The former case includes numpy whereas the latter does not. Neither should since no extras were specified.

To Reproduce

Using this example pyproject.toml, notice how the numpy dep is optional but in a bracket:

[tool.poetry]
name = "example"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"

numpy = [
  { version = "*", optional = true },
]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

run poetry lock

run cyclonedx-py poetry -o sbom.json --of json --sv 1.4

Expected behavior

The contents of sbom.json to not include numpy. E.g., the expected output below was generated by defining the numpy dependency like so: numpy = { version = "*", optional = true }:

{
  "dependencies": [
    {
      "ref": "example"
    }
  ],
  "metadata": {
    "component": {
      "bom-ref": "example",
      "description": "",
      "name": "example",
      "type": "application",
      "version": "0.1.0"
    },
    "timestamp": "2025-01-14T20:18:24.420375+00:00",
    "tools": [
      {
        "externalReferences": [
          {
            "type": "build-system",
            "url": "https://github.com/CycloneDX/cyclonedx-python/actions"
          },
          {
            "type": "distribution",
            "url": "https://pypi.org/project/cyclonedx-bom/"
          },
          {
            "type": "documentation",
            "url": "https://cyclonedx-bom-tool.readthedocs.io/"
          },
          {
            "type": "issue-tracker",
            "url": "https://github.com/CycloneDX/cyclonedx-python/issues"
          },
          {
            "type": "license",
            "url": "https://github.com/CycloneDX/cyclonedx-python/blob/main/LICENSE"
          },
          {
            "type": "release-notes",
            "url": "https://github.com/CycloneDX/cyclonedx-python/blob/main/CHANGELOG.md"
          },
          {
            "type": "vcs",
            "url": "https://github.com/CycloneDX/cyclonedx-python/"
          },
          {
            "type": "website",
            "url": "https://github.com/CycloneDX/cyclonedx-python/#readme"
          }
        ],
        "name": "cyclonedx-py",
        "vendor": "CycloneDX",
        "version": "5.1.1"
      },
      {
        "externalReferences": [
          {
            "type": "build-system",
            "url": "https://github.com/CycloneDX/cyclonedx-python-lib/actions"
          },
          {
            "type": "distribution",
            "url": "https://pypi.org/project/cyclonedx-python-lib/"
          },
          {
            "type": "documentation",
            "url": "https://cyclonedx-python-library.readthedocs.io/"
          },
          {
            "type": "issue-tracker",
            "url": "https://github.com/CycloneDX/cyclonedx-python-lib/issues"
          },
          {
            "type": "license",
            "url": "https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/LICENSE"
          },
          {
            "type": "release-notes",
            "url": "https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/CHANGELOG.md"
          },
          {
            "type": "vcs",
            "url": "https://github.com/CycloneDX/cyclonedx-python-lib"
          },
          {
            "type": "website",
            "url": "https://github.com/CycloneDX/cyclonedx-python-lib/#readme"
          }
        ],
        "name": "cyclonedx-python-lib",
        "vendor": "CycloneDX",
        "version": "8.5.0"
      }
    ]
  },
  "serialNumber": "urn:uuid:9f17742f-841c-4b03-b32a-e12d9249d25e",
  "version": 1,
  "$schema": "http://cyclonedx.org/schema/bom-1.4.schema.json",
  "bomFormat": "CycloneDX",
  "specVersion": "1.4"
}

Screenshots or output-paste

Whats actually happening is numpy is added to the sbom even though i didn't specify any extras that include it (full output here):

Image

Environment

  • cyclonedx-py version: 5.1.1
  • Python version: 3.10
  • Poetry version: 1.8.3
  • OS: Darwin

Additional context

I used numpy to simplify the example here, but this becomes a real issue for more complex cases like torch. torch(cpu) and torch(cuda) have different dependencies, and the way you'd allow a user to select either or would use those brackets, something like this:

torch = [
  { version = "2.2.0+cpu", markers = "extra == 'cpu' and extra != 'cuda'", source = "pytorch-cpu", optional = true},
  { version = "2.2.0", markers = "extra == 'cuda' and extra != 'cpu' ", source = "pypi", optional = true },
]

When generating an sbom on something like this, even when just the cpu extra is given, all the cuda/nvidia dependencies are included too (and they should not be since torch 2.2.0+cpu does not depend on any nvidia stuff).

@rocketrose99 rocketrose99 added the bug Something isn't working label Jan 14, 2025
@jkowalleck
Copy link
Member

jkowalleck commented Jan 15, 2025

@rocketrose99 , thank you for the report.

Are you interested in providing a fix for the bug?
If so, please let me know, so we can discuss further steps.

@jkowalleck
Copy link
Member

i will work on a fix soon

@jkowalleck jkowalleck self-assigned this Jan 21, 2025
@jkowalleck
Copy link
Member

jkowalleck commented Jan 21, 2025

I tried to reproduce, and saw the following behavior:

  • poetry did not install any of the optional dependencies.
  • as described in the ticket: SBOM did include the dependencies unexpectedly

[tool.poetry.dependencies]
python  = "^3.8"
cyclonedx-python-lib = {version = ">=8,<9", optional = true}
py-serializable = [
  {version = "*", optional = true}
]

[tool.poetry.extras]
# no extras - see https://github.com/CycloneDX/cyclonedx-python/issues/840

debug output:

DEBUG    | CDX > args: {'command': 'poetry', 'groups_without': [], 'groups_with': [], 'groups_only': [], 'no_dev': False, 'extras': [], 'all_extras': False, 'mc_type': <ComponentType.APPLICATION: 'application'>, 'project_directory': 'tests/_data/infiles/poetry/with-optionals-no-extra/lock20/', 'short_purls': False, 'outfile': <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, 'schema_version': <SchemaVersion.V1_5: (1, 5)>, 'output_format': <OutputFormat.JSON: 1>, 'output_reproducible': False, 'should_validate': True, '_bbc': <class 'cyclonedx_py._internal.poetry.PoetryBB'>}
INFO     | CDX > Generating SBOM ...
DEBUG    | CDX.PoetryBB > use_groups: frozenset({'main', 'dev'})
DEBUG    | CDX.PoetryBB > use_extras: frozenset()
DEBUG    | CDX.PoetryBB > root-component: <Component bom-ref=<BomRef 'with-optionals-no-extra' id=136575604274768>, group=None, name=with-optionals-no-extra, version=0.1.0, type=ComponentType.APPLICATION>
DEBUG    | CDX.PoetryBB > lock_version: (2, 0)
DEBUG    | CDX.PoetryBB > root-component depends on python
DEBUG    | CDX.PoetryBB > root-component depends on cyclonedx-python-lib
DEBUG    | CDX.PoetryBB > skip optional unused dependency: cyclonedx-python-lib
DEBUG    | CDX.PoetryBB > root-component depends on py-serializable
INFO     | CDX.PoetryBB > add component for package 'py-serializable'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'py-serializable@1.1.2' id=136575604317392>, group=None, name=py-serializable, version=1.1.2, type=ComponentType.LIBRARY>
INFO     | CDX.PoetryBB > add component for package 'defusedxml'
DEBUG    | CDX.PoetryBB > add component: <Component bom-ref=<BomRef 'defusedxml@0.7.1' id=136575604312016>, group=None, name=defusedxml, version=0.7.1, type=ComponentType.LIBRARY>
INFO     | CDX > Serializing SBOM: 1.5/JSON
INFO     | CDX > Validating result to schema: 1.5/JSON
DEBUG    | CDX > result is schema-valid
INFO     | CDX > Writing to: <stdout>
DEBUG    | CDX > Wrote 6285 bytes to <stdout>

remarks:
i see DEBUG | CDX.PoetryBB > skip optional unused dependency: cyclonedx-python-lib
but i dont see the same for py-serializable

@jkowalleck jkowalleck changed the title [BUG] optional dependencies are treated as root dependencies when surrounded by brackets [BUG] multi-declare (optional) dependencies are treated as root dependencies when surrounded by brackets Jan 21, 2025
@jkowalleck
Copy link
Member

the bug was not that optional were not detected, but that multi-constraint handling (multiple version/instruction/constraint declaration for one and the same dependency) was poorly implemented.

@jkowalleck jkowalleck changed the title [BUG] multi-declare (optional) dependencies are treated as root dependencies when surrounded by brackets [BUG] multi-declare (optional) dependencies are treated as root dependencies Jan 21, 2025
jkowalleck added a commit that referenced this issue Jan 21, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
#842)

fixes [#840](#840)

---------

Signed-off-by: Jan Kowalleck <jan.kowalleck@gmail.com>
@jkowalleck
Copy link
Member

jkowalleck commented Jan 21, 2025

@rocketrose99
Copy link
Author

Thank you so much! Sorry for going silent, my husband had a surgery over the weekend so I appreciate you taking care of this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working source: poetry
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants