fix: Fix handling of "format" in package includes initialization

[finswimmer]

Ensure that "format" is always processed as a list when initializing package includes. If not explicit set default to sdist and wheel.

Resolves: python-poetry/poetry#9961

• Added tests for changed code.
• Updated documentation for changed code.

Summary by Sourcery

Bug Fixes:

• Fix handling of "format" in package includes initialization.

Summary by Sourcery

Bug Fixes:

• Resolve an issue where package includes were not correctly handled if the "format" field was not a list.

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request fixes the handling of the "format" key in package includes initialization by ensuring that "format" is always processed as a list. If the "format" key is not explicitly set, it defaults to ["sdist", "wheel"] for packages and ["sdist"] for includes.

Sequence diagram for package format initialization

sequenceDiagram
    participant Factory as Poetry Factory
    participant Package as Project Package

    Factory->>Factory: _prepare_formats(items, default_formats)
    Note over Factory: Process format as list
    Factory->>Factory: _configure_package_poetry_specifics()
    Factory->>Package: include = prepare_formats(includes, ['sdist'])
    Factory->>Package: packages = prepare_formats(packages, ['sdist', 'wheel'])

Class diagram showing Factory and Package relationship

Unable to render rich display

Could not find a suitable point for the given distance

For more information, see https://docs.github.com/get-started/writing-on-github/working-with-advanced-formatting/creating-diagrams#creating-mermaid-diagrams

classDiagram
    class Factory {
        +_prepare_formats(items: list, default_formats: list)
        +_configure_package_poetry_specifics(package: ProjectPackage, tool_poetry: dict)
    }

    class ProjectPackage {
        +include: list
        +packages: list
        +build_config: dict
        +exclude: list
    }

    Factory ..> ProjectPackage: configures

File-Level Changes

Change	Details	Files
Added default value handling for "format" key in package includes.	Added the "_prepare_formats" method to handle default values for the "format" key. Modified the "_configure_package_poetry_specifics" method to use the "_prepare_formats" method.	src/poetry/core/factory.py
Updated package includes initialization to use the new "format" handling.	Simplified the initialization logic for packages and includes in the "builder.py" file. Updated test cases in "test_factory.py" to reflect the new default "format" values.	src/poetry/core/masonry/builders/builder.py tests/test_factory.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time. You can also use
  this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[Secrus]

A test would be in order

[sourcery-ai]

Hey @finswimmer - I've reviewed your changes - here's some feedback:

Overall Comments:

• Please update the documentation to reflect the format parameter behavior and default values, as indicated in the PR checklist.

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟡 Testing: 1 issue found
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

suggestion (testing): Add a test case for an invalid format.

It would be beneficial to add a test case that uses an invalid format value (e.g., an integer, a dictionary, or an invalid string) to ensure that the code handles such scenarios gracefully and raises an appropriate error.

Suggested implementation:

    package = poetry.package
    assert "format" not in package.packages[0]

    if given_format:
        package.packages[0]["format"] = given_format  # type: ignore[index]

    module = Module(
        name=package.name, directory=str(directory), packages=package.packages
    )
    assert len(module.includes) == 1
    assert module.includes[0].formats == expected_formats

def test_module_with_invalid_format(poetry):
    directory = poetry.file.parent
    package = poetry.package

    # Test with an integer format (invalid)
    package.packages[0]["format"] = 42  # type: ignore[index]

    with pytest.raises(ValueError) as exc_info:
        Module(
            name=package.name, directory=str(directory), packages=package.packages
        )
    assert "Invalid format value" in str(exc_info.value)

    # Test with a dictionary format (invalid)
    package.packages[0]["format"] = {"key": "value"}  # type: ignore[index]

    with pytest.raises(ValueError) as exc_info:
        Module(
            name=package.name, directory=str(directory), packages=package.packages
        )
    assert "Invalid format value" in str(exc_info.value)

Note that this test assumes that the Module class already has validation that raises ValueError for invalid formats. If it doesn't, you'll need to:

1. Add format validation in the Module class
2. Adjust the error message assertions to match the actual error messages used in the Module class
3. Import pytest if it's not already imported at the top of the file

[dimbleby]

wasn't the point of #773 to default this only to "sdist"?

[finswimmer]

In my understanding this was about the behavior of format for the include keyword and not for packages. 🤷I guess @radoering can say more about it.

[dimbleby]

can confirm that I find this confusing

so I see, #773 had Builder._module() patch up the package and include objects to add default values if not present - but there is nothing corresponding to this in poetry's RunCommand._module().

Perhaps a better way would be to kill both birds with one stone and do this defaulting at the time the package definition is read, I think in poetry-core's Factory._configure_package_poetry_specifics. Then it is always available for everyone and no further special casing is needed.

[radoering]

Sounds better than having two spots for the default handling.

[finswimmer]

Ah, I guess you found the places I was looking for initially. I'm on mobile so I have to take a look at later/tomorrow.

[radoering]

Maybe, the fix should be downstream. Give me some time to think about it. Currently, all the default handling is in

poetry-core/src/poetry/core/masonry/builders/builder.py

Lines 49 to 65 in 61081dd

	packages: list[dict[str, str | dict[str, str]]] = []
	includes: list[dict[str, str | dict[str, str]]] = []
	for source_list, target_list, default in [
	(self._package.packages, packages, ["sdist", "wheel"]),
	(self._package.include, includes, ["sdist"]),
	]:
	for item in source_list:
	# Default to including in both sdist & wheel
	# if the `format` key is not provided.
	formats = item.get("format", default)
	if not isinstance(formats, list):
	formats = [formats]
	if self.format and self.format not in formats:
	continue
	target_list.append({**item, "format": formats})

That is why it does not work for poetry run. We have different defaults for package includes and include includes. See python-poetry/poetry#9691 for details.

[radoering]

I am wondering if the downstream tests show some unwanted behavior. I remember these tests were the reason I did not do this in the factory but I do not remember if it is a real issue or just the tests... It might be that we are producing unwanted changes in the pyproject.toml now.

[finswimmer]

I am wondering if the downstream tests show some unwanted behavior. I remember these tests were the reason I did not do this in the factory but I do not remember if it is a real issue or just the tests... It might be that we are producing unwanted changes in the pyproject.toml now.

My previous implementation was problematic because it changes the data read from the toml directly. I changed it, so that the toml data stays untouched and only the data that goes into the package.packages attribute is changed.

The only downstream test that is failing, is the same, that we need to adopt in poetry-core. I'm not sure how useful it is, to have this test in poetry itself.

[radoering]

Looks good now.

I'm not sure how useful it is, to have this test in poetry itself.

Also not sure. It might make sense because downstream we inherit from Factory and may want to check that we did not break something.

Can you prepare a downstream PR? (I think we can merge this with the failing test but we should update downstream to use poetry-core's master branch shortly afterwards so that we keep the time of failing downstream tests short.)

[radoering]

One more thing: Can we move the default handling completely into Factory? Add default handling for package.include and remove

poetry-core/src/poetry/core/masonry/builders/builder.py

Lines 49 to 65 in 61081dd

	packages: list[dict[str, str | dict[str, str]]] = []
	includes: list[dict[str, str | dict[str, str]]] = []
	for source_list, target_list, default in [
	(self._package.packages, packages, ["sdist", "wheel"]),
	(self._package.include, includes, ["sdist"]),
	]:
	for item in source_list:
	# Default to including in both sdist & wheel
	# if the `format` key is not provided.
	formats = item.get("format", default)
	if not isinstance(formats, list):
	formats = [formats]
	if self.format and self.format not in formats:
	continue
	target_list.append({**item, "format": formats})

[finswimmer]

@sourcery-ai review

[sourcery-ai]

Hey @finswimmer - I've reviewed your changes - here's some feedback:

Overall Comments:

• Please update the documentation to describe the default format values (sdist+wheel for packages, sdist for includes) and that the format field should be a list.

Here's what I looked at during the review

• 🟡 General issues: 1 issue found
• 🟢 Security: all looks good
• 🟡 Testing: 2 issues found
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

suggestion (testing): Test explicit format exclusion

Add a test case where "format" explicitly excludes a default format (e.g., format: [] or format: ['wheel'] for a package include) to verify that the package is correctly excluded from the specified format.

Suggested implementation:

    assert package.include == [
        {"path": "extra_dir/vcs_excluded.py", "format": ["sdist", "wheel"]},
        {"path": "notes.txt", "format": ["sdist"]},
        {"path": "format_excluded.txt", "format": ["wheel"]},
    ]

You'll also need to:

1. Create a 'format_excluded.txt' file in the test directory structure
2. Add the corresponding entry in the test setup/fixture that creates this test file
3. Update any related test data that defines the package includes to include this new file

[finswimmer]

Can you prepare a downstream PR?

I prepared one here: python-poetry/poetry#9992

One more thing: Can we move the default handling completely into Factory?

Done. (At least I think so 😀 )

[abn]

@sourcery-ai review

[sourcery-ai]

Hey @finswimmer - I've reviewed your changes and they look great!

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}