Fix nightly-release workflow and tweak Poetry package naming #208
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rather than setup.py which was removed, tweak package naming
zackproser
reviewed
Oct 2, 2023
| currentDate=$(date +%Y%m%d%H%M%S) | ||
| versionNumber=$(cat $versionFile) | ||
| devVersion="${versionNumber}.dev${currentDate}" | ||
| echo "$devVersion" > $versionFile |
There was a problem hiding this comment.
This all looks reasonable to me. I was trying to think of some edge cases where it would be possible to perform a partial or incomplete version bump, but theoretically if any of these subcommand were to fail, the step itself would fail and then the GitHub Action job would fail.
zackproser
reviewed
Oct 2, 2023
| @@ -22,8 +22,11 @@ exclude = ''' | |||
| ''' | |||
|
|
|||
| [tool.poetry] | |||
| name = "pinecone" | |||
| name = "pinecone-client" | |||
There was a problem hiding this comment.
Interesting. I went back and forth on this name value in my original Poetry PR from pinecone-client back to pinecone to get everything working. However your explanation in the PR description makes sense to me.
zackproser
reviewed
Oct 2, 2023
Comment on lines
+27
to
+29
| packages = [ | ||
| { include="pinecone", from="." }, | ||
| ] |
There was a problem hiding this comment.
Read up on this and it makes sense to me.
zackproser
approved these changes
Oct 2, 2023
jhamon
approved these changes
Oct 3, 2023
1 task
austin-denoble
added a commit
that referenced
this pull request
Oct 4, 2023
## Problem `nightly-release` workflow is still failing after the previous "fix": #208. This time it's due to `make package` failing on `poetry build`, because `poetry` wasn't setup properly in the workflow. https://github.com/pinecone-io/pinecone-python-client/actions/runs/6399744230/job/17372332268 > Run make package poetry build make: poetry: No such file or directory make: *** [Makefile:22: package] Error [12](https://github.com/pinecone-io/pinecone-python-client/actions/runs/6399744230/job/17372332268#step:9:13)7 Error: Process completed with exit code 2. While fixing this I noticed we're also not installing Poetry in the `publish-to-pypi` workflow, so calling `poetry version` and `make` commands there would fail. We need to make sure we're installing Poetry properly before using `make` in our workflows, as they call Poetry under the hood. ## Solution - Add a new `setup-poetry` action that encapsulates the install + dependencies bits. I noticed we were doing this explicitly in several workflows and thought it called for an action. - Fix `nightly-release` worfklow: call the new action before calling `make package`. - Fix `release-to-pypi` worfklow: call the new action before calling `poetry version`, and `make` commands for publishing. - Clean up / use the action in `testing` worfklow. ## Type of Change - [X] Infrastructure change (CI configs, etc) ## Test Plan Validate PR workflows pass as expected. Merge and check `nightly-release` to make sure we're in good shape.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
After the poetry migration in #193 the
PyPI Release: Nightly (pinecone-client-nightly)has been failing: https://github.com/pinecone-io/pinecone-python-client/actions/runs/6374232183/job/17298658421The reason is we removed the
setup.pyfile as a part of the migration andnightly-release.yamlused to adjust the module name viased -i 's/pinecone-client/pinecone-client-nightly/g' setup.py. To achieve something similar now that we've changed our build / packaging process to use Poetry, we need to look at updating thepyproject.tomlfile.While looking at correcting this, I realized the resulting build artifacts from
make package(which now callspoetry build) now have different package names which may cause issues with PyPI:pinecone-client-2.2.4.tar.gzpinecone-2.2.4.tar.gzpinecone_client-2.2.4-py3-none-any.whlpinecone-2.2.4-py3-none-any.whlSolution
nightly-release.yamlto callsed -i ...on thepyproject.tomlfile instead ofsetup.py. This should swap outname: pinecone-clientforname: pinecone-client-nightlyunder[tools.poetry]which should fix up the package naming.packagessection topyproject.tomlto define the top-level pinecone package entrypoint. Poetry uses thenamefield by default unless this is specified which is why we were ending up with artifacts calledpinecone-...rather thanpinecone-client-....Note:
Poetry swaps the hyphen in
pinecone-clientfor an underscore in the resulting build output artifacts.New packages:
pinecone_client-2.2.4.tar.gzpinecone_client-2.2.4-py3-none-any.whlThe
.whlnaming is equivalent, and thetar.gzhas an underscore rather than a hyphen. I think ultimately this shouldn't matter given the guidance aroundnamehere:https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#name
Type of Change
Test Plan
Describe specific steps for validating this change.