ci: Add GitHub artifact attestations to package distribution

[matthewfeickert]

Description

• Add generation of GitHub artifact attestations to built sdist and wheel before upload.
  c.f.:
  • https://github.blog/2024-05-02-introducing-artifact-attestations-now-in-public-beta/
  • https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds
• Add verification of artifact attestation before publishing to PyPI using the gh attestation verify CLI API, added in v2.49.0.

Checklist Before Requesting Reviewer

• Tests are passing
• "WIP" removed from the title of the pull request
• Selected an Assignee for the PR to be responsible for the log summary

Before Merging

For the PR Assignees:

• Summarize commit messages into a comprehensive review of the PR

* Add generation of GitHub artifact attestations to built sdist and wheel
  before upload.
  c.f.:
   - https://github.blog/2024-05-02-introducing-artifact-attestations-now-in-public-beta/
   - https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds
* Add verification of artifact attestation before publishing to PyPI
  using the 'gh attestation verify' CLI API, added in v2.49.0.
   - c.f. https://github.com/cli/cli/releases/tag/v2.49.0

[matthewfeickert]

Currently blocked by actions/runner-images#9784, without updating ubuntu-latest to ubuntu-24.04.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.21%. Comparing base (ecd0613) to head (1b0191c).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2473   +/-   ##
=======================================
  Coverage   98.21%   98.21%           
=======================================
  Files          69       69           
  Lines        4543     4543           
  Branches      804      804           
=======================================
  Hits         4462     4462           
  Misses         48       48           
  Partials       33       33           

Flag	Coverage Δ
contrib	97.79% <ø> (ø)
doctest	98.08% <ø> (ø)
unittests-3.10	96.23% <ø> (ø)
unittests-3.11	96.23% <ø> (ø)
unittests-3.12	96.23% <ø> (ø)
unittests-3.8	96.25% <ø> (ø)
unittests-3.9	96.27% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[matthewfeickert]

GitHub CLI is updated to v2.49.2 in actions/runner-images#9784 and so should work now.

[matthewfeickert]

So it seems that the most important takeaway is described in https://github.blog/2024-05-02-introducing-artifact-attestations-now-in-public-beta/#a-more-secure-future, and while it seems that this should mean that the number of attestations generated isn't relevant as everything will be machine generated, it seems that it might(?) be useful to keep https://github.com/scikit-hep/pyhf/attestations containing mostly only releases.

@woodruffw @di, not to bug you with things that go beyond https://github.com/sigstore/sigstore-python, but is this the correct take (that we should only be publishing attestations for releases)?

[matthewfeickert]

Also cc @sethmlarson given very helpful feedback on scientific-python/summit-2024#9 (comment).

[woodruffw]

Thanks for the ping @matthewfeickert!

IMO, the number of attestations generated is not currently relevant: these attestations don't appear on PyPI and are "generic" in nature (i.e. don't express a specific intent to build, release, etc.), so having multiple of them doesn't accomplish much at the moment.

However, to take a step back: we're building PEP 740 on the same foundations (Sigstore + sigstore-python), and it will have support for multiple attestations, e.g. for distinguishing publish, release, and also third-party counter-attestations. That PEP uses the same underlying technologies as GitHub's Attestations feature but requires attestations to be uploaded to PyPI as well, which means we'll be building it into the standard PyPA tooling.

That's still a little out and not directly related to what you asked though 😅. But the TL;DR: IMO one GitHub attestation is fine, and in the future PEP 740 will also allow your tools to (automatically) load multiple attestations per-file onto PyPI.

[matthewfeickert]

we're building PEP 740 on the same foundations (Sigstore + sigstore-python), and it will have support for multiple attestations, e.g. for distinguishing publish, release, and also third-party counter-attestations. That PEP uses the same underlying technologies as GitHub's Attestations feature but requires attestations to be uploaded to PyPI as well, which means we'll be building it into the standard PyPA tooling.

Amazing! Thank you very much for all of this work and this is actually quite useful to know about. 👍 I also now have some reading to do. :) https://peps.python.org/pep-0740/

But the TL;DR: IMO one GitHub attestation is fine, and in the future PEP 740 will also allow your tools to (automatically) load multiple attestations per-file onto PyPI.

Okay thanks this is useful. My question was more in the context of the fact that https://github.com/scikit-hep/pyhf/attestations is a log of all attestations, and if I was to remove the if guards that I place here it would generate attestations for each run of the CI, which just seems like it would be noisy. So while not a problem I'll leave the if guards in and do similar things as I propagate this out across other Scikit-HEP libraries and to Scientific Python libraries. 👍

[matthewfeickert]

Permissions added here given https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds#generating-build-provenance-for-binaries

[matthewfeickert]

@meeseeksdev backport to release/v0.7.x