Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for DependabotAlert APIs #2879

Merged
merged 11 commits into from Jan 26, 2024
Merged

Conversation

coopernetes
Copy link
Contributor

@coopernetes coopernetes commented Jan 20, 2024

Fixes #2872

Related #2347 (Dependabot alerts are generated from the underlying dependency graph)

This PR includes the following to bring in all the supported operations for the Dependabot alert APIs:

  • Add new types for DependabotAlert & related classes for advisory & vulnerability information. DependabotAlert is a subclass of AdvisoryBase. Reused existing types for CVSS & CWE since they are shared attributes.
  • Add get_dependabot_alert for Repository
  • Add get_dependabot_alerts for Repository using PaginatedList
  • Add update_dependabot_alert for Repository
  • Add get_dependabot_alerts for Organization using PaginatedList
  • Add get_dependabot_alerts for Enterprise using PaginatedList GitHub's endpoint doesn't work from my limited testing
  • Decoupled AdvisoryVulnerability from AdvisoryBase and moved that specific attribute into RepositoryAdvisory & GlobalAdvisory. Most of the vulnerability information is similar between advisories and Dependabot alerts but there were a few differences in the JSON and I decided to split them up from the base class.
  • Test coverage (TYPE_CHECKING imports seem to be missed but not sure how those would get coverage)
  • Ensure documentation is complete for new types (verified new functions on Repository & Organization have the correct docstrings)
  • Generate appropriate copyright notices on new or changed files

Details on my refactoring amongst the other security-related modules/classes:

After reviewing the existing types for AdvisoryBase, GlobalAdvisory, RepositoryAdvisory and AdvisoryVulnerability, it's clear that the Dependabot alert API looks very similar to those other endpoints with some annoyingly minor differences. For example:

  • dependency.package for a Dependabot alert is identical to AdvisoryVulnerabilityPackage but has additional fields (manifest_path and scope) so I couldn't reuse the entirety of AdvisoryVulnerability. There's likely a place for a Union type but I didn't have a good sense on the best way to write that.
  • vulnerabilities list for a Dependabot alert is slightly different than the Repository/GlobalAdvisory vulnerabilities (the former includes severity and first_patched_version object, the latter uses patched_versions & contains vulnerable_functions)
  • references are their own list of dicts for Dependabot alerts (with a single field called "url" 🤦) but are just a list of strings for GlobalAdvisory

Instead of monkeying around with too much of the existing modules to try & find all the common attributes and create new base modules, I kept most of the Dependabot specific API types as separate modules & classes. I was able to reuse AdvisoryVulnerabilityPackage.

The Dependabot Alert API takes its information from the dependency graph. However, there is no explicit dependency graph endpoints available via GitHub's REST API. Included "subclasses" which DependabotAlert is an aggregate of.
@codecov-commenter
Copy link

codecov-commenter commented Jan 20, 2024

Codecov Report

Attention: 9 lines in your changes are missing coverage. Please review.

Comparison is base (d0caa3c) 96.70% compared to head (fe726ea) 96.69%.

❗ Current head fe726ea differs from pull request most recent head aab2b21. Consider uploading reports for the commit aab2b21 to get more accurate results

Files Patch % Lines
github/DependabotAlert.py 96.11% 4 Missing ⚠️
github/DependabotAlertAdvisory.py 95.83% 1 Missing ⚠️
github/DependabotAlertVulnerability.py 96.96% 1 Missing ⚠️
github/Organization.py 94.11% 1 Missing ⚠️
github/Repository.py 96.77% 1 Missing ⚠️
github/RepositoryAdvisory.py 87.50% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2879      +/-   ##
==========================================
- Coverage   96.70%   96.69%   -0.01%     
==========================================
  Files         142      147       +5     
  Lines       14610    14848     +238     
==========================================
+ Hits        14129    14358     +229     
- Misses        481      490       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@coopernetes
Copy link
Contributor Author

coopernetes commented Jan 20, 2024

@EnricoMi thanks for taking a look at the reported enhancement request. When you get a chance, can you give a quick 👀 on the few changed files in this PR? I've added the list of tasks I will chip away at as part of adding this support but wanted to ensure I was using the correct patterns.

I also reviewed the Contributing guide which helped immensely to get the first test case recorded and written 👍

In the meantime, I'll adapt the new class using CodeScanAlert as a guide since that module seems to closely match what the Dependabot alert API does. Thanks!

DependabotAlert includes a mix of similar properties to RepositoryAdvisory, GlobalAdvisory and AdvisoryBase. However, there are certain fields that have different data so we need separate types for them.

add remaining attributes for DependabotAlert
…Repository

move vulnerabilities out of AdvisoryBase, remove typeddicts
@coopernetes coopernetes marked this pull request as ready for review January 22, 2024 03:39
Comment on lines 1258 to 1260
if is_defined(state):
assert isinstance(state, str), state
assert state in allowed_states, f"State can be one of {', '.join(allowed_states)}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd simplify this pattern to

Suggested change
if is_defined(state):
assert isinstance(state, str), state
assert state in allowed_states, f"State can be one of {', '.join(allowed_states)}"
assert state in allowed_states + [NotSet], f"State can be one of {', '.join(allowed_states)}"

if is_defined(state):
assert isinstance(state, str), state
assert state in allowed_states, f"State can be one of {', '.join(allowed_states)}"
url_parameters["state"] = state
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd replace the

        if is_defined(state):
            url_parameters["state"] = state
...

pattern with

        url_parameters = NotSet.remove_unset_items(
            {
                "state": state,
                "severity": severity,
                "ecosystem": ecosystem,
                "package": package,
                "scope": scope,
                "sort": sort,
                "direction": direction,
            }
        )

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, applied the same for Repository.get_dependabot_alerts.

Copy link
Collaborator

@EnricoMi EnricoMi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@EnricoMi
Copy link
Collaborator

Thanks for this high-quality PR! This is ready to go, right?

@coopernetes
Copy link
Contributor Author

Yep, good to go! Thanks @EnricoMi

@EnricoMi EnricoMi merged commit 14af705 into PyGithub:main Jan 26, 2024
15 checks passed
@EnricoMi
Copy link
Collaborator

Your PR was really great, I could use some help reviewing PRs in PyGithub.

Is there a chance that from time to time you could have a look at open PRs and guide less experienced authors towards an approvable state?

lettuce-bot bot added a commit to lettuce-financial/github-bot-signed-commit that referenced this pull request Jan 30, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [PyGithub](https://togithub.com/pygithub/pygithub) | `==2.1.1` ->
`==2.2.0` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/PyGithub/2.2.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/PyGithub/2.2.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/PyGithub/2.1.1/2.2.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/PyGithub/2.1.1/2.2.0?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>pygithub/pygithub (PyGithub)</summary>

###
[`v2.2.0`](https://togithub.com/PyGithub/PyGithub/releases/tag/v2.2.0)

[Compare
Source](https://togithub.com/pygithub/pygithub/compare/v2.1.1...v2.2.0)

#### Breaking Changes

The `github.Comparison.Comparison` instance returned by
`Repository.compare` provides a `commits` property that used to return a
`list[github.Commit.Commit]`, which has now been changed to
`PaginatedList[github.Commit.Commit]`. This breaks user code that
assumes a `list`:

```python
commits = repo.compare("v0.6", "v0.7").commits
no_of_commits = len(commits)  # will raise a TypeError
```

This will raise a `TypeError: object of type 'PaginatedList' has no
len()`, as the returned `PaginatedList`
does not support the `len()` method. Use the `totalCount` property
instead:

```python
commits = repo.compare("v0.6", "v0.7").commits
no_of_commits = commits.totalCount
```

#### New features

-   Add support to call GraphQL API

#### Improvements

- Add parent_team_id, maintainers and notification_setting for creating
and updating teams. by
[@&#8203;Cheshirez](https://togithub.com/Cheshirez) in
[PyGithub/PyGithub#2863
- Add support for issue reactions summary by
[@&#8203;smuzaffar](https://togithub.com/smuzaffar) in
[PyGithub/PyGithub#2866
- Support for DependabotAlert APIs by
[@&#8203;coopernetes](https://togithub.com/coopernetes) in
[PyGithub/PyGithub#2879
- Derive GraphQL URL from base_url by
[@&#8203;EnricoMi](https://togithub.com/EnricoMi) in
[PyGithub/PyGithub#2880
- Make `Repository.compare().commits` return paginated list by
[@&#8203;EnricoMi](https://togithub.com/EnricoMi) in
[PyGithub/PyGithub#2882
- Add missing branch protection fields by
[@&#8203;treee111](https://togithub.com/treee111) in
[PyGithub/PyGithub#2873
- Add `include_all_branches` to `create_repo_from_template` of
`AuthenticatedUser` and `Organization` by
[@&#8203;janssonoskar](https://togithub.com/janssonoskar) in
[PyGithub/PyGithub#2871
- Add and update organisation dependabot secrets by
[@&#8203;mohy01](https://togithub.com/mohy01) in
[PyGithub/PyGithub#2316
- Add missing params to `Organization.create_repo` by
[@&#8203;tekumara](https://togithub.com/tekumara) in
[PyGithub/PyGithub#2700
- Update allowed values for `Repository` collaborator permissions by
[@&#8203;flying-sheep](https://togithub.com/flying-sheep) in
[PyGithub/PyGithub#1996
- Support editing PullRequestReview by
[@&#8203;ColasGael](https://togithub.com/ColasGael) in
[PyGithub/PyGithub#2851
- Update attributes after calling `PullRequestReview.dismiss` by
[@&#8203;ColasGael](https://togithub.com/ColasGael) in
[PyGithub/PyGithub#2854
- Add `request_cve` on `RepositoryAdvisories` by
[@&#8203;JLLeitschuh](https://togithub.com/JLLeitschuh) in
[PyGithub/PyGithub#2855
- Filter collaborators of a repository by permissions by
[@&#8203;notmicaelfilipe](https://togithub.com/notmicaelfilipe) in
[PyGithub/PyGithub#2792
- Set pull request to auto merge via GraphQL API by
[@&#8203;heitorpolidoro](https://togithub.com/heitorpolidoro) in
[PyGithub/PyGithub#2816
- Support Environment Variables and Secrets by
[@&#8203;AndrewJDawes](https://togithub.com/AndrewJDawes) in
[PyGithub/PyGithub#2848
- Update workflow.get_runs & pullrequest.add_to_assignees function
signature by [@&#8203;sd-kialo](https://togithub.com/sd-kialo) in
[PyGithub/PyGithub#2799
- Add `GithubObject.last_modified_datetime` to have `last_modified` as a
`datetime` by [@&#8203;chouetz](https://togithub.com/chouetz) in
[PyGithub/PyGithub#2772
- Add support for global advisories and unify some shared logic with
repository advisories by
[@&#8203;crimsonknave](https://togithub.com/crimsonknave) in
[PyGithub/PyGithub#2702
- Add `internal` as valid Repository visibility value by
[@&#8203;AndrewJDawes](https://togithub.com/AndrewJDawes) in
[PyGithub/PyGithub#2806
- Add support for issue comments reactions summary by
[@&#8203;smuzaffar](https://togithub.com/smuzaffar) in
[PyGithub/PyGithub#2813

#### Bug Fixes

- Add a bunch of missing urllib.parse.quote calls by
[@&#8203;ExplodingCabbage](https://togithub.com/ExplodingCabbage) in
[PyGithub/PyGithub#1976
- Fix Variable and Secret url bugs by
[@&#8203;AndrewJDawes](https://togithub.com/AndrewJDawes) in
[PyGithub/PyGithub#2835

#### Maintenance

- Update the class name for NetrcAuth in the examples by
[@&#8203;vinnybod](https://togithub.com/vinnybod) in
[PyGithub/PyGithub#2860
- Move build to PEP517 by [@&#8203;trim21](https://togithub.com/trim21)
in
[PyGithub/PyGithub#2800
- Use new type assert functions in `Repository` by
[@&#8203;trim21](https://togithub.com/trim21) in
[PyGithub/PyGithub#2798
- PyTest: Move config to pyproject.toml by
[@&#8203;Borda](https://togithub.com/Borda) in
[PyGithub/PyGithub#2859
- codespell: ignore-words-list by
[@&#8203;Borda](https://togithub.com/Borda) in
[PyGithub/PyGithub#2858
- Improve fix-headers.py script by
[@&#8203;EnricoMi](https://togithub.com/EnricoMi) in
[PyGithub/PyGithub#2728
- Remove dependency on python-dateutil by
[@&#8203;lazka](https://togithub.com/lazka) in
[PyGithub/PyGithub#2804
- CI: update precommit & apply by
[@&#8203;Borda](https://togithub.com/Borda) in
[PyGithub/PyGithub#2600
- Docs: Fix parameter order according to Version 2.1.0 by
[@&#8203;nad182](https://togithub.com/nad182) in
[PyGithub/PyGithub#2786
- Add missing GitHub classes to docs by
[@&#8203;EnricoMi](https://togithub.com/EnricoMi) in
[PyGithub/PyGithub#2783
- CI: Fix mypy error by ignoring override by
[@&#8203;EnricoMi](https://togithub.com/EnricoMi) in
[PyGithub/PyGithub#2779

**Full Changelog**:
PyGithub/PyGithub@v2.1.1...v2.2.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/lettuce-financial/github-bot-signed-commit).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xNTMuMiIsInVwZGF0ZWRJblZlciI6IjM3LjE1My4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
@coopernetes
Copy link
Contributor Author

@EnricoMi Apologies for the late reply! Absolutely, I am available to help with triage & reviewing PRs to support the library. Feel free to assign me PRs or issues and don't hesitate to send me an email if you need to discuss anything else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dependabot alert API support
3 participants