Enable OpenSSF Scorecard Github Action and Badge

[joycebrum]

Component

No response

Is your feature request related to a problem? Please describe

Hi, I am Joyce from Google and I'm working on behalf of the Open Source Security Foundation to help essential open-source projects improve their supply-chain security. Considering how systemd is relevant to countless projects, I want to offer my support to improve the project's supply-chain security.

According to Open Source Security and Risk Analysis Report, 84% of all codebases have at least one vulnerability, with an average of 158 per codebase. The majority have been in the code for more than 2 years and have documented solutions available.

Even in large tech companies, the tedious process of reviewing code for vulnerabilities falls down the priority list, and there is little insight into known vulnerabilities and solutions that companies can draw on.

That’s where the OpenSSF Scorecards tool is helping. The tool's focus is to help maintainers understand their project's security posture and assess the risks that dependencies could introduce.

Describe the solution you'd like

The tool I want to suggest here is the Scorecards. Scorecards runs dozens of automated security checks to help maintainers better understand their project's supply-chain security posture. It is developed by the OpenSSF, [in partnership with GitHub][sc-blog].

To make maintainers' lives easier, the OpenSSF has also developed the Scorecard GitHub Action. It is very lightweight and runs on every change to the repository's main branch. The results of its checks are available on the project's security dashboard, and include suggestions on how to solve any issues (see examples below). The Action does not run or interact with any workflows, but merely parses them to identify possible vulnerabilities. This Action has been adopted by 1800+ projects already, having some prominent users like Tensorflow, Angular, Flutter, sos.dev and deps.dev.

As I've seen in the systemd project, you scored extremely high, being on top 0.1%! Congratulations to all the team for all the work to improve the overall security of the project, such as implementing the OpenSSF Security Best Practices, OSS-Fuzzing, Security Policy, etc.

Considering this scenario, the scorecard action and badge would be a great way to show all your commitment to security posture and also guarantee that future changes on the project won't affect negatively the project's security posture, or even improve even more the security posture of the project.

Would you be interested in a PR which adds this Action? Optionally, it can also publish your results to the OpenSSF REST API, which allows a badge with the project's score to be added to its README.

In case of doubts or concerns you can try to check Scorecards FAQ. Anyway, feel free to reach me out.

Describe alternatives you've considered

No response

The systemd version you checked that didn't have the feature you are asking for

No response

[bluca]

Hi, thanks for taking the time to post this

Would you be interested in a PR which adds this Action? Optionally, it can also publish your results to the OpenSSF REST API, which allows a badge with the project's score to be added to its README.

Sounds interesting to me, as long as it's not invasive - do you have an example of an existing project doing this to look at?

[joycebrum]

Hi, thanks for taking the time to post this

Would you be interested in a PR which adds this Action? Optionally, it can also publish your results to the OpenSSF REST API, which allows a badge with the project's score to be added to its README.

Sounds interesting to me, as long as it's not invasive - do you have an example of an existing project doing this to look at?

I have some PR merget at mihaimaruseac/hindent#595 and an example of a project that have asked for the "minimum features" ethereum/solc-js#667

[evverx]

@joycebrum it seems to me that the scorecard project isn't particularly interested in fixing its false positives, making it work with PRs and so on. That json dashboard (with its promotional links) has never been made human-friendly either. I wonder if it would make sense to wait for those issues to ever be addressed or should it be removed instead? I don't think this action is useful.

[evverx]

As far as I understand GOOST is responsible for this stuff so it would be great if someone could help with google/oss-fuzz#10090 (comment)

[evverx]

ossf/scorecard#2018 (comment) was supposed to be addressed in Q1 (which ended a month ago). I suspect it's safe to assume that those issues aren't going to be fixed.

[joycebrum]

As far as I understand GOOST is responsible for this stuff so it would be great if someone could help with google/oss-fuzz#10090 (comment)

About this, AFAIK it is not a problem to grant write permission to security-events to oss-fuzz because in the scorecard documetation https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions it says it can be granted to "recognized action for uploading SARIF results." and the oss-fuzz certainly will be the case.

Thus, I think we don't need to worry about that.

[evverx]

the oss-fuzz certainly will be the case

I'm not sure about that. systemd has been "insecure" because of CIFuzz/CFLite (provided by OSS-Fuzz) for more than a year

      "details": [
        "Warn: third-party GitHubAction not pinned by hash: .github/workflows/cflite_pr.yml:28: update your workflow using https://app.stepsecurity.io/secureworkflow/systemd/systemd/cflite_pr.yml/main?enable=pin",
        "Warn: third-party GitHubAction not pinned by hash: .github/workflows/cflite_pr.yml:34: update your workflow using https://app.stepsecurity.io/secureworkflow/systemd/systemd/cflite_pr.yml/main?enable=pin",
        "Warn: third-party GitHubAction not pinned by hash: .github/workflows/cifuzz.yml:41: update your workflow using https://app.stepsecurity.io/secureworkflow/systemd/systemd/cifuzz.yml/main?enable=pin",
        "Warn: third-party GitHubAction not pinned by hash: .github/workflows/cifuzz.yml:51: update your workflow using https://app.stepsecurity.io/secureworkflow/systemd/systemd/cifuzz.yml/main?enable=pin",
        "Warn: containerImage not pinned by hash: .clusterfuzzlite/Dockerfile:1: pin your Docker image by updating gcr.io/oss-fuzz-base/base-builder:v1 to gcr.io/oss-fuzz-base/base-builder:v1@sha256:bdc13ad999951672cd6dac25a1c5251739f980564d484619d9f79d7910fa08c0",

and it has never been addressed even though it has been reported a gazillion times.

[evverx]

Just to clarify scorecard isn't fully non-sensical here because I rolled up my sleeves and fixed scorecard (instead of systemd) back in the day: https://github.com/ossf/scorecard/commits?author=evverx but I can't fix all its stuff obviously.

[joycebrum]

About the false positives: have you tried using the "dismiss alert" github function?

I though it should dismiss the alert automatically even in future runs in order to preventing you for being notified over a already dismissed alert. If you tried, does it not work?

This way the action would be useful on only notifying you about real possible fix/improvements and not bothering you with the "false positive" ones.

[joycebrum]

I'm not sure about that. systemd has been "insecure" because of CIFuzz/CFLite (provided by OSS-Fuzz) for more than a year

Unfortunately it reports not pinned by hash even for oss-fuzz (we've discussed some possible solutions with both teams this week), but I believe that's not the case for token permissions at least. It would allow a security-event write and, if not, it definetly should.

[evverx]

have you tried using the "dismiss alert" github function?

I think it was discussed somewhere and I think that maintainers shouldn't waste their time on that. Those are obvious false positives that should be fixed by scorecard.

Anyway I opened #27530.

[bluca]

About the false positives: have you tried using the "dismiss alert" github function?

I though it should dismiss the alert automatically even in future runs in order to preventing you for being notified over a already dismissed alert. If you tried, does it not work?

This way the action would be useful on only notifying you about real possible fix/improvements and not bothering you with the "false positive" ones.

Yes we've already dismissed the non-useful ones, so this is not a really a problem, and it flagged a few improvements that were then made. Human-readable scorecard would be really good to have though, so please do update us when that's coming. Thanks!

[evverx]

it flagged a few improvements that were then made

Which ones? The only thing it could have flagged was that unmaintained release action that can't even handle releases properly: #27209 (comment).

[evverx]

Speaking of that release action looks like scorecard could actually flag it if something like ossf/scorecard-action#1107 was implemented close to scorecard-action.

Anyway I'm still curious. What exactly did it flag and what was improved because of that?

[evverx]

Apparently it doesn't even flag those unsigned releases: ossf/scorecard#2763 (comment). systemd should get its well-deserved 0 there.

[evverx]

To judge from

"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration"

the "branch-protection" check isn't set up either. Once it's on it should downgrade the systemd score further.

[evverx]

I though it should dismiss the alert automatically even in future runs in order to preventing you for being notified over a already dismissed alert

I took a look at the security dashboard and as far as I can see the same alerts pop up over and over again (regardless of whether they are dismissed or not). I'm not sure who exactly keeps "fixing" them but the right fix for "differential-shellcheck" at least is to add it to the list of legit code scanning tools (which was discussed in google/oss-fuzz#10090 (comment)):

[evverx]

More generally alerts like that were supposed to be addressed in ossf/scorecard#2338 but it was apparently half-implemented and then abandoned.

[evverx]

Some issues got reopened and hopefully should be addressed in the foreseeable future.

ossf/scorecard#2979 was just opened (the timing is amazing I must say) and should bring human-readable dashboards like https://clomonitor.io/scorecard?platform=github.com&org=systemd&repo=systemd&theme=light. (those dashboards would be much more useful without false positives and false negatives though).

The issue where support for PRs is discussed hasn't been updated yet.

[evverx]

@joycebrum I'm not sure when the dashboard should be officially released but since I think you probably participate in scorecard meetings (where everything is decided/announced as far as I understand) could you open a PR pointing the badge to that official dashboard once it officially lands?

I think it should also be possible to remove the SARIF part of the action to prevent scorecard from producing the false positives in the security tab over and over again. Its "security-events" permissions can be dropped too. The SARIF part would make sense if the action could be run on PRs though to be able to get diffs but it seems it isn't going to happen any time soon.

Also it seems Dependabot doesn't bump versions embedded in comments when at least one workflow doesn't embed them: 43a2214#diff-649a84adf982ddf68273f1bb0de882a20e97b06632b5432a6a1504a3f23cb2e6. I think either the comments should be removed or all the workflows should be updated.

Anyway I'll go ahead and close #27530 for now. The false positives should be addressed in the foreseeable future hopefully.

[joycebrum]

Sure @evverx ! I'll open a PR updating the badge once it is released!

About the dependabot I think it should be actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # v3.0.0 to work