Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve OpenSSF Scorecard checks #12031

Open
2 of 7 tasks
agilgur5 opened this issue Oct 18, 2023 · 1 comment
Open
2 of 7 tasks

Improve OpenSSF Scorecard checks #12031

agilgur5 opened this issue Oct 18, 2023 · 1 comment
Assignees
Labels
area/build Build or GithubAction/CI issues area/docs Incorrect, missing, or mistakes in docs solution/suggested A solution to the bug has been suggested. Someone needs to implement it. type/feature Feature request type/security Security related

Comments

@agilgur5
Copy link
Member

agilgur5 commented Oct 18, 2023

Summary

Closely related to #9769 and #11953

Can see our scores for OpenSSF Scorecard from the badge link here: https://api.securityscorecards.dev/projects/github.com/argoproj/argo-workflows

Overall Score: 6.5 -> 8.6

Scorecard JSON output, pretty-printed
{
  "date": "2023-10-16",
  "repo": {
    "name": "github.com/argoproj/argo-workflows",
    "commit": "165f57fd1b40256cdd41c41cf3fc2b9e4664c9fe"
  },
  "scorecard": {
    "version": "v4.13.0-21-g8eaf0d76",
    "commit": "8eaf0d7647a3f50d80615812c72a277fd568fb6d"
  },
  "score": 6.5,
  "checks": [
    {
      "name": "Maintained",
      "score": 10,
      "reason": "30 commit(s) out of 30 and 18 issue activity out of 30 found in the last 90 days -- score normalized to 10",
      "details": null,
      "documentation": {
        "short": "Determines if the project is \"actively maintained\".",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#maintained"
      }
    },
    {
      "name": "Code-Review",
      "score": 10,
      "reason": "all changesets reviewed",
      "details": null,
      "documentation": {
        "short": "Determines if the project requires human code review before pull requests (aka merge requests) are merged.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#code-review"
      }
    },
    {
      "name": "License",
      "score": 10,
      "reason": "license file detected",
      "details": [
        "Info: License file found in expected location: LICENSE:1",
        "Info: FSF or OSI recognized license: LICENSE:1"
      ],
      "documentation": {
        "short": "Determines if the project has defined a license.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#license"
      }
    },
    {
      "name": "CII-Best-Practices",
      "score": 5,
      "reason": "badge detected: passing",
      "details": null,
      "documentation": {
        "short": "Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#cii-best-practices"
      }
    },
    {
      "name": "Branch-Protection",
      "score": -1,
      "reason": "internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration",
      "details": null,
      "documentation": {
        "short": "Determines if the default and release branches are protected with GitHub's branch protection settings.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#branch-protection"
      }
    },
    {
      "name": "Signed-Releases",
      "score": 8,
      "reason": "5 out of 5 artifacts are signed or have provenance",
      "details": [
        "Warn: release artifact v3.5.0 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/124999540",
        "Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/130455044",
        "Warn: release artifact v3.5.0-rc2 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/121928138",
        "Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/126960660",
        "Warn: release artifact v3.4.11 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/120364554",
        "Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/125063416",
        "Warn: release artifact v3.5.0-rc1 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/117361096",
        "Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/121637664",
        "Warn: release artifact v3.4.10 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/117350773",
        "Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/121631838"
      ],
      "documentation": {
        "short": "Determines if the project cryptographically signs release artifacts.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#signed-releases"
      }
    },
    {
      "name": "Security-Policy",
      "score": 10,
      "reason": "security policy file detected",
      "details": [
        "Info: security policy file detected: SECURITY.md:1",
        "Info: Found linked content: SECURITY.md:1",
        "Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1",
        "Info: Found text in security policy: SECURITY.md:1"
      ],
      "documentation": {
        "short": "Determines if the project has published a security policy.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#security-policy"
      }
    },
    {
      "name": "Dangerous-Workflow",
      "score": 10,
      "reason": "no dangerous workflow patterns detected",
      "details": null,
      "documentation": {
        "short": "Determines if the project's GitHub Action workflows avoid dangerous patterns.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#dangerous-workflow"
      }
    },
    {
      "name": "Pinned-Dependencies",
      "score": -1,
      "reason": "internal error: error parsing shell code: Dockerfile.windows:1:5: (( can only be used to open an arithmetic cmd",
      "details": null,
      "documentation": {
        "short": "Determines if the project has declared and pinned the dependencies of its build process.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#pinned-dependencies"
      }
    },
    {
      "name": "Token-Permissions",
      "score": 0,
      "reason": "detected GitHub workflow tokens with excessive permissions",
      "details": [
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/changelog.yaml:9",
        "Warn: jobLevel 'contents' permission set to 'write': .github/workflows/changelog.yaml:16: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/ci-build.yaml:17",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/dependabot-reviewer.yml:6",
        "Warn: jobLevel 'contents' permission set to 'write': .github/workflows/dependabot-reviewer.yml:13: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
        "Warn: topLevel 'contents' permission set to 'write': .github/workflows/docs.yaml:16: Visit https://app.stepsecurity.io/secureworkflow/argoproj/argo-workflows/docs.yaml/master?enable=permissions\nTick the 'Restrict permissions for GITHUB_TOKEN'\nUntick other options\nNOTE: If you want to resolve multiple issues at once, you can visit https://app.stepsecurity.io/securerepo instead. (Low effort)",
        "Warn: no topLevel permission defined: .github/workflows/pr.yaml:1: Visit https://app.stepsecurity.io/secureworkflow/argoproj/argo-workflows/pr.yaml/master?enable=permissions\nTick the 'Restrict permissions for GITHUB_TOKEN'\nUntick other options\nNOTE: If you want to resolve multiple issues at once, you can visit https://app.stepsecurity.io/securerepo instead. (Low effort)",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/release.yaml:20",
        "Warn: jobLevel 'contents' permission set to 'write': .github/workflows/release.yaml:278: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/sdks.yaml:8",
        "Warn: jobLevel 'packages' permission set to 'write': .github/workflows/sdks.yaml:15: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
        "Warn: jobLevel 'contents' permission set to 'write': .github/workflows/sdks.yaml:16: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/snyk.yml:11"
      ],
      "documentation": {
        "short": "Determines if the project's workflows follow the principle of least privilege.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#token-permissions"
      }
    },
    {
      "name": "SAST",
      "score": 0,
      "reason": "SAST tool is not run on all commits -- score normalized to 0",
      "details": [
        "Warn: 0 commits out of 30 are checked with a SAST tool",
        "Warn: CodeQL tool not detected"
      ],
      "documentation": {
        "short": "Determines if the project uses static code analysis.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#sast"
      }
    },
    {
      "name": "Fuzzing",
      "score": 0,
      "reason": "project is not fuzzed",
      "details": [
        "Warn: no OSSFuzz integration found: Follow the steps in https://github.com/google/oss-fuzz to integrate fuzzing for your project.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)",
        "Warn: no OneFuzz integration found: Follow the steps in https://github.com/microsoft/onefuzz to start fuzzing for your project.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)",
        "Warn: no GoBuiltInFuzzer integration found: Follow the steps in https://go.dev/doc/fuzz/ to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no PythonAtherisFuzzer integration found: Follow the steps in https://github.com/google/atheris to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no CLibFuzzer integration found: Follow the steps in https://llvm.org/docs/LibFuzzer.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no CppLibFuzzer integration found: Follow the steps in https://llvm.org/docs/LibFuzzer.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no SwiftLibFuzzer integration found: Follow the steps in https://google.github.io/oss-fuzz/getting-started/new-project-guide/swift-lang/ to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no RustCargoFuzzer integration found: Follow the steps in https://rust-fuzz.github.io/book/cargo-fuzz.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no JavaJazzerFuzzer integration found: Follow the steps in https://github.com/CodeIntelligenceTesting/jazzer to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no ClusterFuzzLite integration found: Follow the steps in https://github.com/google/clusterfuzzlite to integrate fuzzing as part of CI.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)",
        "Warn: no HaskellPropertyBasedTesting integration found: Use one of the following frameworks to fuzz your project:\nQuickCheck: https://hackage.haskell.org/package/QuickCheck\nhedgehog: https://hedgehog.qa/\nvalidity: https://github.com/NorfairKing/validity\nsmallcheck: https://hackage.haskell.org/package/smallcheck\nhspec: https://hspec.github.io/\ntasty: https://hackage.haskell.org/package/tasty (High effort)",
        "Warn: no TypeScriptPropertyBasedTesting integration found: Use fast-check: https://github.com/dubzzz/fast-check (High effort)",
        "Warn: no JavaScriptPropertyBasedTesting integration found: Use fast-check: https://github.com/dubzzz/fast-check (High effort)"
      ],
      "documentation": {
        "short": "Determines if the project uses fuzzing.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#fuzzing"
      }
    },
    {
      "name": "Binary-Artifacts",
      "score": 10,
      "reason": "no binaries found in the repo",
      "details": null,
      "documentation": {
        "short": "Determines if the project has generated executable (binary) artifacts in the source repository.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#binary-artifacts"
      }
    },
    {
      "name": "Packaging",
      "score": 10,
      "reason": "publishing workflow detected",
      "details": [
        "Info: GitHub/GitLab publishing workflow used in run https://api.github.com/repos/argoproj/argo-workflows/actions/runs/6540276280: .github/workflows/ci-build.yaml:36"
      ],
      "documentation": {
        "short": "Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#packaging"
      }
    },
    {
      "name": "Vulnerabilities",
      "score": 0,
      "reason": "17 existing vulnerabilities detected",
      "details": [
        "Warn: Project is vulnerable to: GO-2022-0646",
        "Warn: Project is vulnerable to: GHSA-67hx-6x53-jw92",
        "Warn: Project is vulnerable to: GHSA-q8gg-vj6m-hgmj",
        "Warn: Project is vulnerable to: GHSA-w573-4hg7-7wgq",
        "Warn: Project is vulnerable to: GHSA-3wcq-x3mq-6r9p",
        "Warn: Project is vulnerable to: GHSA-74fj-2j2h-c42q",
        "Warn: Project is vulnerable to: GHSA-pw2r-vq6v-hr8c",
        "Warn: Project is vulnerable to: GHSA-43f8-2h32-f4cj",
        "Warn: Project is vulnerable to: GHSA-f4c9-cqv8-9v98",
        "Warn: Project is vulnerable to: GHSA-896r-f27r-55mw",
        "Warn: Project is vulnerable to: GHSA-f8q6-p94x-37v3",
        "Warn: Project is vulnerable to: GHSA-hrpp-h998-j3pp",
        "Warn: Project is vulnerable to: GHSA-p8p7-x288-28g6",
        "Warn: Project is vulnerable to: GHSA-72xf-g2v4-qvf3",
        "Warn: Project is vulnerable to: GHSA-g78m-2chm-r7qv",
        "Warn: Project is vulnerable to: GHSA-j8xg-fqg3-53r7",
        "Warn: Project is vulnerable to: GHSA-6fc8-4gx4-v693"
      ],
      "documentation": {
        "short": "Determines if the project has open, known unfixed vulnerabilities.",
        "url": "https://github.com/ossf/scorecard/blob/8eaf0d7647a3f50d80615812c72a277fd568fb6d/docs/checks.md#vulnerabilities"
      }
    }
  ]
}

More specifically, here's a few scores to improve:

  1. Branch Protection - Score: -1

    • This currently gets an error: internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration. I don't think we can actually improve this as it requires an admin PAT to check that branch protection is set properly. Unless there's something new with the branch rules feature that allows them to be checked publicly
  2. Signed Releases - Score: 8

    • We need to add provenance to our releases, which should also get us to SLSA Level 3. I'll probably file a separate issue for this to summarize some of the work for that that I've started on and collabbed with CD and SIG Security folks on
  3. Pinned Dependencies - Score: -1 -> 5

    • This is actually giving an error, not sure why though: internal error: error parsing shell code: Dockerfile.windows:1:5: (( can only be used to open an arithmetic cmd

      • I believe it's erroring on parsing Windows PowerShell; it likely assumes Linux shell and therefore can't parse the Windows commands properly
      • EDIT: Asked about this in CNCF #security-slam Slack, hoping there's a way to ignore it maybe? Or maybe it should auto-detect Windows PowerShell
        • EDIT2: See issue in Scorecard BUG: Runtime error on Pinned-Dependencies check causes a -1 on its score ossf/scorecard#3316 and PR 🐛 Pinned-Dependencies continues on error ossf/scorecard#3515

          • Here's the Scorecard JSON output when running the PR:
          Pinned Dependencies check JSON
          "details": [
              "Warn: Possibly incomplete results: error parsing shell code: Dockerfile.windows:1:5: (( can only be used to open an arithmetic cmd",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/changelog.yaml:21",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/changelog.yaml:29",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:134",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:135",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:378",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:379",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:397",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:398",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:31",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:36",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:116",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:117",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:154",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:155",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:157",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:166",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:217",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:218",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:224",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:231",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:250",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:341",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yaml:342",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/dependabot-reviewer.yml:18",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/docs.yaml:24",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/docs.yaml:25",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/docs.yaml:28",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/docs.yaml:31",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/docs.yaml:42",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/docs.yaml:49",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/pr.yaml:19",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:32",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:35",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:38",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:43",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:52",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:58",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:100",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:102",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:108",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:150",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:152",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:158",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:165",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:214",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:220",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:248",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:254",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:287",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:288",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:291",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yaml:295",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:300",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yaml:343",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/sdks.yaml:24",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/snyk.yml:36",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/snyk.yml:37",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/snyk.yml:44",
              "Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/snyk.yml:23",
              "Warn: third-party GitHubAction not pinned by hash: .github/workflows/snyk.yml:25",
              "Warn: containerImage not pinned by hash: Dockerfile:6",
              "Warn: containerImage not pinned by hash: Dockerfile:27",
              "Warn: containerImage not pinned by hash: Dockerfile:46",
              "Warn: containerImage not pinned by hash: Dockerfile:56",
              "Warn: containerImage not pinned by hash: Dockerfile:66",
              "Warn: containerImage not pinned by hash: Dockerfile:79",
              "Warn: containerImage not pinned by hash: Dockerfile:90",
              "Warn: containerImage not pinned by hash: Dockerfile:102",
              "Warn: containerImage not pinned by hash: Dockerfile.windows:13",
              "Warn: containerImage not pinned by hash: Dockerfile.windows:31",
              "Warn: containerImage not pinned by hash: Dockerfile.windows:43",
              "Warn: containerImage not pinned by hash: Dockerfile.windows:58",
              "Warn: containerImage not pinned by hash: test/e2e/images/argosay/v1/Dockerfile:1",
              "Warn: containerImage not pinned by hash: test/e2e/images/argosay/v2/Dockerfile:1",
              "Warn: chocoCommand not pinned by hash: Dockerfile.windows:24-25",
              "Warn: chocoCommand not pinned by hash: Dockerfile.windows:24-25",
              "Warn: downloadThenRun not pinned by hash: .devcontainer/pre-build.sh:5",
              "Warn: downloadThenRun not pinned by hash: .devcontainer/pre-build.sh:16",
              "Warn: downloadThenRun not pinned by hash: .github/workflows/ci-build.yaml:126",
              "Warn: downloadThenRun not pinned by hash: .github/workflows/ci-build.yaml:242",
              "Info: Pip installs are pinned",
              "Info: npm installs are pinned",
              "Info: go installs are pinned"
          ],
          "score": 4,
          "reason": "dependency not pinned by hash detected -- score normalized to 4",
          "name": "Pinned-Dependencies",
          "documentation": {
              "url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies",
              "short": "Determines if the project has declared and pinned the dependencies of its build process."
          }
    • Our dependencies for JS and Go are all pinned, but I think we could improve Dockerfiles by using digests and GitHub Actions by using SHAs (as both have mutable tags, so a SemVer number could just be arbitrarily changed by an attacker)

  4. Token Permissions - Score: 0 -> 10

    • These just need some tiny modifications so that top-level is always read-only and only jobs that need them specify other permissions. We have some GH Action Workflows that are just one job, so this just makes those more secure in case another job were to be added.
    • EDIT: Completed in ci: ensure least privilege permissions for GHA tokens #12035
  5. SAST - Score: 0

  6. Fuzzing - Score: 0

  7. Vulnerabilities - Score: 0 -> 9

Use Cases

OpenSSF is being pushed by Google and CNCF and is also part of CLOMonitor checks that are similarly pushed. We have badges for these already, so we should strive to improve our scores on them.
We should also aim to have a more secure, trusted supply chain, which OpenSSF checks help us to achieve.


Message from the maintainers:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

@agilgur5
Copy link
Member Author

agilgur5 commented Oct 25, 2023

Some results are in and after #12035, #12036, and #12058, our Overall Score is now 8.3! Nearly 2 points higher 🙂

New/Current Scorecard JSON output, pretty-printed
{
  "date": "2023-10-23",
  "repo": {
    "name": "github.com/argoproj/argo-workflows",
    "commit": "08096fc0512ed57a89e4a95ced56512631d8c94b"
  },
  "scorecard": {
    "version": "v4.13.0-29-g49c0eed3",
    "commit": "49c0eed3a423f00c872b5c3c9f1bbca9e8aae799"
  },
  "score": 8.3,
  "checks": [
    {
      "name": "Maintained",
      "score": 10,
      "reason": "30 commit(s) out of 30 and 20 issue activity out of 30 found in the last 90 days -- score normalized to 10",
      "details": null,
      "documentation": {
        "short": "Determines if the project is \"actively maintained\".",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#maintained"
      }
    },
    {
      "name": "Code-Review",
      "score": 10,
      "reason": "all changesets reviewed",
      "details": null,
      "documentation": {
        "short": "Determines if the project requires human code review before pull requests (aka merge requests) are merged.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#code-review"
      }
    },
    {
      "name": "License",
      "score": 10,
      "reason": "license file detected",
      "details": [
        "Info: License file found in expected location: LICENSE:1",
        "Info: FSF or OSI recognized license: LICENSE:1"
      ],
      "documentation": {
        "short": "Determines if the project has defined a license.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#license"
      }
    },
    {
      "name": "CII-Best-Practices",
      "score": 5,
      "reason": "badge detected: passing",
      "details": null,
      "documentation": {
        "short": "Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#cii-best-practices"
      }
    },
    {
      "name": "Branch-Protection",
      "score": -1,
      "reason": "internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration",
      "details": null,
      "documentation": {
        "short": "Determines if the default and release branches are protected with GitHub's branch protection settings.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#branch-protection"
      }
    },
    {
      "name": "Signed-Releases",
      "score": 8,
      "reason": "5 out of 5 artifacts are signed or have provenance",
      "details": [
        "Warn: release artifact v3.4.12 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/125983584",
        "Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/131544568",
        "Warn: release artifact v3.5.0 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/124999540",
        "Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/130455044",
        "Warn: release artifact v3.5.0-rc2 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/121928138",
        "Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/126960660",
        "Warn: release artifact v3.4.11 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/120364554",
        "Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/125063416",
        "Warn: release artifact v3.5.0-rc1 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/117361096",
        "Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/121637664"
      ],
      "documentation": {
        "short": "Determines if the project cryptographically signs release artifacts.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#signed-releases"
      }
    },
    {
      "name": "Dangerous-Workflow",
      "score": 10,
      "reason": "no dangerous workflow patterns detected",
      "details": null,
      "documentation": {
        "short": "Determines if the project's GitHub Action workflows avoid dangerous patterns.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#dangerous-workflow"
      }
    },
    {
      "name": "Security-Policy",
      "score": 10,
      "reason": "security policy file detected",
      "details": [
        "Info: security policy file detected: SECURITY.md:1",
        "Info: Found linked content: SECURITY.md:1",
        "Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1",
        "Info: Found text in security policy: SECURITY.md:1"
      ],
      "documentation": {
        "short": "Determines if the project has published a security policy.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#security-policy"
      }
    },
    {
      "name": "Token-Permissions",
      "score": 10,
      "reason": "GitHub workflow tokens follow principle of least privilege",
      "details": [
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/changelog.yaml:10",
        "Warn: jobLevel 'contents' permission set to 'write': .github/workflows/changelog.yaml:17: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/ci-build.yaml:17",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/dependabot-reviewer.yml:6",
        "Warn: jobLevel 'contents' permission set to 'write': .github/workflows/dependabot-reviewer.yml:13: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/docs.yaml:16",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/pr.yaml:12",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/release.yaml:20",
        "Warn: jobLevel 'contents' permission set to 'write': .github/workflows/release.yaml:278: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/sdks.yaml:8",
        "Warn: jobLevel 'packages' permission set to 'write': .github/workflows/sdks.yaml:15: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
        "Warn: jobLevel 'contents' permission set to 'write': .github/workflows/sdks.yaml:16: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
        "Info: topLevel 'contents' permission set to 'read': .github/workflows/snyk.yml:11"
      ],
      "documentation": {
        "short": "Determines if the project's workflows follow the principle of least privilege.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#token-permissions"
      }
    },
    {
      "name": "Pinned-Dependencies",
      "score": -1,
      "reason": "internal error: error parsing shell code: Dockerfile.windows:1:5: (( can only be used to open an arithmetic cmd",
      "details": null,
      "documentation": {
        "short": "Determines if the project has declared and pinned the dependencies of its build process.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#pinned-dependencies"
      }
    },
    {
      "name": "Binary-Artifacts",
      "score": 10,
      "reason": "no binaries found in the repo",
      "details": null,
      "documentation": {
        "short": "Determines if the project has generated executable (binary) artifacts in the source repository.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#binary-artifacts"
      }
    },
    {
      "name": "SAST",
      "score": 0,
      "reason": "SAST tool is not run on all commits -- score normalized to 0",
      "details": [
        "Warn: 0 commits out of 30 are checked with a SAST tool",
        "Warn: CodeQL tool not detected"
      ],
      "documentation": {
        "short": "Determines if the project uses static code analysis.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#sast"
      }
    },
    {
      "name": "Fuzzing",
      "score": 0,
      "reason": "project is not fuzzed",
      "details": [
        "Warn: no OSSFuzz integration found: Follow the steps in https://github.com/google/oss-fuzz to integrate fuzzing for your project.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)",
        "Warn: no OneFuzz integration found: Follow the steps in https://github.com/microsoft/onefuzz to start fuzzing for your project.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)",
        "Warn: no GoBuiltInFuzzer integration found: Follow the steps in https://go.dev/doc/fuzz/ to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no PythonAtherisFuzzer integration found: Follow the steps in https://github.com/google/atheris to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no CLibFuzzer integration found: Follow the steps in https://llvm.org/docs/LibFuzzer.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no CppLibFuzzer integration found: Follow the steps in https://llvm.org/docs/LibFuzzer.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no SwiftLibFuzzer integration found: Follow the steps in https://google.github.io/oss-fuzz/getting-started/new-project-guide/swift-lang/ to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no RustCargoFuzzer integration found: Follow the steps in https://rust-fuzz.github.io/book/cargo-fuzz.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no JavaJazzerFuzzer integration found: Follow the steps in https://github.com/CodeIntelligenceTesting/jazzer to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
        "Warn: no ClusterFuzzLite integration found: Follow the steps in https://github.com/google/clusterfuzzlite to integrate fuzzing as part of CI.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)",
        "Warn: no HaskellPropertyBasedTesting integration found: Use one of the following frameworks to fuzz your project:\nQuickCheck: https://hackage.haskell.org/package/QuickCheck\nhedgehog: https://hedgehog.qa/\nvalidity: https://github.com/NorfairKing/validity\nsmallcheck: https://hackage.haskell.org/package/smallcheck\nhspec: https://hspec.github.io/\ntasty: https://hackage.haskell.org/package/tasty (High effort)",
        "Warn: no TypeScriptPropertyBasedTesting integration found: Use fast-check: https://github.com/dubzzz/fast-check (High effort)",
        "Warn: no JavaScriptPropertyBasedTesting integration found: Use fast-check: https://github.com/dubzzz/fast-check (High effort)"
      ],
      "documentation": {
        "short": "Determines if the project uses fuzzing.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#fuzzing"
      }
    },
    {
      "name": "Packaging",
      "score": 10,
      "reason": "publishing workflow detected",
      "details": [
        "Info: GitHub/GitLab publishing workflow used in run https://api.github.com/repos/argoproj/argo-workflows/actions/runs/6621149784: .github/workflows/ci-build.yaml:144"
      ],
      "documentation": {
        "short": "Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#packaging"
      }
    },
    {
      "name": "Vulnerabilities",
      "score": 9,
      "reason": "1 existing vulnerabilities detected",
      "details": [
        "Warn: Project is vulnerable to: GO-2022-0646"
      ],
      "documentation": {
        "short": "Determines if the project has open, known unfixed vulnerabilities.",
        "url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#vulnerabilities"
      }
    }
  ]
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build Build or GithubAction/CI issues area/docs Incorrect, missing, or mistakes in docs solution/suggested A solution to the bug has been suggested. Someone needs to implement it. type/feature Feature request type/security Security related
Projects
None yet
Development

No branches or pull requests

1 participant