Introducing Scorecard result viewer

[tegioz]

Hi! 👋

This is Sergio from CLOMonitor 🙂

Last week, when we were adding a new check to CLOMonitor for the presence of the OpenSSF Scorecard badge in the repositories' README files, we realized that the badge link points to the Scorecard API result endpoint. We wanted to display that information nicely in CLOMonitor, so we've built a small viewer for the report's json data.

We thought this could be useful outside of CLOMonitor, so instead of adding it as part of the CLOMonitor's UI, we've built it as an independent page that we could donate to you if you'd like. It's just a single self-contained HTML document that can be controlled via some query string parameters. It could serve as a nice badge link destination, for example 😇

You can see it live here. We'll probably tweak a few details over the next couple of days (i.e. some improvements for mobile devices), but it's mostly looking good.

If you'd rather it to continue living only in CLOMonitor for now, please feel free to use the instance we are serving if you'd like, even for the badges 😉 And of course any feedback to improve it is welcomed!

BTW thank you all for the great work done here! Most of CLOMonitor's security checks rely on Scorecard and it has worked great for us ❤️

[naveensrinivasan]

@tegioz This is AMAZING!!! Thank you!!

@ossf/scorecard-maintainers 👀

[evverx]

Thanks! Those dashboards look great on all sorts of devices (including watches).

I wonder if it would be possible to sort the checks by "severity"? I think the "vulnerabilities" check (which is currently at the bottom) should be higher than "CII-Best-Practices-Badge". An additional option would be to sort by "severity" and within those groups by "scores" (if the use case is to make it easier to figure out what needs addressing).

Also depending on who looks at dashboards things like

look scary because in most contexts it means that something is really wrong somewhere. As far as I understand the green line kind of indicates that it's good but maybe it could somehow signal that "10" is good more prominently.

Looks like the text can't be copy-pasted:

clomonitor-scorecard.mov

(it's Chrome Version 113.0.5672.92 (Official Build) (x86_64))

Anyway it doesn't really matter as long as it isn't json. Thanks again!

[spencerschrock]

I think the page looks great!

If it's a static page, it could certainly be hosted on the Netlify portion of Scorecard's website.
I'd have to think about whether or not this is better than keeping it with clomonitor.

[tegioz]

Thanks! And thank you for the feedback @evverx!

We've just deployed some additional changes:

• Ability to sort checks by multiple criteria
  • Controlled by the new sort_by and sort_direction params
• Added extra symbol to the check score to make its meaning clearer
• Added tooltips to check score and risk level
• Fixed issue selecting text

[evverx]

Now that I can sort the checks I think -1 is tricky in terms of visualization. They are kind of neutral but some of them (like "packaging") are more neutral than "branch-protection" (and I think "branch-protection" is almost always -1 because adding a PAT isn't worth it). I have to admit I don't know how to convey it. Maybe those nuances would be better conveyed by the scorecard output somehow.

Anyway all I can say is that those dashboards are great. Thank you! I think badges should certainly point to them (once it's decided where they should officially live).

[spencerschrock]

In the scorecard default output we replace -1 with a ?, which is one possibility.

[evverx]

I think something like "? Packaging" indicates that it's unknown whether a project publishes packages on GitHub while in reality it usually means that packages aren't published and this check isn't applicable to that particular project at all (i.e. I think "n/a Packaging" makes more sense). "Branch-Protection" is different (though there "n/a" could mean "not available" instead of "not applicable" but if it's almost always "n/a" it isn't clear why it should be shown at all).

Also looking at

it seems -1 is also used for internal errors. I think ideally they should be in a separate group to make it easier for whoever looks at dashboards to identify scorecard issues and direct bug reports/complains to the right place.

[tegioz]

We've replaced -1 with ? for now, to match scorecard's output. But we'd be happy to switch to something else if needed. Entries with ? have been pushed to the bottom of the list as well.

[evverx]

Entries with ? have been pushed to the bottom of the list as well

I think as long as they are grouped at the bottom "?" should do. It's certainly better than "n/a"'s scattered all over the place I was proposing here :-)

It seems in dark mode selected areas aren't highlighted:

dark-mode-selection.mov

(on an absolutely unrelated note, I wonder what "CLO" stands for? My guess was that it's some level objectives but I can't seem to figure out what "C" is. Sorry for the off-topic. I'm just curious).

[tegioz]

It seems in dark mode selected areas aren't highlighted:

Thanks, we'll take care of it soon 👍

(on an absolutely unrelated note, I wonder what "CLO" stands for? My guess was that it's some level objectives but I can't seem to figure out what "C" is. Sorry for the off-topic. I'm just curious).

It comes up every now and then, please see cncf/clomonitor#746 🙂 CLOTributor is another member of the CLO family 😉

[tegioz]

Hi @evverx

It seems in dark mode selected areas aren't highlighted

In what OS are you experiencing this issue? We can't reproduce it in any browser in MacOS. We've also observed an alignment issue in the risk level badge in your video, so we'd like to take a look at that as well.

[evverx]

I've just checked and as far as I can tell that laptop comes with macOS Big Sur 11.7.7. Newer laptops seem to be fine (or maybe the brightness settings aren't tweaked there :-)). Looks like it's some weird macOS-specific corner case affecting the dashboard dark mode with the default system dark mode.

[evverx]

It's really weird. On YouTube it's dark. GitHub seems to get it around though.

Anyway I'm not sure it needs fixing. I assume YouTube probably did their research and concluded that this case isn't common (and I'd agree with that conclusion :-))

[tegioz]

Thanks for looking into it @evverx 🙂

[evverx]

This issue has been open for 2 weeks so I'm wondering what is holding this back? Due to #1972 I don't have access to the meeting notes (where I assume dashboards would have been mentioned) so I'm kind of in the dark here. As I mentioned elsewhere since the dashboards are great and can be used to view/sort the scorecard output they should also make it possible to cut off the SARIF part of the scorecard action and prevent scorecard alerts from popping up over and over again in the security tab.

[spencerschrock]

Super happy to see the feedback + iterations here.

This issue has been open for 2 weeks so I'm wondering what is holding this back?

I think some of it had to do with the OSS NA conference which multiple maintainers went to. Now that I'm back (and from a personal vacation) I'm ready to see this resolved as well.

In terms of adoption, the underlying API the badges currently link to wont be changing, so it's really just a matter of:

1. deciding where this is hosted
2. updating the scorecard action README to include instructions for linking the badge to the new page, and making a release note for others to do the same. And of course there's nothing stopping individuals from doing this now for repos they care about.

For the other @ossf/scorecard-maintainers, the only decision is hosting on the netlify portion of https://securityscorecards.dev/, or keeping it on clomonitor.io?

[naveensrinivasan]

+1

I recommend using https://securityscorecards.dev/. It enables larger enterprises to whitelist a single DNS entry, which is a great advantage.

[evverx]

@spencerschrock got it. Thanks!

One last thing. Looking at https://github.com/ossf/scorecard/milestone/5 it seems something called "Structured results" is on its way. As far as I understand the idea is to dump even more json stuff in the foreseeable future. I wonder if it's going to be used by the scorecard action by default? If so it seems it can break the dashboards (because as far as I understand that format isn't expected) and all that json is going to end up in every check and that would bring the UI back to that human-unfriendly format again (unless it's parsed somehow and formatted somehow).

[spencerschrock]

Without going too much off-topic, I don't see that being an immediate problem.

As far as I understand the idea is to dump even more json stuff in the foreseeable future. I wonder if it's going to be used by the scorecard action by default?

Changing the JSON output structure is something we're very conservative with, so it will be its own output format (currently gated behind an experimental env flag). If any change to the default scorecard action format were to occur, I would expect a major version change to accompany it.

would bring the UI back to that human-unfriendly format again

Our goal is the opposite, to split up the various checks into sub-components so consumers can look at the bits they care about and tune out the noise. But yes it would require a change to dashboards/visualizations at some point in the future.

[evverx]

Without going too much off-topic. I don't see that being an immediate problem

I'm not sure why it would be offtopic in an issue where the scorecard UI is discussed. I agree it's not exactly an immediate problem.

split up the various checks into sub-components so consumers can look at the bits they care about and tune out the noise

I went through those issues and I think the idea is to make it easier for machines (not people) to consume that. The UI that actual maintainers can use has never been discussed there.

it would require a change to dashboards/visualizations at some point in the future

It would but it isn't even included in that milestone (or any other milestone for that matter) as far as I can see.

[spencerschrock]

I went through those issues and I think the idea is to make it easier for machines (not people) to consume that. The UI that actual maintainers can use has never been discussed there.
It would but it isn't even included in that milestone (or any other milestone for that matter) as far as I can see.

I'd say both it helps with both, but you're right there was no tracker. #3078

+1
I recommend using https://securityscorecards.dev/. It enables larger enterprises to whitelist a single DNS entry, which is a great advantage.

Without any further objections, I think the decision is to donate the static page to https://github.com/ossf/scorecard-webapp. I know we have a directory for static files (link), but I'm not personally familiar with the router/mux we have setup. At the very least it would be accessible at https://securityscorecards.dev/scorecard.html if simply dropped in.

It's just a single self-contained HTML document

Are changes made directly to the file, or are there source files which get built into the page?

[evverx]

I'd say both it helps with both

I agree and I suspect there're consumers who are probably planning to build some stuff (including all sort of UIs with queries and all kinds of bells and whistles) but my concern was that the UI that maintainers can use wouldn't be a priority. At least until the CLOMonitor folks built those amazing dashboards it was somehow totally acceptable to dump json and call it a day. Time will tell I guess.

Anyway I'm not sure what else I can say here so I'd like to thank the CLOMonitor folks once again!

[tegioz]

Without any further objections, I think the decision is to donate the static page to https://github.com/ossf/scorecard-webapp. I know we have a directory for static files (link), but I'm not personally familiar with the router/mux we have setup. At the very least it would be accessible at https://securityscorecards.dev/scorecard.html if simply dropped in.

That sounds great to us! @cynthia-sg and I would be happy to help if there is something to fix on it. And eventually when the JSON output changes we could work with you to support the new format.

Are changes made directly to the file, or are there source files which get built into the page?

Yes, changes are made directly to the file, there is no build process. We've a small fix pending that we'll push on Monday.

[seberg]

Just a 👍, it is really a downer to press on the badge and get not useful json. If the scorecard is to improve workflows, the badge needs this to make it discoverable/convenient to actually see what to improve (compared to having to use the CLI or so).

[spencerschrock]

That sounds great to us! @cynthia-sg and I would be happy to help if there is something to fix on it. And eventually when the JSON output changes we could work with you to support the new format.

Fantastic. Thanks for the viewer and support!

Yes, changes are made directly to the file, there is no build process. We've a small fix pending that we'll push on Monday.

Sounds good! Do you mind sending the PR to https://github.com/ossf/scorecard-webapp? Dropping it in as scorecards-site/static/viewer/index.html should make it accessible at:
https://securityscorecards.dev/viewer/?platform=...

[tegioz]

No worries! PR is ready: ossf/scorecard-webapp/pull/406 🙂

[spencerschrock]

This has been merged into the site. There may be one or two small tweaks around other supported paths (#ossf/scorecard-webapp#415) before announcing this with the next scorecard-action release.