Print a checksum of the artifact when uploading and downloading it

[jez]

Is your feature request related to a problem? Please describe.

I'm trying to debug an apparent problem in either my Buildkite pipeline or in the buildkite-agent binary where one of our artifacts is getting corrupted. It would be useful to know at what point the corruption happens (before the upload? after the download? etc.)

Describe the solution you'd like

When running buildkite-agent artifact upload or buildkite-agent artifact download, print a checksum of the file (for example, a SHA256 checksum). Right now, the only "checksum" printed is the abbreviated size of the artifact (e.g. 36 KiB).

I'm attempting to debug an issue where the artifact uploaded from one step has a size equal (down to the last byte) to the same artifact when downloaded on a subsequent step, but somehow certain bytes are different.

A checksum would be useful to know if somehow the archive is getting corrupted after the download step.

Describe alternatives you've considered

The current alternative is to just print the (rounded) size of the artifact, which is not precise enough to detect a difference in size of e.g. one byte, nor enough to detect a difference when the sizes are equal.

Additional context

If you're curious about the underlying corruption in the artifacts, see here:

• Where we upload it
• Where we download it
• Where we realize it's corrupted

[moskyb]

hi @jez! this is a great idea. we actually already calculate checksums for artifact uploads, so adding a log line that prints out those checksums would probably be relatively straightforward. would you be interested in submitting a PR to that effect?

[colincoghill]

I agree adding checksums would help with diagnosing this kind of problem.

Could the specific problem you're trying to diagnose be: #2774

[jez]

That was the exact problem in the end, yep!