Improve in-toto's test setup

[adityasaky]

Please fill in the fields below to submit an issue or feature request. The
more information that is provided, the better.

Description of issue or feature request:

The in-toto team currently maintains four implementations: this one, https://github.com/in-toto/in-toto-golang, https://github.com/in-toto/in-toto-java, and https://github.com/in-toto/in-toto-rs. While they were all written using the in-toto spec as a reference, it would be useful to test each implementation against other implementations to verify compatibility. For example, we want metadata generated by in-toto-golang to be verifiable by in-toto-python, and so on.

Further, in-toto implementations currently do not test against incompatible releases of dependencies. If a user updates a dependency to a version that has a breaking change, in-toto's last release may be affected.

Current behavior:

No cross implementation testing, no ongoing integration testing with updated dependencies.

Expected behavior:

Cross implementation testing with a starter set of checks, integration testing of latest in-toto release with dependencies.

[OliverShang]

Hi @adityasaky, I am writing to express my interest in participating in the GSOC 2023 program. I am very excited about the "Improve in-toto's test setup" project idea, which focuses on creating a cross-implementation test. Is there anything else that I need to know or that needs to be taken care of?

Thanks.

[PradyumnaKrishna]

I just remembered, in-toto-python DSSE implementation differs a little bit. According to DSSE spec

"signatures": [{
  "keyid": "<KEYID>",
  "sig": "<Base64(SIGNATURE)>"
}]

The SIGNATURE itself should be base64 encoded that is not in the case of DSSE implementation for python (sig is hex). This will bring incompatibility between in-toto-python and other metadata.

The plan was to use sig as hex internally (to make signing and verifying compatible) and use base64 while exporting DSSE Envelope to json, here. Which I forgot to implement.
https://github.com/secure-systems-lab/securesystemslib/blob/aa3f6064168db9b0f09ca761ec222804cd6939d2/securesystemslib/dsse.py#L67-L76

[adityasaky]

@OliverShang apologies, this went missing in my notifications. I suggest familiarizing yourself with in-toto and its Python + Go implementations. Play with in-toto/demo as well to get a feel for what the project does. If you have questions, feel free to ask them here or in the CNCF Slack on #in-toto.

[PradyumnaKrishna]

I have done some kind of local setup for cross compatibility between python and golang versions. I found both versions are compatible but two major issues right now.

1. in-toto-golang does not provide defaults for excluding artifacts in-toto-golang#207: creates a minor compatibility issue.
2. in-toto-golang omits serializing empty private field in keyval that changes the signature and fails the verification. Older in-toto-golang versions were compatible but after in-toto/in-toto-golang@7bb17dc the layout verification fails for Metablock.

   inspection failed: invalid signature: crypto/rsa: verification error

I am not sure about how we are going to setup the test environment for different python and golang versions.

[adityasaky]

@lukpueh wdyt of setting up a separate repository for these tests? The CI could pull from in-toto-python and in-toto-golang's main branches periodically as well as the latest releases 🤔