Test in Docker with Alpine Linux on CI

[EliahKagan]

As suggested in #1745 (comment), this adds a test job that runs the tests inside a Docker container.

This can be done with Ubuntu in the container, but instead I used Alpine Linux in the container. This provides a greater variety of platforms GitPython tests on, potentially uncovering some bugs that would otherwise go undetected. In particular, Alpine Linux does not satisfy some of the assumptions that are sometimes made about platforms, as it does not, by default, have bash; its ps and related tools are not procps; it does not have GNU coreutils or GNU libc; and so forth.

No tests had to be marked xfail for this. In particular, as noted in #1745 (comment), we have no tests for #1756, so that ps is not procps in Alpine Linux does not currently result in any failures (and if there were such a test then the existing macOS jobs would still be sufficient for that).

This adds only one test job, testing only one version of Python. Currently that version is Python 3.11, since the Alpine Linux python3 apk package provides 3.11 at this time. The specified Docker image is alpine:latest, and the version will presumably advance to Python 3.12 eventually. I added only one job for two reasons:

• This is easiest, since one version of Python 3 is packaged officially in Alpine Linux's downstream repositories, and it makes sense to test with that.
• I don't want to create an excessive number of new jobs, since it seems likely that some more jobs covering interesting cases might be added, or at least might be worth considering, in the future. (One possible such case is discussed below.)

I was able to use the container key to do this, so that the workflow is still expressed in separate steps in the same style as the other test workflows, such that the operations that are analogous are represented by steps written in the same (or very similar) way across all workflows. I think the benefit of readability and comparability justifies this at this time, but depending what is done in the future, it might end up being changed to pack most of the work into a smaller number of steps (maybe one) that use uses: docker://..., or run: docker run ... or docker-run-action, or something else.

Specific reasons for existing design and implementation decisions, including for the strange and somewhat hacky sudo usage, are detailed in commit messages.

---

I've proposed this job as its own pull request, rather than waiting and attempting to introduce it with others, because I think this makes sense to review by itself (and I can incorporate any feedback into future changes), and because if I understand #1745 (comment) correctly it seems to be valued as a feature by itself (which I do agree with). I think that even if there are to be more Docker test jobs, they are better added in a subsequent PR.

However, I'm even more motivated by a goal that this PR does not itself achieve: testing on a bigendian architecture. Because GitPython uses git for most of its work, it shouldn't have a problem with that, but maybe some parts of gitdb/smmap that GitPython uses (for which GitPython's use is under test), or more importantly some seemingly innocuous aspects of encoding and decoding, might inadvertently assume little-endianness.

The good news is that no such bugs appear to be present. I have not yet gotten this to work on CI, but on my local amd64 system I am able to enable emulation and run s390x images. This is with no hardware acceleration, and some of the performance tests are too slow for me even to have waited for them to finish (which does not mean they would have a problem on an actual s390x machine). But when I tell pytest to deselect the performance tests, the remaining tests run in a reasonable time and all pass.

I'd like to do that on CI, which is not hard to do. If possible I'd like to use a workflow of this form, with the container key, and have it be a matrix job with both architectures, with special logic for enabling emulation and for skipping the performance tests for non-amd64 architectures (which would, initially and also probably forever, be just the one other architecture, s390x). I haven't managed to get it working that way, though. Using container:, the container is supposed to be able to start before any steps run. Even if I can manage to get emulation enabled without being able to easily run commands directly on the runner, I suspect I can't get that to happen early enough that using container: is both effective and reasonable.

If Docker test jobs for both amd64 and s390x architectures are to exist, I would want both to be written the same way, even if it makes the amd64 one less nice than in this PR. That way, it would be easier to compare their output to troubleshoot any future failures (as well as to understand how they are set up). If you have thoughts on whether or not this could be valuable, and also how much of a problem (if any) it would be to end up later converting the workflow to look more like that with fewer steps and not using container:, then that can guide me. However, whether or not you have guidance about that, I think this PR itself can probably stand on its own.

(I considered posting this section of the PR description as an issue instead, but I figured I'd put it here in case requested changes to this PR would substantially affect it or make it moot. and also because it provides a possible reason, beyond that discussed in #1745 (comment), to have Docker jobs like this one in the first place.)

[Byron]

Thank you so much for this wonderful addition!

GitPython really is tested rigorously by now with the suite improving over time, given that features aren't usually added, and with the platform coverage increasing. So despite being in maintenance mode, it does manage to maintain and increase its quality. Thank you, again and again, for your tremendous work with this!

---

I appreciate the early PR and believe this setup can change as it has to, and according to your preferences, in the future without problems.

Reading about having a test-matrix for this, I thought maybe multiple containers can run at the same time so they are pre-started? The CI agent should have enough memory to this.

[EliahKagan]

Reading about having a test-matrix for this, I thought maybe multiple contains can run at the same time so they are pre-started? The CI agent should have enough memory to this.

A GitHub Actions job can run multiple containers at a time without problems. This happens automatically when a workflow defines a job using the container key (which causes most steps listed for the job to run in the container it specifies), but also has one or more steps with a uses key specifying a container action. The container specified in the container key or its subkeys is started first, before any steps of any kind are run.

But this is unrelated to the matrix strategy feature, since separate jobs always run on separate runners. (GitHub-hosted runners are virtual machines that are created and destroyed on demand per-job. With self-hosted runners, two jobs may run on the same runner, but not, I don't think, concurrently.)

A job with a container key tries to start that container before it runs any steps, but the GitHub-hosted GHA runners don't have emulation enabled. This can be enabled easily, with a command like:

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

However, that or a similar command has to actually run on the runner, or its effect otherwise achieved on the runner. Giving it as the value of a run key in a script step would ordinarily achieve this, but not when the job has a container key, which causes most steps, including all steps that run commands with run, to run in the container specified in the container key. In addition, even if there is a way to do it, I'm not sure it would be adequately clear. For this reason, I am unsure if there's any reasonable way to use a container key while having an amd64 runner run an s390x container with QEMU emulation.

[Byron]

Thanks for the clarification - it's definitely quite a bit more complicated 😅.