docker/install: Support `version: master` #438

vvoland · 2024-09-06T10:27:34Z

related to: Provide binaries using container images setup-docker-action#81

Add support for installing Docker master packages from moby/moby-bin and dockereng/cli-bin images.

This could also allow to install arbitrary version from these images but for now it's only used for master.

Test run: https://github.com/docker/moby-private/actions/runs/10773693004/job/29874033854?pr=9#step:2:336

src/hubRepository.ts

crazy-max · 2024-09-06T10:42:25Z

src/docker/install.ts

+    core.info(`Downloading Docker ${this.version} from ${this.channel}`);
+
+    this._version = this.version;
+    if (this.version == 'master') {


I think it would be better to rely on csv values so we can either choose archive or image format:

v24.0.9 similar to type=archive,version=v24.0.9

type=image,tag=master

type=image,tag=24.0.9

WDYT?

But that would be smth to handle in setup-docker action as well.

Do we want to have another input variable for cli version alone?
Like:

version: type=image,tag=master # and optional repository=moby/moby-bin-someotherepo cli-version: type=archive,version=v20.10.25 # if not specified, will default to tag from `version` an

or just sth like

version: | component=engine,type=image,tag=master component=cli,type=archive,version=v20.10.25

Or, if we want to go in the direction of: docker/setup-docker-action#81

Then maybe we should have a separate image input:

images: | moby/moby-bin:master dockereng/cli-bin:master

that would be mutually exclusive with version?

Do we want to have another input variable for cli version alone?

I think we should keep both under the same terminology instead of being separated. Don't think we want to allow custom registry or image repository imo.

Or, if we want to go in the direction of: crazy-max/ghaction-setup-docker#81

I think csv values like #438 (comment) are easier to maintain so we don't introduce a new input and keep backward compat with current version input. I will update this proposal if we are ok with this. WDYT?

My main concern here was the ability to install a different engine and cli version. I think it's fine to leave the image repository hardcoded, but it could be useful to be able to install different versions of CLI and Engine.

but it could be useful to be able to install different versions of CLI and Engine.

Ok if there is this use case then what do you think of:

v24.0.9 > type=archive,version=v24.0.9 or type=archive,engine_version=v24.0.9,cli_version=v24.0.9 if you want specific engine/cli version

Same for images:

type=image,tag=24.0.9 > type=image,engine_tag=24.0.9,cli_tag=24.0.9

SGTM, although it's not an urgent use-case so we can handle that in a follow up.

Updated to support the CSV syntax.
Example run: https://github.com/docker/moby-private/actions/runs/10773693004/job/29874033854?pr=9#step:2:336

src/docker/install.ts

vvoland · 2024-09-09T16:55:09Z

Also opened a PR on the ghaction to make use of the new options: docker/setup-docker-action#106

crazy-max

Along the change for source can you add new cases in https://github.com/docker/actions-toolkit/blob/main/__tests__/docker/install.test.ts to download from image?

src/docker/install.ts

crazy-max · 2024-10-16T07:45:17Z

https://github.com/docker/actions-toolkit/actions/runs/11353388604/job/31578295350?pr=438#step:11:583

Downloading from moby/moby-bin and dockereng/cli-bin tag: 27.3.1
FAIL __tests__/docker/install.test.itg.ts (355.416 s)
  ● install › install docker { type: 'image', tag: '27.3.1' }

    expect(received).resolves.not.toThrow()

    Received promise rejected instead of resolved
    Rejected to value: [Error: Cannot find manifest for darwin/amd64/]

Hum right, on macos runners it will use darwin looking at https://github.com/docker/actions-toolkit/pull/438/files#diff-0f2ffb1716f62956ee604fae748f3369d867fa5c39a60388479728f013479b46R120. Might need to enforce linux in this case but not for cli.

vvoland · 2024-10-16T07:46:47Z

Or just skip the daemon on darwin and only download the CLI?

OTOH, this could be confusing behavior as it will not error out and won't be able to connect to the daemon..

crazy-max · 2024-10-16T08:02:16Z

Or just skip the daemon on darwin and only download the CLI?

OTOH, this could be confusing behavior as it will not error out and won't be able to connect to the daemon..

Yeah provisioning for macos is a bit different. It downloads static bins from download.docker.com: https://github.com/docker/actions-toolkit/actions/runs/11289888751/job/31400662984#step:11:17 which only contains cli as expected.

But then during lima provisioning it installs the engine using get.docker.com script:

actions-toolkit/src/docker/assets.ts

Line 224 in a59a5f8

    
               curl -fsSL https://get.docker.com | sh -s -- --channel {{dockerBinChannel}} --version {{dockerBinVersion}}

so platform normalization would match the VM.

vvoland · 2024-10-16T08:10:10Z

Right, so we'd need to also handle the image download inside the lima provisioning script. I think it's easiest to just use undock there? The lima provisioning is already doing a lot of stuff, so the extra binary won't hurt there 😅

EDIT: Actually it might be easier to just mount the binaries to lima vm, let me try that.

crazy-max · 2024-10-16T10:43:48Z

Can you rebase to fix the QEMU issue related to #459

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

vvoland · 2024-10-17T10:41:04Z

Weird qemu failure:

[    0.000000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns
[    0.000000] APIC: Switch to symmetric I/O mode setup
[    0.000000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.000000] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
[    0.000000] ...trying to set up timer (IRQ0) through the 8259A ...
[    0.000000] ..... (found apic 0 pin 2) ...
[    0.016000] ....... failed.
[    0.016000] ...trying to set up timer as Virtual Wire IRQ...
[    0.032000] ..... failed.
[    0.032000] ...trying to set up timer as ExtINT IRQ...
[    0.044000] ..... failed :(.
[    0.044000] Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option.

Is this known to happen sometimes? 🤔

vvoland · 2024-10-17T10:41:44Z

The previous failure on macos-12 was related to the long /var/tmp path - it didn't mount correctly into lima vm.
Fixed by copying to a shorter path in .lima directory.

EDIT: Looks like it's still the case 🫠

Use undock inside lima to pull the image content. We could mount the downloaded binaries from the host, but for some reason lima mounts are not always mounted when the provisioning script is run. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

vvoland · 2024-10-18T13:16:22Z

Ok, looks like lima mounts are not always mounted before the provisioning script is run.
I changed the darwin lima install to pull the binaries with undock instead.
Should be good to go!

crazy-max

Ok, looks like lima mounts are not always mounted before the provisioning script is run.

Argh I will take a look if we can fix that as follow-up so we can just use undock in docker install and avoid hub requests.

But let's get this in in the meantime!

vvoland force-pushed the install-from-binimage branch from ea3d0b7 to 91a2ddd Compare September 6, 2024 10:30

crazy-max reviewed Sep 6, 2024

View reviewed changes

src/hubRepository.ts Show resolved Hide resolved

vvoland force-pushed the install-from-binimage branch 2 times, most recently from 5d3b32c to 2941b11 Compare September 6, 2024 10:40

crazy-max reviewed Sep 6, 2024

View reviewed changes

vvoland commented Sep 9, 2024

View reviewed changes

src/docker/install.ts Outdated Show resolved Hide resolved

vvoland force-pushed the install-from-binimage branch from 7f4cb8d to 9a7c8c8 Compare September 9, 2024 13:02

vvoland mentioned this pull request Sep 9, 2024

Support downloading binaries from docker images docker/setup-docker-action#106

Merged

crazy-max requested changes Sep 18, 2024

View reviewed changes

src/docker/install.ts Outdated Show resolved Hide resolved

crazy-max reviewed Sep 18, 2024

View reviewed changes

src/docker/install.ts Outdated Show resolved Hide resolved

vvoland force-pushed the install-from-binimage branch 4 times, most recently from 2c74c55 to 5a0fe00 Compare October 15, 2024 19:53

vvoland force-pushed the install-from-binimage branch from 5a0fe00 to 97bef8d Compare October 16, 2024 07:45

vvoland force-pushed the install-from-binimage branch from 97bef8d to 20a30f2 Compare October 16, 2024 07:49

vvoland force-pushed the install-from-binimage branch 8 times, most recently from 9662b9d to 23c945d Compare October 16, 2024 10:32

vvoland force-pushed the install-from-binimage branch from 81d3249 to 668161b Compare October 17, 2024 10:18

vvoland force-pushed the install-from-binimage branch 17 times, most recently from 58cfeb4 to 396c98a Compare October 18, 2024 12:55

vvoland force-pushed the install-from-binimage branch from 396c98a to e3d0e4e Compare October 18, 2024 12:57

vvoland requested a review from crazy-max October 18, 2024 13:16

crazy-max approved these changes Oct 27, 2024

View reviewed changes

crazy-max merged commit 517a5c5 into docker:main Oct 27, 2024
42 checks passed

crazy-max mentioned this pull request Oct 28, 2024

undock install #473

Merged

crazy-max mentioned this pull request Dec 6, 2024

Provide binaries using container images docker/setup-docker-action#81

Closed

crazy-max mentioned this pull request Jan 19, 2025

docker(install): use undock to extract image #567

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docker/install: Support `version: master` #438

docker/install: Support `version: master` #438

vvoland commented Sep 6, 2024 •

edited

Loading

crazy-max Sep 6, 2024

crazy-max Sep 6, 2024 •

edited

Loading

vvoland Sep 6, 2024

vvoland Sep 6, 2024

crazy-max Sep 9, 2024 •

edited

Loading

vvoland Sep 9, 2024

crazy-max Sep 9, 2024

vvoland Sep 9, 2024

vvoland Sep 9, 2024

vvoland commented Sep 9, 2024

crazy-max left a comment •

edited

Loading

crazy-max commented Oct 16, 2024 •

edited

Loading

vvoland commented Oct 16, 2024 •

edited

Loading

crazy-max commented Oct 16, 2024

vvoland commented Oct 16, 2024 •

edited

Loading

crazy-max commented Oct 16, 2024 •

edited

Loading

vvoland commented Oct 17, 2024

vvoland commented Oct 17, 2024 •

edited

Loading

vvoland commented Oct 18, 2024

crazy-max left a comment

docker/install: Support version: master #438

docker/install: Support version: master #438

Conversation

vvoland commented Sep 6, 2024 • edited Loading

crazy-max Sep 6, 2024

Choose a reason for hiding this comment

crazy-max Sep 6, 2024 • edited Loading

Choose a reason for hiding this comment

vvoland Sep 6, 2024

Choose a reason for hiding this comment

vvoland Sep 6, 2024

Choose a reason for hiding this comment

crazy-max Sep 9, 2024 • edited Loading

Choose a reason for hiding this comment

vvoland Sep 9, 2024

Choose a reason for hiding this comment

crazy-max Sep 9, 2024

Choose a reason for hiding this comment

vvoland Sep 9, 2024

Choose a reason for hiding this comment

vvoland Sep 9, 2024

Choose a reason for hiding this comment

vvoland commented Sep 9, 2024

crazy-max left a comment • edited Loading

Choose a reason for hiding this comment

crazy-max commented Oct 16, 2024 • edited Loading

vvoland commented Oct 16, 2024 • edited Loading

crazy-max commented Oct 16, 2024

vvoland commented Oct 16, 2024 • edited Loading

crazy-max commented Oct 16, 2024 • edited Loading

vvoland commented Oct 17, 2024

vvoland commented Oct 17, 2024 • edited Loading

vvoland commented Oct 18, 2024

crazy-max left a comment

Choose a reason for hiding this comment

docker/install: Support `version: master` #438

docker/install: Support `version: master` #438

vvoland commented Sep 6, 2024 •

edited

Loading

crazy-max Sep 6, 2024 •

edited

Loading

crazy-max Sep 9, 2024 •

edited

Loading

crazy-max left a comment •

edited

Loading

crazy-max commented Oct 16, 2024 •

edited

Loading

vvoland commented Oct 16, 2024 •

edited

Loading

vvoland commented Oct 16, 2024 •

edited

Loading

crazy-max commented Oct 16, 2024 •

edited

Loading

vvoland commented Oct 17, 2024 •

edited

Loading