Docker 25.03 - Updating Docker results in IP Error

[edrikk]

Description

Hi,

I had been running docker 24.0.X successfully with current setup without issue, however docker 25 started the below. I saw that fixes had gone into version including the 25.0.3 item to resolve MAC address issues.
I have also followed the instructions around recreating all the dockers once on 25.0.3 but my issue remains.

My setup is as follows:

• I have a macvlan-network setup with IPV4 Subnet - 192.168.40.0/22 and IPV4 Gateway - 192.168.41.1. I have defined IPV4 IP Range - 192.168.42.252/32.
• I have portainer managing my dockers, with each docker stack defining the related docker's hard coded MAC and IP Address. For example:

version: "2.4"

services:
  librespeed:
    container_name: LibreSpeed
    image: adolfintel/speedtest:latest
    networks:
      macvlan-network:
         ipv4_address: 192.168.41.35
    hostname: speedtest
    domainname: MYFQDN.com
    mac_address: 02:42:c0:a8:29:23
    dns: 192.168.41.1
    restart: unless-stopped
    environment:
      - WEBPORT=80
networks:
  macvlan-network:
    external: true
    name: macvlan-network

• I also have Watchtower which regularly updates my dockers.

Since upgrading from 24.0.X (and currently on 25.0.3), it appears that the first(? or one of the first) upgrades will be given the .252 address (even thouigh it should get a hardcoded/static IP as defined in the compose/stack). Since this is the only IP Address allowed in my range, the next updated dockers will fail to load with the error message:

Creating /NameOfDocker
Error response from daemon: no available IPv4 address on this network's address pools: macvlan-network (a36........f1).

At this point, I have to manually delete the docker container and delete the docker image of both the docker taking up the "wrongly assigned" IP address, as well as the containers that failed to install, then go into the stack/compose and redeploy them. They will then be given the correct static IP addresses, and all is well.

Until the next set of updates.

Reproduce

Per above,

Expected behavior

Dockers should be given the \static MAC and IP address as defined in their docker compose/stack. This was the correct behavior in 24.0.X which has regressed in 25.0.X

docker version

Client: Docker Engine - Community
 Version:           25.0.3
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        4debf41
 Built:             Tue Feb  6 21:14:25 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          25.0.3
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       f417435
  Built:            Tue Feb  6 21:14:25 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.5
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 16
  Running: 15
  Paused: 0
  Stopped: 1
 Images: 16
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.1.0-18-amd64
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 31.06GiB
 Name: GypsyPC
 ID: X76E:4OGO:P4TZ:NJT3:KUZK:BELS:4CQB:SD2N:NNKI:BEBQ:2CL6:TDMV
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

[akerouanton]

@edrikk Thanks for reporting. This is the CLI repo but your issue is about something wrong (presumably) at the Engine level. Could you reopen in moby/moby please?

Also, it'd be helpful if you could enable debug logs and paste the "form data" log line that matches the request sent by Watchtower.

[edrikk]

Thank you, and my apologies.
I have opened an issue in the appropriate repo.

Closing this item.