mac_address is being ignored in docker-compose files #10393
Comments
|
I can also confirm I was impacted by this issue, when I was unable to restore my containers through the Portainer stack interface. When I had attempted to do so, they got randomly assigned, incrementing MAC addresses from Docker / Portainer. Workaround
A sample compose file of two containers that was impacted by this: version: '2'
networks:
macvlan_net:
external:
name: macvlan-network
web_portal_network:
external:
name: web_portal_network
services:
plex:
container_name: plex
image: plexinc/pms-docker:latest
restart: unless-stopped
mac_address: 00:1D:84:00:50:20
networks:
macvlan_net:
ipv4_address: 10.0.2.132
environment:
- HOSTNAME=REMOVED_REMOVED_REMOVED
- PLEX_CLAIM=REMOVED_REMOVED_REMOVED
- ADVERTISE_IP=REMOVED_REMOVED_REMOVED
- TZ=UTC
volumes:
- /path/to/data/data:/data
devices:
- /dev/dri:/dev/dri
plex-private:
container_name: plex-private
image: plexinc/pms-docker:latest
restart: unless-stopped
mac_address: 00:1D:84:00:50:21
networks:
macvlan_net:
ipv4_address: 10.0.2.133
environment:
- HOSTNAME=REMOVED_REMOVED_REMOVED
- PLEX_CLAIM=REMOVED_REMOVED_REMOVED
- ADVERTISE_IP=REMOVED_REMOVED_REMOVED
- TZ=UTC
volumes:
- /path/to/data/data:/data
devices:
- /dev/dri:/dev/driThis is a blocking issue for me with Portainer, I am unable to deploy containers using
|
|
My workaround was to manually compile the latest moby on the system then it seemed to function again. |
|
I am also seeing the same issue. I ended up rolling back to 2.17.1 to restore functionality. Seems it was introduced with 2.18.0 Rather run latest patches/versions. Please advice on an eta for the fix. Sample compose doc. Seeing the issue across all of them: ` |
|
Still happening: |
|
I had this issue in a different context. I found the following and it led to the solution of my problem.
See here. So I switched from version: '3.8' # Example version
services:
your_service:
image: your_image
mac_address: your_mac_addressto version: '3.8' # Example version
services:
your_service:
image: your_image
networks:
default:
mac_address: your_mac_addressNow, the mac address is correctly set in my container. |
|
I wasn't able to get my portainer to use that change, it kept complaining about the YAML.
|
|
Also unable to verify. Did not work for me
|
|
Is there any other workaround? I've lost the ability to deploy stacks from my portainer with this bug. |
|
Thanks for raising this, I am going to follow this up internally and test this. I will give updates here |
|
@urda @BillBinAz @bastilam What version of docker are you running? I see you have mentioned version above but I want to confirm you current version. Please run |
|
Here is some version info for you: From portainer: |
|
I used to be able to deploy and maintain all my stack within the portainer UI, but now they no longer accept the custom |
|
[~] # docker version Server: Docker Engine - Community |
|
I tested this and it's an issue with an older version of Docker Compose, an updated version will be included in the 2.20 release which will resolve this issue. |
I'm eager to see the fix and try again in-place. Thank you for helping me escalate this issue. |
|
Is there an ETA on the 2.20 release? |
I can't give dates but we are now in code freeze, so it's close :) |
|
Thanks just was wondering if it's the next release or if there will be more 2.19.* etc... before it happens. |
|
Are you able to point to the fix in github I can't seem to find it. Can I manually build it in some way? |
Hey, so you can try our dev image of 2.20 please keep in mind this it not live and is not suitable for production and should only be used for testing, please let me know how this goes for you. Business Edition |
|
When I tried to start a container in 2.20 I got Forbidden - CSRF token invalid. I compiled moby again after breaking my setup by accidentally updating to stock dockerd from packages. Seems like moby compose is broken just like stock dockerd for the mac_address setting. It is giving the portainer a random mac address so it's not currenlty using it's static IP which the broswer would have cache for. The browser cache is fresh for that IP it is on now. I saw an issue about it from 2016 and they said to clear cache to fix it. This doesn't seem like that would work for this CSRF Token issue. I looked under settings to see if there was a simple "Disable csrf token" but no luck there. I tried to also redeploy the compose and got the same error. |
Just trying to unravel your comment. So the issue you are facing is specifc to your compose? I tested the macvlan in 2.20 and confirmed that worked. I tested the below in 2.19.4 and confirmed it failed, but in 2.20 deployed fine. |
|
Sorry, There are two things here Portainer's container MAC Address (which is a dockerd issue not yours) and the portainer containers MAC Address that portainer deploys. As I also set the mac_address of my portainer container which it gets the IP address from a dnsmasq server via the static mac address like this: dnsmasq.conf Dockerd currently seemingly also has this issue with mac_address. I worked around it way above many months ago by switching to a moby dockerd build which somehow fixed the issue for portainer as well... A few days ago after doing a system update. I had a version mismatch error that said I was trying to use a docker client of version 1.13.1 and it needed to be at least 1.24 for docker to start any containers. The issue was with a plugin I was using called docker-net-dhcp which lets me use dnsmasq to pass the IP's off to dockerd Me not remembering I manually updated dockerd with the moby version, overwrote it with the system package version and then after fixing the plugin version issue the mac address issue was back as well. 😢 So I then tried to rebuild the dockerd moby version to see if it would fix it again, but it seems it also has this issue now as well. When I ran my compose directly not through portainer it still wouldn't give it a static mac address being on the latest version of dockerd v25. they now changed where you place the mac_address line to under the network servicer where it used to be in the container service before v25: https://docs.docker.com/compose/compose-file/05-services/#mac_address-1 When I try placing mac_address under the docker service it processes it but it just gives it a random mac like it wasn't there. When I place it under the network service it won't start with the error shown in the comment at the end of the config below: # Portainer Container for Docker
version: '3.9'
services:
portainer:
container_name: portainer
hostname: portainer
#image: portainer/portainer-ce:latest
image: portainerci/portainer:2.20
restart: 'no'
labels:
- "com.centurylinklabs.watchtower.enable=true"
#restart: unless-stopped
security_opt:
- no-new-privileges:true
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /var/lib/libvirt/images/sas/docker-storage/portainer/portainer-data:/data
ports:
- "8000:8000"
- "9443:9443"
mac_address: de:ad:be:ef:00:01
networks:
- dhcp
networks:
dhcp:
name: dbrv100
external: true
#mac_address: de:ad:be:ef:00:01 #ERROR: networks.dhcp Additional property mac_address is not allowedSo I just left it with the random MAC and checked the dnsmasq leases and saw the new IP for portainer was dhcp'd to a random IP address so I connected to that. It was supposed to be .42 but it was .190 as dockerd gave it a random MAC instead of the one that was set in the compose file. I connected to the new IP and was met with 2.20 as it said in the corner Community Edition 2.20.0 When I add this to the stack editor in portainer 2.20 Name: testmac version: '3.8'
services:
nginx:
image: nginx:latest
networks:
default:
mac_address: de:ad:be:ef:00:02I get a notification: Error Deployment error Unable to deploy stack If I try to start one of my other existing containers I get the |
|
I tested your yaml outside of portainer and it does indeed set the MAC address properly. So it must be an issue with using external network ie Any ideas about that CSRF Token invalid? |
|
Figured out the syntax I had was just wrong should have been like this for version 25: version: '3.8'
services:
nginx:
image: nginx:latest
networks:
dhcp:
mac_address: de:ad:be:ef:00:01
networks:
dhcp:
name: dbrv100
external: trueLooks like it is working now yay! "Networks": {
"dbrv100": {
"IPAMConfig": {},
"Links": null,
"Aliases": [
"mactest-nginx-1",
"nginx",
"############"
],
"MacAddress": "de:ad:be:ef:00:01",
"NetworkID": "0b...",
"EndpointID": "75...",
"Gateway": "10.0.42.1",
"IPAddress": "10.0.42.121",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"mactest-nginx-1",
"nginx",
"############"
]
}
} |
|
Great, thanks for testing that and providing feedback that it's now working. Please remember 2.20 is not currently released and the version I provided is just for testing :) |
Don’t worry, I’m ready to test the production change |
|
Because of this error #11283 I was never able to properly test this in 2.20 so I went back to 2.19.4 Inside portainer 2.19.4 was docker-compose version 2.20.2 sudo pacman -Sy busybox
docker cp $(which busybox) portainer:/
docker exec -it portainer /busybox sh
~ # ./docker-compose version
Docker Compose version v2.20.2
~ # ./docker --version
Docker version 20.10.21, build baeda1f
mv docker-compose docker-compose.bak
exitthen did this: wget -O docker-compose https://github.com/docker/compose/releases/latest/download/docker-compose-linux-x86_64
chmod +x docker-compose
./docker-compose version
Docker Compose version v2.24.6
docker cp docker-compose portainer:/
Successfully copied 61.4MB to portainer:/Checking... docker exec -it portainer /busybox sh
~ # ./docker-compose version
Docker Compose version v2.24.6
~ # exitthen tried to deploy again and it worked! "Networks": {
"dbrv100": {
"IPAMConfig": {},
"Links": null,
"Aliases": [
"piper",
"wyoming-piper",
"############",
"wyomingpiper"
],
"MacAddress": "de:ad:be:ef:00:01",
"NetworkID": "0bae16efd85f9c4ba4656774be8acc713537bceebcb212b16af223487ecd2de0",
"EndpointID": "5bf4be51906438c63ff7affb86e91845dc260c87f670131b31fb452ea94a807a",
"Gateway": "10.0.42.1",
"IPAddress": "10.0.42.69",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"DriverOpts": null,
"DNSNames": [
"piper",
"wyoming-piper",
"############",
"wyomingpiper"
]
}
}Make sure after updating docker-compose that your compose yaml format looks like this: services:
someservice:
networks:
somenetwork:
mac_address: de:ad:be:ef:00:01You may also need docker engine v25 as stated in the documentation. mkdir code
cd code
wget -qO- https://download.docker.com/linux/static/stable/x86_64/docker-25.0.3.tgz | tar xvz
cd docker
sudo systemctl stop docker
sudo mv /usr/bin/dockerd /usr/bin/dockerd.bak
sudo cp dockerd /usr/bin/
sudo systemctl start docker
/usr/bin/dockerd --version
Docker version 25.0.3, build 4debf41
docker exec -it portainer /busybox sh
~ # mv docker docker.bak
~ # exit
docker cp docker portainer:/
Successfully copied 35.6MB to portainer:/
docker exec -it portainer /busybox sh
~ # ./docker --version
Docker version 25.0.3, build 4debf41
~ # exitYAML Layout shown here I was told by Moby staff that even though services.name.mac_address is depreciated that it should still technically work and to maybe bring that up with their issue tracker. I have already switched all my stacks to the new format though services.name.networks.mac_address as that seems better going forward. |
|
I had a few containers getting random MAC's but after redeploying their stacks it fixed them up.
Is there a way to get an optional MAC address column to show in the list of containers beside the IP column? |
|
Case anyone has watchtower running I think they use an older version of docker-compose as well. After I redeployed, it was fine but then again today I had a bunch of updates from watchtower, and like clockwork, all those containers have bad MAC addresses. Redeploy from the modified v25 portainer and they get their proper static MACs back. moby/moby#47441 moby/moby#47474 (comment) |
|
Is there an estimated release date* for 2.20 @Nick-Portainer ? |
No offical date I can provide, but the team is very close :) |
|
If 2.20 is not to be the LTS, what will be the LTS version @Nick-Portainer ? |
Hi, 2.21 is planned to be our first LTS release. - Check out the blog post for more details - https://www.portainer.io/blog/portainer-2.20-release |
Before you start please confirm the following.
Problem Description
This is the docker-compose used for setting the mac_address line. This used to work fine until recently. If I run this outside of Portainer with
docker-compose upit generates the MAC address fine.I am not using a swarm, which another user was having an issue with the same problem:
#9152
Does this have to do with the issue?
moby/moby#46406
Random generated MAC ================^
I use static set IP addresses by MAC dished out from DNSMasq DHCP so this breaks all the services for me as they just get random IP addresses instead of their statically defined ones.
Expected Behavior
Generate the mac_address specified.
Actual Behavior
Generates random MAC address.
Steps to Reproduce
Add a new stack, name your stack, dump the yaml above in the web editor, deploy the stack.
Portainer logs or screenshots
Client:
Version: 24.0.5
API version: 1.43
Go version: go1.20.6
Git commit: ced0996600
Built: Wed Jul 26 21:44:58 2023
OS/Arch: linux/amd64
Context: default
Server:
Engine:
Version: 24.0.5
API version: 1.43 (minimum version 1.12)
Go version: go1.20.6
Git commit: a61e2b4c9c
Built: Wed Jul 26 21:44:58 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.7.6
GitCommit: 091922f03c2762540fd057fba91260237ff86acb.m
runc:
Version: 1.1.9
GitCommit:
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Portainer version
2.19.1
Portainer Edition
Community Edition (CE)
Platform and Version
Docker 24.0.5 | docker-compose 2.20.3
OS and Architecture
Arch Linux 6.5.5-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 23 Sep 2023 22:55:13 +0000 x86_64 GNU/Linux
Browser
Chrome Version 117.0.5938.92 (Official Build) (64-bit)
What command did you use to deploy Portainer?
Pushed the "Deploy the stack" button on the WebUI.Additional Information
I noticed the version of compose the containers say they used seem to be perpetually set to when they were first deployed:
com.docker.compose.version | 2.20.2
Maybe this is just a database entry as when you hit redeploy it doesn't update that field. As I have 2.20.3 of compose installed or does Portainer have it's own docker-compose resource i.e. not /usr/bin/docker-compose etc?
I believe docker-compose had an issue moby/moby#46406 with setting mac_address in this version. It said it was fixed 3 weeks ago maybe that is something? Does docker-compose need to be updated in Portainer to take advantage of this fix or it should use the system /usr/bin/docker-compose etc?
Thank you.
The text was updated successfully, but these errors were encountered: