Edit '/etc/hosts' to add/remove IPv6 records.

[robmry]

- What I did

Only add built-in records ('ip6-localhost', 'ip6-mcastprefix', ...) to a container's '/etc/hosts' when the container is connected to an IPv6 network - and remove them when it's no longer connected.

Fixes #35954

- How I did it

Use the existing etchosts functionality to add/remove IPv6-related built-in records from /etc/hosts when an endpoint is added/removed from a container's sandbox.

IPv6 records are always added at the top of the /etc/hosts file. This changes the order of entries, the IPv4 loopback host is currently always added so it now comes after the IPv6 - the ordering was checked by some tests, which I've updated.

A matching follow-up change will probably be needed to add/remove the IPv4 localhost record according to whether the container has IPv4 connectivity.

The built-in IPv6 hostnames may be DNS-resolvable when running in 'rootless' mode because, in that mode, entries in the host's /etc/hosts are used to respond to DNS requests from the container.

- How to verify it

New integration test, and updated unit tests.

With live-restore enabled, restarted the daemon, checked that previously-running containers were still updated as expected when IPv6 connectivity changed.

- Description for the changelog

Edit '/etc/hosts' to add/remove IPv6 records when IPv6 connectivity changes.

- A picture of a cute animal (not mandatory but encouraged)

[akerouanton]

This looks pretty good. I just left two comments about code comments that need to be updated.

[akerouanton]

As you noted in your comment, IPv6 addresses aren't removed explicitly but they're still removed. I think it's okay to keep this behavior but your comment is much needed to clarify what's going on. Can you remove this TODO and keep the rest of the comment please?

[robmry]

Sure - done.

[robmry]

For discussion ...

Editing /etc/hosts has risks because the "etchosts" code writes the file in-place, moving it into place isn't an option with the bind mount. Lots of discussion on that in #17190

This seems less-dangerous than that change, which edited the file for each new container, because:

• Updates are serialised
  • But, could still collide with an edit inside the container, and a reader could see a partial file.
• It'll only happen:
  • during container construction (no problem), and
  • when networks are connected/disconnected, only if they have different IPv6-ness.

Other ideas for this change weren't great, but maybe there's another way to do it? ...

• a container-create, or engine-wide, option to say IPv6 will never be required
  • but the IPv6-related flags are already confusing
• just enable '::1' unconditionally
  • but that's explicitly not what users are asking for
• look at sysctl settings to work out whether IPv6 is disabled system-wide
  • but I'm not sure there's a reliable way to tell (?)

[akerouanton]

If we want to prevent concurrent writes to /etc/hosts from within the container, I think we need to look at fd leases. I think this is the right mechanism for what we want to do. The issue reported by Tim Hockin in this comment #17190 (comment) doesn't exist anymore -- I've tested it myself with the following Go code:

package main

import (
	"os"
	"time"

	"golang.org/x/sys/unix"
)

func main() {
	f, err := os.Open("hosts")
	if err != nil {
		panic(err)
	}
	_, err = unix.FcntlInt(f.Fd(), unix.F_SETLEASE, unix.F_WRLCK)
	if err != nil {
		panic(err)
	}
	time.Sleep(3 * time.Second)
}

Looking at the kernel's source code & git history, I think this was solved by either or both:

• torvalds/linux@c568d68 (released in v4.9)
• torvalds/linux@387e374 (released in v5.3)

Based on my notes (although I'm not sure they're entirely exact / up-to-date), both fixes will be available on all distros we support on July 1st, once RHEL/CentOS 7.9 & 8 reach EOL 🎉 At this point, I think we shouldn't care much about these systems (ie. either use a file lock, use rename for atomic update, or do nothing special).

[robmry]

I've added a second commit that makes a best-effort attempt to acquire a write lease on /etc/hosts - it's only for a Linux host and, even on Linux, if it fails the file will be updated anyway.

If there's an open descriptor for the file, read or write, the request for a write lease fails immediately. If that happens this new code will retry once, after a 10ms sleep (an arbitrary but short time, plenty of time for a normal reader to have gone away, but not long enough to cause a noticeable delay if the file is held open).

If the lease is obtained, it will ensure that nothing in the container gets hold of the file while it's being updated. In other cases (as agreed in the maintainer's call on 21st Dec), it's unlikely to be a problem - the file is tiny, the change is infrequent and corresponds to a network config change that might be expected to cause disruption. And, the file was being updated anyway to add/remove the container name - although that's probably not necessary when the internal DNS resolver is running.

[akerouanton]

I think you also need to return NotImplementedError if fnctl returns EINVAL (to correctly handle RHEL & CentOS 7 / 8).

[robmry]

Ok, yes - that should stop it from logging anything on those platforms. I wasn't sure that was what we'd want, because on Linux we might assume no-log means it got a lease. But, it will prevent any unnecessary concern about what "opened without lease" means, and actual problems that need investigating are very unlikely.

[akerouanton]

LGTM

[robmry]

Closed in favour of #47062