Synchronized file shares stalling and "locks" the entire environment

[NiklasBr]

Description

Upon creating a file share for my project it will scan the files and work for a while, then it will silently error out and everything in Docker will stop working until the tile share is deleted.

Reproduce

1. Create a share for the workspace.
2. Start the Docker Compose workspace.
3. Do some development.
4. Notice how everything stops.
5. Go into the settings, and see errors like Host: var/cache/dev/translations/catalogue.pt_BR.Y83vERT.php unable to create file: unable to relocate staged file: file exists

It should be expected that files in cache and tmp directories are purged occasionally while in development.

Expected behavior

It should be faster than without. Or at least as slow as without file shares.

docker version

Client:
 Cloud integration: v1.0.35+desktop.13
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:44:56 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.30.0 (149282)
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:04 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

### docker info

```bash
Client:
 Version:    26.1.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.0-desktop.1
    Path:     /Users/nikbr/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.0-desktop.2
    Path:     /Users/nikbr/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.29
    Path:     /Users/nikbr/.docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/nikbr/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.23
    Path:     /Users/nikbr/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     /Users/nikbr/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.1.0
    Path:     /Users/nikbr/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/nikbr/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.8.0
    Path:     /Users/nikbr/.docker/cli-plugins/docker-scout
WARNING: Plugin "/Users/nikbr/.docker/cli-plugins/docker-scan" is not valid: failed to fetch metadata: fork/exec /Users/nikbr/.docker/cli-plugins/docker-scan: no such file or directory

Server:
 Containers: 26
  Running: 13
  Paused: 0
  Stopped: 13
 Images: 53
 Server Version: 26.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.6.26-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 9
 Total Memory: 23.44GiB
 Name: docker-desktop
 ID: 4bacd5ea-52d7-4544-a700-b29136ca8a38
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/nikbr/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

### Diagnostics ID

39263439-8ED3-425A-A6C1-A4B1DD929F05/20240513142557

### Additional Info

_No response_

[xenoscopic]

Hi @NiklasBr, the file exists error on the host is typically due to a case conflict. Specifically, files that can co-exist on Linux's case sensitive filesystems can't co-exist on macOS' case-insensitive filesystem. You can exclude these problematic files using a .syncignore file (see this comment for an example). A similar approach for your cache directories might help.

That being said, the existence of case conflicts shouldn't cause the sync to halt. Looking at the diagnostic bundle you sent (thanks!), the synchronization appears to be operating. Can you clarify what you mean by "everything stops?"

[NiklasBr]

By "everything stops" I mean the container(s) become unresponsive, they do not respond to HTTP requests, you cannot connect to the CLI, or otherwise interact with them. It's like they are hard blocked by some process.

[AyoubZahid]

i have this same issue, and when i modify a file it does not get modified inside the container even if i delete the container and regenerate it again, i need to restart docker entirely to get my modification inside the container

[xenoscopic]

Thanks for the clarification. Do either of you have a reproducer or list of tools/frameworks that you could share that might trigger the issue? Synchronized File Shares are fairly orthogonal to anything that might freeze a container, so it may be an unrelated issue, but I wouldn't rule anything out yet.

If possible, it would also be very helpful to know if the freezing issue happens with Docker Desktop 4.29.

[NiklasBr]

At the moment we are using a variant of https://github.com/pimcore/skeleton/

[DougPlumley]

I'm not 100% sure I'm having the same issue, but I've noticed that containers keep becoming unresponsive since updating to 4.30.0.

[NiklasBr]

@xenoscopic any progress?

It's growing more and more problematic because even though no containers are running it is impossible to delete shares.

[AyoubZahid]

for me i rolled back to version 4.24 that does not have those issues.

[xenoscopic]

Hey @NiklasBr, I haven't had a chance yet to investigate further. Is there any chance you'd be available for a Zoom call to try debugging the issue directly?

it is impossible to delete shares.

Also, can you clarify whether the "Delete" button for the shares is greyed out or if it simply has no effect?

[NiklasBr]

Sure invite me at niklas.brunberg@cag.se.

It's gray, even when I stop all containers and Quit and re-launch the application:

[xenoscopic]

In this case it looks like the Delete button is greyed out because of the containers using the share (which are currently stopped, but still have the share bind-mounted).

I'll send you an email to schedule a debug session.

[xenoscopic]

This may be related to and/or a duplicate of #7288, but I'll leave it open until more information is available.