Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

volumes: Implement subpath mount #45687

Merged
merged 12 commits into from Jan 19, 2024
Merged

Conversation

vvoland
Copy link
Contributor

@vvoland vvoland commented Jun 2, 2023

- What I did
Make it possible to mount subdirectory of a named volume.

- How I did it

VolumeOptions now has a Subpath field which allows to specify a path relative to the volume that should be mounted as a destination.

Symlinks are supported, but they cannot escape the base volume directory.
A protection against the Time-of-check to Time-of-use is implemented for Linux and Windows to make it not possible to replace the mount source with a symlink that escapes the volume directory.

On Linux, all subpath components are opened with openat, relative to the base volume directory and checked against the volume escape. The final file descriptor is mounted from the /proc/self/fd/ to a temporary mount point owned by the daemon and then passed to the underlying container runtime. Temporary mountpoint is removed after the container is started.

On Windows, all components of the path are locked before the check, and released once the path is already mounted.

- How to verify it
TestRunMountVolumeSubdir integration test

- Description for the changelog
Add Subpath field to the VolumeOptions making it possible to mount a subpath of a volume.

- A picture of a cute animal (not mandatory but encouraged)
image

@vvoland vvoland added area/api kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny impact/changelog area/volumes labels Jun 2, 2023
@vvoland vvoland added this to the 25.0.0 milestone Jun 2, 2023
@vvoland vvoland force-pushed the volume-mount-subpath branch 6 times, most recently from 162300a to e346dc5 Compare June 2, 2023 13:54
volume/mounts/mounts.go Outdated Show resolved Hide resolved
@vvoland vvoland force-pushed the volume-mount-subpath branch 2 times, most recently from 0176b5a to 1765550 Compare June 2, 2023 15:48
@vvoland vvoland force-pushed the volume-mount-subpath branch 14 times, most recently from b301fa1 to ac302cb Compare June 7, 2023 15:57
@vvoland vvoland modified the milestones: v-future, 26.0.0 Jan 19, 2024
To make it easier to distinguish if an output variable is modified.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
`VolumeOptions` now has a `Subpath` field which allows to specify a path
relative to the volume that should be mounted as a destination.

Symlinks are supported, but they cannot escape the base volume
directory.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
All subpath components are opened with openat, relative to the base
volume directory and checked against the volume escape.
The final file descriptor is mounted from the /proc/self/fd/<fd> to a
temporary mount point owned by the daemon and then passed to the
underlying container runtime.
Temporary mountpoint is removed after the container is started.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Handle EINTR by retrying the syscall.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
For use as a soft fallback if Openat2 is not available.
Source: https://github.com/kubernetes/kubernetes/blob/55fb1805a1217b91b36fa8fe8f2bf3a28af2454d/pkg/volume/util/subpath/subpath_linux.go

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Adapts the function source code to the Moby codebase.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
All components of the path are locked before the check, and
released once the path is already mounted.
This makes it impossible to replace the mounted directory until it's
actually mounted in the container.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
@vvoland vvoland marked this pull request as ready for review January 19, 2024 17:27
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - let's get this one in 🎉

@zmweske
Copy link

zmweske commented Apr 19, 2024

is the long syntax required or would it be possible to utilize short syntax? It currently seems to only work with long syntax but it would make sense to me that if a named volume exists with a subpath, it should work. If a named volume doesn't exist, it will look for folders in ./

See: https://docs.docker.com/compose/compose-file/05-services/#volumes

services:
  example:
    image: example:latest
    volumes:
      - named-volume/subpath:/internal/dir:rw  # would utilize a named volume that exists in the compose file already
      - local-dir/subpath:/internal/dir:rw  # would use a local dir as no named volume exists
      - ./local-dir2/subpath:/internal/dir2:rw  # functionally the same as above
...
volumes:
  named-volume:
    driver_opts:
      type: cifs
      o: "uid=1000,username=username,password=${CIFS_PWD},iocharset=utf8,noperm,nobrl,vers=3.0"
      device: "//192.168.10.10/example/storage"

Thoughts?

@neersighted
Copy link
Member

This is the wrong issue to discuss Compose's implementation. That being said, we're deliberately avoiding short/ambiguous formats (e.g. the implicit behavior you describe) for new features like this, for (hopefully obvious) reasons of explicit being better than implicit.

Less typing sounds nice, until it leads to a bevy of issue reports, wasted hours debugging, or (in the worst case) security issues.

Please open a discussion on https://github.com/compose-spec/compose-spec for questions, or open an issue if you have a concrete proposal/change to suggest.

@thaJeztah
Copy link
Member

Yeah, I don't think we want to overload the short-form syntax more. Also not sure if such a format should be defined by the compose-spec first (as there's an expectation for the short-form format to be the same as is used for -v short-form).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api area/volumes impact/changelog kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny status/1-design-review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[feature] Allow mounting sub-directories of named volumes
9 participants