Beam SDK docker containers on Dataflow VMs are currently launched in privileged mode.
On Mon, Jan 30, 2023 at 9:52 AM Robert Bradshaw via user < user@beam.apache.org> wrote: > Different idea: is it possible to serve this data via another protocol > (e.g. sftp) rather than requiring a mount? > > On Mon, Jan 30, 2023 at 9:26 AM Chad Dombrova <chad...@gmail.com> wrote: > > > > Hi Robert, > > I know very little about the FileSystem classes, but I don’t think it’s > possible for a process running in docker to create an NFS mount without > running in privileged [1] mode, which cannot be done with Dataflow. The > other ways of gaining access to a mount are: > > > > A. the node running docker has the NFS mount itself and passes it along > using docker run --volume. > > B. the mount is created within the container by using docker run --mount. > > > > Neither of these are possible with Dataflow. > > > > Here’s a full example of how an NFS mount can be created when running > docker: > > > > docker run -it --network=host \ > > --mount > 'type=volume,src=pipe-nfs-test,dst=/Volumes/pipe-nfs-test,volume-driver=local,volume-opt=type=nfs,volume-opt=device=:/pipe,"volume-opt=o=addr=turbohal.luma.mel,vers=3"' > \ > > luma/pipe-shell -- bash > > > > In my ideal world, I would make a PR to add support for the docker > --mount flag to Beam for the runners that I can control, and the Dataflow > team would add support on their end. > > > > Let me know if I'm missing anything. > > > > > https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities > > > > thanks, > > -chad >
