On 1/21/20 1:22 PM, Dr. David Alan Gilbert (git) wrote:
From: "Dr. David Alan Gilbert" <dgilb...@redhat.com>
Hi,
This is a full set for virtiofsd - a daemon
that implements the user space side of virtiofs.
I've addressed most review comments from V1; we've got
95 Reviewed-by's out of 110 - I'd like to get a PULL out ASAP.
The set pulls in a big chunk of the upstream libfuse library
(unmodified so that it's easy to check it really is upstream),
chops all the stuff out we don't need and then adds the
new transport we need. I've formatted everything into qemu's code style -
using indent and the clang tools for the files I've imported.
We can't just link with libfuse, since we have to make ABI incompatible
changes for the new transport and it's quite invasive; the library is
designed to be the basis for multiple filesystems, but all on the same
transport.
Running this daemon is typically done with:
./virtiofsd --socket-path=/path/socket -o source=/path/to/fs
connected to a qemu that's then started with:
-chardev socket,id=char0,path=/path/socket -device
vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs
and then in the guest mount with:
mount -t virtiofs myfs /mnt
Our development branch is: https://gitlab.com/virtio-fs/qemu/tree/virtio-fs-dev
FWIW this branch contains more than this series, because it failed to
build on 32-bit:
$ make virtiofsd
CC tools/virtiofsd/fuse_lowlevel.o
tools/virtiofsd/fuse_lowlevel.c: In function 'do_removemapping':
tools/virtiofsd/fuse_lowlevel.c:1956:60: error: format '%ld' expects
argument of type 'long int', but argument 4 has type 'unsigned int'
[-Werror=format=]
"do_removemapping: invalid in, expected %d * %ld, has %ld
- %ld\n",
^
tools/virtiofsd/fuse_lowlevel.c:1956:69: error: format '%ld' expects
argument of type 'long int', but argument 5 has type 'size_t {aka
unsigned int}' [-Werror=format=]
"do_removemapping: invalid in, expected %d * %ld, has %ld
- %ld\n",
^
tools/virtiofsd/fuse_lowlevel.c:1956:75: error: format '%ld' expects
argument of type 'long int', but argument 6 has type 'size_t {aka
unsigned int}' [-Werror=format=]
"do_removemapping: invalid in, expected %d * %ld, has %ld
- %ld\n",
^
And the offending patch is not included in the 109:
$ git show 62d82cfbe5a
commit 62d82cfbe5a2ad05ae3215f2461b76aa9915108d
Author: Dr. David Alan Gilbert <dgilb...@redhat.com>
Date: Fri Jul 6 18:03:49 2018 +0100
DAX: virtiofsd: Add setup/remove mappings fuse commands
Consider pushing tags with version, so it is easier to diff
(virtio-fs-dev got force-pushed).
We don't need libslirp/capstone/dtc but they are still built:
$ make virtiofsd
GEN config-host.h
make[1]: Entering directory 'slirp'
CC slirp/src/state.o
CC slirp/src/tcp_timer.o
CC slirp/src/dhcpv6.o
CC slirp/src/ip_input.o
CC slirp/src/ip6_icmp.o
CC slirp/src/bootp.o
CC slirp/src/ip6_input.o
CC slirp/src/slirp.o
CC slirp/src/vmstate.o
CC slirp/src/ip_output.o
CC slirp/src/ncsi.o
CC slirp/src/tcp_output.o
CC slirp/src/ndp_table.o
CC slirp/src/version.o
CC slirp/src/misc.o
CC slirp/src/ip6_output.o
CC slirp/src/mbuf.o
CC slirp/src/tftp.o
CC slirp/src/arp_table.o
CC slirp/src/util.o
CC slirp/src/socket.o
CC slirp/src/sbuf.o
CC slirp/src/stream.o
CC slirp/src/dnssearch.o
CC slirp/src/udp.o
CC slirp/src/tcp_input.o
CC slirp/src/if.o
CC slirp/src/cksum.o
CC slirp/src/tcp_subr.o
CC slirp/src/udp6.o
CC slirp/src/ip_icmp.o
AR slirp/libslirp.a
make[1]: Leaving directory 'slirp'
DEP dtc/libfdt/fdt_overlay.c
DEP dtc/libfdt/fdt_addresses.c
DEP dtc/libfdt/fdt_empty_tree.c
DEP dtc/libfdt/fdt_strerror.c
DEP dtc/libfdt/fdt_rw.c
DEP dtc/libfdt/fdt_sw.c
DEP dtc/libfdt/fdt_wip.c
DEP dtc/libfdt/fdt_ro.c
DEP dtc/libfdt/fdt.c
DEP dtc/util.c
DEP dtc/fdtoverlay.c
DEP dtc/fdtput.c
DEP dtc/fdtget.c
DEP dtc/fdtdump.c
DEP convert-dtsv0-lexer.lex.c
DEP dtc/srcpos.c
BISON dtc-parser.tab.c
DEP dtc-parser.tab.c
LEX dtc-lexer.lex.c
DEP dtc-lexer.lex.c
DEP dtc/treesource.c
DEP dtc/livetree.c
DEP dtc/fstree.c
DEP dtc/flattree.c
DEP dtc/dtc.c
DEP dtc/data.c
DEP dtc/checks.c
CC libfdt/fdt.o
CC libfdt/fdt_ro.o
CC libfdt/fdt_wip.o
CC libfdt/fdt_sw.o
CC libfdt/fdt_rw.o
CC libfdt/fdt_strerror.o
CC libfdt/fdt_empty_tree.o
CC libfdt/fdt_addresses.o
CC libfdt/fdt_overlay.o
AR libfdt/libfdt.a
arm-linux-gnueabihf-ar: creating libfdt/libfdt.a
a - libfdt/fdt.o
a - libfdt/fdt_ro.o
a - libfdt/fdt_wip.o
a - libfdt/fdt_sw.o
a - libfdt/fdt_rw.o
a - libfdt/fdt_strerror.o
a - libfdt/fdt_empty_tree.o
a - libfdt/fdt_addresses.o
a - libfdt/fdt_overlay.o
CC cs.o
CC utils.o
CC SStream.o
CC MCInstrDesc.o
CC MCRegisterInfo.o
CC arch/ARM/ARMDisassembler.o
CC arch/ARM/ARMInstPrinter.o
CC arch/ARM/ARMMapping.o
CC arch/ARM/ARMModule.o
CC arch/AArch64/AArch64BaseInfo.o
CC arch/AArch64/AArch64Disassembler.o
CC arch/AArch64/AArch64InstPrinter.o
CC arch/AArch64/AArch64Mapping.o
CC arch/AArch64/AArch64Module.o
CC arch/Mips/MipsDisassembler.o
CC arch/Mips/MipsInstPrinter.o
CC arch/Mips/MipsMapping.o
CC arch/Mips/MipsModule.o
CC arch/PowerPC/PPCDisassembler.o
CC arch/PowerPC/PPCInstPrinter.o
CC arch/PowerPC/PPCMapping.o
CC arch/PowerPC/PPCModule.o
CC arch/Sparc/SparcDisassembler.o
CC arch/Sparc/SparcInstPrinter.o
CC arch/Sparc/SparcMapping.o
CC arch/Sparc/SparcModule.o
CC arch/SystemZ/SystemZDisassembler.o
CC arch/SystemZ/SystemZInstPrinter.o
CC arch/SystemZ/SystemZMapping.o
CC arch/SystemZ/SystemZModule.o
CC arch/SystemZ/SystemZMCTargetDesc.o
CC arch/X86/X86DisassemblerDecoder.o
CC arch/X86/X86Disassembler.o
CC arch/X86/X86IntelInstPrinter.o
CC arch/X86/X86ATTInstPrinter.o
CC arch/X86/X86Mapping.o
CC arch/X86/X86Module.o
CC arch/XCore/XCoreDisassembler.o
CC arch/XCore/XCoreInstPrinter.o
CC arch/XCore/XCoreMapping.o
CC arch/XCore/XCoreModule.o
CC MCInst.o
AR libcapstone.a
CC tools/virtiofsd/buffer.o
CC tools/virtiofsd/fuse_opt.o
CC tools/virtiofsd/fuse_log.o
CC tools/virtiofsd/fuse_lowlevel.o
...
FYI Paolo queued this patch, so libfdt shouldn't get compiled anymore:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg671667.html
This code is going into tools/virtiofsd based on the previous long
discussion.
Thank you to all those who have contributed code and to those who have
reviewed the previous version.
Dave
Dr. David Alan Gilbert (35):
virtiofsd: Pull in upstream headers
virtiofsd: Pull in kernel's fuse.h
virtiofsd: Add auxiliary .c's
virtiofsd: Add fuse_lowlevel.c
virtiofsd: Add passthrough_ll
virtiofsd: Trim down imported files
virtiofsd: Format imported files to qemu style
virtiofsd: Fix fuse_daemonize ignored return values
virtiofsd: Fix common header and define for QEMU builds
virtiofsd: Trim out compatibility code
virtiofsd: Add options for virtio
virtiofsd: Open vhost connection instead of mounting
virtiofsd: Start wiring up vhost-user
virtiofsd: Add main virtio loop
virtiofsd: get/set features callbacks
virtiofsd: Start queue threads
virtiofsd: Poll kick_fd for queue
virtiofsd: Start reading commands from queue
virtiofsd: Send replies to messages
virtiofsd: Keep track of replies
virtiofsd: Add Makefile wiring for virtiofsd contrib
virtiofsd: Fast path for virtio read
virtiofs: Add maintainers entry
virtiofsd: Plumb fuse_bufvec through to do_write_buf
virtiofsd: Pass write iov's all the way through
virtiofsd: cap-ng helpers
docs: Add docs/tools
virtiofsd: Handle reinit
virtiofsd: Handle hard reboot
virtiofsd: Kill threads when queues are stopped
vhost-user: Print unexpected slave message types
contrib/libvhost-user: Protect slave fd with mutex
virtiofsd: Clean up inodes on destroy
libvhost-user: Fix some memtable remap cases
virtiofsd: Convert lo_destroy to take the lo->mutex lock itself
Eric Ren (1):
virtiofsd: fix incorrect error handling in lo_do_lookup
Eryu Guan (3):
virtiofsd: print log only when priority is high enough
virtiofsd: convert more fprintf and perror to use fuse log infra
virtiofsd: stop all queue threads on exit in virtio_loop()
Jiufei Xue (1):
virtiofsd: support nanosecond resolution for file timestamp
Liu Bo (6):
virtiofsd: fix error handling in main()
virtiofsd: cleanup allocated resource in se
virtiofsd: fix memory leak on lo.source
virtiofsd: add helper for lo_data cleanup
virtiofsd: enable PARALLEL_DIROPS during INIT
Virtiofsd: fix memory leak on fuse queueinfo
Masayoshi Mizuma (4):
virtiofsd: Add ID to the log with FUSE_LOG_DEBUG level
virtiofsd: Add timestamp to the log with FUSE_LOG_DEBUG level
virtiofsd: Prevent multiply running with same vhost_user_socket
virtiofsd: add some options to the help message
Miklos Szeredi (10):
virtiofsd: passthrough_ll: add fallback for racy ops
virtiofsd: passthrough_ll: add renameat2 support
virtiofsd: passthrough_ll: disable readdirplus on cache=never
virtiofsd: passthrough_ll: control readdirplus
virtiofsd: rename unref_inode() to unref_inode_lolocked()
virtiofsd: fail when parent inode isn't known in lo_do_lookup()
virtiofsd: extract root inode init into setup_root()
virtiofsd: passthrough_ll: clean up cache related options
virtiofsd: passthrough_ll: use hashtable
virtiofsd: passthrough_ll: fix refcounting on remove/rename
Misono Tomohiro (1):
virtiofsd: Fix data corruption with O_APPEND write in writeback mode
Peng Tao (1):
virtiofsd: do not always set FUSE_FLOCK_LOCKS
Stefan Hajnoczi (37):
virtiofsd: remove mountpoint dummy argument
virtiofsd: remove unused notify reply support
virtiofsd: add -o source=PATH to help output
virtiofsd: add --fd=FDNUM fd passing option
virtiofsd: make -f (foreground) the default
virtiofsd: add vhost-user.json file
virtiofsd: add --print-capabilities option
virtiofsd: passthrough_ll: add lo_map for ino/fh indirection
virtiofsd: passthrough_ll: add ino_map to hide lo_inode pointers
virtiofsd: passthrough_ll: add dirp_map to hide lo_dirp pointers
virtiofsd: passthrough_ll: add fd_map to hide file descriptors
virtiofsd: validate path components
virtiofsd: add fuse_mbuf_iter API
virtiofsd: validate input buffer sizes in do_write_buf()
virtiofsd: check input buffer size in fuse_lowlevel.c ops
virtiofsd: prevent ".." escape in lo_do_lookup()
virtiofsd: prevent ".." escape in lo_do_readdir()
virtiofsd: use /proc/self/fd/ O_PATH file descriptor
virtiofsd: sandbox mount namespace
virtiofsd: move to an empty network namespace
virtiofsd: move to a new pid namespace
virtiofsd: add seccomp whitelist
virtiofsd: set maximum RLIMIT_NOFILE limit
virtiofsd: fix libfuse information leaks
virtiofsd: add security guide document
virtiofsd: add --syslog command-line option
virtiofsd: use fuse_lowlevel_is_virtio() in fuse_session_destroy()
virtiofsd: prevent fv_queue_thread() vs virtio_loop() races
virtiofsd: make lo_release() atomic
virtiofsd: prevent races with lo_dirp_put()
virtiofsd: rename inode->refcount to inode->nlookup
virtiofsd: add man page
virtiofsd: introduce inode refcount to prevent use-after-free
virtiofsd: process requests in a thread pool
virtiofsd: prevent FUSE_INIT/FUSE_DESTROY races
virtiofsd: fix lo_destroy() resource leaks
virtiofsd: add --thread-pool-size=NUM option
Vivek Goyal (6):
virtiofsd: Make fsync work even if only inode is passed in
virtiofsd: passthrough_ll: create new files in caller's context
virtiofsd: Parse flag FUSE_WRITE_KILL_PRIV
virtiofsd: Drop CAP_FSETID if client asked for it
virtiofsd: Support remote posix locks
virtiofsd: Reset O_DIRECT flag during file open
Xiao Yang (2):
vitriofsd/passthrough_ll: fix fallocate() ifdefs
virtiofsd/passthrough_ll: Pass errno to fuse_reply_err()
piaojun (2):
virtiofsd: add definition of fuse_buf_writev()
virtiofsd: use fuse_buf_writev to replace fuse_buf_write for better
performance
.gitignore | 1 +
MAINTAINERS | 8 +
Makefile | 27 +-
Makefile.objs | 1 +
configure | 16 +
contrib/libvhost-user/libvhost-user.c | 57 +-
contrib/libvhost-user/libvhost-user.h | 6 +
docs/index.rst | 1 +
docs/interop/vhost-user.json | 4 +-
docs/tools/conf.py | 16 +
docs/tools/index.rst | 14 +
docs/tools/virtiofsd-security.rst | 118 +
hw/virtio/vhost-user.c | 2 +-
include/standard-headers/linux/fuse.h | 891 ++++++
scripts/update-linux-headers.sh | 1 +
tools/virtiofsd/50-qemu-virtiofsd.json.in | 5 +
tools/virtiofsd/Makefile.objs | 12 +
tools/virtiofsd/buffer.c | 351 +++
tools/virtiofsd/fuse.h | 1249 +++++++++
tools/virtiofsd/fuse_common.h | 860 ++++++
tools/virtiofsd/fuse_i.h | 115 +
tools/virtiofsd/fuse_log.c | 40 +
tools/virtiofsd/fuse_log.h | 74 +
tools/virtiofsd/fuse_lowlevel.c | 2779 +++++++++++++++++++
tools/virtiofsd/fuse_lowlevel.h | 2023 ++++++++++++++
tools/virtiofsd/fuse_misc.h | 60 +
tools/virtiofsd/fuse_opt.c | 449 +++
tools/virtiofsd/fuse_opt.h | 272 ++
tools/virtiofsd/fuse_signals.c | 97 +
tools/virtiofsd/fuse_virtio.c | 985 +++++++
tools/virtiofsd/fuse_virtio.h | 33 +
tools/virtiofsd/helper.c | 349 +++
tools/virtiofsd/passthrough_helpers.h | 51 +
tools/virtiofsd/passthrough_ll.c | 3006 +++++++++++++++++++++
tools/virtiofsd/seccomp.c | 164 ++
tools/virtiofsd/seccomp.h | 16 +
tools/virtiofsd/virtiofsd.texi | 104 +
37 files changed, 14242 insertions(+), 15 deletions(-)
create mode 100644 docs/tools/conf.py
create mode 100644 docs/tools/index.rst
create mode 100644 docs/tools/virtiofsd-security.rst
create mode 100644 include/standard-headers/linux/fuse.h
create mode 100644 tools/virtiofsd/50-qemu-virtiofsd.json.in
create mode 100644 tools/virtiofsd/Makefile.objs
create mode 100644 tools/virtiofsd/buffer.c
create mode 100644 tools/virtiofsd/fuse.h
create mode 100644 tools/virtiofsd/fuse_common.h
create mode 100644 tools/virtiofsd/fuse_i.h
create mode 100644 tools/virtiofsd/fuse_log.c
create mode 100644 tools/virtiofsd/fuse_log.h
create mode 100644 tools/virtiofsd/fuse_lowlevel.c
create mode 100644 tools/virtiofsd/fuse_lowlevel.h
create mode 100644 tools/virtiofsd/fuse_misc.h
create mode 100644 tools/virtiofsd/fuse_opt.c
create mode 100644 tools/virtiofsd/fuse_opt.h
create mode 100644 tools/virtiofsd/fuse_signals.c
create mode 100644 tools/virtiofsd/fuse_virtio.c
create mode 100644 tools/virtiofsd/fuse_virtio.h
create mode 100644 tools/virtiofsd/helper.c
create mode 100644 tools/virtiofsd/passthrough_helpers.h
create mode 100644 tools/virtiofsd/passthrough_ll.c
create mode 100644 tools/virtiofsd/seccomp.c
create mode 100644 tools/virtiofsd/seccomp.h
create mode 100644 tools/virtiofsd/virtiofsd.texi
