Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-30 Thread Stefan Hajnoczi
On Thu, Mar 25, 2021 at 06:58:56PM +0100, Ilya Maximets wrote: > On 3/25/21 5:43 PM, Stefan Hajnoczi wrote: > > On Thu, Mar 25, 2021 at 12:00:11PM +0100, Ilya Maximets wrote: > >> On 3/25/21 10:35 AM, Stefan Hajnoczi wrote: > >>> On Wed, Mar 24, 2021 at 02:11:3

Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-18 Thread Stefan Hajnoczi
On Wed, Mar 17, 2021 at 09:25:26PM +0100, Ilya Maximets wrote: Hi, Some questions to understand the problems that SocketPair Broker solves: > Even more configuration tricks required in order to share some sockets > between different containers and not only with the host, e.g. to > create service c

Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-19 Thread Stefan Hajnoczi
On Thu, Mar 18, 2021 at 08:47:12PM +0100, Ilya Maximets wrote: > On 3/18/21 6:52 PM, Stefan Hajnoczi wrote: > > On Wed, Mar 17, 2021 at 09:25:26PM +0100, Ilya Maximets wrote: > >> And some housekeeping usually required for applications in case the > >> socket serve

Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-19 Thread Stefan Hajnoczi
On Thu, Mar 18, 2021 at 09:14:27PM +0100, Ilya Maximets wrote: > On 3/18/21 8:47 PM, Ilya Maximets wrote: > > On 3/18/21 6:52 PM, Stefan Hajnoczi wrote: > >> On Wed, Mar 17, 2021 at 09:25:26PM +0100, Ilya Maximets wrote: > >> BTW what is the security model of the b

Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-19 Thread Stefan Hajnoczi
Hi Ilya, By the way, it's not clear to me why dpdkvhostuser is deprecated. If OVS is restarted then existing vhost-user connections drop with an error but QEMU could attempt to reconnect to the UNIX domain socket which the new OVS instance will set up. Why is it impossible to reconnect when OVS ow

Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-19 Thread Stefan Hajnoczi
On Fri, Mar 19, 2021 at 04:37:01PM +0100, Ilya Maximets wrote: > On 3/19/21 3:16 PM, Stefan Hajnoczi wrote: > > On Thu, Mar 18, 2021 at 09:14:27PM +0100, Ilya Maximets wrote: > >> On 3/18/21 8:47 PM, Ilya Maximets wrote: > >>> On 3/18/21 6:52 PM, Stefan Hajnoczi wrot

Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-19 Thread Stefan Hajnoczi
On Fri, Mar 19, 2021 at 04:29:21PM +0100, Ilya Maximets wrote: > On 3/19/21 3:05 PM, Stefan Hajnoczi wrote: > > On Thu, Mar 18, 2021 at 08:47:12PM +0100, Ilya Maximets wrote: > >> On 3/18/21 6:52 PM, Stefan Hajnoczi wrote: > >>> On Wed, Mar 17, 2021 at 09:25:2

Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-24 Thread Stefan Hajnoczi
On Tue, Mar 23, 2021 at 04:54:57PM -0400, Billy McFall wrote: > On Tue, Mar 23, 2021 at 3:52 PM Ilya Maximets wrote: > > > On 3/23/21 6:57 PM, Adrian Moreno wrote: > > > > > > > > > On 3/19/21 6:21 PM, Stefan Hajnoczi wrote: > > >> On Fri, Ma

Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-24 Thread Stefan Hajnoczi
On Wed, Mar 24, 2021 at 02:11:31PM +0100, Ilya Maximets wrote: > On 3/24/21 1:05 PM, Stefan Hajnoczi wrote: > > On Tue, Mar 23, 2021 at 04:54:57PM -0400, Billy McFall wrote: > >> On Tue, Mar 23, 2021 at 3:52 PM Ilya Maximets wrote: > >> > >>>

Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-25 Thread Stefan Hajnoczi
On Wed, Mar 24, 2021 at 02:11:31PM +0100, Ilya Maximets wrote: > On 3/24/21 1:05 PM, Stefan Hajnoczi wrote: > > On Tue, Mar 23, 2021 at 04:54:57PM -0400, Billy McFall wrote: > >> On Tue, Mar 23, 2021 at 3:52 PM Ilya Maximets wrote: > >>> On 3/23/21 6:57 PM, Adrian M

Re: [dpdk-dev] [RFC 0/4] SocketPair Broker support for vhost and virtio-user.

2021-03-25 Thread Stefan Hajnoczi
On Thu, Mar 25, 2021 at 12:00:11PM +0100, Ilya Maximets wrote: > On 3/25/21 10:35 AM, Stefan Hajnoczi wrote: > > On Wed, Mar 24, 2021 at 02:11:31PM +0100, Ilya Maximets wrote: > >> On 3/24/21 1:05 PM, Stefan Hajnoczi wrote: > >>> On Tue, Mar 23, 2021 at 04:54:

[dpdk-dev] [RFC 00/24] vhost: add virtio-vhost-user transport

2018-01-19 Thread Stefan Hajnoczi
bind.py to support virtio-vhost-user: 4aee6f653 usertools: add virtio-vhost-user devices to dpdk-devbind.py Finally, vhost-scsi seems broken to me so two workarounds were needed so it can be tested again: b9d17bfaf WORKAROUND revert virtio-net mq vring deletion cadb25e7d WORKAROUND

[dpdk-dev] [RFC 01/24] vhost: move vring_call() into trans_af_unix.c

2018-01-19 Thread Stefan Hajnoczi
vhost-user code. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/Makefile| 2 +- lib/librte_vhost/vhost.h | 35 +++- lib/librte_vhost/trans_af_unix.c | 49 lib/librte_vhost/vhost.c | 1 + 4 files changed

[dpdk-dev] [RFC 02/24] vhost: move AF_UNIX code from socket.c to trans_af_unix.c

2018-01-19 Thread Stefan Hajnoczi
ut this is a temporary measure and will be cleaned up in later patches. By simply moving code unmodified in this patch it will be easier to review the actual refactoring that follows. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 65 + lib/librte_vhost/socket.c

[dpdk-dev] [RFC 03/24] vhost: allocate per-socket transport state

2018-01-19 Thread Stefan Hajnoczi
. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 9 + lib/librte_vhost/socket.c| 6 -- lib/librte_vhost/trans_af_unix.c | 5 + 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h index

[dpdk-dev] [RFC 05/24] vhost: move start_server/client() calls to trans_af_unix.c

2018-01-19 Thread Stefan Hajnoczi
Introduce a vhost_transport_ops->socket_start() interface so the transport can begin establishing vhost-user connections. This is part of the AF_UNIX transport refactoring and removes AF_UNIX code from vhost.h and socket.c. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhos

[dpdk-dev] [RFC 04/24] vhost: move socket_fd and un sockaddr into trans_af_unix.c

2018-01-19 Thread Stefan Hajnoczi
socket_cleanup() to vhost_transport_ops. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 31 +- lib/librte_vhost/socket.c| 10 ++-- lib/librte_vhost/trans_af_unix.c | 55 3 files changed, 72 insertions

[dpdk-dev] [RFC 07/24] vhost: move vhost_user_reconnect_init() into trans_af_unix.c

2018-01-19 Thread Stefan Hajnoczi
The socket reconnection code is highly specific to AF_UNIX, so move the remaining pieces of it into trans_af_unix.c. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 10 +++--- lib/librte_vhost/socket.c| 4 lib/librte_vhost/trans_af_unix.c | 9

[dpdk-dev] [RFC 06/24] vhost: move vhost_user_connection to trans_af_unix.c

2018-01-19 Thread Stefan Hajnoczi
. Move struct vhost_user_connection to trans_af_unix.c and conn_list/conn_mutex into struct af_unix_socket. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 19 ++--- lib/librte_vhost/socket.c| 36 ++-- lib/librte_vhost/trans_af_unix.c | 60

[dpdk-dev] [RFC 08/24] vhost: move vhost_user.fdset to trans_af_unix.c

2018-01-19 Thread Stefan Hajnoczi
The fdset is used by the AF_UNIX transport code but other transports may not need it. Move it to trans_af_unix.c and then make struct vhost_user private again since nothing outside socket.c needs it. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 15 --- lib

[dpdk-dev] [RFC 09/24] vhost: pass vhost_transport_ops through vhost_new_device()

2018-01-19 Thread Stefan Hajnoczi
turn towards vhost_user.c and its mixture of vhost-user protocol processing and socket I/O. The socket I/O will be moved into trans_af_unix.c so that other transports can be added that don't use file descriptors. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 2 +- lib/li

[dpdk-dev] [RFC 12/24] vhost: move slave_req_fd field to AF_UNIX transport

2018-01-19 Thread Stefan Hajnoczi
. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 27 -- lib/librte_vhost/trans_af_unix.c | 41 +++- lib/librte_vhost/vhost.c | 3 ++- lib/librte_vhost/vhost_user.c| 18 +- 4 files changed, 68

[dpdk-dev] [RFC 11/24] vhost: extract vhost_user.c socket I/O into transport

2018-01-19 Thread Stefan Hajnoczi
The core vhost-user protocol code should not do socket I/O because the details are transport-specific. Move code to send and receive vhost-user messages into trans_af_unix.c. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 26 lib/librte_vhost/vhost_user.h

[dpdk-dev] [RFC 10/24] vhost: embed struct virtio_net inside struct vhost_user_connection

2018-01-19 Thread Stefan Hajnoczi
these structs together. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 11 +- lib/librte_vhost/trans_af_unix.c | 44 +--- lib/librte_vhost/vhost.c | 10 - 3 files changed, 34 insertions(+), 31 deletions(-) diff

[dpdk-dev] [RFC 13/24] vhost: move mmap/munmap to AF_UNIX transport

2018-01-19 Thread Stefan Hajnoczi
lable from vhost_user.c. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 17 +++ lib/librte_vhost/vhost_user.h| 3 ++ lib/librte_vhost/trans_af_unix.c | 78 + lib/librte_vhost/vhost_user.c| 95 ++---

[dpdk-dev] [RFC 14/24] vhost: move librte_vhost to drivers/

2018-01-19 Thread Stefan Hajnoczi
to librte_vhost without exporting all private vhost.h symbols. Signed-off-by: Stefan Hajnoczi --- Is there a better way of giving a PCI driver access to librte_vhost transport (not a public API that apps should use)? --- drivers/Makefile| 2 ++ {lib => driv

[dpdk-dev] [RFC 15/24] vhost: add virtio pci framework

2018-01-19 Thread Stefan Hajnoczi
it. Or perhaps we could backport SPDK's lib/virtio. I don't have time to do either right now so I've just copied the code, removed virtio-net and ethdev parts, and renamed symbols to avoid link errors. Signed-off-by: Stefan Hajnoczi --- drivers/librte_vhost/Makefile |

[dpdk-dev] [RFC 16/24] vhost: remember a vhost_virtqueue's queue index

2018-01-19 Thread Stefan Hajnoczi
Currently the only way of determining a struct vhost_virtqueue's index is to search struct virtio_net->virtqueue[] for its address. Stash the index in struct vhost_virtqueue so we won't have to search the array. This new field will be used by virtio-vhost-user. Signed-off-by: St

[dpdk-dev] [RFC 18/24] vhost: add RTE_VHOST_USER_VIRTIO_TRANSPORT flag

2018-01-19 Thread Stefan Hajnoczi
Extend the API to support the virtio-vhost-user transport as an alternative to the AF_UNIX transport. The caller provides a PCI DomBDF address: rte_vhost_driver_register(":00:04.0", RTE_VHOST_USER_VIRTIO_TRANSPORT); Signed-off-by: Stefan Hajnoczi --

[dpdk-dev] [RFC 17/24] vhost: add virtio-vhost-user transport

2018-01-19 Thread Stefan Hajnoczi
intended use case is for running DPDK inside a guest. Other guests can communicate via DPDK's "vhost" vdev driver. For more information on virtio-vhost-user, see https://wiki.qemu.org/Features/VirtioVhostUser. Signed-off-by: Stefan Hajnoczi --- drivers/librte_vhost/Makefile

[dpdk-dev] [RFC 19/24] net/vhost: add virtio-vhost-user support

2018-01-19 Thread Stefan Hajnoczi
The new virtio-transport=0|1 argument enables virtio-vhost-user support: testpmd ... --pci-whitelist :00:04.0 \ --vdev vhost,iface=:00:04.0,virtio-transport=1 Signed-off-by: Stefan Hajnoczi --- drivers/net/vhost/rte_eth_vhost.c | 13 + 1 file changed, 13

[dpdk-dev] [RFC 20/24] examples/vhost_scsi: add --socket-file argument

2018-01-19 Thread Stefan Hajnoczi
The default filename built into examples/vhost_scsi may not be convenient. Allow the user to specify the full UNIX domain socket path on the command-line. Signed-off-by: Stefan Hajnoczi --- examples/vhost_scsi/vhost_scsi.c | 93 1 file changed, 75

[dpdk-dev] [RFC 21/24] examples/vhost_scsi: add virtio-vhost-user support

2018-01-19 Thread Stefan Hajnoczi
The new --virtio-vhost-user-pci command-line argument uses virtio-vhost-user instead of the default AF_UNIX transport. Signed-off-by: Stefan Hajnoczi --- examples/vhost_scsi/vhost_scsi.c | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/examples/vhost_scsi

[dpdk-dev] [RFC 22/24] usertools: add virtio-vhost-user devices to dpdk-devbind.py

2018-01-19 Thread Stefan Hajnoczi
The virtio-vhost-user PCI adapter is not detected in any existing group of devices supported by dpdk-devbind.py. Add a new "Others" group for miscellaneous devices like this one. Signed-off-by: Stefan Hajnoczi --- usertools/dpdk-devbind.py | 8 1 file changed, 8 insertion

[dpdk-dev] [RFC 24/24] WORKAROUND examples/vhost_scsi: avoid broken EVENT_IDX

2018-01-19 Thread Stefan Hajnoczi
The EVENT_IDX code in DPDK is broken. It's missing the signalled_used_valid flag that handles the corner cases (startup and wrapping). Disable it for now. Signed-off-by: Stefan Hajnoczi --- examples/vhost_scsi/vhost_scsi.c | 1 - 1 file changed, 1 deletion(-) diff --git a/examples/vhost

[dpdk-dev] [RFC 23/24] WORKAROUND revert virtio-net mq vring deletion

2018-01-19 Thread Stefan Hajnoczi
The virtio-net mq vring deletion code should be in virtio_net.c, not in the generic vhost_user.c code where it breaks non-virtio-net devices. Signed-off-by: Stefan Hajnoczi --- drivers/librte_vhost/vhost_user.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/librte_vhost

[dpdk-dev] [PATCH 1/2] vhost: add flag for built-in virtio_net.c driver

2018-01-31 Thread Stefan Hajnoczi
ned-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 3 +++ lib/librte_vhost/socket.c | 15 +++ lib/librte_vhost/vhost.c | 17 - lib/librte_vhost/virtio_net.c | 14 ++ 4 files changed, 48 insertions(+), 1 deletion(-) diff --git a/lib/li

[dpdk-dev] [PATCH 0/2] vhost: fix VIRTIO_NET_F_MQ vhost_scsi breakage

2018-01-31 Thread Stefan Hajnoczi
2 for details. Patch 1 puts the infrastructure in place to distinguish between the built-in virtio_net.c driver and generic vhost device backend usage. Patch 2 fixes the regression by handling VIRTIO_NET_F_MQ only when the built-in virtio_net.c driver is in use. Stefan Hajnoczi (2): vhost: add

[dpdk-dev] [PATCH 2/2] vhost: only drop vqs with built-in virtio_net.c driver

2018-01-31 Thread Stefan Hajnoczi
host net device backends. Use the new VIRTIO_DEV_BUILTIN_VIRTIO_NET flag to check whether virtio_net.c is being used. This fixes examples/vhost_scsi. Cc: Maxime Coquelin Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost_user.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) d

[dpdk-dev] [PATCH] examples/vhost_scsi: drop unimplemented EVENT_IDX feature bit

2018-01-31 Thread Stefan Hajnoczi
si so that vring call works again. Cc: Changpeng Liu Cc: Junjie Chen Signed-off-by: Stefan Hajnoczi --- examples/vhost_scsi/vhost_scsi.c | 1 - 1 file changed, 1 deletion(-) diff --git a/examples/vhost_scsi/vhost_scsi.c b/examples/vhost_scsi/vhost_scsi.c index da01ad378..3cb4383e9 100644 --- a/exa

[dpdk-dev] [PATCH 1/8] vhost: add security model documentation to vhost_user.c

2018-02-05 Thread Stefan Hajnoczi
Input validation is not applied consistently in vhost_user.c. This suggests that not everyone has the same security model in mind when working on the code. Make the security model explicit so that everyone can understand and follow the same model when modifying the code. Signed-off-by: Stefan

[dpdk-dev] [PATCH 2/8] vhost: avoid enum fields in VhostUserMsg

2018-02-05 Thread Stefan Hajnoczi
fragile and it's unlikely that anyone changing the code would remember this. A security hole can be introduced accidentally. This patch switches VhostUserMsg.request fields to uint32_t to avoid the portability and potential security issues. Signed-off-by: Stefan Hajnoczi --- lib/librte_vh

[dpdk-dev] [PATCH 0/8] vhost: input validation enhancements

2018-02-05 Thread Stefan Hajnoczi
says. Stefan Hajnoczi (8): vhost: add security model documentation to vhost_user.c vhost: avoid enum fields in VhostUserMsg vhost: validate untrusted memory.nregions field vhost: clear out unused SCM_RIGHTS file descriptors vhost: reject invalid log base mmap_offset values vhost: fix

[dpdk-dev] [PATCH 3/8] vhost: validate untrusted memory.nregions field

2018-02-05 Thread Stefan Hajnoczi
Check if memory.nregions is valid right away. This eliminates the possibility of bugs when memory.nregions is used later on in vhost_user_set_mem_table(). Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost_user.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/lib/librte_vhost

[dpdk-dev] [PATCH 6/8] vhost: fix msg->payload union typo in vhost_user_set_vring_addr()

2018-02-05 Thread Stefan Hajnoczi
vhost_user_set_vring_addr() uses the msg->payload.addr union member, not msg->payload.state. Luckily the offset of the 'index' field is identical in both structs, so there was never any buggy behavior. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost_user.c | 2 +- 1

[dpdk-dev] [PATCH 7/8] vhost: validate virtqueue size

2018-02-05 Thread Stefan Hajnoczi
Check the virtqueue size constraints so that invalid values don't cause bugs later on in the code. For example, sometimes the virtqueue size is stored as unsigned int and sometimes as uint16_t, so bad things happen if it is ever larger than 65535. Signed-off-by: Stefan Hajnoczi ---

[dpdk-dev] [PATCH 4/8] vhost: clear out unused SCM_RIGHTS file descriptors

2018-02-05 Thread Stefan Hajnoczi
match. Refactoring vhost_user.c to pass around and check the number of file descriptors everywhere would make the code more complex. It is simpler for read_fd_message() to set unused elements in fds[] to -1. This way close(-1) is called and no harm is done. Signed-off-by: Stefan Hajnoczi --

[dpdk-dev] [PATCH 5/8] vhost: reject invalid log base mmap_offset values

2018-02-05 Thread Stefan Hajnoczi
If the log base mmap_offset is larger than mmap_size then it points outside the mmap region. We must not write to memory outside the mmap region, so validate mmap_offset in vhost_user_set_log_base(). Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost_user.c | 9 + 1 file changed

[dpdk-dev] [PATCH 8/8] vhost: check for memory_size + mmap_offset overflow

2018-02-05 Thread Stefan Hajnoczi
If memory_size + mmap_offset overflows then the memory region is bogus. Do not use the overflowed mmap_size value for mmap(). Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost_user.c | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/lib/librte_vhost

[dpdk-dev] [PATCH] vhost: introduce rte_vhost_vring_call()

2017-12-21 Thread Stefan Hajnoczi
icate it. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/rte_vhost.h | 15 +++ examples/vhost/virtio_net.c| 11 ++- examples/vhost_scsi/vhost_scsi.c | 6 +++--- lib/librte_vhost/vhost.c | 27 +++ lib/librte_

[dpdk-dev] [PATCH v2 0/2] vhost: introduce rte_vhost_vring_call()

2018-01-02 Thread Stefan Hajnoczi
v2: * Add internal vhost_vring_call() helper function [Maxime] These patches eliminate code duplication for vhost_virtqueue->callfd users by introducing rte_vhost_vring_call() (public API) and vhost_vring_call() (librte_vhost-internal API). Stefan Hajnoczi (2): vhost: add vhost_vring_c

[dpdk-dev] [PATCH v2 1/2] vhost: add vhost_vring_call() helper

2018-01-02 Thread Stefan Hajnoczi
Extract the callfd eventfd signal operation so virtio_net.c does not have to repeat it multiple times. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/vhost.h | 12 lib/librte_vhost/virtio_net.c | 23 +++ 2 files changed, 15 insertions(+), 20 deletions

[dpdk-dev] [PATCH v2 2/2] vhost: introduce rte_vhost_vring_call()

2018-01-02 Thread Stefan Hajnoczi
icate it. Signed-off-by: Stefan Hajnoczi --- lib/librte_vhost/rte_vhost.h | 15 +++ examples/vhost/virtio_net.c| 11 ++- examples/vhost_scsi/vhost_scsi.c | 6 +++--- lib/librte_vhost/vhost.c | 21 + lib/librte_