Re: [PATCH vhost v9 05/12] virtio_ring: split: virtqueue_add_split() support premapped

2023-05-17 Thread Jason Wang
On Wed, May 17, 2023 at 10:23 AM Xuan Zhuo wrote: > > virtqueue_add_split() only supports virtual addresses, dma is completed > in virtqueue_add_split(). > > In some scenarios (such as the AF_XDP scenario), the memory is allocated > and DMA is completed in advance, so it is necessary for us to sup

Re: [PATCH vhost v9 04/12] virtio_ring: virtqueue_add() support premapped

2023-05-17 Thread Jason Wang
On Wed, May 17, 2023 at 10:23 AM Xuan Zhuo wrote: > > virtuque_add() adds parameter premapped. I wonder if this patch is over simplified. Maybe it can be squashed with the patch that implements the premapped logic. Thanks > > Signed-off-by: Xuan Zhuo > --- > drivers/virtio/virtio_ring.c | 9

Re: [PATCH vhost v9 03/12] virtio_ring: check use_dma_api before unmap desc for indirect

2023-05-17 Thread Jason Wang
On Wed, May 17, 2023 at 10:23 AM Xuan Zhuo wrote: > > Inside detach_buf_split(), if use_dma_api is false, > vring_unmap_one_split_indirect will be called many times, but actually > nothing is done. So this patch check use_dma_api firstly. > > Signed-off-by: Xuan Zhuo Acked-by: Jason Wang Thank

Re: [PATCH vhost v9 02/12] virtio_ring: simplify the reference of desc state inside detach_buf_split()

2023-05-17 Thread Jason Wang
On Wed, May 17, 2023 at 10:23 AM Xuan Zhuo wrote: > > The purpose of this is to simplify the reference to state. It is > convenient for subsequent commit. It's better to be verbose, e.g how it can simplify the following patches. Thanks > > Signed-off-by: Xuan Zhuo > --- > drivers/virtio/virt

Re: [PATCH vhost v9 01/12] virtio_ring: put mapping error check in vring_map_one_sg

2023-05-17 Thread Jason Wang
On Wed, May 17, 2023 at 10:23 AM Xuan Zhuo wrote: > > This patch put the dma addr error check in vring_map_one_sg(). > > The benefits of doing this: > > 1. make vring_map_one_sg more simple, without calling >vring_mapping_error to check the return value. > 2. reduce one judgment of vq->use_dma

Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base

2023-05-17 Thread Jason Wang
On Wed, May 17, 2023 at 3:00 PM Stefano Garzarella wrote: > > On Wed, May 17, 2023 at 7:26 AM Jason Wang wrote: > > > > On Wed, May 17, 2023 at 2:26 AM Shannon Nelson > > wrote: > > > > > > On 5/16/23 12:49 AM, Stefano Garzarella wrote: > > > > On Mon, May 15, 2023 at 01:41:12PM -0700, Shannon

Re: [RFC PATCH 1/8] signal: Dequeue SIGKILL even if SIGNAL_GROUP_EXIT/group_exec_task is set

2023-05-17 Thread Eric W. Biederman
Long story short. In the patch below the first hunk is a noop. The code you are bypassing was added to ensure that process termination (aka SIGKILL) is processed before any other signals. Other than signal processing order there are not any substantive differences in the two code paths. With

Re: [RFC PATCH 1/8] signal: Dequeue SIGKILL even if SIGNAL_GROUP_EXIT/group_exec_task is set

2023-05-17 Thread Eric W. Biederman
Mike Christie writes: > This has us deqeue SIGKILL even if SIGNAL_GROUP_EXIT/group_exec_task is > set when we are dealing with PF_USER_WORKER tasks. > When a vhost_task gets a SIGKILL, we could have outstanding IO in flight. > We can easily stop new work/IO from being queued to the vhost_task, b

Re: [RFC PATCH 8/8] fork/vhost_task: remove no_files

2023-05-17 Thread Mike Christie
On 5/17/23 7:09 PM, Mike Christie wrote: > + CLONE_THREAD | CLONE_FILES, CLONE_SIGHAND, Sorry. I tried to throw this one in last second so we could see that we can also see that we can now use CLONE_FILES like io_uring. It will of course not compile. _

Re: [RFC PATCH 2/8] vhost/vhost_task: Hook vhost layer into signal handler

2023-05-17 Thread Mike Christie
On 5/17/23 7:16 PM, Linus Torvalds wrote: > On Wed, May 17, 2023 at 5:09 PM Mike Christie > wrote: >> >> + __set_current_state(TASK_RUNNING); >> + rc = get_signal(&ksig); >> + set_current_state(TASK_INTERRUPTIBLE); >> + return rc; > > The games with current_state seem nons

Re: Re: Re: [PATCH v2 1/2] virtio: abstract virtqueue related methods

2023-05-17 Thread zhenwei pi via Virtualization
On 5/17/23 18:39, Michael S. Tsirkin wrote: On Wed, May 17, 2023 at 04:35:55PM +0800, zhenwei pi wrote: On 5/17/23 15:46, Christoph Hellwig wrote: On Wed, May 17, 2023 at 03:43:03PM +0800, zhenwei pi wrote: I have a plan to introduce 'Virtio Over Fabrics'(TCP&RDMA) as Virtio transport, as me

Re: [RFC PATCH 2/8] vhost/vhost_task: Hook vhost layer into signal handler

2023-05-17 Thread Linus Torvalds
On Wed, May 17, 2023 at 5:09 PM Mike Christie wrote: > > + __set_current_state(TASK_RUNNING); > + rc = get_signal(&ksig); > + set_current_state(TASK_INTERRUPTIBLE); > + return rc; The games with current_state seem nonsensical. What are they all about? get_signal() shouldn

[RFC PATCH 8/8] fork/vhost_task: remove no_files

2023-05-17 Thread Mike Christie
The vhost_task can now support the worker being freed from under the device when we get a SIGKILL or the process exits without closing devices. We no longer need no_files so this removes it. Signed-off-by: Mike Christie --- include/linux/sched/task.h | 1 - kernel/fork.c | 10 ++---

[RFC PATCH 6/8] vhost-scsi: Add callback to stop and wait on works

2023-05-17 Thread Mike Christie
This moves the scsi code we use to stop new works from being queued and wait on running works to a helper which is used by the vhost layer when the vhost_task is being killed by a SIGKILL. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 23 +++ 1 file changed, 15 inse

[RFC PATCH 7/8] vhost-net: Add callback to stop and wait on works

2023-05-17 Thread Mike Christie
This moves the net code we use to stop new works from being queued and wait on running works to a helper which is used by the vhost layer when the vhost_task is being killed by a SIGKILL. Signed-off-by: Mike Christie --- drivers/vhost/net.c | 13 ++--- 1 file changed, 10 insertions(+), 3

[RFC PATCH 5/8] vhost: Add callback that stops new work and waits on running ones

2023-05-17 Thread Mike Christie
When the vhost_task gets a SIGKILL we want to stop new work from being queued and also wait for and handle completions for running work. For the latter, we still need to use the vhost_task to handle the completing work so we can't just exit right away. But, this has us kick off the stopping and flu

[RFC PATCH 4/8] vhost-net: Move vhost_net_open

2023-05-17 Thread Mike Christie
This moves vhost_net_open so in the next patches we can pass vhost_dev_init a new helper which will use the stop/flush functions. There is no functionality changes in this patch. Signed-off-by: Mike Christie --- drivers/vhost/net.c | 134 ++-- 1 file chang

[RFC PATCH 1/8] signal: Dequeue SIGKILL even if SIGNAL_GROUP_EXIT/group_exec_task is set

2023-05-17 Thread Mike Christie
This has us deqeue SIGKILL even if SIGNAL_GROUP_EXIT/group_exec_task is set when we are dealing with PF_USER_WORKER tasks. When a vhost_task gets a SIGKILL, we could have outstanding IO in flight. We can easily stop new work/IO from being queued to the vhost_task, but for IO that's already been se

[RFC PATCH 2/8] vhost/vhost_task: Hook vhost layer into signal handler

2023-05-17 Thread Mike Christie
This patch has vhost use get_signal to handle freezing and sort of handle signals. By the latter I mean that when we get SIGKILL, our parent will exit and call our file_operatons release function. That will then stop new work from breing queued and wait for the vhost_task to handle completions for

[RFC PATCH 3/8] fork/vhost_task: Switch to CLONE_THREAD and CLONE_SIGHAND

2023-05-17 Thread Mike Christie
This is a modified version of Linus's patch which has vhost_task use CLONE_THREAD and CLONE_SIGHAND and allow SIGKILL and SIGSTOP. I renamed the ignore_signals to block_signals based on Linus's comment where it aligns with what we are doing with the siginitsetinv p->blocked use and no longer calli

[RFC PATCH 0/8] vhost_tasks: Use CLONE_THREAD/SIGHAND

2023-05-17 Thread Mike Christie
This patch allows the vhost and vhost_task code to use CLONE_THREAD, CLONE_SIGHAND and CLONE_FILES. It's a RFC because I didn't do all the normal testing, haven't coverted vsock and vdpa, and I know you guys will not like the first patch. However, I think it better shows what we need from the signa

Re: [PATCH v11 8/8] vhost: use vhost_tasks for worker threads

2023-05-17 Thread Mike Christie
On 5/17/23 12:09 PM, Oleg Nesterov wrote: > IIUC, Mike is going to send the next version? So I think we can delay > the further discussions until then. Yeah, I'm working on a version that supports signals so it will be easier to discuss with the vhost devs and you, Christian and Eric.

Re: [PATCH v11 8/8] vhost: use vhost_tasks for worker threads

2023-05-17 Thread Oleg Nesterov
On 05/16, Eric W. Biederman wrote: > > Oleg Nesterov writes: > > >> There is this bit in complete_signal when SIGKILL is delivered to any > >> thread in the process. > >> > >>t = p; > >>do { > >>task_clear_jobctl_pending(t, > >>

Re: [PATCH 0/7] riscv: Memory Hot(Un)Plug support

2023-05-17 Thread David Hildenbrand
On 12.05.23 16:57, Björn Töpel wrote: From: Björn Töpel Memory Hot(Un)Plug support for the RISC-V port == Introduction To quote "Documentation/admin-guide/mm/memory-hotplug.rst": "Memory hot(un)plug allows for increasing and decreasing

Re: Re: [PATCH v2 1/2] virtio: abstract virtqueue related methods

2023-05-17 Thread Michael S. Tsirkin
On Wed, May 17, 2023 at 04:35:55PM +0800, zhenwei pi wrote: > > > On 5/17/23 15:46, Christoph Hellwig wrote: > > On Wed, May 17, 2023 at 03:43:03PM +0800, zhenwei pi wrote: > > > I have a plan to introduce 'Virtio Over Fabrics'(TCP&RDMA) as Virtio > > > transport, as mentioned in cover letter of

Re: Re: Re: Re: [PATCH v2 0/2] virtio: abstract virtqueue related methods

2023-05-17 Thread Michael S. Tsirkin
On Wed, May 17, 2023 at 02:21:09PM +0800, zhenwei pi wrote: > On 5/17/23 14:10, Michael S. Tsirkin wrote: > > On Wed, May 17, 2023 at 12:58:10PM +0800, zhenwei pi wrote: > > > On 5/17/23 11:57, Michael S. Tsirkin wrote: > > > > On Wed, May 17, 2023 at 11:51:03AM +0800, zhenwei pi wrote: > > > > >

Re: Re: [PATCH v2 1/2] virtio: abstract virtqueue related methods

2023-05-17 Thread zhenwei pi via Virtualization
On 5/17/23 15:46, Christoph Hellwig wrote: On Wed, May 17, 2023 at 03:43:03PM +0800, zhenwei pi wrote: I have a plan to introduce 'Virtio Over Fabrics'(TCP&RDMA) as Virtio transport, as mentioned in cover letter of this series: 3 weeks ago, I posted a proposal 'Virtio Over Fabrics': https://l

Re: [PATCH v2 1/2] virtio: abstract virtqueue related methods

2023-05-17 Thread Christoph Hellwig
On Wed, May 17, 2023 at 03:43:03PM +0800, zhenwei pi wrote: > I have a plan to introduce 'Virtio Over Fabrics'(TCP&RDMA) as Virtio > transport, as mentioned in cover letter of this series: > 3 weeks ago, I posted a proposal 'Virtio Over Fabrics': > https://lists.oasis-open.org/archives/virtio-comme

Re: Re: [PATCH v2 1/2] virtio: abstract virtqueue related methods

2023-05-17 Thread zhenwei pi via Virtualization
On 5/17/23 15:39, Christoph Hellwig wrote: On Wed, May 17, 2023 at 10:54:23AM +0800, zhenwei pi wrote: All the vring based virtqueue methods could be abstratct in theory, MST suggested that add/get bufs and kick functions are quite perfmance sensitive, so export these functions from virtio_ri

Re: [PATCH v2 1/2] virtio: abstract virtqueue related methods

2023-05-17 Thread Christoph Hellwig
On Wed, May 17, 2023 at 10:54:23AM +0800, zhenwei pi wrote: > All the vring based virtqueue methods could be abstratct in theory, > MST suggested that add/get bufs and kick functions are quite perfmance > sensitive, so export these functions from virtio_ring.ko, drivers > still call them in a fast

Re: [PATCH v2 2/3] vhost: support PACKED when setting-getting vring_base

2023-05-17 Thread Stefano Garzarella
On Wed, May 17, 2023 at 7:26 AM Jason Wang wrote: > > On Wed, May 17, 2023 at 2:26 AM Shannon Nelson wrote: > > > > On 5/16/23 12:49 AM, Stefano Garzarella wrote: > > > On Mon, May 15, 2023 at 01:41:12PM -0700, Shannon Nelson wrote: > > >> On 5/9/23 1:46 AM, Stefano Garzarella wrote: > > >>> On M