Re: [PATCH v2] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Anton Ivanov
On 04/07/2024 23:55, Johannes Berg wrote: On Thu, 2024-07-04 at 21:16 +0100, Anton Ivanov wrote: - qi->queue_depth = 0; + wmb(); /* Ensure that RX processing elsewhere sees the changes */ + atomic_set(&qi->queue_depth, 0); } I don't understand this. prep_queue_for_rx() is

Re: [PATCH v2] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Johannes Berg
On Thu, 2024-07-04 at 21:16 +0100, Anton Ivanov wrote: > > > - qi->queue_depth = 0; > > > + wmb(); /* Ensure that RX processing elsewhere sees the changes */ > > > + atomic_set(&qi->queue_depth, 0); > > > } > > I don't understand this. > > > > prep_queue_for_rx() is called by vector_mmsg_rx(),

Re: [PATCH v2] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Anton Ivanov
On 04/07/2024 17:12, Johannes Berg wrote: On Thu, 2024-07-04 at 13:22 +0100, anton.iva...@cambridgegreys.com wrote: @@ -675,11 +657,20 @@ static void prep_queue_for_rx(struct vector_queue *qi) struct vector_private *vp = netdev_priv(qi->dev); struct mmsghdr *mmsg_vector = qi->mms

[PATCH v8 6/7] um: clear all memory in new userspace processes

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg With the change to use execve() we can now safely clear the memory up to STUB_START as rseq will not be trying to use memory in that region. Also, on 64 bit the previous changes should mean that there is no usable memory range above the stub. Make the change and remove the co

[PATCH v8 7/7] um: Add 4 level page table support

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg The larger memory space is useful to support more applications inside UML. One example for this is ASAN instrumentation of userspace applications which requires addresses that would otherwise not be available. Signed-off-by: Benjamin Berg --- v7: - Reword options and fix d

[PATCH v8 2/7] um: use execveat to create userspace MMs

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg Using clone will not undo features that have been enabled by libc. An example of this already happening is rseq, which could cause the kernel to read/write memory of the userspace process. In the future the standard library might also use mseal by default to protect itself, wh

[PATCH v8 3/7] um: Fix stub_start address calculation

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg The calculation was wrong as it only subtracted one and then rounded down for alignment. However, this is incorrect if host_task_size is not already aligned. This probably worked fine because on 64 bit the host_task_size is bigger than returned by os_get_top_address. Signed-

[PATCH v8 5/7] um: Discover host_task_size from envp

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg When loading the UML binary, the host kernel will place the stack at the highest possible address. It will then map the program name and environment variables onto the start of the stack. As such, an easy way to figure out the host_task_size is to use the highest pointer to a

[PATCH v8 4/7] um: Limit TASK_SIZE to the addressable range

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg We may have a TASK_SIZE from the host that is bigger than UML is able to address with a three-level pagetable. Guard against that by clipping the maximum TASK_SIZE to the maximum addressable area. Signed-off-by: Benjamin Berg --- v7: Fix integer overflow on 32 bit with 3-l

[PATCH v8 1/7] um: Add generic stub_syscall1 function

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg The 64bit version did not have a stub_syscall1 function yet. Add it as it will be useful to implement a static binary for stub loading. Signed-off-by: Benjamin Berg --- arch/x86/um/shared/sysdep/stub_64.h | 11 +++ 1 file changed, 11 insertions(+) diff --git a/arch

[PATCH v8 0/7] Increased address space for 64 bit

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg The new version of the patchset uses execveat on a memfd instead of cloning twice to disable rseq. This should be much more robust going forward as it will also avoid issues with other new features like mseal. This patchset fixes a few bugs, adds a new method of discovering t

Re: [PATCH v7 2/7] um: use execveat to create userspace MMs

2024-07-04 Thread Benjamin Berg
On Thu, 2024-07-04 at 18:49 +0200, Johannes Berg wrote: > On Thu, 2024-07-04 at 18:27 +0200, Benjamin Berg wrote: > > > > + /* set a nice name */ > > + stub_syscall2(__NR_prctl, PR_SET_NAME, (unsigned long)"uml-userspace"); > > Is that even needed when you're passing it as argv[0] in execve()

Re: [PATCH v7 6/7] um: clear all memory in new userspace processes

2024-07-04 Thread Johannes Berg
On Thu, 2024-07-04 at 18:27 +0200, Benjamin Berg wrote: > From: Benjamin Berg > > With the change to use execve() we can now safely clear the memory up to > STUB_START as rseq will not be trying to use memory in that region. Also, > on 64 bit the previous changes should mean that there is no usab

Re: [PATCH v7 2/7] um: use execveat to create userspace MMs

2024-07-04 Thread Johannes Berg
On Thu, 2024-07-04 at 18:27 +0200, Benjamin Berg wrote: > > + /* set a nice name */ > + stub_syscall2(__NR_prctl, PR_SET_NAME, (unsigned long)"uml-userspace"); Is that even needed when you're passing it as argv[0] in execve()? But whatever, it's fine, just wondering. > + /* setup sig

[PATCH v7 6/7] um: clear all memory in new userspace processes

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg With the change to use execve() we can now safely clear the memory up to STUB_START as rseq will not be trying to use memory in that region. Also, on 64 bit the previous changes should mean that there is no usable memory range above the stub. Make the change and remove the co

[PATCH v7 7/7] um: Add 4 level page table support

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg The larger memory space is useful to support more applications inside UML. One example for this is ASAN instrumentation of userspace applications which requires addresses that would otherwise not be available. Signed-off-by: Benjamin Berg --- v7: - Reword options and fix d

[PATCH v7 5/7] um: Discover host_task_size from envp

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg When loading the UML binary, the host kernel will place the stack at the highest possible address. It will then map the program name and environment variables onto the start of the stack. As such, an easy way to figure out the host_task_size is to use the highest pointer to a

[PATCH v7 3/7] um: Fix stub_start address calculation

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg The calculation was wrong as it only subtracted one and then rounded down for alignment. However, this is incorrect if host_task_size is not already aligned. This probably worked fine because on 64 bit the host_task_size is bigger than returned by os_get_top_address. Signed-

[PATCH v7 4/7] um: Limit TASK_SIZE to the addressable range

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg We may have a TASK_SIZE from the host that is bigger than UML is able to address with a three-level pagetable. Guard against that by clipping the maximum TASK_SIZE to the maximum addressable area. Signed-off-by: Benjamin Berg --- v7: Fix integer overflow on 32 bit with 3-l

[PATCH v7 0/7] Increased address space for 64 bit

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg The new version of the patchset uses execveat on a memfd instead of cloning twice to disable rseq. This should be much more robust going forward as it will also avoid issues with other new features like mseal. This patchset fixes a few bugs, adds a new method of discovering t

[PATCH v7 2/7] um: use execveat to create userspace MMs

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg Using clone will not undo features that have been enabled by libc. An example of this already happening is rseq, which could cause the kernel to read/write memory of the userspace process. In the future the standard library might also use mseal by default to protect itself, wh

[PATCH v7 1/7] um: Add generic stub_syscall1 function

2024-07-04 Thread Benjamin Berg
From: Benjamin Berg The 64bit version did not have a stub_syscall1 function yet. Add it as it will be useful to implement a static binary for stub loading. Signed-off-by: Benjamin Berg --- arch/x86/um/shared/sysdep/stub_64.h | 11 +++ 1 file changed, 11 insertions(+) diff --git a/arch

Re: [PATCH v2] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Johannes Berg
On Thu, 2024-07-04 at 13:22 +0100, anton.iva...@cambridgegreys.com wrote: > > @@ -675,11 +657,20 @@ static void prep_queue_for_rx(struct vector_queue *qi) > struct vector_private *vp = netdev_priv(qi->dev); > struct mmsghdr *mmsg_vector = qi->mmsg_vector; > void **skbuff_vector =

[GIT PULL] Kselftest fixes for v6.10

2024-07-04 Thread Mickaël Salaün
Hi Linus, This PR fixes a few kselftests [1]. This has been in linux-next for a week and rebased to add Mark Brown's Tested-by. The race condition found while writing this fix is not new and seems specific to UML's hostfs (I also tested against ext4 and btrfs without being able to trigger this i

[PATCH v2] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread anton . ivanov
From: Anton Ivanov UML vector drivers use ring buffer structures which map preallocated skbs onto mmsg vectors for use with sendmmsg and recvmmsg. They are designed around a single consumer, single producer pattern allowing simultaneous enqueue and dequeue. Lock debugging with preemption showed

Re: [PATCH 14/26] block: move the nonrot flag to queue_limits

2024-07-04 Thread Simon Fernandez
Hi folks, how can I unsubscribe from this group.? Thanks in advance. S > On 24 Jun 2024, at 18:08, Keith Busch wrote: > > On Mon, Jun 17, 2024 at 08:04:41AM +0200, Christoph Hellwig wrote: >> -#define blk_queue_nonrot(q) test_bit(QUEUE_FLAG_NONROT, &(q)->queue_flags) >> +#define blk_queue_nonrot

Re: [PATCH] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Anton Ivanov
On 04/07/2024 10:55, Johannes Berg wrote: On Thu, 2024-07-04 at 10:52 +0100, Anton Ivanov wrote: There is an extra issue there - stats. I need to double-check the locking when they are being fetched. Same story - these need atomics. Otherwise we can potentially get the same ABBA lock issue

[PATCH] um: remove variable stack array in os_rcv_fd_msg()

2024-07-04 Thread Johannes Berg
From: Johannes Berg When generalizing this, I was in the mindset of this being "userspace" code, but even there we should not use variable arrays as the kernel is moving away from allowing that. Simply reserve (but not use) enough space for the maximum two descriptors we might need now, and retu

Re: [PATCH] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Johannes Berg
On Thu, 2024-07-04 at 11:55 +0200, Johannes Berg wrote: > On Thu, 2024-07-04 at 10:52 +0100, Anton Ivanov wrote: > > > > > > There is an extra issue there - stats. I need to double-check the locking > > > when > > > they are being fetched. > > > > Same story - these need atomics. Otherwise we ca

Re: [PATCH] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Johannes Berg
On Thu, 2024-07-04 at 10:52 +0100, Anton Ivanov wrote: > > > > There is an extra issue there - stats. I need to double-check the locking > > when > > they are being fetched. > > Same story - these need atomics. Otherwise we can potentially get the same > ABBA > lock issue when they are being fe

Re: [PATCH] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Johannes Berg
On Thu, 2024-07-04 at 10:45 +0100, Anton Ivanov wrote: > > (it probably also never even executes twice unless you actually have SMP > > or preemption?) > > It does. If half of the vector is at the end of the array which is used to > imitate a ring buffer and the other half is at the beginning. Qu

Re: [PATCH] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Anton Ivanov
On 04/07/2024 10:45, Anton Ivanov wrote: On 04/07/2024 10:17, Johannes Berg wrote: Hi Anton, Thanks for taking a look! Also thanks for the other explanation. On Thu, 2024-07-04 at 08:21 +0100, anton.iva...@cambridgegreys.com wrote: From: Anton Ivanov UML vector drivers use ring buffer

Re: [PATCH] um: register power-off handler

2024-07-04 Thread Anton Ivanov
On 04/07/2024 10:21, Johannes Berg wrote: On Thu, 2024-07-04 at 07:58 +0100, Anton Ivanov wrote: +__initcall(register_power_off); Anton Ivanov Heh, I thought I saw an Acked-by from you but then patchwork didn't seem to pick it up, I guess I'll treat it as one anyway? :) Yes. Sorry. Too hi

Re: [PATCH] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Anton Ivanov
On 04/07/2024 10:17, Johannes Berg wrote: Hi Anton, Thanks for taking a look! Also thanks for the other explanation. On Thu, 2024-07-04 at 08:21 +0100, anton.iva...@cambridgegreys.com wrote: From: Anton Ivanov UML vector drivers use ring buffer structures which map preallocated skbs onto

Re: [PATCH] um: register power-off handler

2024-07-04 Thread Johannes Berg
On Thu, 2024-07-04 at 07:58 +0100, Anton Ivanov wrote: > > > +__initcall(register_power_off); > > Anton Ivanov Heh, I thought I saw an Acked-by from you but then patchwork didn't seem to pick it up, I guess I'll treat it as one anyway? :) johannes

Re: [PATCH] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread Johannes Berg
Hi Anton, Thanks for taking a look! Also thanks for the other explanation. On Thu, 2024-07-04 at 08:21 +0100, anton.iva...@cambridgegreys.com wrote: > From: Anton Ivanov > > UML vector drivers use ring buffer structures which map > preallocated skbs onto mmsg vectors for use with sendmmsg > and

[PATCH] um: vector: Replace locks guarding queue depth with atomics

2024-07-04 Thread anton . ivanov
From: Anton Ivanov UML vector drivers use ring buffer structures which map preallocated skbs onto mmsg vectors for use with sendmmsg and recvmmsg. They are designed around a single consumer, single producer pattern allowing simultaneous enqueue and dequeue. Lock debugging with preemption showed

Re: [PATCH] um: vector: always reset vp->opened

2024-07-04 Thread Anton Ivanov
On 03/07/2024 17:46, Johannes Berg wrote: From: Johannes Berg If open fails, we have already set vp->opened, but it's not reset so that any further attempts will just return -ENXIO. Reset vp->opened even if close has nothing to do. This helps e.g. with slirp4netns handling only a single con

Re: [PATCH v2] um: vector: remove vp->lock

2024-07-04 Thread Anton Ivanov
On 03/07/2024 17:46, Johannes Berg wrote: From: Johannes Berg This lock is useless, all the places that are using it for some locking will already hold the RTNL. Just remove it. Signed-off-by: Johannes Berg --- v2: remove unused flags variables --- arch/um/drivers/vector_kern.c | 16 +---