[PATCH 0/2] sched/fair: remote load updates for idle CPUs

into the future. This field is used to determine the need for triggering the newly-added NOHZ kick. So if such newly-idle balances are happening often enough, no additional CPU wakeups are required to keep all the CPUs' loads updated. Brendan Jackman (1): sched/fair: Update blocked

[PATCH 1/2] sched: force update of blocked load of idle cpus

use PELT half life] [Moved update_blocked_averges call outside rebalance_domains to simplify code] Signed-off-by: Brendan Jackman --- kernel/sched/fair.c | 71 +--- kernel/sched/sched.h | 1 + 2 files changed, 63 insertions(+), 9 deletions(-) diff

[PATCH 2/2] sched/fair: Update blocked load from newly idle balance

out taking the rq lock. Cc: Dietmar Eggemann Cc: Vincent Guittot Cc: Ingo Molnar Cc: Morten Rasmussen Cc: Peter Zijlstra Signed-off-by: Brendan Jackman --- kernel/sched/core.c | 1 + kernel/sched/fair.c | 41 +++-- kernel/sched/sched.h | 1 + 3 files c

Re: [PATCH RFC 5/5] sched/fair: remove impossible condition from find_idlest_group_cpu

o which > 'cpu' belongs to is chosen. So we're always guaranteed to call > find_idlest_group_cpu with a group to which cpu is non-local. This makes one > of > the conditions in find_idlest_group_cpu an impossible one, which we can get > rid > off. > > Cc: Ingo Moln

[PATCH bpf-next v3] bpf: Propagate stack bounds to registers in atomics w/ BPF_FETCH

abend Signed-off-by: Brendan Jackman --- Difference from v2->v3 [1]: * Fixed missing ENABLE_ATOMICS_TESTS check. Difference from v1->v2: * Reworked commit message to clarify this only affects stack memory * Added the Suggested-by * Added a C-based test. [1]: https://lore.kernel.o

[PATCH bpf-next v2] bpf: Propagate memory bounds to registers in atomics w/ BPF_FETCH

stead propagate bounds smartly. A simple test is added with an infinite loop that can only be proved unreachable if this propagation is present. This is implemented both with C and directly in test_verifier using assembly. Suggested-by: John Fastabend Signed-off-by: Brendan Jackman --- Difference fr

Re: [PATCH bpf-next v2] bpf: Propagate memory bounds to registers in atomics w/ BPF_FETCH

On Tue, 2 Feb 2021 at 00:27, Alexei Starovoitov wrote: > > On Mon, Feb 1, 2021 at 7:00 AM Brendan Jackman wrote: > > + > > +SEC("fentry/bpf_fentry_test1") > > +int BPF_PROG(sub, int x) > > +{ > > + int a = 0; > > + int b

[RFC] security: replace indirect calls with static calls

: James Morris Cc: p...@google.com Cc: ja...@google.com Cc: pet...@infradead.org Cc: rafael.j.wyso...@intel.com Cc: keesc...@chromium.org Cc: thgar...@chromium.org Cc: kpsi...@google.com Cc: paul.renauld.e...@gmail.com Signed-off-by: Paul Renauld Signed-off-by: KP Singh

[PATCH bpf-next] bpf: Clarify return value of probe str helpers

Signed-off-by: Brendan Jackman --- include/uapi/linux/bpf.h | 10 +- tools/include/uapi/linux/bpf.h | 10 +- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 77d7c1bb2923..a1ad32456f89 100644 --- a/in

[PATCH bpf-next] bpf: Fix a verifier message for alloc size helper arg

The error message here is misleading, the argument will be rejected unless it is a known constant. Signed-off-by: Brendan Jackman --- kernel/bpf/verifier.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 17270b8404f1

[PATCH bpf-next] libbpf: Expose libbpf ringbufer epoll_fd

This allows the user to do their own manual polling in more complicated setups. Signed-off-by: Brendan Jackman --- tools/lib/bpf/libbpf.h | 1 + tools/lib/bpf/ringbuf.c | 6 ++ 2 files changed, 7 insertions(+) diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h index 6909ee81113a

Re: [PATCH bpf-next v3 12/14] bpf: Pull tools/build/feature biz into selftests Makefile

On Fri, Dec 04, 2020 at 11:00:24AM -0800, Andrii Nakryiko wrote: > On Fri, Dec 4, 2020 at 1:41 AM Brendan Jackman wrote: > > > > On Thu, Dec 03, 2020 at 01:01:27PM -0800, Andrii Nakryiko wrote: > > > On Thu, Dec 3, 2020 at 8:07 AM Brendan Jackman > > > wrote:

Re: [PATCH bpf-next v3 10/14] bpf: Add bitwise atomic instructions

On Fri, Dec 04, 2020 at 07:21:22AM -0800, Yonghong Song wrote: > > > On 12/4/20 1:36 AM, Brendan Jackman wrote: > > On Thu, Dec 03, 2020 at 10:42:19PM -0800, Yonghong Song wrote: > > > > > > > > > On 12/3/20 8:02 AM, Brendan Jackman

Re: [PATCH bpf-next v3 13/14] bpf: Add tests for new BPF atomic operations

On Fri, Dec 04, 2020 at 11:49:22AM -0800, Andrii Nakryiko wrote: > On Fri, Dec 4, 2020 at 7:29 AM Yonghong Song wrote: > > > > > > > > On 12/4/20 1:45 AM, Brendan Jackman wrote: > > > On Thu, Dec 03, 2020 at 11:06:31PM -0800, Yonghong Song wrote: > > &g

[PATCH bpf-next v4 00/11] Atomics for eBPF

v3: https://lore.kernel.org/bpf/x8kn7na7bjc7a...@google.com/ [2] Visualisation of eBPF opcode space: https://gist.github.com/bjackman/00fdad2d5dfff601c1918bc29b16e778 Brendan Jackman (11): bpf: x86: Factor out emission of ModR/M for *(reg + off) bpf: x86: Factor out emission of REX byte

[PATCH bpf-next v4 01/11] bpf: x86: Factor out emission of ModR/M for *(reg + off)

The case for JITing atomics is about to get more complicated. Let's factor out some common code to make the review and result more readable. NB the atomics code doesn't yet use the new helper - a subsequent patch will add its use as a side-effect of other changes. Signed-off-by: Brend

[PATCH bpf-next v4 05/11] bpf: Move BPF_STX reserved field check into BPF_STX verifier code

I can't find a reason why this code is in resolve_pseudo_ldimm64; since I'll be modifying it in a subsequent commit, tidy it up. Signed-off-by: Brendan Jackman --- kernel/bpf/verifier.c | 13 ++--- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/kernel/bpf/ve

[PATCH bpf-next v4 08/11] bpf: Pull out a macro for interpreting atomic ALU operations

switch case means that we need an extra conditional branch to differentiate them) in favour of compact and (relatively!) simple C code. Acked-by: Yonghong Song Signed-off-by: Brendan Jackman --- kernel/bpf/core.c | 80 +++ 1 file changed, 39 insertions

[PATCH bpf-next v4 02/11] bpf: x86: Factor out emission of REX byte

The JIT case for encoding atomic ops is about to get more complicated. In order to make the review & resulting code easier, let's factor out some shared helpers. Signed-off-by: Brendan Jackman --- arch/x86/net/bpf_jit_comp.c | 39 ++--- 1 file ch

[PATCH bpf-next v4 03/11] bpf: x86: Factor out a lookup table for some ALU opcodes

A later commit will need to lookup a subset of these opcodes. To avoid duplicating code, pull out a table. The shift opcodes won't be needed by that later commit, but they're already duplicated, so fold them into the table anyway. Signed-off-by: Brendan Jackman --- arch/x86/net/bpf_

[PATCH bpf-next v4 07/11] bpf: Add instructions for atomic_[cmp]xchg

rn-old-value is easier to JIT. Signed-off-by: Brendan Jackman --- arch/x86/net/bpf_jit_comp.c| 8 include/linux/filter.h | 22 ++ include/uapi/linux/bpf.h | 4 +++- kernel/bpf/core.c | 20 kernel/bpf/disasm.c

[PATCH bpf-next v4 06/11] bpf: Add BPF_FETCH field / create atomic_fetch_add instruction

The BPF_FETCH field can be set in bpf_insn.imm, for BPF_ATOMIC instructions, in order to have the previous value of the atomically-modified memory location loaded into the src register after an atomic op is carried out. Suggested-by: Yonghong Song Signed-off-by: Brendan Jackman --- arch/x86

[PATCH bpf-next v4 10/11] bpf: Add tests for new BPF atomic operations

ect's data section, which tells the userspace object whether to skip the atomics test. Signed-off-by: Brendan Jackman --- tools/testing/selftests/bpf/Makefile | 10 + .../selftests/bpf/prog_tests/atomics.c| 246 ++ tools/testing/selftests/bpf/progs/atomics

[PATCH bpf-next v4 11/11] bpf: Document new atomic instructions

Document new atomic instructions. Signed-off-by: Brendan Jackman --- Documentation/networking/filter.rst | 26 ++ 1 file changed, 26 insertions(+) diff --git a/Documentation/networking/filter.rst b/Documentation/networking/filter.rst index 1583d59d806d..26d508a5e038

[PATCH bpf-next v4 09/11] bpf: Add bitwise atomic instructions

ectly support the fetch_ version these operations, so we need to generate a CMPXCHG loop in the JIT. This requires the use of two temporary registers, IIUC it's safe to use BPF_REG_AX and x86's AUX_REG for this purpose. Signed-off-by: Brendan Jackman --- arch/x86/net/bpf_j

[PATCH bpf-next v4 04/11] bpf: Rename BPF_XADD and prepare to encode other atomics in .imm

possible (doesn't break existing valid BPF progs) because the immediate field is currently reserved MBZ and BPF_ADD is zero. All uses are removed from the tree but the BPF_XADD definition is kept around to avoid breaking builds for people including kernel headers. Signed-off-by: Brendan Ja

Re: [PATCH bpf-next v3 10/14] bpf: Add bitwise atomic instructions

On Mon, Dec 07, 2020 at 07:58:09AM -0800, Yonghong Song wrote: > > > On 12/7/20 3:28 AM, Brendan Jackman wrote: > > On Fri, Dec 04, 2020 at 07:21:22AM -0800, Yonghong Song wrote: > > > > > > > > > On 12/4/20 1:36 AM, Brendan Jackman wrote: > &

Re: [PATCH bpf-next v4 04/11] bpf: Rename BPF_XADD and prepare to encode other atomics in .imm

Hi John, thanks a lot for the reviews! On Mon, Dec 07, 2020 at 01:56:53PM -0800, John Fastabend wrote: > Brendan Jackman wrote: > > A subsequent patch will add additional atomic operations. These new > > operations will use the same opcode field as the existing XADD, with &g

Re: [PATCH bpf-next v4 06/11] bpf: Add BPF_FETCH field / create atomic_fetch_add instruction

On Mon, Dec 07, 2020 at 05:41:05PM -0800, Yonghong Song wrote: > > > On 12/7/20 8:07 AM, Brendan Jackman wrote: > > The BPF_FETCH field can be set in bpf_insn.imm, for BPF_ATOMIC > > instructions, in order to have the previous value of the > > atomically-modified memo

Re: [PATCH bpf-next v4 06/11] bpf: Add BPF_FETCH field / create atomic_fetch_add instruction

On Mon, Dec 07, 2020 at 09:31:40PM -0800, John Fastabend wrote: > Brendan Jackman wrote: > > The BPF_FETCH field can be set in bpf_insn.imm, for BPF_ATOMIC > > instructions, in order to have the previous value of the > > atomically-modified memory location loaded into the src

Re: [PATCH bpf-next v4 10/11] bpf: Add tests for new BPF atomic operations

On Mon, Dec 07, 2020 at 07:18:57PM -0800, Yonghong Song wrote: > > > On 12/7/20 8:07 AM, Brendan Jackman wrote: > > The prog_test that's added depends on Clang/LLVM features added by > > Yonghong in commit 286daafd6512 (was https://reviews.llvm.org/D72184 ). > &g

Re: [PATCH bpf-next v4 10/11] bpf: Add tests for new BPF atomic operations

On Tue, Dec 08, 2020 at 08:38:04AM -0800, Yonghong Song wrote: > > > On 12/8/20 4:41 AM, Brendan Jackman wrote: > > On Mon, Dec 07, 2020 at 07:18:57PM -0800, Yonghong Song wrote: > > > > > > > > > On 12/7/20 8:07 AM, Brendan Jackman wrote: > >

Re: [PATCH bpf-next v3 12/14] bpf: Pull tools/build/feature biz into selftests Makefile

On Mon, Dec 07, 2020 at 06:19:12PM -0800, Andrii Nakryiko wrote: > On Mon, Dec 7, 2020 at 3:00 AM Brendan Jackman wrote: > > > > On Fri, Dec 04, 2020 at 11:00:24AM -0800, Andrii Nakryiko wrote: > > > On Fri, Dec 4, 2020 at 1:41 AM Brendan Jackman > > > wrot

Re: [PATCH bpf-next] bpf: Fix a verifier message for alloc size helper arg

Sorry, duplicate - seems I had my mail client in HTML mode the first time around. On Tue, 12 Jan 2021 at 14:14, KP Singh wrote: > > On Tue, Jan 12, 2021 at 1:39 PM Brendan Jackman wrote: > > > > The error message here is misleading, the argument will be rejected > &

[PATCH bpf-next v6 00/11] Atomics for eBPF

ad2d5dfff601c1918bc29b16e778 [3] Comment from John about propagating bounds in verifier: https://lore.kernel.org/bpf/5fcf0fbcc8aa8_9ab320853@john-XPS-13-9370.notmuch/ [4] Mail from Andrii about not supporting old Clang in selftests: https://lore.kernel.org/bpf/CAEf4BzYBddPaEzRUs=jaWSo5kbf=lzd

[PATCH bpf-next v6 01/11] bpf: x86: Factor out emission of ModR/M for *(reg + off)

Fastabend Signed-off-by: Brendan Jackman --- arch/x86/net/bpf_jit_comp.c | 43 + 1 file changed, 25 insertions(+), 18 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 796506dcfc42..30526776fa78 100644 --- a/arch/x86/net/bpf_

[PATCH bpf-next v6 02/11] bpf: x86: Factor out emission of REX byte

The JIT case for encoding atomic ops is about to get more complicated. In order to make the review & resulting code easier, let's factor out some shared helpers. Signed-off-by: Brendan Jackman Acked-by: John Fastabend --- arch/x86/net/bpf_jit_co

[PATCH bpf-next v6 03/11] bpf: x86: Factor out a lookup table for some ALU opcodes

A later commit will need to lookup a subset of these opcodes. To avoid duplicating code, pull out a table. The shift opcodes won't be needed by that later commit, but they're already duplicated, so fold them into the table anyway. Signed-off-by: Brendan Jackman Acked-by: John

[PATCH bpf-next v6 07/11] bpf: Add instructions for atomic_[cmp]xchg

old-value is easier to JIT, so that's what we use. Signed-off-by: Brendan Jackman Acked-by: Yonghong Song --- arch/x86/net/bpf_jit_comp.c| 8 include/linux/filter.h | 2 ++ include/uapi/linux/bpf.h | 4 +++- kernel/bpf/core.c | 20 +++

[PATCH bpf-next v6 06/11] bpf: Add BPF_FETCH field / create atomic_fetch_add instruction

The BPF_FETCH field can be set in bpf_insn.imm, for BPF_ATOMIC instructions, in order to have the previous value of the atomically-modified memory location loaded into the src register after an atomic op is carried out. Suggested-by: Yonghong Song Signed-off-by: Brendan Jackman Acked-by: John

[PATCH bpf-next v6 11/11] bpf: Document new atomic instructions

Document new atomic instructions. Acked-by: Yonghong Song Signed-off-by: Brendan Jackman --- Documentation/networking/filter.rst | 31 + 1 file changed, 31 insertions(+) diff --git a/Documentation/networking/filter.rst b/Documentation/networking/filter.rst index

[PATCH bpf-next v6 10/11] bpf: Add tests for new BPF atomic operations

ect's data section, which tells the userspace object whether to skip the atomics test. Acked-by: Yonghong Song Signed-off-by: Brendan Jackman --- tools/testing/selftests/bpf/Makefile | 2 + .../selftests/bpf/prog_tests/atomics.c| 246 ++ tools/testing/sel

[PATCH bpf-next v6 08/11] bpf: Pull out a macro for interpreting atomic ALU operations

switch case means that we need an extra conditional branch to differentiate them) in favour of compact and (relatively!) simple C code. Acked-by: Yonghong Song Signed-off-by: Brendan Jackman --- kernel/bpf/core.c | 80 +++ 1 file changed, 39 insertions

[PATCH bpf-next v6 05/11] bpf: Move BPF_STX reserved field check into BPF_STX verifier code

I can't find a reason why this code is in resolve_pseudo_ldimm64; since I'll be modifying it in a subsequent commit, tidy it up. Signed-off-by: Brendan Jackman Acked-by: Yonghong Song Acked-by: John Fastabend --- kernel/bpf/verifier.c | 13 ++--- 1 file changed, 6 insert

[PATCH bpf-next v6 04/11] bpf: Rename BPF_XADD and prepare to encode other atomics in .imm

possible (doesn't break existing valid BPF progs) because the immediate field is currently reserved MBZ and BPF_ADD is zero. All uses are removed from the tree but the BPF_XADD definition is kept around to avoid breaking builds for people including kernel headers. Signed-off-by: Brendan Ja

[PATCH bpf-next v6 09/11] bpf: Add bitwise atomic instructions

ectly support the fetch_ version these operations, so we need to generate a CMPXCHG loop in the JIT. This requires the use of two temporary registers, IIUC it's safe to use BPF_REG_AX and x86's AUX_REG for this purpose. Signed-off-by: Brendan Jackman Acked-by: Yonghong Song --

Re: [PATCH bpf-next] docs: bpf: Fixup atomics documentation

Thanks for the review :) On Mon, 18 Jan 2021 at 16:33, Jonathan Corbet wrote: > > On Mon, 18 Jan 2021 11:36:43 +0000 > Brendan Jackman wrote: > > > This fixues up the markup to fix a warning, be more consistent with > > use of monospace, and use the correct .rst synta

[PATCH bpf-next v2 0/2] BPF docs fixups

Difference from v1->v2 [1]: * Split into 2 patches * Avoided unnecessary ': ::' in .rst source * Tweaked wording of the -mcpu=v3 bit a little more [1] https://lore.kernel.org/bpf/ca+i-1c1lvkjfqlbyk6siiqhxfy0jcr7ubcamj4jced0a9aw...@mail.gmail.com/T/#t Brendan Jackman (2)

[PATCH bpf-next v2 1/2] docs: bpf: Fixup atomics markup

/CA+i-1C3cEXqxcXfD4sibQfx+dtmmzvOzruhk8J5pAw3g5v=k...@mail.gmail.com/T/#t Signed-off-by: Brendan Jackman --- Documentation/networking/filter.rst | 19 ++- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/Documentation/networking/filter.rst b/Documentation/netwo

[PATCH bpf-next] bpf: Propagate memory bounds to registers in atomics w/ BPF_FETCH

PF_DW, BPF_REG_10, BPF_REG_0, -8), instead of one: BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0), Because BPF_ST_MEM doesn't seem to set the stack slot type to 0 when storing an immediate. Signed-off-by: Brendan Jackman --- kernel/bpf/verifier.c

[PATCH bpf-next v2 2/2] docs: bpf: Clarify -mcpu=v3 requirement for atomic ops

Alexei pointed out [1] that this wording is pretty confusing. Here's an attempt to be more explicit and clear. [1] https://lore.kernel.org/bpf/CAADnVQJVvwoZsE1K+6qRxzF7+6CvZNzygnoBW9tZNWJELk5c=q...@mail.gmail.com/T/#m07264fc18fdc43af02fc1320968afefcc73d96f4 Signed-off-by: Brendan Ja

[PATCH bpf-next] docs: bpf: Fixup atomics documentation

using here. NB this conflicts with Lukas' patch at [1], here where I've added `::` to fix the warning, I also kept the original ':' which appears in the output text. [1] https://lore.kernel.org/bpf/CA+i-1C3cEXqxcXfD4sibQfx+dtmmzvOzruhk8J5pAw3g5v=k...@mail.gmail.com/T

[PATCH bpf-next v7 00/11] Atomics for eBPF

github.com/bjackman/00fdad2d5dfff601c1918bc29b16e778 [3] Comment from John about propagating bounds in verifier: https://lore.kernel.org/bpf/5fcf0fbcc8aa8_9ab320853@john-XPS-13-9370.notmuch/ [4] Mail from Andrii about not supporting old Clang in selftests: https://lore.kernel.org/bpf/CAE

[PATCH bpf-next v7 01/11] bpf: x86: Factor out emission of ModR/M for *(reg + off)

Fastabend Signed-off-by: Brendan Jackman --- arch/x86/net/bpf_jit_comp.c | 43 + 1 file changed, 25 insertions(+), 18 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 796506dcfc42..30526776fa78 100644 --- a/arch/x86/net/bpf_

[PATCH bpf-next v7 02/11] bpf: x86: Factor out emission of REX byte

The JIT case for encoding atomic ops is about to get more complicated. In order to make the review & resulting code easier, let's factor out some shared helpers. Signed-off-by: Brendan Jackman Acked-by: John Fastabend --- arch/x86/net/bpf_jit_co

[PATCH bpf-next v7 07/11] bpf: Add instructions for atomic_[cmp]xchg

old-value is easier to JIT, so that's what we use. Signed-off-by: Brendan Jackman Acked-by: Yonghong Song --- arch/x86/net/bpf_jit_comp.c| 8 include/linux/filter.h | 2 ++ include/uapi/linux/bpf.h | 4 +++- kernel/bpf/core.c | 20 +++

[PATCH bpf-next v7 03/11] bpf: x86: Factor out a lookup table for some ALU opcodes

A later commit will need to lookup a subset of these opcodes. To avoid duplicating code, pull out a table. The shift opcodes won't be needed by that later commit, but they're already duplicated, so fold them into the table anyway. Signed-off-by: Brendan Jackman Acked-by: John

[PATCH bpf-next v7 04/11] bpf: Rename BPF_XADD and prepare to encode other atomics in .imm

possible (doesn't break existing valid BPF progs) because the immediate field is currently reserved MBZ and BPF_ADD is zero. All uses are removed from the tree but the BPF_XADD definition is kept around to avoid breaking builds for people including kernel headers. Signed-off-by: Brendan Ja

[PATCH bpf-next v7 08/11] bpf: Pull out a macro for interpreting atomic ALU operations

switch case means that we need an extra conditional branch to differentiate them) in favour of compact and (relatively!) simple C code. Acked-by: Yonghong Song Signed-off-by: Brendan Jackman --- kernel/bpf/core.c | 80 +++ 1 file changed, 39 insertions

[PATCH bpf-next v7 11/11] bpf: Document new atomic instructions

Document new atomic instructions. Acked-by: Yonghong Song Signed-off-by: Brendan Jackman --- Documentation/networking/filter.rst | 31 + 1 file changed, 31 insertions(+) diff --git a/Documentation/networking/filter.rst b/Documentation/networking/filter.rst index

[PATCH bpf-next v7 05/11] bpf: Move BPF_STX reserved field check into BPF_STX verifier code

I can't find a reason why this code is in resolve_pseudo_ldimm64; since I'll be modifying it in a subsequent commit, tidy it up. Signed-off-by: Brendan Jackman Acked-by: Yonghong Song Acked-by: John Fastabend --- kernel/bpf/verifier.c | 13 ++--- 1 file changed, 6 insert

[PATCH bpf-next v7 10/11] bpf: Add tests for new BPF atomic operations

ect's data section, which tells the userspace object whether to skip the atomics test. Acked-by: Yonghong Song Signed-off-by: Brendan Jackman --- tools/testing/selftests/bpf/Makefile | 2 + .../selftests/bpf/prog_tests/atomics.c| 246 ++ tools/testing/sel

[PATCH bpf-next v7 09/11] bpf: Add bitwise atomic instructions

ectly support the fetch_ version these operations, so we need to generate a CMPXCHG loop in the JIT. This requires the use of two temporary registers, IIUC it's safe to use BPF_REG_AX and x86's AUX_REG for this purpose. Signed-off-by: Brendan Jackman Acked-by: Yonghong Song --

[PATCH bpf-next v7 06/11] bpf: Add BPF_FETCH field / create atomic_fetch_add instruction

The BPF_FETCH field can be set in bpf_insn.imm, for BPF_ATOMIC instructions, in order to have the previous value of the atomically-modified memory location loaded into the src register after an atomic op is carried out. Suggested-by: Yonghong Song Signed-off-by: Brendan Jackman Acked-by: John

Re: [PATCH bpf-next v3 07/14] bpf: Add BPF_FETCH field / create atomic_fetch_add instruction

On Thu, Dec 03, 2020 at 09:27:04PM -0800, Yonghong Song wrote: > On 12/3/20 8:02 AM, Brendan Jackman wrote: [...] > > diff --git a/kernel/bpf/disasm.c b/kernel/bpf/disasm.c > > index 37c8d6e9b4cc..3ee2246a52ef 100644 > > --- a/kernel/bpf/disasm.c > > +++ b/kernel/bpf/dis

Re: [PATCH bpf-next v3 08/14] bpf: Add instructions for atomic_[cmp]xchg

O Thu, Dec 03, 2020 at 09:34:23PM -0800, Yonghong Song wrote: > On 12/3/20 8:02 AM, Brendan Jackman wrote: > > This adds two atomic opcodes, both of which include the BPF_FETCH > > flag. XCHG without the BPF_FETCh flag would naturally encode > > BPF_FETCh => BPF_FETC

Re: [PATCH bpf-next v3 09/14] bpf: Pull out a macro for interpreting atomic ALU operations

On Thu, Dec 03, 2020 at 10:30:18PM -0800, Yonghong Song wrote: > > > On 12/3/20 8:02 AM, Brendan Jackman wrote: > > Since the atomic operations that are added in subsequent commits are > > all isomorphic with BPF_ADD, pull out a macro to avoid the > > interpreter bec

Re: [PATCH bpf-next v3 10/14] bpf: Add bitwise atomic instructions

On Thu, Dec 03, 2020 at 10:42:19PM -0800, Yonghong Song wrote: > > > On 12/3/20 8:02 AM, Brendan Jackman wrote: > > This adds instructions for > > > > atomic[64]_[fetch_]and > > atomic[64]_[fetch_]or > > atomic[64]_[fetch_]xor > > > > All

Re: [PATCH bpf-next v3 12/14] bpf: Pull tools/build/feature biz into selftests Makefile

On Thu, Dec 03, 2020 at 01:01:27PM -0800, Andrii Nakryiko wrote: > On Thu, Dec 3, 2020 at 8:07 AM Brendan Jackman wrote: > > > > This is somewhat cargo-culted from the libbpf build. It will be used > > in a subsequent patch to query for Clang BPF atomics suppo

Re: [PATCH bpf-next v3 13/14] bpf: Add tests for new BPF atomic operations

On Thu, Dec 03, 2020 at 11:06:31PM -0800, Yonghong Song wrote: > On 12/3/20 8:02 AM, Brendan Jackman wrote: [...] > > diff --git a/tools/testing/selftests/bpf/prog_tests/atomics_test.c > > b/tools/testing/selftests/bpf/prog_tests/atomics_test.c > > new file mode 100644 &

Re: [PATCH v2 bpf-next 02/13] bpf: x86: Factor out emission of REX byte

On Sat, Nov 28, 2020 at 05:14:05PM -0800, Alexei Starovoitov wrote: > On Fri, Nov 27, 2020 at 05:57:27PM +0000, Brendan Jackman wrote: > > The JIT case for encoding atomic ops is about to get more > > complicated. In order to make the review & resulting code easier, > &

Re: [PATCH v2 bpf-next 01/13] bpf: x86: Factor out emission of ModR/M for *(reg + off)

On Sat, Nov 28, 2020 at 05:15:52PM -0800, Alexei Starovoitov wrote: > On Fri, Nov 27, 2020 at 05:57:26PM +0000, Brendan Jackman wrote: > > +/* Emit the ModR/M byte for addressing *(r1 + off) and r2 */ > > +static void emit_modrm_dstoff(u8 **pprog, u32 r1, u32 r2, int off) > &g

Re: [PATCH v2 bpf-next 05/13] bpf: Rename BPF_XADD and prepare to encode other atomics in .imm

On Fri, Nov 27, 2020 at 07:43:46PM -0800, Yonghong Song wrote: > > > On 11/27/20 9:57 AM, Brendan Jackman wrote: > > diff --git a/Documentation/networking/filter.rst > > b/Documentation/networking/filter.rst > [...] > > diff --git a/drivers/net/ethernet/netronom

Re: [PATCH v2 bpf-next 07/13] bpf: Add BPF_FETCH field / create atomic_fetch_add instruction

On Fri, Nov 27, 2020 at 08:15:49PM -0800, Yonghong Song wrote: > > > On 11/27/20 9:57 AM, Brendan Jackman wrote: [...] > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > > index e8b41ccdfb90..cd4c03b25573 100644 > > --- a/kernel/bpf/verifier.c >

Re: [PATCH v2 bpf-next 08/13] bpf: Add instructions for atomic_[cmp]xchg

On Fri, Nov 27, 2020 at 09:25:53PM -0800, Yonghong Song wrote: > > > On 11/27/20 9:57 AM, Brendan Jackman wrote: > > This adds two atomic opcodes, both of which include the BPF_FETCH > > flag. XCHG without the BPF_FETCh flag would naturally encode > > BPF_FETCH

Re: [PATCH v2 bpf-next 08/13] bpf: Add instructions for atomic_[cmp]xchg

On Sat, Nov 28, 2020 at 05:27:48PM -0800, Alexei Starovoitov wrote: > On Fri, Nov 27, 2020 at 05:57:33PM +0000, Brendan Jackman wrote: > > > > /* atomic op type fields (stored in immediate) */ > > -#define BPF_FETCH 0x01/* fetch previous value into src reg */ > >

Re: [PATCH v2 bpf-next 10/13] bpf: Add instructions for atomic[64]_[fetch_]sub

On Mon, Nov 30, 2020 at 09:18:09AM -0800, Yonghong Song wrote: > On 11/28/20 5:34 PM, Alexei Starovoitov wrote: > > On Fri, Nov 27, 2020 at 09:35:07PM -0800, Yonghong Song wrote: > > > On 11/27/20 9:57 AM, Brendan Jackman wrote: [...] > > > > +#define BPF_

Re: [PATCH v2 bpf-next 12/13] bpf: Add tests for new BPF atomic operations

On Mon, Nov 30, 2020 at 07:55:02PM -0800, Yonghong Song wrote: > On 11/27/20 9:57 AM, Brendan Jackman wrote: [...] > > diff --git a/tools/testing/selftests/bpf/Makefile > > b/tools/testing/selftests/bpf/Makefile > > index 3d5940cd110d..5eadfd09037d 100644 > > --- a/

Re: [PATCH v2 bpf-next 01/13] bpf: x86: Factor out emission of ModR/M for *(reg + off)

Tue, Dec 01, 2020 at 09:50:00PM -0800, Alexei Starovoitov wrote: > On Tue, Dec 1, 2020 at 4:14 AM Brendan Jackman wrote: > > > > On Sat, Nov 28, 2020 at 05:15:52PM -0800, Alexei Starovoitov wrote: > > > On Fri, Nov 27, 2020 at 05:57:26PM +, Brendan Jackman wrote: >

Re: [PATCH v2 bpf-next 02/13] bpf: x86: Factor out emission of REX byte

On Tue, Dec 01, 2020 at 09:48:36PM -0800, Alexei Starovoitov wrote: > On Tue, Dec 1, 2020 at 4:12 AM Brendan Jackman wrote: > > > > On Sat, Nov 28, 2020 at 05:14:05PM -0800, Alexei Starovoitov wrote: > > > On Fri, Nov 27, 2020 at 05:57:27PM +, Brendan Jackman wrote:

Re: [PATCH v2 bpf-next 10/13] bpf: Add instructions for atomic[64]_[fetch_]sub

On Tue, Dec 01, 2020 at 09:55:22PM -0800, Alexei Starovoitov wrote: > On Tue, Dec 1, 2020 at 4:38 AM Brendan Jackman wrote: > > > > I guess it's also worth remembering other archs might have an atomic > > subtract. > > which one? > arm64 LSE implements atomi

Re: [PATCH v2 bpf-next 12/13] bpf: Add tests for new BPF atomic operations

On Tue, Dec 01, 2020 at 06:22:50PM -0800, Andrii Nakryiko wrote: > On Fri, Nov 27, 2020 at 10:01 AM Brendan Jackman wrote: [...] > > + > > +static void test_xchg(void) > > +{ > > + struct atomics_test *atomics_skel = NULL; > > nit: = NULL is unnecessary [..

[PATCH bpf-next] tools/resolve_btfids: Fix some error messages

Add missing newlines and fix polarity of strerror argument. Signed-off-by: Brendan Jackman --- tools/bpf/resolve_btfids/main.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c index dfa540d8a02d

[PATCH bpf-next] bpf: Fix cold build of test_progs-no_alu32

This object lives inside the trunner output dir, i.e. tools/testing/selftests/bpf/no_alu32/btf_data.o At some point it gets copied into the parent directory during another part of the build, but that doesn't happen when building test_progs-no_alu32 from clean. Signed-off-by: Brendan Ja

[PATCH bpf-next v3 00/14] Atomics for eBPF

. This approach also allows the new eBPF instructions to map directly to single x86 instructions. [1] Previous patchset: https://lore.kernel.org/bpf/20201123173202.1335708-1-jackm...@google.com/ [2] Visualisation of eBPF opcode space: https://gist.github.com/bjackman/00fdad2d5dfff601c1918bc29b

[PATCH bpf-next v3 02/14] bpf: x86: Factor out emission of REX byte

The JIT case for encoding atomic ops is about to get more complicated. In order to make the review & resulting code easier, let's factor out some shared helpers. Signed-off-by: Brendan Jackman Change-Id: I66dbd5ad0bf6f820901fb73d6b2c6a63e00483b1 --- arch/x86/net/bpf_jit_co

[PATCH bpf-next v3 01/14] bpf: x86: Factor out emission of ModR/M for *(reg + off)

The case for JITing atomics is about to get more complicated. Let's factor out some common code to make the review and result more readable. NB the atomics code doesn't yet use the new helper - a subsequent patch will add its use as a side-effect of other changes. Signed-off-by: Brend

[PATCH bpf-next v3 07/14] bpf: Add BPF_FETCH field / create atomic_fetch_add instruction

This value can be set in bpf_insn.imm, for BPF_ATOMIC instructions, in order to have the previous value of the atomically-modified memory location loaded into the src register after an atomic op is carried out. Suggested-by: Yonghong Song Signed-off-by: Brendan Jackman Change-Id

[PATCH bpf-next v3 04/14] bpf: x86: Factor out a lookup table for some ALU opcodes

Signed-off-by: Brendan Jackman --- arch/x86/net/bpf_jit_comp.c | 33 +++-- 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 171ce539f6b9..ee7905051ee9 100644 --- a/arch/x86/net/bpf_jit_comp.c

[PATCH bpf-next v3 08/14] bpf: Add instructions for atomic_[cmp]xchg

rn-old-value is easier to JIT. Signed-off-by: Brendan Jackman Change-Id: I3f19ad867dfd08515eecf72674e6fdefe28424bb --- arch/x86/net/bpf_jit_comp.c| 8 include/linux/filter.h | 20 include/uapi/linux/bpf.h | 4 +++- kernel/bpf/core.c |

[PATCH bpf-next v3 06/14] bpf: Move BPF_STX reserved field check into BPF_STX verifier code

I can't find a reason why this code is in resolve_pseudo_ldimm64; since I'll be modifying it in a subsequent commit, tidy it up. Change-Id: I3410469270f4889a3af67612bd6c2e7979ab4da1 Signed-off-by: Brendan Jackman --- kernel/bpf/verifier.c | 13 ++--- 1 file changed, 6 insert

[PATCH bpf-next v3 03/14] bpf: x86: Factor out function to emit NEG

There's currently only one usage of this but implementation of atomic_sub add another. Change-Id: Ia56743ec26ff5e7bcde8ae94fa17fef92d418d2b Signed-off-by: Brendan Jackman --- arch/x86/net/bpf_jit_comp.c | 23 ++- 1 file changed, 18 insertions(+), 5 deletions(-) diff --

[PATCH bpf-next v3 09/14] bpf: Pull out a macro for interpreting atomic ALU operations

switch case means that we need an extra conditional branch to differentiate them) in favour of compact and (relatively!) simple C code. Change-Id: I8cae5b66e75f34393de6063b91c05a8006fdd9e6 Signed-off-by: Brendan Jackman --- kernel/bpf/core.c | 79 +++ 1

[PATCH bpf-next v3 13/14] bpf: Add tests for new BPF atomic operations

er to skip the atomics test. Change-Id: Iecc12f35f0ded4a1dd805cce1be576e7b27917ef Signed-off-by: Brendan Jackman --- tools/testing/selftests/bpf/Makefile | 4 + .../selftests/bpf/prog_tests/atomics_test.c | 262 ++ .../selftests/bpf/progs/atomics_test.c

[PATCH bpf-next v3 05/14] bpf: Rename BPF_XADD and prepare to encode other atomics in .imm

possible (doesn't break existing valid BPF progs) because the immediate field is currently reserved MBZ and BPF_ADD is zero. All uses are removed from the tree but the BPF_XADD definition is kept around to avoid breaking builds for people including kernel headers. Signed-off-by: Brendan Ja

[PATCH bpf-next v3 10/14] bpf: Add bitwise atomic instructions

ectly support the fetch_ version these operations, so we need to generate a CMPXCHG loop in the JIT. This requires the use of two temporary registers, IIUC it's safe to use BPF_REG_AX and x86's AUX_REG for this purpose. Change-Id: I340b10cecebea8cb8a52e3606010cde547a10ed4 Signed-off-by:

[PATCH bpf-next v3 14/14] bpf: Document new atomic instructions

Change-Id: Ic70fe9e3cb4403df4eb3be2ea5ae5af53156559e Signed-off-by: Brendan Jackman --- Documentation/networking/filter.rst | 26 ++ 1 file changed, 26 insertions(+) diff --git a/Documentation/networking/filter.rst b/Documentation/networking/filter.rst index

[PATCH bpf-next v3 11/14] tools build: Implement feature check for BPF atomics in Clang

Change-Id: Ia15bb76f7152fff2974e38242d7430ce2987a71e Cc: Arnaldo Carvalho de Melo Cc: Jiri Olsa Cc: Quentin Monnet Cc: "Frank Ch. Eigler" Cc: Stephane Eranian Cc: Namhyung Kim Cc: Thomas Hebb Change-Id: Ie2c3832eaf050d627764071d1927c7546e7c4b4b Signed-off-by: Brendan Jackman

[PATCH bpf-next v3 12/14] bpf: Pull tools/build/feature biz into selftests Makefile

This is somewhat cargo-culted from the libbpf build. It will be used in a subsequent patch to query for Clang BPF atomics support. Change-Id: I9318a1702170eb752acced35acbb33f45126c44c Signed-off-by: Brendan Jackman --- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf

Re: [PATCH bpf-next v3 00/14] Atomics for eBPF

On Thu, Dec 03, 2020 at 04:02:31PM +, Brendan Jackman wrote: [...] > [1] Previous patchset: > https://lore.kernel.org/bpf/20201123173202.1335708-1-jackm...@google.com/ Sorry, bogus link. That's v1, here's v2: https://lore.kernel.org/bpf/20201127175738.1085417-1-jackm...@google.com/

  1   2   3   >