On Sat, Oct 5, 2019 at 8:09 PM Jerin Jacob <jerinjac...@gmail.com> wrote: > > > >> > > >> This might be one avenue if all kernel JIT contributors would be on > > >> board. > > >> Another option I'm wondering could be to extend the bpf() syscall in > > >> order > > >> to pass down a description of context and helper mappings e.g. via BTF > > >> and > > >> let everything go through the verifier in the kernel the usual way (I > > >> presume > > >> one goal might be that you want to assure that the generated BPF code > > >> passes > > >> the safety checks before running the prog), then have it JITed and > > >> extract > > >> the generated image in order to use it from user space. Kernel would have > > >> to make sure it never actually allows attaching this program in the > > >> kernel. > > >> Generated opcodes can already be retrieved today (see below). Such infra > > >> could potentially help bpf-gcc folks as well as they expressed desire to > > >> have some sort of a simulator for their gcc BPF test suite.. and it would > > >> allow for consistent behavior of the BPF runtime. Just a thought. > > > > > > This idea looks good. This can remove the verifier code also from DPDK. > > > > Right, definitely makes sense to have consolidation also on this one as well > > aside from the JITs, and pushing to the kernel and receiving back the JITed > > image seems quite nice and would be generic to open up many other use cases > > outside of networking. From app pov, it's just an implementation detail > > where > > to get that BPF opcode image from anyway. > > > > > A couple of downsides I can think of, > > > > > > # We may need to extend the kernel verifier to understand the user-space > > > address > > > and its symbols for CALL and MEM access operations. > > > > Yep, that part would be needed, potentially BTF could be of help here as > > well > > for the description of the user space runtime environment like context, > > helpers > > etc, so that JIT knows how to handle this. > > Though, We can not conclude the following non-technical aspects in > this forum like > # Dual license Linux Kernel BPF code as GPL-BSD as a separate library. > # DPDK BPF support for FreeBSD and Windows OS, Treatment for Other OS? > # Need a different treatment for old Linux kernels. > This would call for immediate DPDK release to follow the existing semantics. > > Yes, For the long term, Using the Kernel JITed EBPF program for > userspace will be helpful. At least DPDK can use it in the future. > A couple of other things to consider when someone does this > # https://github.com/iovisor/ubpf can also benefit from this. > # We need to think about how to support tail call in userspace > # It is possible to have burst mode support in userspace as an > improvement (dealing with 4 packets at the time with SIMD). > Dealing SIMD in kernel space will be an issue or dealing with such > improvement in general. > # Kernel verifier and dealing with address has a lot of security > requirements that may not apply for userspace, and therefore some of > the > optimization specific to userspace needs to consider some way > # Based on my understanding the Linux and DPDK JITed code, following > optimizations may need/have a different path > > a) Userspace JIT has to deal with a 64bit address space. Kernel BPF > code can assume the Kernel virtual address space range and optimize. > b) I see, Kernel JIT always makes stack size as non zero even though > BPF applications are not using the stack. I am not sure why it > is that(Could be some security issue). There could be some > optimization not to push the stack pointer in the prologue if EBPF > program does > not use stack > c) In DPDK JIT compiler, In this first pass, we are checking the > actual registers really in use and based on that we are crafting > prologue and > epilogue at runtime to improve the performance. It could be just > implementation detail, but not sure why Linux kernel is not doing such > optimization.
Just pinging back to see if anyone interested in the common library. Answers to the above questions and mainly this below thead will make us forward progress for common library if there is still interest in collaboration. http://mails.dpdk.org/archives/dev/2019-October/146063.html
