On Sun, May 2, 2021 at 8:21 AM Ryutaroh Matsumoto <ryuta...@ict.e.titech.ac.jp> wrote: > > Sorry for a bit late response. > > > I would not expect any change in performance from omitting unused drivers. > > If turning off the other platforms has a performance impact, this could > > still > > mean that there is a serious performance regression where we do not > > expect it. > > I do not know if you meant CONFIG_ARCH_* by "drivers". > Removal of all CONFIG_ARCH_* other than CONFIG_ARCH_BCM2835 disables > CONFIG_GENERIC_IRQ_MIGRATION=y > CONFIG_GENERIC_IRQ_CHIP=y > CONFIG_IRQ_FASTEOI_HIERARCHY_HANDLERS=y
The way it generally works is that each platform option only allows you to enable additional platform specific drivers that don't make sense elsewhere. E.g. The generic irq chip infrastructure is library code that is used by certain drivers but not others, so the expectation is that they would not change the performance. If they do, that may be considered a bug. > CONFIG_NUMA=n & CONFIG_HOTPLUG_CPU=n disable > CONFIG_HAVE_SETUP_PER_CPU_AREA > and CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK, > and enable CONFIG_ARCH_FLATMEM_ENABLE. > > Those changes could have some impact... These can have a small performance impact, though it should mostly make NUMA machines worse, rather than making non-NUMA machines better. > > The impact of CONFIG_DEBUG_PREEMPT is also higher than I expected > > here, it may be worth asking on the linux-rt-users list about what the > > expected cost on arm64 hardware is. > > I believe they are very well aware of this, see > https://wiki.linuxfoundation.org/realtime/documentation/howto/applications/preemptrt_setup > > There recommendation seems(?) CONFIG_DEBUG_PREEMPT=n > for better performance. Ok, in that case it might help to change the Kconfig description that today recommends turning it on: |config DEBUG_PREEMPT | bool "Debug preemptible kernel" | depends on DEBUG_KERNEL && PREEMPTION && TRACE_IRQFLAGS_SUPPORT | default y | help | If you say Y here then the kernel will use a debug variant of the | commonly used smp_processor_id() function and will print warnings | if kernel code uses it in a preemption-unsafe way. Also, the kernel | will detect preemption count underflows. In particular the "default y" makes it sound like this has very little impact. > > Can you check whether there are any other differences in the .config > > file besides CONFIG_PARAVIRT that may cause the difference, and > > that you didn't mix up the results? > > I believe no. > The reason of the difference may come from: > * The number of measurement is too few (2 times). > * Measured speed depends on the IPv6 network of ISP, which I cannot make > constant. > The RPi4B is used for processing real network traffic and my family complains > if it is down for too long... Right. In that case, the other numbers are probably also less reliable than the variance between runs suggests. > > I see you do a couple of things in this fragment. One of them is the > > CONFIG_BPF_JIT_ALWAYS_ON=y option that might result in > > a significant difference if you actually use BPF (otherwise it makes > > no difference). > > I believe the measured speed depends on nftables, ipv4-ipv6 tunnel, > macvlan driver, Ethernet driver and the general network stack, not > including BPF. Ok. > My net if config is: > ip6tnl1 (tunnel) binds to myve1 (macvlan), and > myve1 binds to eth0, and eth0 has absolutely no IPv4 or IPv6 address. > The reason of using macvlan is to use multiple macvlan and macvtap > interfaces binding to eth0. Ok. FWIW, this driver also lacks support for IFF_UNICAST_FLT, which means using macvlan/macvtap puts the device into promiscuous mode, and every frame on the wire will have to be processed coming into the device. > > I also see that you enable a number debugging options, including > > CONFIG_UBSAN_SANITIZE_ALL=y, which I would expect to make > > the kernel significantly slower when turned on. Is this one enabled > > in the other kernels as well, or did you find that it has a positive > > effect here? > > As far as I see, CONFIG_UBSAN=y and CONFIG_UBSAN_SANITIZE_ALL=y > have not decreased the performance noticeablly (for my personal use cases). > So I choose to turn on them when I have chance to build a kernel. > As far as I can recall CONFIG_UBSAN related options did not > decrease the YouTube playing by firefox-esr. > For build of user-space applications, I have not seen " subjectively > noticeable" > performance difference by UBSAN. So I routinely use -fanitize=undefined. > ASAN and MSAN are terribly slow, as we know well. The overhead of ubsan is very workload specific, I've seen other cases in which it matters a lot. > > As mentioned above, turning off the unused platforms /should/ not > > make a difference other than code size. Do you get different > > results if you drop all the CONFIG_ARCH_*=n lines from the > > fragment? If you do, I would consider that a problem in the > > upstream kernel that needs to be investigated further. > > Having look at arch/arm64/Kconfig.platforms, I see some options > depending on CONFIG_ARCH_*. Besides the ones > mentioned at the beginning, they include > IRQ_DOMAIN_HIERARCHY > ARM_GIC > > The *IRQ* and ARM_GIC config options can have some impact on the performance, > if a use case includes lots of HW interrupts, as I am using it > > I am ready to re-build a Debian kernel with only CONFIG_ARCH_* > (except CONFIG_ARCH_BCM2835) disabled. Ok. As I said, I don't think the IRQ options would matter here, but turning off PREEMPT_RT should help a lot if there are too many interrupts. More importantly, you can play around with changing the IRQ coalescing numbers for 'ethtool -C' (if the driver supports that). Setting the coalescing options higher generally improves throughput because more work can be done per interrupt, but setting it too high can add latency from buffer bloat. Arnd
