This patch series introduces multikernel architecture support, enabling multiple independent kernel instances to coexist and communicate on a single physical machine. Each kernel instance can run on dedicated CPU cores while sharing the underlying hardware resources.
The multikernel architecture provides several key benefits: - Better resource utilization than traditional VM (KVM, Xen etc.) - Better performance than containers without noisy neighbor issues - Improved fault isolation between different kernels - Potential zero-down kernel update with KHO (Kernel HandOver) Architecture Overview: The implementation leverages kexec infrastructure to load and manage multiple kernel images, with each kernel instance assigned to specific CPU cores. Inter-kernel communication is facilitated through a dedicated IPI framework that allows kernels to coordinate and share information when necessary. Key Components: 1. Enhanced kexec subsystem with dynamic kimage tracking 2. Generic physical memory allocation for multikernel and kernel instances 3. Multikernel-specific kernfs for managing kernel instances 4. Device-tree based mechanism for static resource allocation 5. Device-tree overlay based mechanism for dynamic resource allation 6. KHO based mechanism for device-tree sharing between kernels 7. Generic IPI communication framework for inter-kernel messaging 8. Proc interface (/proc/kimage) for monitoring loaded kernel images The implementation maintains full backward compatibility with existing kexec functionality while adding the new multikernel capabilities. For the complete roadmap, please see: https://docs.google.com/document/d/1yneO6O6C_z0Lh3A2QyT8XsH7ZrQ7-naGQT-rpdjWa_g/ For the Proof-of-Concept demo (prior to device-tree), please see: https://www.youtube.com/watch?v=bmC4JRTSDKE For user-space components, especially device-tree specification, please check kerf project: https://github.com/multikernel/kerf IMPORTANT NOTES: 1) This is a Request for Comments (RFC) submission. While the core architecture is functional, there are numerous implementation details that need improvement. The primary goal is to gather feedback on the high-level design and overall approach rather than focus on specific coding details at this stage. 2) This patch series represents only the foundational framework for multikernel support. It establishes the basic infrastructure and communication mechanisms. We welcome the community to build upon this foundation and develop their own solutions based on this framework. 3) Testing has been limited to the author's development machine using hard-coded boot parameters and specific hardware configurations. Community testing across different hardware platforms, configurations, and use cases would be greatly appreciated to identify potential issues and improve robustness. Obviously, don't use this code beyond testing. Signed-off-by: Cong Wang <[email protected]> Changes since RFC V1: - Switched from kexec_load() to kexec_file_load() - Introduced generic multikernel physical memory allocation and per-instance virtual memory allocation - Introduced kernfs interface for managing kernel instances - Introduced device-tree and KHO framework for resource management and sharing - Introduced messaging over IPI framework, preparing for dynamic resource allocation and zero-downtime upgrade - Grouped multikernel code in the kernel/multikernel/ directory - Reorganized and rebased all the patches --- Cong Wang (16): kexec: Introduce multikernel support via kexec x86: Introduce SMP INIT trampoline for multikernel CPU bootstrap multikernel: Introduce basic multikernel subsystem infrastructure x86: Introduce MULTIKERNEL_VECTOR for inter-kernel communication x86: Introduce arch_cpu_physical_id() to obtain physical CPU ID multikernel: Introduce physical memory reservation and allocation kexec: Implement dynamic kimage tracking multikernel: Introduce device-tree based kernfs interface kexec: Integrate multikernel instance management with kexec subsystem Documentation: Add multikernel usage kexec: Add /proc/kimage interface for kimage tracking multikernel: Introduce per-instance memory allocation interface kernel: Introduce generic multikernel IPI communication framework multikernel: Add messaging layer for inter-kernel communication kexec: Integrate multikernel support with kexec_file_load() multikernel: Integrate Kexec HandOver framework for DTB preservation Documentation/multikernel/usage.rst | 215 ++++++++ arch/powerpc/kexec/crash.c | 8 +- arch/x86/include/asm/idtentry.h | 3 + arch/x86/include/asm/irq_vectors.h | 1 + arch/x86/include/asm/smp.h | 7 + arch/x86/kernel/Makefile | 1 + arch/x86/kernel/crash.c | 4 +- arch/x86/kernel/head64.c | 5 + arch/x86/kernel/idt.c | 3 + arch/x86/kernel/kexec-bzimage64.c | 16 +- arch/x86/kernel/setup.c | 11 +- arch/x86/kernel/smp.c | 17 + arch/x86/kernel/smpboot.c | 159 ++++++ arch/x86/kernel/trampoline_64_bsp.S | 288 +++++++++++ arch/x86/kernel/vmlinux.lds.S | 6 + drivers/of/kexec.c | 20 +- include/linux/kexec.h | 30 +- include/linux/kexec_handover.h | 40 ++ include/linux/multikernel.h | 659 ++++++++++++++++++++++++ include/uapi/linux/kexec.h | 5 + include/uapi/linux/reboot.h | 2 +- kernel/Kconfig.kexec | 2 + kernel/Makefile | 1 + kernel/kexec.c | 60 ++- kernel/kexec_core.c | 384 ++++++++++++++ kernel/kexec_file.c | 158 +++++- kernel/kexec_handover.c | 197 ++++++- kernel/multikernel/Kconfig | 26 + kernel/multikernel/Makefile | 9 + kernel/multikernel/core.c | 532 +++++++++++++++++++ kernel/multikernel/dts.c | 466 +++++++++++++++++ kernel/multikernel/internal.h | 4 + kernel/multikernel/ipi.c | 471 +++++++++++++++++ kernel/multikernel/kernfs.c | 772 ++++++++++++++++++++++++++++ kernel/multikernel/kho.c | 356 +++++++++++++ kernel/multikernel/mem.c | 376 ++++++++++++++ kernel/multikernel/messaging.c | 278 ++++++++++ kernel/reboot.c | 10 + 38 files changed, 5572 insertions(+), 30 deletions(-) create mode 100644 Documentation/multikernel/usage.rst create mode 100644 arch/x86/kernel/trampoline_64_bsp.S create mode 100644 include/linux/multikernel.h create mode 100644 kernel/multikernel/Kconfig create mode 100644 kernel/multikernel/Makefile create mode 100644 kernel/multikernel/core.c create mode 100644 kernel/multikernel/dts.c create mode 100644 kernel/multikernel/internal.h create mode 100644 kernel/multikernel/ipi.c create mode 100644 kernel/multikernel/kernfs.c create mode 100644 kernel/multikernel/kho.c create mode 100644 kernel/multikernel/mem.c create mode 100644 kernel/multikernel/messaging.c -- 2.34.1
