Igor, Thanks for the well thought out comment. Do you have a suggestion for a fast way to write to disk? Since the design requires random access perhaps just a random access file?
Claude On Thu, May 23, 2024 at 1:17 PM Igor Soarez <i...@soarez.me> wrote: > Hi Claude, > > Thanks for writing this KIP. This issue seems particularly > thorny, and I appreciate everyone's effort to address this. > > I want to share my concern with the KIP's proposal of the > use of memory mapped files – mmap is Java's achilles heel, > Kafka should make less use of it, not more. > > The JVM often needs to stop all application threads (aka > mutator threads) before some operations, such as GC, > optimizations, redefinitions, internal cleanups and various > other internal reasons. This is known as Safepointing. > > Because the JVM cannot forcefully stop threads, it must instead > wait for each thread to observe the Safepointing request, > mark themselves as safe and stop. > A single delayed thread can leave the whole JVM hanging, waiting. > > Reads and writes to memory mapped files can trigger system interrupts, > which can block on IO for prolonged amounts of time. > One particualrly bad example is hitting the page cage dirty ratio, > and having to flush all of the page cage, in a potentially large > (high RAM) system into a potentially slow filesystem. > I have seen pauses as extreme as 1 minute, and others have reported > There are other public reports on this. [1][2] > > Safepointing in the JVM is designed with mechanisms to prevent having > to wait for a single busy thread: Threads mark themselves as safe before > waiting on locks, before system calls, before doing JNI, etc, and upon > returning they check if a Safepoint is ongoing. > So if a read or write syscall takes a bit longer that's fine, the JVM > won't halt for Safepointing, it will proceed knowing that any thread stuck > on a syscall will stop if necessary when it returns. > But there's no protection against long system interrups. > From the JVM's perspective the use of mmap is just a simple memory access, > so there's no Safepointing protection around that. > The kernel does know nor care for Java's Safepointing, and does not treat > halting a single unsuspecting thread for a longer period of time with > the severity that it may imply during a JVM Safepoint. > > So for this reason, I urge you to consider alternatives to the use > of memory mapped files. > > Best, > > -- > Igor > > https://groups.google.com/g/mechanical-sympathy/c/LFrJPhyVOJ4 > > https://groups.google.com/g/mechanical-sympathy/c/tepoA7PRFRU/m/7HbSINaFBgAJ > >
