Hi, I am developing a data structure that resides in a large memory buffer. It is important that I be able to mutate the data structure without interfering with concurrent reads, and then swap it in at a convenient time.
There is a set of initial data stored in a file. I use mmap(2) to map the file into memory. At some later point, I would like to run incremental updates. This involves copying the data structure, mutating it, then publishing it. Only a very small percentage of the data structure is changed, but it can change at random. Currently, I accomplish this by mapping a new anonymous segment, copying the data, and modifying it. However this is wasteful both in time (copying all that data takes a fair amount of time) and memory (most of the data did not actually change, but now we have two copies). I would like to utilize kernel support for copy-on-write to eliminate both of these bottlenecks. Ideally, I would be able to: 1) create a new COW mapping of an existing anonymous mapping that shares pages initially and I can write to without affecting the original 2) At the time of creating that new mapping, extend the buffer with initially-zero pages (in case I need more space to append to the end of the data structure in the new version) The mmap and mremap documentation get me tantalizingly close — I can resize the segment I already have, I can allocate file-backed copy on write segments. Also, forking while holding a private mapping seems to provide other bits of the functionality (COW of anonymous segments). But I can’t seem to figure out how to fit the puzzle pieces together. Also, I’m running in the JVM, so forking is not a possibility. There’s a couple of unanswered StackOverflow questions in a similar vein, so I hope I’m not just missing something obvious. The most relevant is: http://stackoverflow.com/questions/16965505/allocating-copy-on-write-memory-within-a-process And it looks like this dance is even possible on Mach, with some clever use of ‘vm_remap'. I’m hoping that I’m just missing the magic incantation to pass to mmap to achieve this behavior. I wasn’t brave enough to subscribe to LKML just yet, so please CC me on replies! TIA, Steven Schlansker -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
