On Tue, Aug 11, 2026 at 05:27:21PM +0200, Christian Brauner wrote: > A coredump generated via the coredump socket ends up transferring > zeroed data when a mapping contains holes. For a large process that > maps a bunch of data that's wasting a ton of work. > > Jacob ran into this and Josef has bitched^wcomplained about this to me > before. I dislike the coredump_filter bit solution in [1] which stops > each PT_LOAD at the last populated page. > > The problem is real though. I don't think coredump_filter is where we > need to solve this. That mask says which kinds of memory to include and > it propagates across fork and exec, whereas what is being selected here > is an encoding mechanism. > > I also think that the usermodehelper - may it swiftly die - isn't really > salvagable for this and it's not the future anyway. The coredump socket > already has a handshake for stuff like this. > > I always had an idea how this would look like but punted on it back > then. So here it is. > > A server that raises COREDUMP_HEADER in coredump_ack->mask doesn't get > the coredump as a plain byte stream but as a sequence of frames. Each > one a struct coredump_frame_header followed by what it describes. A data > frame carries its bytes. If a server also raises COREDUMP_SPARSE, zero > frames are sent for unpopulated mappings. They only indicate how many > zero bytes need to be written and to not include data. Reassembling the > frames gives back the same coredump. A debugger and everything else > still see an ordinary core file and nothing outside the coredump server > has to learn anything.
Hey, Christian, I proposed pretty much this exact solution to Jacob, so thank you for writing it :) There are a couple of reasons we still wanted to explore the coredump_filter solution: 1. Our core dumper application is not really prepared to run as a daemon that listens on a socket, having been written to be a transient usermode helper. But thinking about it more, maybe that's something we could paper over with systemd socket activation? 2. More importantly, we sometimes write core dumps to disk and sometimes upload them to blob storage. For the former, this approach of sending holes over the socket is great. For the latter, we'd now need to wrap the dump in some sort of container supporting sparseness that all consumers then need to reassemble. The coredump_filter approach doesn't require any changes in that pipeline. To be transparent, I still prefer the sparse socket approach, but Jacob has different contraints that I'd love to have addressed: it's a trade-off of more work on the core dump server side and all of its consumers vs. in the debug tooling side, which is mostly already there. Thanks, Omar
