bug#73721: grep perf docs barely mention mem usage

2024-10-09 Thread mark . yagnatinsky--- via Bug reports for GNU grep
This page: https://www.gnu.org/software/grep/manual/html_node/Performance.html discusses how to get good performance out of grep. That is nice, but the perf advice focuses almost entirely on speed. I'm more interested in how to avoid excessive memory usage. After a bit of research, it seems that on

bug#73721: grep perf docs barely mention mem usage

2024-10-09 Thread mark . yagnatinsky--- via Bug reports for GNU grep
Thanks!! Re: wouldn't help much: maybe it wouldn't help with speed. But how could it possibly not help with memory usage? In particular, the "commit charge" is surely far lower, right? With a read() based approach, grep needs to explicitly allocate an array, and as far as the kernel knows, this ar

bug#73721: grep perf docs barely mention mem usage

2024-10-09 Thread Paul Eggert
On 2024-10-09 10:01, mark.yagnatinsky--- via Bug reports for GNU grep wrote: After a bit of research, it seems that once upon a time, grep used mmap where possible, but it no longer does this. Thus, peak memory usage will be proportional to the length of the longest line in the file. Thus, if u

bug#73721: grep perf docs barely mention mem usage

2024-10-09 Thread Paul Eggert
On 2024-10-09 14:12, mark.yagnatin...@barclays.com wrote: With mmap(), the kernel just needs to set up a bit of book-keeping Yes, I'm sure than in some cases mmap would be a win. However, all in all it was found not to be, and I expect that the hassle of maintaining the mmap variant wasn't wo