On 24Apr2022 08:21, Chris Angelico <ros...@gmail.com> wrote: >On Sun, 24 Apr 2022 at 08:18, Cameron Simpson <c...@cskk.id.au> wrote: >> An approach I think you both may have missed: mmap the file and use >> mmap.rfind(b'\n') to locate line delimiters. >> https://docs.python.org/3/library/mmap.html#mmap.mmap.rfind > >Yeah, I made a vague allusion to use of mmap, but didn't elaborate >because I actually have zero idea of how efficient this would be. >Would it be functionally equivalent to the chunking, but with the >chunk size defined by the system as whatever's most optimal? It would >need to be tested.
True. I'd expect better than single byte seek/read though. >I've never used mmap for this kind of job, so it's not something I'm >comfortable predicting the performance of. Fair. But it would be much easier to read code. Cheers, Cameron Simpson <c...@cskk.id.au> -- https://mail.python.org/mailman/listinfo/python-list
