Map the file into RAM by using the mmap module. The file's contents than is availabel as a seachable string.
that's a good idea, but I wonder if it actually saves on memory? I just tried regexing through a 25Mb file and end up with 40Mb as working set (it rose linearly as the loop progessed through the file). Am I actually saving anything by not letting normal vm do its thing?
HTH, Gerald
Robin Becker schrieb:
Is there any way to get regexes to work on non-string/unicode objects. I would like to split large files by regex and it seems relatively hard to do so without having the whole file in memory. Even with buffers it seems hard to get regexes to indicate that they failed because of buffer termination and getting a partial match to be resumable seems out of the question.
What interface does re actually need for its src objects?
-- Robin Becker
-- http://mail.python.org/mailman/listinfo/python-list
