This should be a relatively simple problem, but I haven't quite got the idea of how to go about it. I have a VERY large file that I would like to load a line at a time, do some manipulations on it, and then make it available to as a file-like object for use as input to a database module (psycopg2) that wants a file-like object (with read and readlines methods). I could write the manipulated file out to disk and then read it back in, but that seems wasteful. So, it seems like I need a buffer, a way to fill the buffer and a way to have read and readlines use the buffer. What I can't do is to load the ENTIRE file into a stringio object, as the file is much too large. Any suggestions?
Thanks, Sean -- http://mail.python.org/mailman/listinfo/python-list
