Hi All, I need to store a large number of large objects to file and then access them sequentially. I am talking about a few thousands of objects and each with size of a few hundred kilobytes, and total file size a few gigabytes. I tried shelve, but it is not good at sequentially accessing the data. In essence, shelve.keys() takes forever.
I am wondering if there is a module that can persist a stream of objects without having to load everything into memory. (For this reason, I think Pickle is out, too, because it needs everything to be in memory.) Thanks, GZ -- http://mail.python.org/mailman/listinfo/python-list
