Re: Sequential Object Store

2010-08-09 Thread GZ
Hi Alex, On Aug 7, 6:54 pm, Alex Willmer wrote: > On Aug 7, 5:26 pm, GZ wrote: > > > I am wondering if there is a module that can persist a stream of > > objects without having to load everything into memory. (For this > > reason, I think Pickle is out, too, because it needs everything to be > >

Re: Sequential Object Store

2010-08-07 Thread Alex Willmer
On Aug 7, 5:26 pm, GZ wrote: > I am wondering if there is a module that can persist a stream of > objects without having to load everything into memory. (For this > reason, I think Pickle is out, too, because it needs everything to be > in memory.) >From the pickle docs it looks like you could do

Sequential Object Store

2010-08-07 Thread GZ
Hi All, I need to store a large number of large objects to file and then access them sequentially. I am talking about a few thousands of objects and each with size of a few hundred kilobytes, and total file size a few gigabytes. I tried shelve, but it is not good at sequentially accessing the dat