Forafo San wrote:
> Folks,
> What might be a good replacement for the shelve module, but one that
> can handle a few gigs of data. I'm doing some calculations on daily
> stock prices and the result is a nested list like:
>
> [[date_1, floating result 1],
> [date_2, floating result 2],
> ...
> [date_n, floating result n]]
>
> However, there are about 5,000 lists like that, one for each stock
> symbol.
You might save some memory by using tuples rather than lists:
>>> sys.getsizeof(["01/01/2000", 123.456]) # On a 32-bit system.
40
>>> sys.getsizeof(("01/01/2000", 123.456))
32
By the way, you know that you should never, ever use floats for currency,
right?
http://vladzloteanu.wordpress.com/2010/01/11/why-you-shouldnt-use-float-for-currency-floating-point-issues-explained-for-ruby-and-ror/
http://stackoverflow.com/questions/3730019/why-not-use-double-or-float-to-represent-currency
> Using the shelve module I could easily save them to a file
> ( myshelvefile['symbol_1') = symbol_1_list) and likewise retrieve the
> data. But shelve is deprecated
It certainly is not.
http://docs.python.org/library/shelve.html
http://docs.python.org/py3k/library/shelve.html
Not a word about it being deprecated in either Python 2.x or 3.x.
> AND when a lot of data is written
> shelve was acting weird (refusing to write, filesizes reported with an
> "ls" did not make sense, etc.).
I would like to see this replicated. If it is true, that's a bug in shelve,
but I expect you're probably doing something wrong.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
