I am manipulating lots of log files (about 500,000 files and about 30Gb
in total) to get them into a little SQL db. Part of this process is
"normalisation" and creating tables of common data. I am creating
dictionaries for these in a simple {value,key} form.In terms of memory and performance what are the reasonable limits for a dictionary with a key and a 16 character string? eg; if I read in one of my tables from disk into a dictionary, what sizing is comfortable? 100,000 entries? 1,000,000 entries? Lookup times and memory requirements are my main worries. (Running Python 2.3.4 on RH Ent, dual-Xeon with 2GB memory) -- http://mail.python.org/mailman/listinfo/python-list
