Mag Gam wrote:
Yes, the system has 64Gig of physical memory.
drool ;-).
What I meant was, is it possible to load to a hdf5 dataformat (basically NumPy array) without reading the entire file at first? I would like to splay to disk beforehand so it would be a bit faster instead of having 2 copies in memory.
If you can write hdf5 a line at a time, you should be able to something like <open cvs> <open hdf5> for line in cvs: process line write hdf5 line this assumes 1-1 lines. -- http://mail.python.org/mailman/listinfo/python-list
