Grant Edwards wrote: > On 2006-04-30, David Lees <[EMAIL PROTECTED]> wrote: > >>I want to process large binary files (>2GB) in Python. I have played >>around with prototypes in pure Python and profiled the code. Most of >>the time seems to be spent converting back and forth to and from strings >>using the struct module. Is there a way to directly read into an array >>of integers in Python? > > Perhaps the numarray module?
numpy for new code, please. In particular, numarray is limited by 32-bit APIs even on 64-bit platforms, so a >2GB file will be difficult to process even when using mmap. numpy removed this restriction on 64-bit platforms. 32-bit users will still have to split up the file into <2GB chunks, though. http://numeric.scipy.org/ https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Robert Kern [EMAIL PROTECTED] "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
