On Wed, Apr 2, 2008 at 10:59 AM, Derek Tracy <[EMAIL PROTECTED]> wrote:
> I am trying to write a script that reads in a large binary file (over 2Gb)
> saves the header file (169088 bytes) into one file then take the rest of the
> data and dump it into anther file. I generated code that works wonderfully
> for files under 2Gb in size but the majority of the files I am dealing with
> are over the 2Gb limit
>
> INPUT = open(infile, 'rb')
> header = FH.read(169088)
>
> ary = array.array('H', INPUT.read())
>
> INPUT.close()
>
> OUTF1 = open(outfile1, 'wb')
> OUTF1.write(header)
>
> OUTF2 = open(outfile2, 'wb')
> ary.tofile(OUTF2)
>
>
> When I try to use the above on files over 2Gb I get:
> OverflowError: requested number of bytes is more than a Python string
> can hold
>
> Does anybody have an idea as to how I can get by this hurdle?
>
> I am working in an environment that does not allow me to freely download
> modules to use. Python version 2.5.1
>
>
>
> R/S --
> ---------------------------------
> Derek Tracy
> [EMAIL PROTECTED]
> ---------------------------------
>
I know have 2 solutions, one using
partial
and the other using array
Both are clocking in at the same time (1m 5sec for 2.6Gb), are there
any ways I can optimize either solution? Would turning off the
read/write buff increase speed?
--
---------------------------------
Derek Tracy
[EMAIL PROTECTED]
http://twistedcode.blogspot.com
---------------------------------
--
http://mail.python.org/mailman/listinfo/python-list
