On Thu, 16 May 2013 20:20:26 -0700, lokeshkoppaka wrote:
> I need to write numbers into a file upto 50mb and it should be fast can
> any one help me how to do that?
> i had written the following code..
> ----------------------------------------------------------------------
> def create_file_numbers_old(filename, size): start = time.clock()
>
> value = 0
> with open(filename, "w") as f:
> while f.tell()< size:
> f.write("{0}\n".format(value))
> value += 1
>
> end = time.clock()
>
> print "time taken to write a file of size", size, " is ", (end -start),
> "seconds \n"
> --------------------------------------------------------------------
> it takes about 20sec i need 5 to 10 times less than that.
20 seconds to write how many numbers? If you are doing
create_file_numbers_old(filename, 5)
then 20 seconds is really slow. But if you are doing:
create_file_numbers_old(filename, 50000000000000)
then 20 seconds is amazingly fast.
Try this instead, it may be a little faster:
def create_file_numbers_old(filename, size):
count = value = 0
with open(filename, 'w') as f:
while count < size:
s = '%d\n' % value
f.write(s)
count += len(s)
value += 1
If this is still too slow, you can try three other tactics:
1) pre-calculate the largest integer that will fit in `size` bytes, then
use a for-loop instead of a while loop:
maxn = calculation(...)
with open(filename, 'w') as f:
for i in xrange(maxn):
f.write('%d\n' % i)
2) Write an extension module in C that writes to the file.
3) Get a faster hard drive, and avoid writing over a network.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
