Re: file seek is slow

2010-03-12 Thread CHEN Guang
>> Metalone wrote: >>> I just tried the seek test with Cython. >>> Cython fseek() : 1.059 seconds. 30% slower than 'C' >>> Python f.seek : 1.458 secondds. 80% slower than 'C'. >>> >>> It is amazing to me that Cython generates a 'C' file that is 1478 >>> lines. >> >> PythoidC ( http://pythoidc.goo

Re: file seek is slow

2010-03-12 Thread Metalone
I almost wrote a long reply to all this. In the end it boils down to being concerned about how much overhead there is to calling a 'C' function. I assumed that file.seek() simply delegated to fseek() and thus was one way to test this overhead. However, I now think that it must be doing more and may

Re: file seek is slow

2010-03-12 Thread Antoine Pitrou
Le Tue, 09 Mar 2010 15:56:47 -0800, Metalone a écrit : > for i in xrange(100): >f1.seek(0) This is quite a stupid benchmark to write, since repeatedly seeking to 0 is a no-op. I haven't re-read the file object code recently, but chances are that the Python file object has its own

Re: file seek is slow

2010-03-12 Thread Stefan Behnel
Metalone, 11.03.2010 23:57: I just tried the seek test with Cython. Cython fseek() : 1.059 seconds. 30% slower than 'C' Python f.seek : 1.458 secondds. 80% slower than 'C'. It is amazing to me that Cython generates a 'C' file that is 1478 lines. Well, it generated an optimised Python interfa

Re: file seek is slow

2010-03-12 Thread Stefan Behnel
CHEN Guang, 12.03.2010 08:51: Metalone wrote: I just tried the seek test with Cython. Cython fseek() : 1.059 seconds. 30% slower than 'C' Python f.seek : 1.458 secondds. 80% slower than 'C'. It is amazing to me that Cython generates a 'C' file that is 1478 lines. PythoidC ( http://pythoidc.

Re: file seek is slow

2010-03-11 Thread CHEN Guang
Metalone wrote: > I just tried the seek test with Cython. > Cython fseek() : 1.059 seconds. 30% slower than 'C' > Python f.seek : 1.458 secondds. 80% slower than 'C'. > > It is amazing to me that Cython generates a 'C' file that is 1478 > lines. > PythoidC ( http://pythoidc.googlecode.com )

Re: file seek is slow

2010-03-11 Thread Steve Holden
Metalone wrote: > I just tried the seek test with Cython. > Cython fseek() : 1.059 seconds. 30% slower than 'C' > Python f.seek : 1.458 secondds. 80% slower than 'C'. > > It is amazing to me that Cython generates a 'C' file that is 1478 > lines. > And what response are you seeking to your amaze

Re: file seek is slow

2010-03-11 Thread Metalone
I just tried the seek test with Cython. Cython fseek() : 1.059 seconds. 30% slower than 'C' Python f.seek : 1.458 secondds. 80% slower than 'C'. It is amazing to me that Cython generates a 'C' file that is 1478 lines. #Cython code import time cdef int SEEK_SET = 0 cdef extern from "stdio.h"

Re: file seek is slow

2010-03-11 Thread Metalone
I am assuming that Python delegates the f.seek call to the seek call in the MS C runtime library msvcrt.dll. Does anybody know a nice link to the Python source like was posted above for the BSD 'C' library? Ok, I ran some more tests. C, seek: 0.812 seconds // test from original post

Re: file seek is slow

2010-03-10 Thread sjdevn...@yahoo.com
On Mar 10, 6:01 pm, Neil Hodgson wrote: > Metalone: > > > As it turns out each call is only > > 646 nanoseconds slower than 'C'. > > However, that is still 80% of the time to perform a file seek, > > which I would think is a relatively slow operation compared to just > > making a system call. > >

Re: file seek is slow

2010-03-10 Thread Neil Hodgson
Metalone: > As it turns out each call is only > 646 nanoseconds slower than 'C'. > However, that is still 80% of the time to perform a file seek, > which I would think is a relatively slow operation compared to just > making a system call. A seek may not be doing much beyond setting a current

Re: file seek is slow

2010-03-10 Thread Metalone
Thanks, Tim. Good to know. -- http://mail.python.org/mailman/listinfo/python-list

Re: file seek is slow

2010-03-10 Thread Metalone
f1_seek = f1.seek did not change the performance at all. As it turns out each call is only 646 nanoseconds slower than 'C'. However, that is still 80% of the time to perform a file seek, which I would think is a relatively slow operation compared to just making a system call. -- http://mail.python

Re: file seek is slow

2010-03-09 Thread Tim Roberts
Metalone wrote: > >static void main(int argc, char *argv[]) As a side note, do you realize that this definition is invalid, in two ways? "main" cannot be declared "static". The whole reason we use the special name "main" is so that the startup code in the C run-time can link to it. If "main" i

Re: file seek is slow

2010-03-09 Thread Paul McGuire
This is a pretty tight loop: for i in xrange(100): f1.seek(0) But there is still a lot going on, some of which you can lift out of the loop. The easiest I can think of is the lookup of the 'seek' attribute on the f1 object. Try this: f1_seek = f1.seek for i in xrange(100):

file seek is slow

2010-03-09 Thread Metalone
I ran a comparison that timed 1e6 file seeks. The tests were run with Python 2.6.4 and Microsoft Visual C 6.0 on Windows XP with an Intel 3GHz single processor with hyperthreading. Results: C: 0.812 seconds Python: 1.458 seconds. difference = 0.646 seconds. If the file.seek is removed the