[issue1327] Python 2.4+ spends too much time in PyEval_EvalFrame w/ xmlrpmclib
Eric Sammons added the comment: I have tested 2.4.4 and 2.5.1 from python.org and both suffer from the exact same issue. I have also compared ceval.c from 2.3, the last known working copy and ceval.c from 2.4+ and found that ceval.c has undergone some pretty significant changes. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1327> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1327] Python 2.4+ spends too much time in PyEval_EvalFrame w/ xmlrpmclib
Eric Sammons added the comment: Please find the attached ceval.c. This file was generated from oprofile and gives clear indicators where the simple program below gets hung up. You will find in the oprofile output, from ceval.c, that PyEval_Frame has a total of 1649 26.2789. This indicates with a high degree of probabilty that the changes made in this portion of the code are promblamatic. Now I will be the first to admit that I am by no means a C programmer; however, I can tell you that the code has changed a great deal with regards to xmlrpc and it has impacted performance to the point where it seems to be unusable with current releases of Python where expected output is > 300 records. Please check it out for yourself, using a different xmlrpc server - I currently have access to one only where the number of records is greater than 300. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1327> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1327] Python 2.4+ spends too much time in PyEval_EvalFrame w/ xmlrpmclib
Eric Sammons added the comment: Has anybody else tested this against an RPC server that has more than 300 items to be returned. I find it interesting that before all the xmlrpc code changed everything works fine, but after the code change (re-write) things don't seem to work. As for the bug, I do not claim there is a bug, I am simply pointing out that when using oprofile to profile the test application, most of the time is spent in a specific routine. Now it could be a side affect of xmlrpc, I don't know - I do not claim to be a "programmer". I am looking to turn on verbose mode in xmlrpc. As soon as I do that I will test again and send the results. On Jan 15, 2008 7:05 PM, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > > Fredrik Lundh added the comment: > > Can you switch on verbose mode in xmlrpclib, so you can see *where* the > transfer hangs? > > Arguing that a hanging Python program must be caused by a bug in the > code that *executes* the Python program isn't that meaningful, really. > After all, that code is used to run *all* Python programs, so I think > we'd noticed if it had a tendency to hang unexpectedly... > > > __ > Tracker <[EMAIL PROTECTED]> > <http://bugs.python.org/issue1327> > __ > __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1327> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1327] Python 2.4+ spends too much time in PyEval_EvalFrame w/ xmlrpmclib
Eric Sammons added the comment: I have added the verbose=1 option, the processing hangs in the body: where : for a in l: print a['channel_label'] Is returning the data. Initially it returns data quickly and then slows until it eventually stops. Here is the sample of the body data... data removed... body: ' This information is repeats ~500 times in the server where the program eventually times out. If you require anything further from the verbose output please let me know. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1327> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1327] Python 2.4+ spends too much time in PyEval_EvalFrame w/ xmlrpmclib
Eric Sammons added the comment: Here is the header information that I get: reply: 'HTTP/1.1 200 OK\r\n' header: Date: Tue, 22 Jan 2008 14:42:54 GMT header: Server: Apache header: Content-Length: 183 header: Content-Type: text/xml;charset=UTF-8 Then I get addition send followed by more header info: reply: 'HTTP/1.1 200 OK\r\n' header: Date: Tue, 22 Jan 2008 14:42:54 GMT header: Server: Apache header: Content-Length: 399191 header: Content-Type: text/xml;charset=UTF-8 header: X-Pad: avoid browser bug I am looking for additional Content-Length data. Also, the above was done using Python 2.5.1 (r251:54863, Oct 30 2007, 13:54:11). Thank you! __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1327> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1327] Python 2.4+ spends too much time in PyEval_EvalFrame w/ xmlrpmclib
Eric Sammons added the comment: Some strace data: WORKS: strace -cf ./test.py >/tmp/out Process 9439 detached % time seconds usecs/call callserrors syscall -- --- --- - - 33.100.009125 8 1086 956 open 23.470.006469 7 930 recvfrom 9.670.002667 9 311 278 stat 8.100.002234 14 165 read 3.400.000936 7 133 mmap 3.250.000896 6 144 close 3.240.000892 4520 poll 2.970.000818 5 173 fstat 2.830.000780 1553 munmap 2.130.000588 118 5 3 execve 1.340.000369 939 mprotect 1.130.000312 472 rt_sigaction 0.660.000183 535 futex 0.640.000176 1017 brk 0.610.000169 1214 sendto 0.570.000158 26 6 write 0.500.000139 434 fcntl 0.490.000134 1014 socket 0.400.000111 425 lseek 0.360.000100 714 2 connect 0.340.93 23 4 getdents64 0.270.74 514 2 ioctl 0.170.48 411 gettimeofday 0.060.16 8 2 2 access 0.050.14 7 2 2 readlink 0.040.10 5 2 arch_prctl 0.040.10 5 2 time 0.030.09 9 1 _sysctl 0.030.08 8 1 lstat 0.030.08 4 2 uname 0.020.06 6 1 rt_sigprocmask 0.020.06 6 1 getcwd 0.010.04 4 1 getrlimit 0.010.04 4 1 set_tid_address -- --- --- - - 100.000.027566 3335 1245 total I am still compiling the same data for Python 2.4.3 (#1, Dec 11 2006, 11:39:03). However, this could take hours... as 2.4+ seem to be problematic. __ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue1327> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
