On 09/03/2013 3:51 AM, Peter Otten wrote:
Colin J. Williams wrote:
The program runs correctly under each version, but it runs more slowly
under 3.2.
This is probably due to the fact that the .pyc file is created for the
Python 2.7 execution.
When Python 3.2 is run it fails to create a new .pyc file and if the 2.7
.pyc is offered directly a magic number problem is reported.
(1) .pyc files are only created if a module is imported
(2) The 2.7 .pyc file is put alongside the .py file whereas the 3.2 .pyc is
put into the __pycache__ subfolder. No clash can occur.
A simple example:
$ ls
mod.py
$ cat mod.py
print("hello world")
Run it; no pyc is created:
$ python2.7 mod.py
hello world
$ ls
mod.py
Import it using 2.7:
$ python2.7 -c 'import mod'
hello world
$ ls
mod.py mod.pyc
Import it using 3.2:
$ python3.2 -c 'import mod'
hello world
$ ls
mod.py mod.pyc __pycache__
$ ls __pycache__/
mod.cpython-32.pyc
Run the compiled code:
$ python2.7 mod.pyc
hello world
$ python3.2 __pycache__/mod.cpython-32.pyc
hello world
But I'm with Steven, it's unlikely that the module compilation phase is
responsible for a noticeable slowdown.
Thanks to Steven and Peter for their responses.
My main problem appears to be with:
Profile with Python 2.7
11 25.736 2.340 25.736 2.340
{numpy.linalg.lapack_lite.dgesv}
Profile with Python 3.2
11 152.111 13.828 152.111 13.828 {built-in method dgesv}
In other words, the Python 3.2 linear equation solve takes longer than
with Python 2.7. I'll pursue this with the numpy folk.
There also appears to be a problem with the generation of the .pyc.
Please see the example below:
rem temp.bat
dir *.pyc
del *.pyc
C:\python32\python.exe profiler.py Intel P4 2.8GHz 2MB Ram 221 GB Free
Disk cjw> prof3.txt
dir *.pyc
This is executed with: tmp.bat > tmp.lst
profiler.py contains:
#-------------------------------------------------------------------------------
# Name: profiler.py
# Purpose:
#
# Author: cjw
#
# Created: 17/02/2013
# Copyright: (c) cjw 2013
# Licence: <your licence>
#-------------------------------------------------------------------------------
import cProfile as p, pstats as s, sys
def main():
v= sys.version
statsFile= 'FPSStats' + v[0] + v[2] + '.txt'
p.run('import testFPSpeed; testFPSpeed.main()',
statsFile)
t= s.Stats(statsFile)
t.strip_dirs().sort_stats('cumulative').print_stats(40)
pass
if __name__ == '__main__':
main()
tmp.lst contains:
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>rem temp.bat
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>dir *.pyc
Volume in drive C has no label.
Volume Serial Number is D001-FAC4
Directory of C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>del *.pyc
C:\Documents and Settings\cjw.P4A\My
Documents\devpy\raspi>C:\python32\python.exe profiler.py Intel P4 2.8GHz
2MB Ram 221 GB Free Disk cjw 1>prof3.txt
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>dir *.pyc
Volume in drive C has no label.
Volume Serial Number is D001-FAC4
Directory of C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>rem temp.bat
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>dir *.pyc
Volume in drive C has no label.
Volume Serial Number is D001-FAC4
Directory of C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>del *.pyc
C:\Documents and Settings\cjw.P4A\My
Documents\devpy\raspi>C:\python32\python.exe profiler.py Intel P4 2.8GHz
2MB Ram 221 GB Free Disk cjw 1>prof3.txt
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>dir *.pyc
Volume in drive C has no label.
Volume Serial Number is D001-FAC4
Directory of C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi
tmp.lst contains:
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>rem temp.bat
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>dir *.pyc
Volume in drive C has no label.
Volume Serial Number is D001-FAC4
Directory of C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>del *.pyc
C:\Documents and Settings\cjw.P4A\My
Documents\devpy\raspi>C:\python32\python.exe profiler.py Intel P4 2.8GHz
2MB Ram 221 GB Free Disk cjw 1>prof3.txt
C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi>dir *.pyc
Volume in drive C has no label.
Volume Serial Number is D001-FAC4
Directory of C:\Documents and Settings\cjw.P4A\My Documents\devpy\raspi
From this we see that no .pys was reported for testFPSpeed.py.
However, looking at the directory itself. the .pyc is in fact created.
Thus, the .pyc is, if necessary, generated upon the import of a .py and
so this does not explain the time difference between 2.7 and 3.2.
Thanks to Steven for pointing to the __cache__ directory, I find no
reference to it in the docs.
Colin W.
PS I wish we could format the text in these messages.
--
http://mail.python.org/mailman/listinfo/python-list
