shouldn't list comprehension be faster than for loops?
Hello all I am testing my code with list comprehensions against for loops. the loop: dipList=[float(val[1]) for val in datalist] dip1=[] for dp in dipList: if dp == 90: dip1.append(dp - 0.01) else: dip1.append(dp) listcomp: dipList=[float(val[1]) for val in datalist] dip1=[(dp, dp-0.01)[dp==90.0] for dp in dipList] Tenting the time spent by each approach (using time.clock()), with a file with about 100,000 entries, I get 0.03s for the loop and 0.05s for the listcomp. thoughts? TIA Carlos -- http://mail.python.org/mailman/listinfo/python-list
Re: shouldn't list comprehension be faster than for loops?
> Have you tried this with > > dip1 = [dp - 0.01 if dp == 90 else dp for dp in dipList] > Yes that is better! many thanks! -- http://mail.python.org/mailman/listinfo/python-list
problems with CSV module
Hi all, I'm using csv to read text files, and its working fine, except in two cases: - when there is only one line of text (data) in the file - when there is a blank line after the last data line this is the kind of data: 45 67 89 23 45 06 12 34 67 ... and this is the function: def getData(paths): """get data from file and create lists with values and column names.""" filehandle = paths#[i] # filehandle = os.path.join(dirname, filename) csvfile = open(filehandle,'r') # Open the file and read the contents sample = csvfile.read( 1024 )# Grab a sample csvfile.seek( 0 ) dialect = csv.Sniffer().sniff(sample) # Check for file format with sniffer. csvfile = csv.reader( csvfile, dialect ) if csv.Sniffer().has_header( sample ): #if there is a header colnames = csvfile.next() # label columns from first line datalist = list( csvfile ) # append data to a list else: #if there is NO header datalist = list( csvfile ) # append data to a list colnames = ['col_%i' % i for i in range(len(datalist[0]))] # label columns as col_1, col_2, etc return datalist, colnames TIA for any help. -- http://mail.python.org/mailman/listinfo/python-list
Re: problems with CSV module
Thanks for your prompt response, Neil. > That data doesn't appear to be csv worthy. Why not use str.split > or str.partition? Well, I should have said that this is one kind of data. The function is part of a larger app, and there is the possibility that someone uses headers in the data files, or some other field separator, so I tried to make it more universal. > In Python 2.6 and earlier, you need to open the file in binary > mode. I tried that, no changes > Use: > csvfile = csv.reader(csvfile, dialect=dialect) > dialect is a keyword argument. thanks for pointing that out.it stopped the errors when there s only one data line, but it still can't get the values for that line Carlos -- http://mail.python.org/mailman/listinfo/python-list
speed of numpy.power()?
Hi all, I'd like to hear from you on the benefits of using numpy.power(x,y) over (x*x*x*x..) I looks to me that numpy.power takes more time to run. cheers Carlos -- http://mail.python.org/mailman/listinfo/python-list
Re: speed of numpy.power()?
On 25 ago, 12:40, David Cournapeau wrote: > On Wed, Aug 25, 2010 at 10:59 PM, Carlos Grohmann > Thanks David and Hrvoje. That was the feedback I was looking for. I am using numpy in my app but in some cases I will use math.pow(), as some tests with timeit showed that numpy.power was slower for (x*x*x*x*x). best Carlos -- http://mail.python.org/mailman/listinfo/python-list
help removing pyQt dll from dist created with py2exe
Hello all,
i've been trying to build an .exe with py2exe. After many tentatives,
it worked, but the total space used by the app goes to 30Mb. It is a
simple app, that uses wxpython, matplotlib and numpy. I checked the
library.zip file and notived that there is a pyQt-related file there:
Pyqt - QtGui.pyo - 8 Mb
I'm not using Qt at all, so I assume it would be safe to not have this
file, but I don't see how to do it.
my setup.py file follows.
many thanks
Carlos
#--
from distutils.core import setup
import py2exe
from glob import glob
# Remove the build folder, a bit slower but ensures that build
contains the latest
import shutil
shutil.rmtree("build", ignore_errors=True)
# my setup.py is based on one generated with gui2exe, so data_files is
done a bit differently
data_files = [("Microsoft.VC90.CRT", glob(r'c:\dev\*.*'))]
includes = ['wx', 'os', 'sys', 'csv', 're', 'floatspin',
'scrolledpanel', 'customtreectrl',
'wx.lib.expando', 'wx.lib.pubsub', 'wx.lib.embeddedimage',
'wx.lib.wordwrap', 'types',
'matplotlib', 'matplotlib.pyplot', 'matplotlib.axes',
'matplotlib.figure',
'matplotlib.backends.backend_wxagg',
'mpl_toolkits.axes_grid.axislines', 'mpl_toolkits.axes_grid',
'matplotlib.patches', 'matplotlib.lines',
'matplotlib.text', 'matplotlib.mlab', 'matplotlib.nxutils',
'matplotlib.collections', 'matplotlib.font_manager',
'numpy', 'numpy.ma', 'numpy.linalg', 'math', 'scipy.interpolate'
]
excludes = ['_gtkagg', '_tkagg', 'bsddb', 'curses', 'pywin.debugger',
'pywin.debugger.dbgcon', 'pywin.dialogs', 'tcl',
'Tkconstants', 'Tkinter', 'pydoc', 'doctest', 'test',
'sqlite3',
'bsddb', 'curses', 'email','_fltkagg', '_gtk',
'_gtkcairo',
'_agg2', '_cairo', '_cocoaagg',
'matplotlib.backends.backend_qt4agg','matplotlib.backends.backend_qt4'
]
packages = ['encodings','pytz','scipy']
dll_excludes = ['libgdk-win32-2.0-0.dll', 'libgobject-2.0-0.dll',
'tcl84.dll', 'tk84.dll',
'libgdk_pixbuf-2.0-0.dll', 'libgtk-win32-2.0-0.dll',
'libglib-2.0-0.dll',
'libcairo-2.dll', 'libpango-1.0-0.dll',
'libpangowin32-1.0-0.dll', 'libpangocairo-1.0-0.dll',
'libglade-2.0-0.dll', 'libgmodule-2.0-0.dll',
'libgthread-2.0-0.dll', 'QtGui4.dll', 'QtCore.dll',
'QtCore4.dll'
]
icon_resources = []
bitmap_resources = []
other_resources = []
# add the mpl mpl-data folder and rc file
import matplotlib as mpl
data_files += mpl.get_py2exe_datafiles()
setup(
windows=['OpenStereo.py'],
# compressed and optimize reduce the size
options = {"py2exe": {"compressed": 2,
"optimize": 2,
"includes": includes,
"excludes": excludes,
"packages": packages,
"dll_excludes": dll_excludes,
# using 2 to reduce number of files in dist
folder
# using 1 is not recommended as it often
does not work
"bundle_files": 2,
"dist_dir": 'dist',
"xref": False,
"skip_archive": False,
"ascii": False,
"custom_boot_script": '',
}
},
# using zipfile to reduce number of files in dist
zipfile = r'lib\library.zip',
data_files=data_files
)
--
http://mail.python.org/mailman/listinfo/python-list
Re: help removing pyQt dll from dist created with py2exe
many thanks Almar.
No more pyqt stuff in my dist.
cheers
carlos
On Tue, Sep 14, 2010 at 09:21, Almar Klein wrote:
> Hi,
>
> Have you tried adding "PyQt4", "PyQt4.QtGui" and "PyQt4.QtCore" to your list
> of excludes?
> (Maybe only "PyQt4.QtGui" is sufficient.)
>
> Almar
>
>
> On 14 September 2010 13:02, Carlos Grohmann
> wrote:
>>
>> Hello all,
>>
>> i've been trying to build an .exe with py2exe. After many tentatives,
>> it worked, but the total space used by the app goes to 30Mb. It is a
>> simple app, that uses wxpython, matplotlib and numpy. I checked the
>> library.zip file and notived that there is a pyQt-related file there:
>>
>> Pyqt - QtGui.pyo - 8 Mb
>>
>> I'm not using Qt at all, so I assume it would be safe to not have this
>> file, but I don't see how to do it.
>>
>> my setup.py file follows.
>>
>> many thanks
>>
>> Carlos
>>
>>
>> #--
>> from distutils.core import setup
>> import py2exe
>> from glob import glob
>>
>> # Remove the build folder, a bit slower but ensures that build
>> contains the latest
>> import shutil
>> shutil.rmtree("build", ignore_errors=True)
>>
>> # my setup.py is based on one generated with gui2exe, so data_files is
>> done a bit differently
>> data_files = [("Microsoft.VC90.CRT", glob(r'c:\dev\*.*'))]
>>
>> includes = ['wx', 'os', 'sys', 'csv', 're', 'floatspin',
>> 'scrolledpanel', 'customtreectrl',
>> 'wx.lib.expando', 'wx.lib.pubsub', 'wx.lib.embeddedimage',
>> 'wx.lib.wordwrap', 'types',
>> 'matplotlib', 'matplotlib.pyplot', 'matplotlib.axes',
>> 'matplotlib.figure',
>> 'matplotlib.backends.backend_wxagg',
>> 'mpl_toolkits.axes_grid.axislines', 'mpl_toolkits.axes_grid',
>> 'matplotlib.patches', 'matplotlib.lines',
>> 'matplotlib.text', 'matplotlib.mlab', 'matplotlib.nxutils',
>> 'matplotlib.collections', 'matplotlib.font_manager',
>> 'numpy', 'numpy.ma', 'numpy.linalg', 'math', 'scipy.interpolate'
>> ]
>>
>> excludes = ['_gtkagg', '_tkagg', 'bsddb', 'curses', 'pywin.debugger',
>> 'pywin.debugger.dbgcon', 'pywin.dialogs', 'tcl',
>> 'Tkconstants', 'Tkinter', 'pydoc', 'doctest', 'test',
>> 'sqlite3',
>> 'bsddb', 'curses', 'email','_fltkagg', '_gtk',
>> '_gtkcairo',
>> '_agg2', '_cairo', '_cocoaagg',
>> 'matplotlib.backends.backend_qt4agg','matplotlib.backends.backend_qt4'
>> ]
>>
>> packages = ['encodings','pytz','scipy']
>>
>> dll_excludes = ['libgdk-win32-2.0-0.dll', 'libgobject-2.0-0.dll',
>> 'tcl84.dll', 'tk84.dll',
>> 'libgdk_pixbuf-2.0-0.dll', 'libgtk-win32-2.0-0.dll',
>> 'libglib-2.0-0.dll',
>> 'libcairo-2.dll', 'libpango-1.0-0.dll',
>> 'libpangowin32-1.0-0.dll', 'libpangocairo-1.0-0.dll',
>> 'libglade-2.0-0.dll', 'libgmodule-2.0-0.dll',
>> 'libgthread-2.0-0.dll', 'QtGui4.dll', 'QtCore.dll',
>> 'QtCore4.dll'
>> ]
>>
>> icon_resources = []
>> bitmap_resources = []
>> other_resources = []
>>
>> # add the mpl mpl-data folder and rc file
>> import matplotlib as mpl
>> data_files += mpl.get_py2exe_datafiles()
>>
>> setup(
>> windows=['OpenStereo.py'],
>> # compressed and optimize reduce the size
>> options = {"py2exe": {"compressed": 2,
>> "optimize": 2,
>> "includes": includes,
>> "excludes": excludes,
>> "packages": packages,
>> "dll_excludes": dll_excludes,
>> # using 2 to reduce number of files in dist
>> folder
>> # using 1 is not recommended as it often
>> does not work
>> "bundle_files": 2,
>> "dist_dir": 'dist',
>> "xref": False,
>> "skip_archive": False,
>> "ascii": False,
>> "custom_boot_script": '',
>> }
>> },
>>
>> # using zipfile to reduce number of files in dist
>> zipfile = r'lib\library.zip',
>>
>> data_files=data_files
>> )
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
>
--
Prof. Carlos Henrique Grohmann - Geologist D.Sc.
Institute of Geosciences - Univ. of São Paulo, Brazil
http://www.igc.usp.br/pessoais/guano
http://lattes.cnpq.br/5846052449613692
Linux User #89721
Can’t stop the signal.
--
http://mail.python.org/mailman/listinfo/python-list
