Re: preallocate list

2005-04-14 Thread Jim
John Machin wrote: On Wed, 13 Apr 2005 14:28:51 +0100, Jim <[EMAIL PROTECTED]> wrote: Thanks for the suggestions. I guess I must ensure that this is my bottle neck. def readFactorsIntoList(self,filename,numberLoads): 1. "numberLoads" is not used. factors = [] f = open(self.

Re: preallocate list

2005-04-13 Thread John Machin
On Wed, 13 Apr 2005 14:28:51 +0100, Jim <[EMAIL PROTECTED]> wrote: >Thanks for the suggestions. I guess I must ensure that this is my bottle >neck. > > def readFactorsIntoList(self,filename,numberLoads): 1. "numberLoads" is not used. > factors = [] > f = open(self.basedir + f

Re: preallocate list

2005-04-13 Thread Dan Christensen
Bill Mill <[EMAIL PROTECTED]> writes: > Bill Mill <[EMAIL PROTECTED]> writes: > >> I would profile your app to see that it's your append which is taking >> ages, but to preallocate a list of strings would look like: >> >> ["This is an average length string" for i in range(approx_length)] I don't

Re: preallocate list

2005-04-13 Thread Jim
ivec = n*[None] so that if I use a list element before intializing it, for example ivec[0] += 1 I get an error message File "xxnone.py", line 2, in ? ivec[0] += 1 TypeError: unsupported operand type(s) for +=: 'NoneType' and 'int' This is in the same spirit as Python's (welcome) termination

Re: preallocate list

2005-04-13 Thread Jim
Steven Bethard wrote: Jim wrote: What I really want is a Numeric array but I don't think Numeric supports importing files. Hmmm... Maybe the scipy package? I think scipy.io.read_array might help, but I've never used it. STeVe Sounds promising. I only got Numeric because I wanted scipy but I've h

Re: preallocate list

2005-04-13 Thread Jim
F. Petitjean wrote: Le Wed, 13 Apr 2005 16:46:53 +0100, Jim a écrit : What I really want is a Numeric array but I don't think Numeric supports importing files. Numeric arrays can be serialized from/to files through pickles : import Numeric as N help(N.load) help(N.dump) (and it is space efficient)

Re: preallocate list

2005-04-13 Thread beliavsky
Jim wrote: > Hi all > > Is this the best way to preallocate a list of integers? > listName = range(0,length) For serious numerical work you should use Numeric or Numarray, as others suggested. When I do allocate lists the initial values 0:n-1 are rarely what I want, so I use ivec = n*[None] so t

Re: preallocate list

2005-04-13 Thread F. Petitjean
Le Wed, 13 Apr 2005 16:46:53 +0100, Jim a écrit : > > What I really want is a Numeric array but I don't think Numeric supports > importing files. Numeric arrays can be serialized from/to files through pickles : import Numeric as N help(N.load) help(N.dump) (and it is space efficient) > > Jim --

Re: preallocate list

2005-04-13 Thread Steven Bethard
Jim wrote: What I really want is a Numeric array but I don't think Numeric supports importing files. Hmmm... Maybe the scipy package? I think scipy.io.read_array might help, but I've never used it. STeVe -- http://mail.python.org/mailman/listinfo/python-list

Re: preallocate list

2005-04-13 Thread Jim
Steven Bethard wrote: Jim wrote: .. OK. I've just tried with 4 lines and the code works. With 11000 lines it uses all CPU for at least 30 secs. There must be a better way. Was your test on *just* this function? Or were you doing something with the list produced by this function as well? STeVe

Re: preallocate list

2005-04-13 Thread Jim
[EMAIL PROTECTED] wrote: what about : factors = [map(float, line.split()) for line in file] should be a hell of a lot faster and nicer. for line in f: factor = [] tokens = line.split() for i in tokens: factor.appen

Re: preallocate list

2005-04-13 Thread Jim
Steven Bethard wrote: Jim wrote: Thanks for the suggestions. I guess I must ensure that this is my bottle neck. def readFactorsIntoList(self,filename,numberLoads): factors = [] f = open(self.basedir + filename,'r') line = f.readline() tokens = line.split()

Re: preallocate list

2005-04-13 Thread Steven Bethard
Jim wrote: Thanks for the suggestions. I guess I must ensure that this is my bottle neck. def readFactorsIntoList(self,filename,numberLoads): factors = [] f = open(self.basedir + filename,'r') line = f.readline() tokens = line.split() columns = len(tokens)

Re: preallocate list

2005-04-13 Thread peufeu
what about : factors = [map(float, line.split()) for line in file] should be a hell of a lot faster and nicer. for line in f: factor = [] tokens = line.split() for i in tokens: factor.append(float(i))

Re: preallocate list

2005-04-13 Thread Mike C. Fletcher
Jim wrote: > Thanks for the suggestions. I guess I must ensure that this is my > bottle neck. ... > for line in f: > factor = [] > tokens = line.split() > for i in tokens: > factor.append(float(i)) >

Re: preallocate list

2005-04-13 Thread Jim
Thanks for the suggestions. I guess I must ensure that this is my bottle neck. def readFactorsIntoList(self,filename,numberLoads): factors = [] f = open(self.basedir + filename,'r') line = f.readline() tokens = line.split() columns = len(tokens) if int

Re: preallocate list

2005-04-13 Thread Bill Mill
Just a correction: > I would profile your app to see that it's your append which is taking > ages, but to preallocate a list of strings would look like: > > ["This is an average length string" for i in range(approx_length)] > > My guess is that it won't help to preallocate, but time it and let

Re: preallocate list

2005-04-13 Thread Jim
rbt wrote: Jim wrote: If I have a file with a floating point number on each line, what is the best way of reading them into a list (or other ordered structure)? I was iterating with readline and appending to a list but it is taking ages. Perhaps you should use readlines (notice the s) instead o

Re: preallocate list

2005-04-13 Thread Bill Mill
On 4/13/05, Jim <[EMAIL PROTECTED]> wrote: > Hi all > > Is this the best way to preallocate a list of integers? > listName = range(0,length) > the 0 is unnecessary; range(length) does the same thing. > What about non integers? > arr = [myobject() for i in range(length)] > I've just claimed i

Re: preallocate list

2005-04-13 Thread rbt
Jim wrote: If I have a file with a floating point number on each line, what is the best way of reading them into a list (or other ordered structure)? I was iterating with readline and appending to a list but it is taking ages. Perhaps you should use readlines (notice the s) instead of readline.