Re: Slow down while creating a big list and iterating over it

2010-01-31 Thread MRAB
marc magrans de abril wrote: Hi! ...I have found a good enough solution, although it only works if the number of patterns (clusters) is not very big: def classify(f): THERESHOLD=0.1 patterns={} for l in enumerate(f): found = False for p,c in patterns.items():

Re: Slow down while creating a big list and iterating over it

2010-01-31 Thread marc magrans de abril
Hi! ...I have found a good enough solution, although it only works if the number of patterns (clusters) is not very big: def classify(f): THERESHOLD=0.1 patterns={} for l in enumerate(f): found = False for p,c in patterns.items(): if dist(l,p) < THERESHOLD:

Re: Slow down while creating a big list and iterating over it

2010-01-31 Thread marc magrans de abril
>Find out which pattern is being used on the second iteration and then try it >on the first iteration. Is it just as slow? You were right, the second pattern was 1891 bytes but the first was just 142 :P I will need to put more thought than I expect in the "small script". -- http://mail.python.or

Re: Slow down while creating a big list and iterating over it

2010-01-31 Thread Aliaksandr Abushkevich
Maybe it is a good idea to use Disco (http://discoproject.org/) to process your data. Yours faithfully, Alexander Abushkevich On Sat, Jan 30, 2010 at 10:36 PM, marc magrans de abril wrote: > Dear colleagues, > > I was doing a small program to classify log files for a cluster of > PCs, I just w

Re: Slow down while creating a big list and iterating over it

2010-01-30 Thread MRAB
Alf P. Steinbach wrote: * marc magrans de abril: Dear colleagues, I was doing a small program to classify log files for a cluster of PCs, I just wanted to simplify a quite repetitive task in order to find errors and so. My first naive implementation was something like: patterns = [] wh

Re: Slow down while creating a big list and iterating over it

2010-01-30 Thread Alf P. Steinbach
* marc magrans de abril: Dear colleagues, I was doing a small program to classify log files for a cluster of PCs, I just wanted to simplify a quite repetitive task in order to find errors and so. My first naive implementation was something like: patterns = [] while(logs): patter

Slow down while creating a big list and iterating over it

2010-01-30 Thread marc magrans de abril
Dear colleagues, I was doing a small program to classify log files for a cluster of PCs, I just wanted to simplify a quite repetitive task in order to find errors and so. My first naive implementation was something like: patterns = [] while(logs): pattern = logs[0] new_log