Re: speed of string chunks file parsing

2009-04-06 Thread John Machin
On Apr 6, 11:48 pm, Hyunchul Kim wrote: > Hi, all > > I have a simple script. > Can you improve algorithm of following 10 line script, with a view point > of speed ? > Following script do exactly what I want but I want to improve the speed. So the first thing to do is to try to work out where it

Re: speed of string chunks file parsing

2009-04-06 Thread andrew cooke
[disclaimer - this is just guessing from general knowledge of regular expressions; i don't know any details of python's regexp engine] if your regular expression is the bottleneck rewrite it to avoid lazy matching, references, groups, lookbacks, and perhaps even counted repeats. with a little th

Re: speed of string chunks file parsing

2009-04-06 Thread bearophileHUGS
bearophile: >     cp_regular_expression = re.compile("^a complex regular expression > here$") >     for line in file(inputfile): >         if cp_regular_expression.match(line) and result_seq: Sorry, you can replace that with: cp_regular_expression = re.compile("^a complex regular expression h

Re: speed of string chunks file parsing

2009-04-06 Thread bearophileHUGS
Hyunchul Kim: > Following script do exactly what I want but I want to improve the speed. This may be a bit faster, especially if sequences are long (code untested): import re from collections import deque def scanner1(deque=deque): result_seq = deque() cp_regular_expression = re.compile

Re: speed of string chunks file parsing

2009-04-06 Thread MRAB
Hyunchul Kim wrote: Hi, all I have a simple script. Can you improve algorithm of following 10 line script, with a view point of speed ? Following script do exactly what I want but I want to improve the speed. This parse a file and accumulate lines till a line match a given regular expression

speed of string chunks file parsing

2009-04-06 Thread Hyunchul Kim
Hi, all I have a simple script. Can you improve algorithm of following 10 line script, with a view point of speed ? Following script do exactly what I want but I want to improve the speed. This parse a file and accumulate lines till a line match a given regular expression. Then, when a line m