Re: efficient text file search -solution

2006-09-12 Thread Sion Arrowsmith
noro <[EMAIL PROTECTED]> wrote: >OK, am not sure why, but > >fList=file('somefile').read() >if fList.find('string') != -1: > print 'FOUND' > >works much much faster. > >it is strange since i thought 'for line in file('somefile')' is >optemized and read pages to the memory, Step back and think ab

efficient text file search -solution

2006-09-11 Thread noro
OK, am not sure why, but fList=file('somefile').read() if fList.find('string') != -1: print 'FOUND' works much much faster. it is strange since i thought 'for line in file('somefile')' is optemized and read pages to the memory, i guess not.. George Sakkis wrote: > noro wrote: > > > Is there

Re: efficient text file search.

2006-09-11 Thread George Sakkis
noro wrote: > Is there a more efficient method to find a string in a text file then: > > f=file('somefile') > for line in f: > if 'string' in line: > print 'FOUND' > > ? Is this something you want to do only once for a given file ? The replies so far seem to imply so and in this case

Re: efficient text file search.

2006-09-11 Thread noro
i'm not sure. each line in the text file and an index string. i can sort the file, and use some binary tree search on it. (I need to do a number of searchs). there are 1219137 indexs in the file. so maby a memory efficient sort algorithm is in place. how can mmap help me? is there any fbinary sear

Re: efficient text file search.

2006-09-11 Thread Luuk
"John Machin" <[EMAIL PROTECTED]> schreef in bericht news:[EMAIL PROTECTED] > > Luuk wrote: > [snip] >> some googling turned op the following. >> Second paragraph of chapter 14 of http://www.amk.ca/python/2.1/ > [snip] >> For a fuller discussion of the line I/O changes, see the python-dev >> sum

Re: efficient text file search.

2006-09-11 Thread Steve Holden
noro wrote: > Bill Scherer wrote: > >>noro wrote: >> >> >>>Is there a more efficient method to find a string in a text file then: >>> >>>f=file('somefile') >>>for line in f: >>> if 'string' in line: >>>print 'FOUND' >>> >>>? >>> >>>BTW: >>>does "for line in f: " read a block of line to t

Re: efficient text file search.

2006-09-11 Thread noro
can you add some more info, or point me to a link, i havn't found anything about binary search in mmap() in python documents. the files are very big... thanks amit Bill Scherer wrote: > noro wrote: > > >Is there a more efficient method to find a string in a text file then: > > > >f=file('somefile

Re: efficient text file search.

2006-09-11 Thread John Machin
Luuk wrote: [snip] > some googling turned op the following. > Second paragraph of chapter 14 of http://www.amk.ca/python/2.1/ [snip] > For a fuller discussion of the line I/O changes, see the python-dev summary > for January 1-15, 2001 at http://www.amk.ca/python/dev/2001-01-1.html. That is *HIST

Re: efficient text file search.

2006-09-11 Thread Ant
noro wrote: > Is there a more efficient method to find a string in a text file then: > > f=file('somefile') > for line in f: > if 'string' in line: > print 'FOUND' break ^^^ Add a 'break' after the print statement - that way you won't have to read the entire

Re: efficient text file search.

2006-09-11 Thread Bill Scherer
noro wrote: >Is there a more efficient method to find a string in a text file then: > >f=file('somefile') >for line in f: >if 'string' in line: > print 'FOUND' > >? > >BTW: >does "for line in f: " read a block of line to te memory or is it >simply calls f.readline() many times? > >than

Re: efficient text file search.

2006-09-11 Thread Kent Johnson
noro wrote: > Is there a more efficient method to find a string in a text file then: > > f=file('somefile') > for line in f: > if 'string' in line: > print 'FOUND' Probably better to read the whole file at once if it isn't too big: f = file('somefile') data = f.read() if 'string' in

Re: efficient text file search.

2006-09-11 Thread Luuk
"noro" <[EMAIL PROTECTED]> schreef in bericht news:[EMAIL PROTECTED] > :) > > via python... > > Luuk wrote: >> "noro" <[EMAIL PROTECTED]> schreef in bericht >> news:[EMAIL PROTECTED] >> > Is there a more efficient method to find a string in a text file then: >> > >> > f=file('somefile') >> > for

Re: efficient text file search.

2006-09-11 Thread noro
:) via python... Luuk wrote: > "noro" <[EMAIL PROTECTED]> schreef in bericht > news:[EMAIL PROTECTED] > > Is there a more efficient method to find a string in a text file then: > > > > f=file('somefile') > > for line in f: > >if 'string' in line: > > print 'FOUND' > > > > > yes, more

Re: efficient text file search.

2006-09-11 Thread Luuk
"noro" <[EMAIL PROTECTED]> schreef in bericht news:[EMAIL PROTECTED] > Is there a more efficient method to find a string in a text file then: > > f=file('somefile') > for line in f: >if 'string' in line: > print 'FOUND' > yes, more efficient would be: grep (http://www.gnu.org/softwa

efficient text file search.

2006-09-11 Thread noro
Is there a more efficient method to find a string in a text file then: f=file('somefile') for line in f: if 'string' in line: print 'FOUND' ? BTW: does "for line in f: " read a block of line to te memory or is it simply calls f.readline() many times? thanks amit -- http://mail.py