Re: Keeping track of the N largest values

2010-12-31 Thread Dan Stromberg
On Sat, Dec 25, 2010 at 7:42 AM, Roy Smith wrote: > I'm processing a stream of N numbers and want to keep track of the K > largest.  There's too many numbers in the stream (i.e. N is too large) > to keep in memory at once.  K is small (100 would be typical). > > >From a theoretical point of view,

Re: Keeping track of the N largest values

2010-12-28 Thread Miki
> I'm processing a stream of N numbers and want to keep track of the K > largest. There's too many numbers in the stream (i.e. N is too large) > to keep in memory at once. K is small (100 would be typical). > ... deque can be bounded by maxsize, might fit the bill: >>> from collections imp

Re: Keeping track of the N largest values

2010-12-27 Thread Stefan Sonnenberg-Carstens
Am 26.12.2010 19:51, schrieb Stefan Sonnenberg-Carstens: l = [] K = 10 while 1: a = input() if len(l) == K: l.remove(min(l)) l=[x for x in l if x < a] + [a] + [x for x in l if x > a] print l A minor fault made it into my prog: l = [0] K = 10 while 1: a = input()

Re: Keeping track of the N largest values

2010-12-26 Thread Stefan Sonnenberg-Carstens
Am 25.12.2010 16:42, schrieb Roy Smith: I'm processing a stream of N numbers and want to keep track of the K largest. There's too many numbers in the stream (i.e. N is too large) to keep in memory at once. K is small (100 would be typical). > From a theoretical point of view, I should be able

Re: Keeping track of the N largest values

2010-12-26 Thread Roy Smith
In article , n00m wrote: > from bisect import insort_left > > K = 5 > top = [] > while 1: > x = input() > if len(top) < K: > insort_left(top, x) > elif x > top[0]: > del top[0] > insort_left(top, x) > print top > > > will be enough Hmmm, that's an int

Re: Keeping track of the N largest values

2010-12-26 Thread n00m
from bisect import insort_left K = 5 top = [] while 1: x = input() if len(top) < K: insort_left(top, x) elif x > top[0]: del top[0] insort_left(top, x) print top will be enough -- http://mail.python.org/mailman/listinfo/python-list

Re: Keeping track of the N largest values

2010-12-26 Thread Peter Otten
Roy Smith wrote: > In article , > Duncan Booth wrote: > >> Roy Smith wrote: >> >> > >> > I'm processing a stream of N numbers and want to keep track of the K >> > largest. There's too many numbers in the stream (i.e. N is too large) >> > to keep in memory at once. K is small (100 would be

Re: Keeping track of the N largest values

2010-12-25 Thread John Nagle
On 12/25/2010 8:04 AM, Peter Otten wrote: Roy Smith wrote: I'm processing a stream of N numbers and want to keep track of the K largest. There's too many numbers in the stream (i.e. N is too large) to keep in memory at once. K is small (100 would be typical). http://docs.python.org/library/

Re: Keeping track of the N largest values

2010-12-25 Thread Paul Rubin
Roy Smith writes: >> from heapq import nlargest >> top = nlargest(K, input()) > In addition to finding the K largest values, I *also* need to do some > other processing on all the values The problem with nlargest() > is that it doesn't give me a hook to do that. def processed_input

Re: Keeping track of the N largest values

2010-12-25 Thread Roy Smith
In article , Duncan Booth wrote: > Roy Smith wrote: > > > > > I'm processing a stream of N numbers and want to keep track of the K > > largest. There's too many numbers in the stream (i.e. N is too large) > > to keep in memory at once. K is small (100 would be typical). > > ... > > Is the

Re: Keeping track of the N largest values

2010-12-25 Thread Robert Kern
On 12/25/10 10:42 AM, Roy Smith wrote: I'm processing a stream of N numbers and want to keep track of the K largest. There's too many numbers in the stream (i.e. N is too large) to keep in memory at once. K is small (100 would be typical). From a theoretical point of view, I should be able to

Re: Keeping track of the N largest values

2010-12-25 Thread Duncan Booth
Roy Smith wrote: > > I'm processing a stream of N numbers and want to keep track of the K > largest. There's too many numbers in the stream (i.e. N is too large) > to keep in memory at once. K is small (100 would be typical). > ... > Is there a better way to do this, either from a theoretica

Re: Keeping track of the N largest values

2010-12-25 Thread Peter Otten
Roy Smith wrote: > I'm processing a stream of N numbers and want to keep track of the K > largest. There's too many numbers in the stream (i.e. N is too large) > to keep in memory at once. K is small (100 would be typical). > > From a theoretical point of view, I should be able to do this in N

Keeping track of the N largest values

2010-12-25 Thread Roy Smith
I'm processing a stream of N numbers and want to keep track of the K largest. There's too many numbers in the stream (i.e. N is too large) to keep in memory at once. K is small (100 would be typical). >From a theoretical point of view, I should be able to do this in N log K time. What I'm do