Re: optimizing large dictionaries

2009-01-15 Thread Per Freem
thanks to everyone for the excellent suggestions. a few follow up q's: 1] is Try-Except really slower? my dict actually has two layers, so my_dict[aKey][bKeys]. the aKeys are very small (less than 100) where as the bKeys are the ones that are in the millions. so in that case, doing a Try-Except o

optimizing large dictionaries

2009-01-15 Thread Per Freem
hello i have an optimization questions about python. i am iterating through a file and counting the number of repeated elements. the file has on the order of tens of millions elements... i create a dictionary that maps elements of the file that i want to count to their number of occurs. so i iter

Re: efficient interval containment lookup

2009-01-12 Thread Per Freem
cy so much... thanks. On Jan 13, 12:24 am, brent wrote: > On Jan 12, 8:55 pm, Per Freem wrote: > > > > > On Jan 12, 10:58 pm, Steven D'Aprano > > > wrote: > > > On Mon, 12 Jan 2009 14:49:43 -0800, Per Freem wrote: > > > > thanks for your repli

Re: efficient interval containment lookup

2009-01-12 Thread Per Freem
i forgot to add, my naive_find is: def naive_find(intervals, start, stop): results = [] for interval in intervals: if interval.start >= start and interval.stop <= stop: results.append(interval) return results On Jan 12, 11:55 pm, Per Freem wrote: > On Jan 12, 10:58 p

Re: efficient interval containment lookup

2009-01-12 Thread Per Freem
On Jan 12, 10:58 pm, Steven D'Aprano wrote: > On Mon, 12 Jan 2009 14:49:43 -0800, Per Freem wrote: > > thanks for your replies -- a few clarifications and questions. the > > is_within operation is containment, i.e. (a,b) is within (c,d) iff a > >>= c and b <= d

Re: efficient interval containment lookup

2009-01-12 Thread Per Freem
thanks for your replies -- a few clarifications and questions. the is_within operation is containment, i.e. (a,b) is within (c,d) iff a >= c and b <= d. Note that I am not looking for intervals that overlap... this is why interval trees seem to me to not be relevant, as the overlapping interval pro

efficient interval containment lookup

2009-01-12 Thread Per Freem
hello, suppose I have two lists of intervals, one significantly larger than the other. For example listA = [(10, 30), (5, 25), (100, 200), ...] might contain thousands of elements while listB (of the same form) might contain hundreds of thousands or millions of elements. I want to count how many i