Re: Berkely Db. How to iterate over large number of keys "quickly"

2007-08-02 Thread lazy
On Aug 2, 1:42 pm, Ian Clark <[EMAIL PROTECTED]> wrote: > lazy wrote: > > I have a berkely db and Im using the bsddb module to access it. The Db > > is quite huge (anywhere from 2-30GB). I want to iterate over the keys > > serially. > > I tried using something ba

Re: Berkely Db. How to iterate over large number of keys "quickly"

2007-08-02 Thread lazy
Sorry, Just a small correction, > a way I can tell Python to not load allkeys, but try to access it as > the loop progresses(like in a linked list). I could find any accessor > methonds on bsddb to this with my initial search. I meant, "I couldn't find any accessor methonds on bsddb to do this(i.

Berkely Db. How to iterate over large number of keys "quickly"

2007-08-02 Thread lazy
I have a berkely db and Im using the bsddb module to access it. The Db is quite huge (anywhere from 2-30GB). I want to iterate over the keys serially. I tried using something basic like for key in db.keys() but this takes lot of time. I guess Python is trying to get the list of all keys first and

huge dictionary -> bsddb/pickle question

2007-06-15 Thread lazy
Hi, I have a dictionary something like this, key1=>{key11=>[1,2] , key12=>[6,7] , } For lack of wording, I will call outer dictionary as dict1 and its value(inner dictionary) dict2 which is a dictionary of small fixed size lists(2 items) The key of the dictionary is a string and value is a

Re: url question - extracting (2 types of) domains

2007-05-16 Thread lazy
,.co.il etc ) and then I can find the root domain(significant_domain) atleast for those and I hope majority of them fall into this :) On May 16, 12:32 am, Michael Bentley <[EMAIL PROTECTED]> wrote: > On May 15, 2007, at 9:04 PM, lazy wrote: > > > > > Hi, > > Im trying to e

url question - extracting (2 types of) domains

2007-05-15 Thread lazy
Hi, Im trying to extract the domain name from an url. lets say I call it full_domain and significant_domain(which is the homepage domain) Eg: url=http://en.wikipedia.org/wiki/IPod , full_domain=en.wikipedia.org ,significant_domain=wikipedia.org Using urlsplit (of urlparse module), I will be able

Re: Newbie question about string(passing by ref)

2007-05-10 Thread lazy
PROTECTED]> wrote: > > > On May 10, 5:43 pm, lazy <[EMAIL PROTECTED]> wrote: > > > > I want to pass a string by reference. > > > Don't worry, all function parameters in Python are passed by reference. > > Actually, just to clarify a little bit if y

Newbie question about string(passing by ref)

2007-05-10 Thread lazy
Hi, I want to pass a string by reference. I understand that strings are immutable, but Im not going to change the string in the function, just to aviod the overhead of copying(when pass-by-value) because the strings are long and this function will be called over and over again. I initially thought