Re: fast method accessing large, simple structured data

2008-02-03 Thread Stefan Behnel
agc wrote: > I guess an important feature of what I'm looking for is > some kind of mapping from *exact* title to corresponding article, > i.e. if my data set wasn't so large, I would just keep all my > data in a in-memory Python dictionary, which would be very fast. > > But I have about 2 million

Re: fast method accessing large, simple structured data

2008-02-03 Thread Ivan Illarionov
> Is there some good format that is optimized for search for > just 1 attribute (title) and then returning the corresponding article? I would use Durus (http://www.mems-exchange.org/software/durus/) - simple pythonic object database - and store this data as persistent python dict with Title keys a

Re: fast method accessing large, simple structured data

2008-02-02 Thread agc
On Feb 2, 1:50 pm, John Machin <[EMAIL PROTECTED]> wrote: > agc wrote: > > Hi, > > > I'm looking for a fast way of accessing some simple (structured) data. > > > The data is like this: > > Approx 6 - 10 GB simple XML files with the only elements > > I really care about are the and ones. > > > So

Re: fast method accessing large, simple structured data

2008-02-02 Thread M.-A. Lemburg
On 2008-02-02 21:36, agc wrote: > Hi, > > I'm looking for a fast way of accessing some simple (structured) data. > > The data is like this: > Approx 6 - 10 GB simple XML files with the only elements > I really care about are the and ones. > > So what I'm hoping to do is put this data in a form

Re: fast method accessing large, simple structured data

2008-02-02 Thread John Machin
agc wrote: > Hi, > > I'm looking for a fast way of accessing some simple (structured) data. > > The data is like this: > Approx 6 - 10 GB simple XML files with the only elements > I really care about are the and ones. > > So what I'm hoping to do is put this data in a format so > that I can ac

Re: fast method accessing large, simple structured data

2008-02-02 Thread Diez B. Roggisch
agc schrieb: > Hi, > > I'm looking for a fast way of accessing some simple (structured) data. > > The data is like this: > Approx 6 - 10 GB simple XML files with the only elements > I really care about are the and ones. > > So what I'm hoping to do is put this data in a format so > that I can