Re: InstantiatedIndex help + first impression

2008-11-18 Thread karl wettin
On Wed, Nov 19, 2008 at 3:27 AM, karl wettin <[EMAIL PROTECTED]> wrote: > rewritten query. I.e. this is probably as much a store related expense > as it is a Levenshtein calculation expense. "this is probably *not* as much a store related.." that is. karl ---

Re: InstantiatedIndex help + first impression

2008-11-18 Thread karl wettin
The actual performance depends on how much you load to the index. Can you tell us how many documents and how large these documents are that you have in your index? Compared with RAMDirectory I'vee seen performance boosts of up to 100x in a small index that contains (1-20) Wikipedia sized document

Re: InstantiatedIndex help + first impression

2008-11-16 Thread Darren Govoni
After I switched to InstantiatedIndex from RAMDirectory (but using the reader from my RAMDirectory to create the InstantiatedIndex), I see a less than 25% (.25) improvement in speed. Nowhere near the 100x (100.00) speed mentioned in the documentation. Probably I am doing something wrong. I am usi

Re: InstantiatedIndex help

2008-11-16 Thread Darren Govoni
Yeah. That makes sense. Its not too hard to wrap those extra steps so I can end up with something simpler too. Like: iindex = InstantiatedIndex("path/to/my/index") I'm lazy so the intermediate hoops to jump through clutter my code. Hehe. :) Darren On Sun, 2008-11-16 at 11:46 -0500, Mark Miller

Re: InstantiatedIndex help

2008-11-16 Thread Mark Miller
Can you start with an empty index? Then how about: // Adding these iindex = InstantiatedIndex() ireader = iindex.indexReaderFactory() isearcher = IndexSearcher(ireader) If you want a copy from another IndexReader though, you have to get that reader from somewhere right? - Mark D

Re: InstantiatedIndex help

2008-11-16 Thread Darren Govoni
Hi Mark, Thanks for the tips. Here's what I will try (psuedo-code) endirectory = RAMDirectory("index/dictionary.en") ensearcher = IndexSearcher(endirectory) // Adding these reader = ensearcher.getIndexReader() iindex = InstantiatedIndex(reader) ireader = iindex.indexReade

Re: InstantiatedIndex help

2008-11-16 Thread Mark Miller
Check out the docs at: http://lucene.apache.org/java/2_4_0/api/contrib-instantiated/index.html There is a performance graph there to check out. The code should be fairly straightforward - you can make an InstantiatedIndex thats empty, or seed it with an IndexReader. Then you can make an Inst

InstantiatedIndex help

2008-11-16 Thread Darren Govoni
Hi gang, I am trying to trace the 2.4 API to create an InstantiatedIndex, but its rather difficult to connect directory,reader,search,index etc just reading the javadocs. I have a (POI - plain old index) directory already and want to create a faster InstantiatedIndex and IndexSearcher to q