Re: Big problem with big indexes

2006-10-17 Thread karl wettin
17 okt 2006 kl. 15.55 skrev Ariel Isaac Romero Cartaya: Here are pieces of my source code: public Hits search(String query) throws IOException { for (int i = 0; i < IndexCount; i++) { searchables[i] = new IndexSearcher (RAMIndexsManager.getInstance ().getDir

Re: Big problem with big indexes

2006-10-17 Thread Ariel Isaac Romero Cartaya
Here are pieces of my source code: First of all, I search in all the indexes given a query String with a parallel searcher. As you can see I make a multi field query. Then you can see the index format I use, I store in the index all the fields. My index is optimized. public Hits search

Re: Big problem with big indexes

2006-10-16 Thread Ariel Isaac Romero Cartaya
First af all, what is your machine architecture ??? Do you have a super pc ??? I'm running this on a dual xeon hyperthreading 2,4 Ghz, 1 Gb RAM, HD SATA. I Can not get the times results you get. I think that the problem may be in the structure of my index, for example I use a special analyzer fo

Re: Big problem with big indexes

2006-10-11 Thread Doron Cohen
These times really are not reasonable. But 60K do not seem much for Lucene. I once indexed ~1M docs of ~20K each, that's ~20GB input collection. The result index size was ~2.5GB and the search times for a short query 2-3 words free text (or) query was ~300ms for a "hot" query and ~900ms for a "cold

Re: Big problem with big indexes

2006-10-11 Thread Erick Erickson
Something's extremely not right First of all, I'm running a 1.4G index on a single machine and getting very good results, under 10 seconds even for the most complex queries I'm firing. This is with 870,000 documents, and includes sorting by criteria other than relevance. And using span queri

Big problem with big indexes

2006-10-11 Thread Ariel Isaac Romero Cartaya
Hi everybody: I have a big problem making prallel searches in big indexes. I have indexed with lucene over 60 000 articles, I have distributed the indexes in 10 computers nodes so each index not exceed the 60 MB of size. I makes parallel searches in those indexes but I get the search resu