Re: performance implications for an index with large number of
documents.
hi,
Thank you for all the quick and pertinent responses.
The index is being optimized every hour due to the number of updates.
The JVM has a heap of 2gig and the machine has a total of 4.
Currently the JVM is not configu
hi,
Thank you for all the quick and pertinent responses.
The index is being optimized every hour due to the number of updates.
The JVM has a heap of 2gig and the machine has a total of 4.
Currently the JVM is not configured with -server parameter and the
parallel garbage collection (we are test
Hi Ori,
Before taking drastic rehosting measures, and introducing the associated
software complexity off splitting your application into pieces running
on separate machines, I'd recommend looking at the way your document
data is distributed and the way you're searching them. Here are some
qu
How much RAM do you have? If you're under linux, can you run
something like "iostat -x -d -t 60" and watch your disk usage during
searching? If your disk utilization is high, add more RAM (enough to
hold your index in RAM) and see if the OS cache solves the problem. I
would try this before the c
Hi,
Apologies if this question has being asked before on this list.
I am working on an application with a Lucene index whose performance
(response time for a query) has started degrading as its size has
increase.
The index is made up of approximately 10 million documents that have
11 fields. Th
:
: The index is made up of approximately 10 million documents that have
: 11 fields. The average document size is less then 1k. The index has
: a total of 13 million terms. The total index size is about 2.2 gig.
: The index is being updated relatively aggressively. In a 24hr period
: there may
Lucene scales with the number of unique terms in the index and not the
number of documents nor the size of the documents. Typically, you
should have at most 1 million unique terms for a set of 10 million
documents.
So the fact that you have 13 million unique terms in 10 million
documents tells me
Hi,
Apologies if this question has being asked before on this list.
I am working on an application with a Lucene index whose performance
(response time for a query) has started degrading as its size has
increase.
The index is made up of approximately 10 million documents that have
11 fields. Th
Quite feasible and you'll be very glad you picked Lucene :)
Erik
On Mar 23, 2005, at 8:02 PM, [EMAIL PROTECTED] wrote:
Hi,
We are evaluating to use Lucence as one search engine for almost
100,000 documents (each will be less than 10,000 words)collections, is
it feasible or no sense at all
Hi,
We are evaluating to use Lucence as one search engine for almost 100,000
documents (each will be less than 10,000 words)collections, is it feasible or
no sense at all? someone has similar experience before?
Many thanks for suggestions,
George
10 matches
Mail list logo
