Each time you close indexWriter ram cached documents are flushed to
disk. In case you open and close it per document many one-document
segments are created on disk.
minMergeDocs sets number of documents that are cached in ram.
JM Tinghir wrote:
do you keep your indexWriter open all the time
> do you keep your indexWriter open all the time during process?
I think that might be the real cause. And as it reopen it all the
time, the mergeFactor isn't used at all I guess...
I'll try to modify that.
Thanks.
Jean-Marie Tinghir
-
can you measure "pure" index creation time (without creating XMLs)
and one more question:
do you keep your indexWriter open all the time during process?
JM Tinghir wrote:
Well, it just took 145 minutes to index 2670 files (450 MB) in one
index (29 MB).
It only took 33 minutes when I did it int
I contest to the value of increasing the minMergeDocs.it directly effects
how much IO gets performed in indexing.
Splitting it into multiple indices (if you want to pay the price of
complexity), may well increase your throughput. Assuming you are not utilizing
all of the resources the sys
my previous message lost somewhere :(
reposting
can you measure "pure" index creation time (without creating XMLs)
and one more question:
do you keep your indexWriter open all the time during process?
best way to determine bottlenecks is profiling :)
regards,
Volodymyr Bychkoviak
JM Tinghir w
lease respond to Re: Performance with multi index
[EMAIL PROTECTED]
> Well, it just took 145 minutes to index 2670 files (450 MB) in one
> index (29 MB).
> It only took 33 minutes when I did it into ~10 indexes (global size of 32 MB).
Forgot to add, that it does not only indexes files, it also creates
XML documents. So don't worry if takes 30 minutes to index 450
JM Tinghir wrote:
Could you qualify a bit more about what is slow?
Well, it just took 145 minutes to index 2670 files (450 MB) in one
index (29 MB).
It only took 33 minutes when I did it into ~10 indexes (global size of 32 MB).
I think it took so much time, because it's merged too
> Could you qualify a bit more about what is slow?
Well, it just took 145 minutes to index 2670 files (450 MB) in one
index (29 MB).
It only took 33 minutes when I did it into ~10 indexes (global size of 32 MB).
> Perhaps you need to optimize the index?
Perhaps, never tried it...
JM
--
On Jun 16, 2005, at 4:08 AM, JM Tinghir wrote:
I have a 25 Mb index and was wondering if it would be better to divide
it in about 10 indexes and search in it with MutliSearcher.
Would searching be faster this way?
The indexing would be faster I guess, as it is getting slower and
slower while ind
Hi,
I have a 25 Mb index and was wondering if it would be better to divide
it in about 10 indexes and search in it with MutliSearcher.
Would searching be faster this way?
The indexing would be faster I guess, as it is getting slower and
slower while indexes get bigger.
But searching?
Jean-Marie
11 matches
Mail list logo
