Hi!
After upgrading ES cluster from 6.2 to 7.9 version, we find that force merge
operation will take long time, about double of previous latency.
Based on our investigation, we found the follows is main cause of the
force-merge performance decrease:
* From Lucene 8.0, NormsProducer is
Hi!
After upgrading ES cluster from 6.2 to 7.9 version, we find that force merge
operation will take long time, about double of previous latency.
Based on our investigation, we found the follows is main cause of the
force-merge performance decrease:
* From Lucene 8.0, NormsProducer is added as
Hi!
After upgrading ES cluster from 6.2 to 7.9 version, we find that force merge
operation will take long time, about double of previous latency.
Based on our investigation, we found the follows is main cause of the
force-merge performance decrease:
* From Lucene 8.0, NormsProducer is
On Wed, Sep 30, 2015 at 7:41 PM, McKinley, James T
wrote:
> We really don't have the option of moving to local disk without a significant
> redesign of our systems. However, we do have the possibility of switching to
> iSCSI instead of NFS without changing our hardware, do you happen to know
iSCSI instead of NFS without changing our hardware, do you happen to know
whether iSCSI would be a better protocol for use with Lucene? Thanks!
Jim
From: will martin
Sent: 30 September 2015 06:49
To: java-user@lucene.apache.org
Subject: RE: Lucene 5 : a
Thanks Mike. This is very informative.
-Original Message-
From: Michael McCandless [mailto:luc...@mikemccandless.com]
Sent: Tuesday, September 29, 2015 3:22 PM
To: Lucene Users
Subject: Re: Lucene 5 : any merge performance metrics compared to 4.x?
No, it is not possible to disable
inley, James T [mailto:james.mckin...@cengage.com]
> Sent: Tuesday, September 29, 2015 2:42 PM
> To: java-user@lucene.apache.org
> Subject: Re: Lucene 5 : any merge performance metrics compared to 4.x?
>
> Yes, the indexing workflow is completely separate from the runtime system.
>
any level IMHO.
-Original Message-
From: McKinley, James T [mailto:james.mckin...@cengage.com]
Sent: Tuesday, September 29, 2015 2:42 PM
To: java-user@lucene.apache.org
Subject: Re: Lucene 5 : any merge performance metrics compared to 4.x?
Yes, the indexing workflow is completely separate f
Yes, the indexing workflow is completely separate from the runtime system. The
file system is EMC Isilon via NFS.
Jim
From: will martin
Sent: 29 September 2015 14:29
To: java-user@lucene.apache.org
Subject: RE: Lucene 5 : any merge performance metrics
: Lucene 5 : any merge performance metrics compared to 4.x?
Hi Adrien and Will,
Thanks for your responses. I work with Selva and he's busy right now with
other things, so I'll add some more context to his question in an attempt to
improve clarity.
The merge in question is part of our batc
015 12:08
To: java-user@lucene.apache.org
Subject: RE: Lucene 5 : any merge performance metrics compared to 4.x?
So, if its new, it adds to pre-existing time? So it is a cost that needs to be
understood I think.
And, I'm really curious, what happens to the result of the post merge
checkIntegrit
15 8:46 AM
To: java-user@lucene.apache.org
Subject: Re: Lucene 5 : any merge performance metrics compared to 4.x?
Indeed this is new but I'm a bit surprised this is the source of your issues as
it should be much faster than the merge itself. I don't understand your
proposal to check t
Indeed this is new but I'm a bit surprised this is the source of your
issues as it should be much faster than the merge itself. I don't
understand your proposal to check the index after merge: the goal is to
make sure that we do not propagate corruptions so it's better to check the
index before the
it appears Lucene 5.2 index merge is running checkIntegrity on existing
index prior to merging additional indices.
This seems to be new.
We have an existing checkIndex but this is run post index merge.
Two follow up questions :
* Is there way to turn off built-in checkIntegrity? Just for my under
We observe some merge slowness after we migrated from 4.10 to 5.2.
Is this expected? Any new tunable merge parameters in Lucene 5 ?
-Selva
david m wrote:
A couple of reasons that lead to the merge approach:
- Source documents are written to archive media and retrieval is
relatively slow. Add to that our processing pipeline (including
text extraction)... Retrieving and merging minis is faster than
re-processing and re-indexing f
: based on a retention policy,
documents of type x are to be kept for y years.
One example for constructing a custom index would be for legal
discovery.
Thanks, david.
On 4/18/07, Michael D. Curtin <[EMAIL PROTECTED]> wrote:
d m wrote:
> I'd like to share index merge performance d
ularly relevant because
before I get to the point of making 847 committable, I need a way of
testing merge performance (the factoring in 847 proposes to simplify the
API slightly, so the merge algorithm would be slightly modified). The
stuff David has written gives me some ideas for benchmark tests
d m wrote:
I'd like to share index merge performance data and have a couple
of questions about it...
We (AXS-One, www.axsone.com) build one "master" index per day.
For backup and recovery purposes, we also build many individual
"mini" indexes from the docs added to
Yup, 845 is relevant, as is 847. I haven't had time to digest all that
David wrote yet, but I'm starting. It's particularly relevant because
before I get to the point of making 847 committable, I need a way of
testing merge performance (the factoring in 847 proposes to simplify t
the notion of using IndexWriter.ramSizeInBytes
is of too much use when merging indexes
Erick
On 4/18/07, d m <[EMAIL PROTECTED]> wrote:
I'd like to share index merge performance data and have a couple
of questions about it...
We (AXS-One, www.axsone.com) build one "m
I'd like to share index merge performance data and have a couple
of questions about it...
We (AXS-One, www.axsone.com) build one "master" index per day.
For backup and recovery purposes, we also build many individual
"mini" indexes from the docs added to the master
22 matches
Mail list logo
