er thread creation is heavy or ThreadLocal cleanup from outside is
> > complicated.
> > My test shows NumericDocValues will return wrong value, but sure that
> it's
> > a long value, upper logic can verify whether the value is valid or not.
> >
> > As I described i
e, but sure that it's
> a long value, upper logic can verify whether the value is valid or not.
>
> As I described in earlier mail, in Lucene4.4
> Lucene42DocValuesFormat(in-memory)
> has no problem, DiskDocValuesFormat(in-disk) has problem. Now in
> Lucene4.5, MemoryDocValue
from outside is
complicated.
My test shows NumericDocValues will return wrong value, but sure that it's
a long value, upper logic can verify whether the value is valid or not.
As I described in earlier mail, in Lucene4.4 Lucene42DocValuesFormat(in-memory)
has no problem, DiskDocValuesFormat(in
:28 AM, Duke DAI wrote:
> Hi guys,
>
> Seems I have the same problem with Lucene45DocValuesFormat, no problem with
> MemoryDocValuesFormat. The problem I encountered with Lucene4.4 is with
> DiskDocValuesFormat, no with Lucene42DocValuesFormat.
>
> I dig into a little and
Hi guys,
Seems I have the same problem with Lucene45DocValuesFormat, no problem with
MemoryDocValuesFormat. The problem I encountered with Lucene4.4 is with
DiskDocValuesFormat, no with Lucene42DocValuesFormat.
I dig into a little and found the superficial cause. In SegmentCoreReaders,
there is
> We can't reindex every time we upgrade lucene since our indexes are too
> > large. Should we copy the code from DiskDocValuesFormat and call it
> > CustomDiskDocValuesFormat, and give CustomDiskDocValuesFormat a new name
> so
> > that when we upgrade lucene, we won
time we upgrade lucene since our indexes are too
> large. Should we copy the code from DiskDocValuesFormat and call it
> CustomDiskDocValuesFormat, and give CustomDiskDocValuesFormat a new name so
> that when we upgrade lucene, we won't use an incompatible version of
> DiskDocValu
code from DiskDocValuesFormat and call it
CustomDiskDocValuesFormat, and give CustomDiskDocValuesFormat a new name so
that when we upgrade lucene, we won't use an incompatible version of
DiskDocValuesFormat?
Thanks,
Sean
On Wed, Aug 21, 2013 at 8:44 AM, Robert Muir wrote:
> On Wed, Aug 21
On Wed, Aug 21, 2013 at 11:30 AM, Sean Bridges wrote:
> What is the recommended way to use DiskDocValuesFormat in production if we
> can't reindex when we upgrade?
I'm not going to recommend using any experimental codecs in production, but...
1. with 4.3 jar file: IWC.setCodec(
What is the recommended way to use DiskDocValuesFormat in production if we
can't reindex when we upgrade?
Will the 4.4 version of DDVF be backwards compatible, or should we make our
own copy of DDVF and give it a different codec name to protect ourselves
against incompatible changes?
T
; On Tue, Aug 13, 2013 at 4:54 AM, Duke DAI wrote:
> > Hi experts,
> >
> > I'm upgrading Lucene 4.4 and trying to use DocValues instead of store
> field
> > for performance reason. But due to unknown size of index(depends on
> > customer), so I will use Dis
http://blog.mikemccandless.com
On Tue, Aug 13, 2013 at 4:54 AM, Duke DAI wrote:
> Hi experts,
>
> I'm upgrading Lucene 4.4 and trying to use DocValues instead of store field
> for performance reason. But due to unknown size of index(depends on
> customer), so I will use DiskDocValuesFo
Hi experts,
I'm upgrading Lucene 4.4 and trying to use DocValues instead of store field
for performance reason. But due to unknown size of index(depends on
customer), so I will use DiskDocValuesFormat, especially for some binary
field. Then I wrote my customized Codec:
final Codec
at
> >
> >
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:514)
> >
> > I configured jvm with "-Xmx4096m", and it seems still not enough memory.
> I
> > thought DiskDocValuesFormat puts most of the
pache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:514)
>
> I configured jvm with "-Xmx4096m", and it seems still not enough memory. I
> thought DiskDocValuesFormat puts most of the data on disk and there should
> not be that much memory consumptio
geScheduler.java:541)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:514)
I configured jvm with "-Xmx4096m", and it seems still not enough memory. I
thought DiskDocValuesFormat puts most of the data on disk and there should
not be tha
JAR file with a ZIP
> > > program and check that all files in META-INF/services contain all
> > > entries merged from all Lucene JARs.
> > >
> > > Uwe
> > >
> > > -
> > > Uwe Schindler
> > > H.-H.-Meier-Allee 63, D-28213 Brem
ARs.
> >
> > Uwe
> >
> > -
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: u...@thetaphi.de
> >
> >
> > > -Original Message-
> > > From: Wei Wang [mailto:welshw...@gmail.com]
&g
inal Message-
> > From: Wei Wang [mailto:welshw...@gmail.com]
> > Sent: Sunday, April 14, 2013 11:49 PM
> > To: java-user@lucene.apache.org
> > Subject: Re: DiskDocValuesFormat
> >
> > Yes, I used Maven Shade plugin, but still have this problem. Here is the
e.apache.org
> Subject: Re: DiskDocValuesFormat
>
> Yes, I used Maven Shade plugin, but still have this problem. Here is the
> Maven output during packaging:
>
> [INFO] --- maven-shade-plugin:2.0:shade (default) @ audience-profile-
> indexer --- [INFO] Including commons-collections:co
3, D-28213 Bremen
> http://www.thetaphi.de
> eMail: u...@thetaphi.de
>
>
> > -Original Message-
> > From: Wei Wang [mailto:welshw...@gmail.com]
> > Sent: Sunday, April 14, 2013 11:30 PM
> > To: java-user@lucene.apache.org
> > Subject: Re: DiskDocValuesFormat
>
://www.thetaphi.de
eMail: u...@thetaphi.de
> -Original Message-
> From: Wei Wang [mailto:welshw...@gmail.com]
> Sent: Sunday, April 14, 2013 11:30 PM
> To: java-user@lucene.apache.org
> Subject: Re: DiskDocValuesFormat
>
> Hi Adrien,
>
> The Lucene42Codec works well
Hi Adrien,
The Lucene42Codec works well to generate the index with
DiskDocValuesFormat. But when I tried to merge the index segments by
calling:
IndexWriter iw = new IndexWriter(directory, iw_config);
...
iw.forceMerge(1);
I got the following error message:
Caused by
Hi Adrien,
Thanks for your example. Really helpful!
Wei
On Sat, Apr 13, 2013 at 4:25 AM, Adrien Grand wrote:
> Hi Wei,
>
> On Sat, Apr 13, 2013 at 7:44 AM, Wei Wang wrote:
> > I am trying to use DiskDocValuesFormat for a particular
> > BinaryDocValuesField. It seems ther
Hi Wei,
On Sat, Apr 13, 2013 at 7:44 AM, Wei Wang wrote:
> I am trying to use DiskDocValuesFormat for a particular
> BinaryDocValuesField. It seems there is no good examples showing how to do
> this. The only hint I got from various docs and forums is set some codec in
> IndexW
I am trying to use DiskDocValuesFormat for a particular
BinaryDocValuesField. It seems there is no good examples showing how to do
this. The only hint I got from various docs and forums is set some codec in
IndexWriter. Could someone give a few lines of code snippet and show how to
set
26 matches
Mail list logo
