__
From: Lebiram
To: java-user@lucene.apache.org
Sent: Wednesday, December 24, 2008 2:43:12 PM
Subject: Re: Optimize and Out Of Memory Errors
Hello Mark,
As of the moment the index could not be rebuilt to remove norms.
Right now, I'm trying to figure out what luke is doing by going throug
To: java-user@lucene.apache.org
Sent: Wednesday, December 24, 2008 2:20:23 PM
Subject: Re: Optimize and Out Of Memory Errors
We don't know those norms are "the" problem. Luke is loading norms if its
searching that index. But what else is Luke doing? What else is your App doing?
I
he.org
Sent: Tuesday, December 23, 2008 5:25:30 PM
Subject: Re: Optimize and Out Of Memory Errors
Mark Miller wrote:
Lebiram wrote:
Also, what are norms
Norms are a byte value per field stored in the index that is factored into the
score. Its used for length normalization (shorter docume
r
To: java-user@lucene.apache.org
Sent: Tuesday, December 23, 2008 5:25:30 PM
Subject: Re: Optimize and Out Of Memory Errors
Mark Miller wrote:
> Lebiram wrote:
>> Also, what are norms
> Norms are a byte value per field stored in the index that is factored into
> the score.
Mark Miller wrote:
Lebiram wrote:
Also, what are norms
Norms are a byte value per field stored in the index that is factored
into the score. Its used for length normalization (shorter documents =
more important) and index time boosting. If you want either of those,
you need norms. When norms
Lebiram wrote:
Also, what are norms
Norms are a byte value per field stored in the index that is factored
into the score. Its used for length normalization (shorter documents =
more important) and index time boosting. If you want either of those,
you need norms. When norms are loaded up into a
Field("field5", "groupId" + i, Field.Store.YES,
Field.Index.UN_TOKENIZED));
writer.addDocument(doc);
________________
From: mark harwood
To: java-user@lucene.apache.org
Sent: Tuesday, December 23, 2008 2:42:25 PM
Subject: Re: Optimize a
"groupId" + i, Field.Store.YES,
Field.Index.UN_TOKENIZED));
writer.addDocument(doc);
________
From: mark harwood
To: java-user@lucene.apache.org
Sent: Tuesday, December 23, 2008 2:42:25 PM
Subject: Re: Optimize and Out Of Memory Errors
I've had reports of OOM e
ifferent environments
with large indexes make me think there is still something to be uncovered
here...
- Original Message
From: Michael McCandless
To: java-user@lucene.apache.org
Cc: Utan Bisaya
Sent: Tuesday, 23 December, 2008 14:08:26
Subject: Re: Optimize and Out Of Memory Errors
How many indexed fields do you have, overall, in the index?
If you have a very large number of fields that are "sparse" (meaning
any given document would only have a small subset of the fields), then
norms could explain what you are seeing.
Norms are not stored sparsely, so when segments g
10 matches
Mail list logo
