Re: Modifying the stored norm type

2006-06-22 Thread Yonik Seeley
On 6/22/06, karl wettin <[EMAIL PROTECTED]> wrote: I tried to make a quick and dirty proof of concept, but noticed that no matter what order TermDocs return the documents, the collector get ascending document number order. TermDocs should also always return documents in ascending order for a si

Re: Modifying the stored norm type

2006-06-22 Thread karl wettin
On Wed, 2006-06-21 at 19:32 +0200, Paul Elschot wrote: > > > TermDocs in reversed chronological order > > There is no need to write extra code for that, the documents would be > collected oldest first, newest last. I tried to make a quick and dirty proof of concept, but noticed that no matter wha

Re: Modifying the stored norm type

2006-06-21 Thread Paul Elschot
On Tuesday 20 June 2006 18:42, Dan Climan wrote: > >Paul Elschot <[EMAIL PROTECTED]> > >>On Tuesday 20 June 2006 12:02, Marcus Falck wrote: > >> After a lot of debugging and some API doc reading I have come to the > > conclusion that the static encodeNorm method of the Similarity class > > will en

Re: Modifying the stored norm type

2006-06-21 Thread Paul Elschot
On Wednesday 21 June 2006 12:13, karl wettin wrote: > On Tue, 2006-06-20 at 18:01 +0200, Paul Elschot wrote: > > On Tuesday 20 June 2006 12:02, Marcus Falck wrote: > > > encodeNorm method of the Similarity class will encode my boost value > > into a single byte decimal number. And I will loose a l

Re: Modifying the stored norm type

2006-06-21 Thread Yonik Seeley
On 6/21/06, karl wettin <[EMAIL PROTECTED]> wrote: Marcus is trying to use the norms to enforce results in chronological order when matching a TB-sized corpus. He can't get any speed by sorting on a date field. Once a FieldCache entry is populated, sorting on a DateField should be about the sam

Re: Modifying the stored norm type

2006-06-21 Thread karl wettin
On Tue, 2006-06-20 at 18:01 +0200, Paul Elschot wrote: > On Tuesday 20 June 2006 12:02, Marcus Falck wrote: > encodeNorm method of the Similarity class will encode my boost value > into a single byte decimal number. And I will loose a lot of > resolution and will get severe rounding errors. > Are

Re: Modifying the stored norm type

2006-06-20 Thread Dan Climan
>Paul Elschot <[EMAIL PROTECTED]> >>On Tuesday 20 June 2006 12:02, Marcus Falck wrote: >> After a lot of debugging and some API doc reading I have come to the > conclusion that the static encodeNorm method of the Similarity class > will encode my boost value into a single byte decimal number. >>

Re: Modifying the stored norm type

2006-06-20 Thread Yonik Seeley
On 6/20/06, Marcus Falck <[EMAIL PROTECTED]> wrote: So I guess I will have to get lucene to store a 4 byte norm in the form of a float instead of the single byte? Can you store the float in a field instead? That seems like it might be a bit easier than modifying how lucene stores norms. -Yon

Re: Modifying the stored norm type

2006-06-20 Thread Paul Elschot
On Tuesday 20 June 2006 12:02, Marcus Falck wrote: > Hi again, > > > > After a lot of debugging and some API doc reading I have come to the > conclusion that the static encodeNorm method of the Similarity class > will encode my boost value into a single byte decimal number. > > And I will loos

Re: Modifying the stored norm type

2006-06-20 Thread karl wettin
On Tue, 2006-06-20 at 13:59 +0200, karl wettin wrote: > On Tue, 2006-06-20 at 12:02 +0200, Marcus Falck wrote: > > > So I guess I will have to get lucene to store a 4 byte norm in the > > form of a float instead of the single byte? > > > > Is this do able or is it just madness? And will it slow t

Re: Modifying the stored norm type

2006-06-20 Thread karl wettin
On Tue, 2006-06-20 at 12:02 +0200, Marcus Falck wrote: > So I guess I will have to get lucene to store a 4 byte norm in the > form of a float instead of the single byte? > > Is this do able or is it just madness? And will it slow the search > timings down or will it just eat more memory? It is