Re: Changing field lengthnorm to store length

2014-06-19 Thread Nalini Kartha
Ok, makes sense. Thanks for the info! On Thu, Jun 19, 2014 at 3:05 PM, Robert Muir wrote: > Don't extend that: extend Similarity. > > Some of those implementations actually rely and optimize for the fact > that its a byte and build lookup tables and so on. > > On Thu, Jun 19, 2014 at 6:03 PM, N

Re: Changing field lengthnorm to store length

2014-06-19 Thread Robert Muir
Don't extend that: extend Similarity. Some of those implementations actually rely and optimize for the fact that its a byte and build lookup tables and so on. On Thu, Jun 19, 2014 at 6:03 PM, Nalini Kartha wrote: > Sorry, I meant the encodeNormValue and decodeNormValue methods on the > TFIDFSimi

Re: Changing field lengthnorm to store length

2014-06-19 Thread Nalini Kartha
Sorry, I meant the encodeNormValue and decodeNormValue methods on the TFIDFSimilarity class - public byte encodeNormValue(float f) public float decodeNormValue(byte b) On Thu, Jun 19, 2014 at 12:08 PM, Robert Muir wrote: > No they do not. The method is: > > public abstract long computeNorm(F

Re: Changing field lengthnorm to store length

2014-06-19 Thread Robert Muir
No they do not. The method is: public abstract long computeNorm(FieldInvertState state); On Thu, Jun 19, 2014 at 1:54 PM, Nalini Kartha wrote: > Thanks for the info! > > We're more interested in changing the lengthnorm function vs using > additional stats for scoring so option 2 seems like t

Re: Changing field lengthnorm to store length

2014-06-19 Thread Nalini Kartha
Thanks for the info! We're more interested in changing the lengthnorm function vs using additional stats for scoring so option 2 seems like the right way. It looks like the encode and decode methods deal with bytes right now - would changing those APIs to deal with longs instead be a good idea? I

RE: Changing field lengthnorm to store length

2014-06-19 Thread Uwe Schindler
Hi, You may not need to change the length-norm at all: If you want to support *additional* statistics, add a docvalues field to your index where you can store that information in addition to the Lucene-Default statistics. Based on a function query you can then use it for scoring. In fact, you c