Re: Inconsistent StandardTokenizer behaviour

2005-11-22 Thread yahootintin . 11533894
Cool, I'll take a look at fixing this. --- java-user@lucene.apache.org wrote: > On 21 Nov 2005, at 19:39, [EMAIL PROTECTED] wrote: > > This is the results for the StandardTokenizer: > >input - output token - > > output type > > 1. 1.2 - 1.2 - > > 2. 1.2. - 1.2 - > > >

Re: Inconsistent StandardTokenizer behaviour

2005-11-21 Thread Erik Hatcher
On 21 Nov 2005, at 19:39, [EMAIL PROTECTED] wrote: This is the results for the StandardTokenizer: input - output token - output type 1. 1.2 - 1.2 - 2. 1.2. - 1.2 - 3. a.b - a.b - 4. a.b. - a.b. - 5. www.apache.org - www.apache.org - 6. www.apac

Re: Inconsistent StandardTokenizer behaviour

2005-11-21 Thread yahootintin . 11533894
Sorry for the bad looking table. Retrying... input string - output token (output type) 1. 1.2 - 1.2 () 2. 1.2. - 1.2 () 3. a.b - a.b () 4. a.b. - a.b. () 5. www.apache.org - www.apache.org () 6. www.apache.org. - www.apache.org. () --- java-user@lucene.apache.org wrote: This is the results for

Inconsistent StandardTokenizer behaviour

2005-11-21 Thread yahootintin . 11533894
This is the results for the StandardTokenizer: input - output token - output type 1. 1.2 - 1.2 - 2. 1.2. - 1.2 - 3. a.b - a.b - 4. a.b. - a.b. - 5. www.apache.org - www.apache.org - 6. www.apache.org. - www.apache.org. - Number 6 should still be