Re: ArrayIndexOutOfBoundsException trying to use tokenizer in Lucene 4.1

2013-02-26 Thread Paul Taylor
On 26/02/2013 13:29, Alan Woodward wrote: Hi Paul, You need to call tokenizer.reset() before you call incrementToken() Alan Woodward www.flax.co.uk Hi, thanks that fixes it

Re: ArrayIndexOutOfBoundsException trying to use tokenizer in Lucene 4.1

2013-02-26 Thread Alan Woodward
Hi Paul, You need to call tokenizer.reset() before you call incrementToken() Alan Woodward www.flax.co.uk On 26 Feb 2013, at 12:26, Paul Taylor wrote: > This works in 3.6, but in 4.1 fails whats wrong with the code > > public void testTokenization() throws IOException >{ >StringBu

ArrayIndexOutOfBoundsException trying to use tokenizer in Lucene 4.1

2013-02-26 Thread Paul Taylor
This works in 3.6, but in 4.1 fails whats wrong with the code public void testTokenization() throws IOException { StringBuffer sb = new StringBuffer(); for(char i=0;i<100;i++) { Character c = new Character(i); if(!Character.isWhitespace(c)) {