Re: How to add "tokens" to a stream

2010-06-07 Thread Aad Nales
Yes it does, The key bit is the part with the termAttribute... thanks a lot, cheers, Aad On Mon, Jun 7, 2010 at 4:53 PM, Simon Willnauer wrote: > Hey there, > in lucene 3.0 / 2.9 the Token class has been remove / replaced with an > Attribute based API. A TokenStream operates on Attibutes it ha

Re: How to add "tokens" to a stream

2010-06-07 Thread Simon Willnauer
Hey there, in lucene 3.0 / 2.9 the Token class has been remove / replaced with an Attribute based API. A TokenStream operates on Attibutes it has declared which are eventually accessed by the IndexWriter to create the inverted index. There are Attributes like TermAttribute, PositionIncrementAttribu