eMail: u...@thetaphi.de
> -Original Message-
> From: Daniel Shane [mailto:sha...@lexum.umontreal.ca]
> Sent: Thursday, September 03, 2009 8:55 PM
> To: java-user@lucene.apache.org
> Subject: TokenStream API, Quick Question.
>
> Does a TokenStream have to return
Does a TokenStream have to return always the same number of attributes
with the same underlying classes for all the tokens it generates?
I mean, during the tokenization phase, can the first "token" have a Term
and Offset Attribute and the second "token" only a Type Attribute or
does this mean
