Re: iterate through tokens in standardtokenizer

Thanks a lot for the help.

iterate through tokens in standardtokenizer

Hi, I am trying to use standardTokenizer in a non-lucene project to generate tokens. The previous versions i used supported token.next , getToken to iterate over and retrieve the tokens continuously. 3.0.2 doesnt have that and i cant figure out how to iterate. Can get number of tokens with incremen