Re: Token character positions

2009-11-18 Thread Grant Ingersoll
On Nov 17, 2009, at 10:37 AM, Christopher Tignor wrote: > Hello, > > Hoping someone might clear up a question for me: > > When Tokenizing we provide the start and end character offsets for each > token locating it within the source text. > > If I tokenize the text "word" and then serach for th

Token character positions

2009-11-17 Thread Christopher Tignor
Hello, Hoping someone might clear up a question for me: When Tokenizing we provide the start and end character offsets for each token locating it within the source text. If I tokenize the text "word" and then serach for the term "word" in the same field, how can I recover this character offset i