Hi Ahmet, Sorry, I wasn't very clear. I need a TokenFilter that filters/skips some tokens, just like FilteringTokenFilter. To determine which tokens to filter, I first need to know all tokens. Therefore I need to go through all tokens twice.
I implemented an TwoPassTokenFilter that is very similar to CachingTokenFilter. On the first call to incrementToken() it builds a cache and goes through al tokens for the first pass. The following calls to incrementToken() build the second pass. In the second pass I can use information collected in the first pass. Christian 2014-08-24 13:50 GMT+02:00 Ahmet Arslan <iori...@yahoo.com.invalid>: > Hi, > > Can you elaborate more, what do you mean by "I need to know all tokens > in advance." > > Ahmet > > > On Wednesday, August 20, 2014 6:48 PM, Christian Beil < > christian.a.b...@gmail.com> wrote: > Hey guys, > > I need a TokenFilter that filters some tokens like the > FilteringTokenFilter. > The problem is, in order to do the filtering I need to know all tokens in > advance. > > I thought I'll adapt the CachingTokenFilter in order to collect all tokens > in the first pass. > In the second pass it can use this information to filter the tokens. > > Or is there a better solution to do this? > > Thanks, > Christian > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
