Thanks Benson, I'll have a look.
Mike McCandless
http://blog.mikemccandless.com
On Sat, Sep 7, 2013 at 4:33 PM, Benson Margulies wrote:
> LUCENE-5202. It seems to show the problem of the extra peek. I'm still
> struggling to make sense of the 'problem' of not always calling
> afterPosition();
LUCENE-5202. It seems to show the problem of the extra peek. I'm still
struggling to make sense of the 'problem' of not always calling
afterPosition(); that may be entirely my own confusion.
On Sat, Sep 7, 2013 at 4:21 PM, Michael McCandless
wrote:
> That would be awesome, thanks!
>
> Mike McCand
That would be awesome, thanks!
Mike McCandless
http://blog.mikemccandless.com
On Sat, Sep 7, 2013 at 3:40 PM, Benson Margulies wrote:
> I think I had better build you a test case for this situation, and
> attach it to a JIRA.
>
> On Sat, Sep 7, 2013 at 3:33 PM, Michael McCandless
> wrote:
>>
I think I had better build you a test case for this situation, and
attach it to a JIRA.
On Sat, Sep 7, 2013 at 3:33 PM, Michael McCandless
wrote:
> Something is wrong; I'm not sure what offhand, but calling peekToken
> 10 times should not stack all tokens @ position 0; it should stack the
> token
Something is wrong; I'm not sure what offhand, but calling peekToken
10 times should not stack all tokens @ position 0; it should stack the
tokens at the positions where they occurred. Are you sure the posIncr
att is sometimes 1 (i.e., the position is in fact moving forward for
some tokens)?
next
nextToken() calls peekToken(). That seems to prevent my lookahead
processing from seeing that item later. Am I missing something?
On Fri, Sep 6, 2013 at 9:15 PM, Benson Margulies wrote:
> I think that the penny just dropped, and I should not be using this class.
>
> If I call peekToken 10 times
I think that the penny just dropped, and I should not be using this class.
If I call peekToken 10 times while sitting at token 0, this class will
stack up all 10 of these _at token position 0_. That's not really very
helpful for what I'm doing. I need to borrow code from this class and
not use it.
Michael,
I'm apparently not fully deconfused yet.
I've got a very simple incrementToken function. It calls peekToken to
stack up the tokens.
afterPosition is never called; I expected it to be called as each of
the peeked tokens gets next-ed back out.
I assume that I'm missing something simple.
On Fri, Sep 6, 2013 at 7:31 AM, Michael McCandless
wrote:
>
> On Thu, Sep 5, 2013 at 8:44 PM, Benson Margulies wrote:
> > I'm trying to work through the logic of reading ahead until I've seen
> > marker for the end of a sentence, then applying some analysis to all of the
> > tokens of the sentenc
On Thu, Sep 5, 2013 at 8:44 PM, Benson Margulies wrote:
> I'm trying to work through the logic of reading ahead until I've seen
> marker for the end of a sentence, then applying some analysis to all of the
> tokens of the sentence, and then changing some attributes of each token to
> reflect the r
It's in test-framework only because no "real" TokenFilter uses it yet,
and, it's all very new code :)
My intention was to eventually cutover tricky graph TokenFilters (like
SynFilter), to simplify them, factoring out the common buffering of
tokens by position into LookaheadTokenFilter, but I never
11 matches
Mail list logo
