[il-antlr-interest: 23753] Re: [antlr-interest] Lexer matching non-matching rule

2009-05-17 Thread Jesper Larsson
On Sun, 2009-05-17 at 12:48 +0200, Micha wrote: > On Sunday 17 May 2009 11:01:36 Jesper Larsson wrote: > > Not always so easy, however. My original example was, even more > > simplified, something like this: > > > > FOO:'foo'; > > BAR:'b

[il-antlr-interest: 23750] Re: [antlr-interest] Lexer matching non-matching rule

2009-05-17 Thread Jesper Larsson
> If you want the longest match, then left factor everything and let it > do that: > > A ( B (C|) |) ; > > And set the token type at the appropriate points. Not always so easy, however. My original example was, even more simplified, something like this: FOO:'foo'; BAR:'bar'; FOOZ: '

[il-antlr-interest: 23743] Re: [antlr-interest] Lexer matching non-matching rule

2009-05-16 Thread Jesper Larsson
On Sat, 2009-05-16 at 08:27 +0530, Indhu Bharathi wrote: > This is because on seeing 'f' of foo lexer has two options - 1. IDENT > 2. URL. And it takes the second options since that seems to be longer > that the first alternative. Note that the lexer always tries to match > the longest token possi

[il-antlr-interest: 23727] [antlr-interest] Lexer matching non-matching rule

2009-05-15 Thread Jesper Larsson
Sorry if this is a stupid question, but if it is I hope it has a quick stupid answer. My ANTLR-generated lexer protests about unexpected characters in a situation where it could have matched the input with other rules. My understanding of ANTLR and other tools that generate lexers is that it shou