[il-antlr-interest: 23741] Re: [antlr-interest] Lexer matching non-matching rule

Indhu Bharathi Fri, 15 May 2009 20:00:50 -0700

This is because on seeing 'f' of foo lexer has two options - 1. IDENT 2. URL. And it takes the second options since that seems to be longer that the first alternative. Note that the lexer always tries to match the longest token possible.

After having decided to go for URL, it matches the input with URL and it fails. Lexer doesn't backtrack and hence throws an exception.

In your case you can use semantic predicates to validate whether 'URL' exists before matching the input with URL. You can refactor the code similar to the one shown below:

grammar Test;

options { output=AST; }

file
    : IDENT DOT EOF
    ;

DOT:            '.';

WHITESPACE:     ('\f' | '\n' | '\r' | '\t' | ' ')+
                { $channel = HIDDEN; };

URL_OR_IDENT    :    (FRAG_URL) =>FRAG_URL
            { $type = URL; }
        |    (FRAG_IDENT) => FRAG_IDENT
            { $type = IDENT; }
        ;

fragment IDENT
    :    ;

fragment URL
    :    ;

fragment
FRAG_IDENT:          ('a'..'z' | 'A'..'Z')+;

fragment
FRAG_URL:            ('a'..'z') ('a'..'z' | '0'..'9' | '+' | '-' | '.')* ':'
                ~('\f' | '\n' | '\r' | '\t' | ' ')*;


Hope that helps.

Cheers, Indhu

Michael wrote:

Am Friday 15 May 2009 14:03:16 schrieb Jesper Larsson:

URL:            ('a'..'z') ('a'..'z' | '0'..'9' | '+' | '-' | '.')* ':'
                ~('\f' | '\n' | '\r' | '\t' | ' ')*;


if you leave out the dot in the URL rule it works as it should. 
If the dot is there the IDENT rule is not called but the URL rule is called 
which raises an exception since the colon is missing.
No idea why .


cheers
 Michael

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "il-antlr-interest" group.
To post to this group, send email to il-antlr-interest@googlegroups.com
To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en
-~----------~----~----~----~------~----~------~--~---

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: 
http://www.antlr.org/mailman/options/antlr-interest/your-email-address

[il-antlr-interest: 23741] Re: [antlr-interest] Lexer matching non-matching rule

Reply via email to