[il-antlr-interest: 23837] Re: [antlr-interest] Island Grammars with identical tokens

2009-05-21 Thread Thomas Brandon
On Thu, May 21, 2009 at 11:42 PM, Mike J. Bell wrote: > I have a main grammar that passes control to two different island > grammars to parse a language.  One island grammar is an island because > it's so big and can be used in a context outside the main language, > and the other island has vastl

[il-antlr-interest: 23344] Re: [antlr-interest] case-insensitive parsing

2009-04-23 Thread Thomas Brandon
On Fri, Apr 24, 2009 at 12:19 AM, Andreas Meyer wrote: > Ok ... there are two options: > (2) use island grammars, as advertised on the Wiki > (http://www.antlr.org/wiki/display/ANTLR3/Island+Grammars+Under+Parser+Control). > however, this is quite complicated to set up > > Island grammars are nic

[il-antlr-interest: 23325] Re: [antlr-interest] Relaxed number format lexer problem

2009-04-22 Thread Thomas Brandon
On Thu, Apr 23, 2009 at 2:40 AM, Sam Harwell wrote: > I’m using a heavily relaxed NUMBER token in my lexer so I can provide better > error messages. The problem I’m having occurs when input such as 1..2 is > reached. The '..' should be an OP_RANGE token, not part of the number. > > > > NUMBER > >

[il-antlr-interest: 23258] Re: [antlr-interest] Can I restart lexing from definite positionindocument?

2009-04-17 Thread Thomas Brandon
On Sat, Apr 18, 2009 at 12:04 AM, Sam Harwell wrote: > You should tokenize on a per-line basis. Never allow a token to span > multiple lines, and never allow lookahead/back to cross a newline > boundary. I've documented this process in my blog followed by a email on > this list earlier this week

[il-antlr-interest: 23115] Re: [antlr-interest] Stupid languages, and parsing them

2009-04-11 Thread Thomas Brandon
On Sun, Apr 12, 2009 at 6:01 AM, Sam Barnett-Cormack wrote: > I'm not sure an island grammar would work, as I need the eventual AST of the > "WITH SYNTAX" block to be included in the final AST of the master grammar. > > Unless, that is, I can invoke a full lexer/parser combination, get the tree >

[il-antlr-interest: 23107] Re: [antlr-interest] Stupid languages, and parsing them

2009-04-11 Thread Thomas Brandon
On Sun, Apr 12, 2009 at 4:45 AM, Sam Barnett-Cormack wrote: > Hi all, > > In my ongoing project, I need to parse a really crazy structure that > wants to change the lexing rules dependent on syntactic factors. I hate > this. > > Within the thing I'm talking about, whitespace and comments are hand

[il-antlr-interest: 22946] Re: [antlr-interest] Looping forever on some input

2009-04-02 Thread Thomas Brandon
The problem is most likely your DecimalLiteral rule: DecimalLiteral : ; Non-fragment lexer rules that can match no input will cause infinite loops. This may only occur for certain input as other input will be matched before ANTLR checks whether the troublesome rule can match. Having input

[il-antlr-interest: 22690] Re: [antlr-interest] How can I avoid "mismatched input" error?

2009-03-24 Thread Thomas Brandon
2009/3/24 Gabriel Petrovay : > Hi Indhu, > > I was trying to simplify the example such that I still get the error and the > example is simple enough for everybody to understand the problem. > > Here is the corrected grammar: > > // > grammar k; > options { >

[il-antlr-interest: 22415] Re: [antlr-interest] Good practice for grammar with translated keywords

2009-03-12 Thread Thomas Brandon
If you know ahead of time what language is to be used and all keywords are acceptable identifiers you could use an action in your identifier rule to check a language specific hashtable mapping text to token type and set the resulting token type based on that. e.g. @lexer::members { private Hash

[il-antlr-interest: 22257] Re: [antlr-interest] lookahead DFA too big?

2009-03-05 Thread Thomas Brandon
On Thu, Mar 5, 2009 at 9:50 PM, Andreas Meyer wrote: > Maybe it's possible to partition the set of keywords, but that would be > some effort: figuring out for 800 keywords, where they appear, what is > the context they are used in etc. Note that the problem only appeared > after switching to ANTL

[il-antlr-interest: 21879] Re: [antlr-interest] Writing (for now) just a lexer

2009-02-11 Thread Thomas Brandon
't match the second ('0'..'9')+ loop of the float rule. As ANTLR only considers single tokens it is essentially matching the input against an implicit rule: MTOKENS: (INT_LITERAL|FLOAT_LITERAL|DOT|ID); You can see this rule in the generated mTokens method. Given this

[il-antlr-interest: 21861] Re: [antlr-interest] Writing (for now) just a lexer

2009-02-11 Thread Thomas Brandon
On Wed, Feb 11, 2009 at 7:52 PM, Evan Driscoll wrote: > Hi all, > > I'm teaching a compilers class at University of Wisconsin-Madison. > Traditionally the class has followed sort of a classic sequence of > projects 'write a lexer', 'write a parser', etc., and in the past has > used either JLex/Ja

[il-antlr-interest: 21469] Re: [antlr-interest] ANTLR Questions

2009-01-21 Thread Thomas Brandon
On Wed, Jan 21, 2009 at 11:55 PM, Bruno Marc-Aurele wrote: > Thanks a lot Johannes. > > Has anyone used heterogenous trees? I would like to know how customisable my > AST > can be... I will rewrite my question concerning this so you don't need to look > for it in the previous messages: > > Q: I

[il-antlr-interest: 21154] Re: [antlr-interest] code too large error

2008-12-31 Thread Thomas Brandon
ANTLR combines predicates and the token lookahead when creating DFAs so it will still have a predictor. It will also hoist predicates from called rules into parent rules to disambiguate. So those rules will generate a fairly complex predictor. I would say you are better to just accept any number o

[il-antlr-interest: 21034] Re: [antlr-interest] Flattening lists?

2008-12-20 Thread Thomas Brandon
On Sat, Dec 20, 2008 at 7:46 PM, Gary R. Van Sickle wrote: > > From: Jim Idle > > > > On Mon, 2008-12-15 at 08:44 -0600, Gary R. Van Sickle wrote: > > > > > > Hi all, > > > > Not sure if this is something that should be obvious or > > not, but is there a > > way, completely in AN

[il-antlr-interest: 19557] Re: [antlr-interest] Using Antlr with StringTemplate files

2008-09-17 Thread Thomas Brandon
Until then you can use the remote debugging feature of ANTLRWorks. Generate the code with the -debug command line option (can be set in ANTLRWorks preferences), then run your application code however you want, it will pause waiting for the debugger to connect when you select remote debugging in AN

[il-antlr-interest: 19448] Re: [antlr-interest] lexer: matching float vs int

2008-09-09 Thread Thomas Brandon
na <[EMAIL PROTECTED]> wrote: > Quoting Thomas Brandon <[EMAIL PROTECTED]>: > >> On Tue, Sep 9, 2008 at 2:01 PM, Olya Krachina <[EMAIL PROTECTED]> wrote: >> > Hello, >> > I am new to antlr and i seem to be stuck on this. >> > i need to have 2

[il-antlr-interest: 19444] Re: [antlr-interest] lexer: matching float vs int

2008-09-09 Thread Thomas Brandon
On Tue, Sep 9, 2008 at 2:01 PM, Olya Krachina <[EMAIL PROTECTED]> wrote: > Hello, > I am new to antlr and i seem to be stuck on this. > i need to have 2 datatypes defined: int and float, currently i have them > defined > like this in my .g file: > > INT: ('0'..'9')+; > FLOAT:('0'..'9')*(

[il-antlr-interest: 19317] Re: [antlr-interest] Simple question, Hard answer?

2008-09-03 Thread Thomas Brandon
On Wed, Sep 3, 2008 at 7:13 PM, Bill Mayfield <[EMAIL PROTECTED]> wrote: > Hi, > > Suppose I want to recognize each pattern 'okko' in a random string of > characters. Some examples: > > aoookokkaaa -> should yield one match of 'okko' > aaaokkookko -> should yield two matches of 'okko'

[il-antlr-interest: 19326] Re: [antlr-interest] Fwd: Simple question, Hard answer?

2008-09-03 Thread Thomas Brandon
On Wed, Sep 3, 2008 at 10:28 PM, Bill Mayfield <[EMAIL PROTECTED]> wrote: > > Thomas: This one doesn't work however > > > grammar test; > > start : chaos* pattern* chaos*; > > pattern : T1 T2 T2 T1; > > chaos : T1 | T2; > > T1 : 'o'; > T2 : 'k'; > > It s

[il-antlr-interest: 19324] Re: [antlr-interest] Fwd: Simple question, Hard answer?

2008-09-03 Thread Thomas Brandon
On Wed, Sep 3, 2008 at 7:36 PM, Bill Mayfield <[EMAIL PROTECTED]> wrote: > Okay... but what if this isn't a lexer problem? Suppose I my input is > only made up of 'o' and 'k' > > okkkooo -> 1 match > oookkokkoook -> 2 matches > > what would my grammar look like? > > grammar test; > > s

[il-antlr-interest: 19303] Re: [antlr-interest] Reporting errors from rules

2008-09-02 Thread Thomas Brandon
On Tue, Sep 2, 2008 at 8:54 PM, Florian Weimer <[EMAIL PROTECTED]> wrote: > I'd like to report errors from rules because I want to reuse existing, > non-ANTLR parsers and make the lexer rules a bit more lenient than > what's actually in the language. For instance, in > > IPV4_ADDRESS : ('0' .. '9

[il-antlr-interest: 19299] Re: [antlr-interest] Re : Re : How to get a list of all validoptions for the next token?

2008-09-02 Thread Thomas Brandon
Can't you reuse the ANTLR routines for this, e.g. BaseRecognizer.computeContextSensitiveRuleFOLLOW and BaseRecognizer.computeErrorRecoverySet. Otherwise I think you want to be operating on the follow set stack maintained by ANTLR (RecognizerSharedState.following) rather than the static follow set

[il-antlr-interest: 19220] Re: [antlr-interest] ParseTree::getLine problem

2008-08-28 Thread Thomas Brandon
On Thu, Aug 28, 2008 at 11:11 PM, Peter Bulychev <[EMAIL PROTECTED]> wrote: > Hello. > > I have a parse tree and I want to know, in which lines of parsed file do > nodes of this tree reside. > > ParseTree class itself doesn't have getLine function, but its parent > (BaseTree class) does have. > Un