In message <[EMAIL PROTECTED]>
Simon Cozens <[EMAIL PROTECTED]> wrote:
> Parsing Perl is not easy. :)
You can say that again ;-)
> At some points, you have to say, well, heck, I don't *know* what this token
> is. At the moment, perl guesses, and it guesses reasonably well. But
> guessing something wrongly which you could have got right if you'd read the
> next line strikes me as a little anti-DWIM.
Quite likely you're right. I can't say I have much experience of
parsers that do this but we can always blaze a new trail in our
efforts to parse perl.
> In a sense, though, you're right; this is a general problem. I'm currently
> trying to work out a design for a tokeniser, and it seems to me that
> there's going to be a lot of communicating of "hints" between the
> tokeniser, the lexer and the parser.
You have to be vary careful about downward communication from the
parser to the lexer if there's any lookahead involved as you can
find that you're trying to affect the lexing of tokens which are
already in the lookahead buffer of the parser.
Backtracking may well be better than lookahead here as you can always
jump back a bit after you change the lexer's state ;-)
> Parsing Perl is hard. Trust me. :)
Oh, you did say it again...
Parsing Fortran is fun as well. Whoever decided to allow spaces in
identifiers needs their head read...
Tom
--
Tom Hughes ([EMAIL PROTECTED])
http://www.compton.nu/
...Who's on first?
