I've been frustrated with trying to compose lexers together. The
situation is that I'd like to define a lexer that knows how to give up
gracefully, and to give another lexer a go at processing a port.
Ideally, I'd like to say something like this:
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(define lexer-1
(lexer ...
[else (lexer-2 ip)]))
(define lexer-2
(lexer ...
[else (error 'i-dunno)]))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
As far as I can tell from reading the documentation, I can't make this
work nicely. That is, I do not want lexer-1 to consume a character
when taking an alternative.
So putting:
;;;;;;
(define lexer-1
(lexer ...
[any-char (lexer-2 ip)]))
;;;;;;
despite what the documentation says about how this helps me handle the
error condition, is not a solution that sits right with me. It
creates a non-composable situation with any other lexers I'd like to
work with.
To make it compose, I need to somehow get that character back into the
port. I don't know how to efficiently push that character back into
the port besides using something like racket/port's input-port-append,
which is a solution, but not a simple one nor one that immediately
comes to mind.
The design of the parser-tools/lex library feels like it values
simplicity, so I'm wondering: is there a technical reason why
parser-tools/lex doesn't have an "else"?
____________________
Racket Users list:
http://lists.racket-lang.org/users
