Hi,
I am trying to use the lexer from the parser-tools collection.
I have written a minimal example that exhibits a behavior that seems odd to
me (code is at the end of the email).
The first two tests behave as expected but I would have expected the third
to fail. I understand that it does not, given the way the lexer is
implemented, but I cannot figure out how to change this behavior.
Could someone please give me directions to fix this ?
Thanks in advance.
Regards,
Philippe Mechaï
Sample code:
--------------------
#lang racket/base
(require racket/port
parser-tools/lex
(prefix-in : parser-tools/lex-sre))
(define-tokens data (ID NUM))
(define-empty-tokens delim (EOF))
(define sample-lexer
(lexer
[(eof) 'EOF]
[whitespace (sample-lexer input-port)]
[(:+ alphabetic) (token-ID (string->symbol lexeme))]
[(:+ numeric) (token-NUM (string->number lexeme))]))
(define (test-lexer instr)
(printf "~%Test [~a]:~%" instr)
(with-handlers ([exn:fail? (λ (e)
(eprintf "ERROR: ~a~%" (exn-message e)))])
(with-input-from-string instr
(λ ()
(printf "token: ~a~%" (sample-lexer (current-input-port)))))))
;; TEST 1
(test-lexer "*")
;; TEST 2
(test-lexer "4 a")
;; TEST 3
(test-lexer "4a")
____________________
Racket Users list:
http://lists.racket-lang.org/users
