Hello everyone! I'm new to Scheme, and I need some help wrapping my head around
it.
I want to write a simple tokenizer, which is just a function that takes a
string as input and outputs a list of "tokens", which to keep thing simple is a
tuple of NAME and VALUE.
So, for example:
(tokenize "foo = 1") => '((IDENTIFIER . "foo") (EQUALS . "=") (NUMBER . 1))
I didn't want to use global variables to keep things functional, so my solution
doesn't use any, but I wouldn't mind much using them. This is some code for my
current implementation, I omit some code for brevity as it's quite repetitive
for now:
(define (match-identifier input)
(regexp-match #rx"^[a-zA-Z_]+"))
(define (consume-identifier input)
(let ([match (first (match-identifier input))])
(cons `(IDENTIFIER . ,match) (tokenize (substring input (string-
length match))))))
;; some more matchers and consumers, all pretty similar
;; ...
(define (tokenize input)
(cond
((match-identifier input) (consume-identifier input))
((match-equals input) (consume-equals input))
((match-number input) (consume-number input))
(else '())))
What bothers me is that it's calling the matchers twice for each token, which
isn't very good for performance and it's not pretty :p
I want to get rid of that double-check. I figured I could use some globals in
order to do it but I'd rather not.
Does anyone have a suggestion/fix on my implementation?
Thank you for your time :)
--
You received this message because you are subscribed to the Google Groups
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
