Ah, that did it. Thank you! On Wed, Jan 13, 2016 at 3:38 PM Matthew Flatt <mfl...@cs.utah.edu> wrote:
> You can adjust a port to count characters instead of bytes by using > `port-count-lines!`. > > At Wed, 13 Jan 2016 12:23:24 -0800 (PST), Ben Draut wrote: > > I've been tinkering with a lexer/parser for Lambda calculus expressions. > I'm > > trying to add a feature to highlight unexpected characters, rather than > making > > the user count columns. For example, the lexer will choke on this string, > > because it doesn't have a match for % anywhere: (λx.x %) > > > > I'd like to get output like this: > > > > <Some kind of explanation message>: > > > > (λx.x %) > > ^ > > > > I've hit a problem though where the srclocs returned from the lexer don't > > correspond exactly to string indices when unicode characters (such as λ) > are > > included in the expression. > > > > Take this program for example: > > > > (with-handlers ([exn:fail:read? (λ (e) (printf "ERROR: ~a~n" e))]) > > ((lexer-src-pos [(eof) 'EOF]) (open-input-string "λ"))) > > > > This outputs: > > > > ERROR: #(struct:exn:fail:read lexer: No match found in input starting > with: λ > > #<continuation-mark-set> (#(struct:srcloc #f #f #f 1 2))) > > > > You can see in the srcloc that the position is 1, (as expected) but the > span > > is 2. > > > > So let's imagine I have a lexer that understands λ, but nothing else. > When I > > try to lex the expression "λx", the lexer chokes on the x, and the srcloc > > information has position 3 and span 1. However, (string-length "λx") > evaluates > > to 2. > > > > I'm struggling to figure out how to solve the problem. I'm guessing that > the λ > > is two bytes, and the lexer counts 1 position per byte. So what's the > 'right' > > way to handle this? I could just sub1 from a position for each λ that > appeared > > in the expression beforehand, but that feels sick and wrong. > > > > Is there a way to tell the lexer to count positions on a > > character-by-character basis, rather than bytes? (Or perhaps I'm > > misunderstanding how that works too.) > > > > Any pointers would be appreciated. Thanks! > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Racket Users" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to racket-users+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
