Simplest way is to make the keywords regular expressions that look for a
"word boundary" after the keyword:
(def unambiguous-tokenizer-improved
(insta/parser
"sentence = token (<whitespace> token)*
<token> = keyword | !keyword identifier
whitespace = #'\\s+'
identifier = #'[a-zA-Z]+'
keyword = #'cond\\b' | #'defn\\b'"))
On Mon, Nov 18, 2013 at 1:20 AM, <eliassona...@yahoo.com> wrote:
> Hi,
>
> I'm trying to use instaparse to differentiate between identifiers and
> keywords.
>
> The following code is from the tutorial.
>
> (def unambiguous-tokenizer
> (insta/parser
> "sentence = token (<whitespace> token)*
> <token> = keyword | !keyword identifier
> whitespace = #'\\s+'
> identifier = #'[a-zA-Z]+'
> keyword = 'cond' | 'defn'"))
>
> The above parser works fine for:
>
> (insta/parse unambiguous-tokenizer "cond id defn")
>
> It recognizes cond and defn as keywords and id as identifier.
>
> But if an identifier starts with a keyword such as condid:
>
> (insta/parse unambiguous-tokenizer "condid id defn")
>
> It doesn't work anymore. (I want it to recognize condid as an identifier
> not a misspelled keyword)
>
> Does anybody know how to make that work?
>
> Thanks
>
> --anders
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
