Hi, >From your description I think something like PyParsing might be useful to you.
http://pyparsing.wikispaces.com/ If not then I guess I m not understanding your question right. Hope this helps Nathan On 21/08/07, Robert Dailey <[EMAIL PROTECTED]> wrote: > Hi, > > I am looking for a sort of "tokenizer" for python. I've taken a look at the > tokenize module, but that seems to parse python code from what I read. I > want a tokenizer that works a little like boost::tokenizer, however for > python. Basically I want to be able to pass in an arbitrary string (or line > from readline()) and specify tokens that cause the string to be separated > into parts, much like the regular expression split() method (I think that's > the name of it). Is there anything that already exists that does this, or do > I need to implement it myself with regular expressions? > > Thanks. > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
