Hello all, I've been spending the last few weeks learning Python, and I've just started to use it to write a simple BASIC compiler. I'm writing a mathematical expression parser and wrote a function that would take a string and split it into high level tokens.
The code can be found at http://pastebin.com/f757252ec (have a look at the function tokenize). I was interested to see if the tokenize function could be written in a better way, so as to make better use of the Python language and libraries. One thing that sprung to mind was whether it would be a good idea to perhaps take advantage of regular expressions. I had a look at a few compiler sites and regexps seem to come into that role quite a bit. I've only done a little work on regexps and they seem very powerful, but can easily get out of control! So, any suggestions as to how to make better use of the language writing something like this? Thanks, --Amr -- http://mail.python.org/mailman/listinfo/python-list
