I am pleased to announce the 1.0 version of pytoken. It is available here:
http://code.google.com/p/pytoken/downloads/list What is pytoken Pytoken is a scanner generator. Given an input specification - a bunch of regular expressions - pytoken will generate x86 machine code that recognizes those regular expressions. Pytoken will be most useful for programmers that want to parse complex text files. Pytoken has separate objects for scanners and buffers. Here is a simple example: import pytoken lexer_obj = pytoken.lexer() lexer_obj.add_pattern("a", 1) lexer_obj.add_pattern("b", 2) lexer_obj.compile_to_machine_code() buf = pytoken.lexer_state() buf.set_input("ab") tok = lexer_obj.get_token(buf) assert tok == 1 tok = lexer_obj.get_token(buf) assert tok == 2 Pytoken has been written in a portable fashion - it is designed to support multiple CPU types, even though only the x86 (32 bit) is supported now. -Ram -- http://mail.python.org/mailman/listinfo/python-list
