I just did some timings, and found that using a list instead of a string for tok is significantly slower (it takes 1.5x longer). Using a regex is slightly faster for long strings, and slightly slower for short ones. So, regex wins in both berevity and speed!
-- http://mail.python.org/mailman/listinfo/python-list
