I don't have a whole lot of help for you but it does seem that at some point, this type of application demands asynchronous processing.
I had modest luck with the Background Process outlined here but eventually switched to per minute cron jobs: http://web2py.com/book/default/chapter/04#Background-Processes-and-Task-Queues At some point it would be great of Web2py was easily connectable to Celery (http://celeryproject.org/) or some sort of Delayed Job (https://github.com/tobi/delayed_job). You could either define a "queue" table or use a "status" field in your messages table. The benefit of a separate table is you can delete tasks as they are completed making the db(status='new').select() much faster. If you are up for a project, I was just reading about Redis today (http://redis.io/download). Although not currently supported by DAL (and I'm not sure if it could be shoe-horned in given some of its unique features), it's easy to get started: https://github.com/andymccurdy/redis-py The sample project is a twitter clone: http://redis.io/topics/twitter-clone and there is a Python port: https://github.com/pims/retwis-py
