I won't have time to work out a async proof of concept at this time. I hope to get this after some more real world profiling with my web2py app though. To give you an idea of how an async web framework could feel as natural in programming style as web2py (eg. no call backs all over the place), have a look at Concurrence documentation if you're interested: http://opensource.hyves.org/concurrence/index.html To implement async for web2py is probably for the most part straightforward (monkey patching all the IO). The trouble will be with external libraries that block and can't be monkey patched. For example db drivers. Maybe those blocking calls are best dealt with in a thread pool and queue.
The idea of Comet is to keep the connection open to the client and flow data as it becomes available: http://en.wikipedia.org/wiki/Comet_%28programming%29 It saves the overhead of a client polling at intervals and establishing the connection each time. In a thread per connection model you would need to keep a thread available per client. A thread per client can get expensive quickly and does not scale nicely. After a few hundred connections most servers slow down dramatically because of thread context switching. See also: http://www.kegel.com/c10k.html For most web apps a thread per connection (from a threadpool) won't be a problem but for for things like Ajax email applications or chat / IM it does get troublesome. On May 25, 10:59 pm, mdipierro <mdipie...@cs.depaul.edu> wrote: > On May 25, 9:24 pm, Allard <docto...@gmail.com> wrote: > > > Comet is a nice way to get this done but I wonder how to implement > > comet efficiently in web2py. > > I have never used comet but I do not see any major problem > > > Massimo, does web2py use a threadpool > > under the hood? For comet you would then quickly run out of threads. > > The web server creates a thread pool. for stand alone web2py that > would be Rocket. > You do not run out of them any more than any other web app. > > > > > If you'd try to do this with a thread per connection things would get > > out of hand pretty quickly so the best way is doing the work > > asynchronously like Orbited. Alternatives would be using one of the > > contemporary Python asynchronous libraries. These libraries provide > > monkey patching of synchronous calls like your url fetching. Some > > suggestions: > > > Gevent: now with support of Postgress, probably the fastest out there > > Eventlet: used at Lindenlab / Second Life > > Concurrence: with handy async mysql interface > > Tornado: full async webserver in Python > > > Massimo: what do you think of an asynchronous model for web2py? It'd > > be great to to have asynchronous capabilities. I am writing an app > > that will require quite a bit of client initiated background > > processing (sending emails, resizing images) which I would rather hand > > off to a green thread and not block one the web2py threads. Curious > > about your thoughts. > > I do not think we can use async IO with web2py. async IO as far as I > understand would require a different programming style. > Anyway, if you have a working proof of concept I would like to see it. > > Massimo > > > > > BTW - my first post here. Started to use for web2py for a community > > site and enjoy working in it a lot! Great work. > > > On May 25, 9:39 pm, Candid <roman.bat...@gmail.com> wrote: > > > > Well, actually there is a way for the server to trigger an action in > > > the browser. It's called comet. Of course under the hood it's > > > implemented on top of http, so it's browser who initiates request, but > > > from the developer perspective it looks like there is dual channel > > > connection between the browser and the server, and they both can send > > > messages to each other asynchronously. There are several > > > implementation of comet technology. I've used Orbited (http:// > > > orbited.org/) and it worked quite well for me. > > > > On May 25, 9:00 pm, mdipierro <mdipie...@cs.depaul.edu> wrote: > > > > > I would use a background process that does the work and adds the items > > > > to a database table. The index function would periodically refresh or > > > > pull an updated list via ajax from the database table. there is no way > > > > for te server to trigger an action in the browser unless 1) the > > > > browser initiates it or 2) the client code embeds an ajax http server. > > > > I would stay away from 1 and 2 and > > > > use reload of ajax. > > > > > On May 25, 5:33 pm, Giuseppe Luca Scrofani <glsdes...@gmail.com> > > > > wrote: > > > > > > Hi all, as promised I'm here to prove you are patient and nice :) > > > > > I' have to make this little app where there is a function that read > > > > > the html content of several pages of another website (like a spider) > > > > > and if a specified keyword is found the app refresh a page where there > > > > > is the growing list of "match". > > > > > Now, the spider part is already coded, is called search(), it uses > > > > > twill to log in the target site, read the html of a list of pages, > > > > > perform some searching procedures and keep adding the result to a > > > > > list. I integrated this in a default.py controller and make a call in > > > > > def index(): > > > > > This make the index.html page loading for a long time, because now it > > > > > have to finish to scan all pages before return all results. > > > > > What I want to achieve is to automatically refresh index every 2 > > > > > second to keep in touch with what is going on, seeing the list of > > > > > match growing in "realtime". Even better, if I can use some sort of > > > > > ajax magic to not refresh the entire page... but this is not vital, a > > > > > simple page refresh would be sufficient. > > > > > Question is: I have to use threading to solve this problem? > > > > > Alternative solutions? > > > > > I have to made the list of match a global to read it from another > > > > > function? It would be simpler if I made it write a text file, adding a > > > > > line for every match and reading it from the index controller? If I > > > > > have to use thread it will run on GAE? > > > > > > Sorry for the long text and for my bad english :) > > > > > > gls
