> In the current architecture, a twistd daemon spawns a notebook server > which is responsible for doing "sage" stuff. twistd is fully > asynchronous, but the notebook process itself is a pexpect based > blocking process connected with pipes to twistd. As such, the block > on read by pexpect precludes the sage process servicing asynchronous > events. > > IMHO, this architecture is incorrect and limited... Perhaps this is > part of what is being rethought... if not, I believe it should be. >
As an avid Twisted user, I too thought this initially (why use pexpect, when you could use Twisted). But after looking at this issue further, I think using pexpect is not that bad. Here is why: 1. If you were to use Twisted, while the process was running user's code, Twisted would still block. Using threads (running the Twisted event loop in a thread) only partially solves this problem as the python intepreter can't switch threads while no GIL-releasing C/C++ code is running. We ran into this in early versions of IPython's parallel stuff - it worked great (asynch) until the second we went to do something like diagonalize a matrix using scipy. Then everything would block. We have had to work very hard to get around this GIL induced limitation of using Twisted. 2. Both dsage and parallel ipython clients use Twisted. For this to work, these clients need to run the Twisted reactor in a different thread than user code is executed. Currently, these work fine in the notebook, because they can start the reactor in this way by themselves. If the notebook itself used Twisted, great care would need to be used to make sure these things still worked. You would have to run user code in the main thread and run all the twisted stuff in a different thread. User code needs to be in the main thread if you want users to be able to run real GUI code (I do this sometimes!). To summarize, you could implement this by running user code in the main thread and and the twisted reactor in a second thread *but*, you don't gain much over pexpect because everything will still block when non-GIL releasing C code is run. Furthermore, pexpect is pretty simple and just works. With that said, there are other reasons that doing this using Twisted would be nice. Namely, you could potentially distribute the running of user code to other hosts. That would be very nice. Cheers, Brian --~--~---------~--~----~------------~-------~--~----~ To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://www.sagemath.org -~----------~----~----~----~------~----~------~--~---
