On Wed, Jul 22, 2009 at 2:19 PM, ghtdak<gl...@tarbox.org> wrote: > > > > On Jul 22, 11:18 am, William Stein <wst...@gmail.com> wrote: >> On Wed, Jul 22, 2009 at 11:12 AM, ghtdak<gl...@tarbox.org> wrote: >> >> > On Jul 21, 6:40 pm, "Dr. David Kirkby" <david.kir...@onetel.net> >> > wrote: >> >> ghtdak wrote: >> >> >> > This thread has gotten long and there are many subjects embedded >> >> > within. >> >> >> > One of the problems I've had with the notebook implementation is that >> >> > the sage process supporting the notebook computation blocks on the >> >> > pipe between itself and the twistd server which spawns it. This means >> >> > that one can't build an asynchronous event handler without using >> >> > threads. >> >> >> > More conventional web server architectures use a callback style for >> >> > invocation making it much easier to splice in other events for >> >> > handling by the main thread (this is the general asynchronous >> >> > programming model and the heart of how Twisted works) >> >> >> > Perhaps it is a foregone conclusion that this approach will be taken >> >> > in the rewrite. if not, I'd like to suggest that it is an important >> >> > consideration. >> >> >> > -glenn >> >> >> I don't know about how the web server is implemented. I know it did not >> >> work on my Solaris box, but that is another matter. >> >> >> But actually including Apache might be a sensible choice. A lot of >> >> people know how to administer Apache. It offers a lot of flexibility. >> >> You can for example only serve pages to particular IP addresses. >> >> >> Worth a thought anyway. >> >> > I came across this post from the Twisted folks. It looks like they do >> > WSGI and run Django quite well... >> >> >http://blog.dreid.org/2009/03/twisted-django-it-wont-burn-down-your.html >> >> > At least there are alternatives to apache which might be simpler. >> >> > The core question remains: While the presentation layer support is >> > defined by django, pyjamas etc, the integration with the underlying >> > sage process process is the issue, not the web presentation. >> >> > In the current architecture, a twistd daemon spawns a notebook server >> > which is responsible for doing "sage" stuff. twistd is fully >> > asynchronous, but the notebook process itself is a pexpect based >> > blocking process connected with pipes to twistd. As such, the block >> > on read by pexpect precludes the sage process servicing asynchronous >> > events. >> >> Here's the relevant code that you're talking about (starting at line >> 3007 of worksheet.py): >> >> try: >> done, out, new = S._so_far(wait=wait, >> alternate_prompt=SAGE_END+str(self.synchro())) >> except RuntimeError, msg: >> verbose("Computation was interrupted or failed. >> Restarting.\n%s"%msg) >> self.__comp_is_running = False >> self.start_next_comp() >> return 'w', C >> >> The Sage notebook server does a blocking read from the >> pexpect-controlled subprocess for "wait" seconds. The default value >> of wait is 0.2 seconds. So indeed there is a block, but it is never >> for more than 0.2 seconds. During that time other asynchronous >> events can't be dealt with. > > I'm not sure I understand, so let me restate what I think you're > saying: > > The notebook "server" is doing a blocking read with a timeout. The > server is a twistd process which multiplexes and processes in support > the browsers.
Exactly right. > The "pexpect-controlled subprocess" is where Sage itself is. Yes. > The issue I was raising was with the latter (subprocess) although I > now think the server implementation might also be an issue. > > when twisted spawns a sub-process, typically one registers a callback > to handle information from that sub-process. Twisted does not spawn the subprocess that does the computations. The notebook server spawns that subprocess and controls it using pexpect. > There should be no > reason to check for input. Generally a callback for signals is also > registered to sense the sub-process dying, the pipe being disconnected > etc. Unless I misunderstand (very possible), you're "polling" the set > of spawned Sage sub-processes for data. Correct. > While that doesn't sound good either, thats not the problem I was > describing. > > My primary problem is that the Sage subprocess is blocking forever on > the other side of the pipe when its not computing... Therefore, I > can't have a Sage sub-process that I'm using in the notebook that is > also able to communicate with other processes as I can't > asynchronously receive data (or get timing interrupts). I've gotten > around this in the past by using threads as it was the only choice I > had. Thanks for the clarification. Since I don't really understand the problem, without further clarification I don't think it will get fixed in the near future. William > > -glenn > >> >> >> >> > IMHO, this architecture is incorrect and limited... Perhaps this is >> > part of what is being rethought... if not, I believe it should be. >> >> > A preferable architecture is an event loop which dispatches requests >> > within the sage process. Since Sage is written in python, I would >> > suggest Twisted for this but there might be better alternatives (I'd >> > be surprised, but its possible) >> >> > Using this approach, one could easily add other elements to the core >> > event loop to support asynchronous processing (timers, communication, >> > etc) without threads which are, in this case, unnecessary. Threads >> > when necessary are bad enough, when they're introduced because of >> > unnecessary blocking, one gets all the threading nightmare without any >> > benefit. (reminds me of health care reform) >> >> > Another benefit is that since asynchronous event processing is the >> > widely accepted approach to this type of problem, there are lots of >> > libraries / packages to "make it so". >> >> > -glenn >> >> >> dave >> >> -- >> William Stein >> Associate Professor of Mathematics >> University of Washingtonhttp://wstein.org > > > -- William Stein Associate Professor of Mathematics University of Washington http://wstein.org --~--~---------~--~----~------------~-------~--~----~ To post to this group, send email to sage-devel@googlegroups.com To unsubscribe from this group, send email to sage-devel-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://www.sagemath.org -~----------~----~----~----~------~----~------~--~---
