William, Thanks for clarifying some of the details of pexpect. I do really want to understand this because I am starting to use the notebook more and currently IPython's parallel stuff works fine (there are a few things that need to be fixed on our side to make it easier though).
Moreover, as long as the worksheet and the notebook server are > distinct processes (as they should be, IMHO), the difference between > using pexpect, or xmlrpc, or anything else, for them to communicate is > completely and totally irrelevant, since it is a black box to the > entire rest of the program. > I agree with you that for the rest of the program (the notebook) this detail is completely hidden. But I guess I don't quite follow your statement that the differences of using pexpect/twisted to manage this are irrelevant. In my mind there is a big different between pexpect and twisted: * pexpect simply controls and observes the worksheet process (I now understand that this can be asynchronous). The worksheet process doesn't have *any* custom code to enable this to work and probably doesn't even import pexpect (unless it does so for a completely different reason - like talking to Mathematica, etc.). * To get Twisted to make two processes talk over TCP/IP (I am ignoring Twisted's ability to talk to a process in the same manner as pexpect, which I think it might be able to do - are you thinking of this?) *both* processes must start the Twisted reactor. Thus, if you wanted the notebook server and a worksheet process to talk over xmlrpc or pb, the worksheet process must be re-designed to run the Twisted reactor. To use IPython or dsage in a context like that, the Twisted reactor and user code must be run in different threads. This is a super subtle aspect of using Twisted in a blocking manner like the dsage and IPython clients do. I can give more details about this aspect if needed. Also, to correct another possible misconception, communication between > a processes and a subprocess using pexpect is not blocking. The > master processes can listen for however long it wants to the > subprocess, then stop listening. That's why when you do > > for i in range(10): > sleep(1) > print(i) > > in the Sage notebook, you see the output as it is computed. The > notebook server just uses pexpect to "peak" at the output of the > subprocess doing the actual work and look to see what has been output > so far. > Not that you mention this I vaguely remember this about pexpect. > Another misconception is that pexpect is restricted to local > processes. It's easy to control a process via pexpect over the > network via ssh. This has been in Sage since 2005, and can already > be used for worksheet subprocesses *now* as long as you have a shared > filesystem (just use the server_pool option). Here is an example on > the command line. I have ssh keys setup so I can do "ssh > sage.math.washington.edu" and login without typing a password. I > start Sage on my laptop in a coffee shop, and make a connection to a > remote Sage that gets started running on sage.math, and I run a > calculation. > Wow, I had no idea that this was possible! I will definitely keep this in mind as it is very nice. > flat:sageuse wstein$ sage > ---------------------------------------------------------------------- > | Sage Version 4.1.1, Release Date: 2009-08-14 | > | Type notebook() for the GUI, and license() for information. | > ---------------------------------------------------------------------- > sage: s = Sage(server="sage.math.washington.edu") > No remote temporary directory (option server_tmpdir) specified, using > /tmp/ on sage.math.washington.edu > sage: s.eval("2+2") > '4' > sage: s.eval("os.system('uname -a')") > 'Linux sage.math.washington.edu 2.6.24-23-server #1 SMP Wed Apr 1 > 22:14:30 UTC 2009 x86_64 GNU/Linux\n0' > sage: > > > The above used pexpect. You can even interact with remote objects: > > sage: e = s("EllipticCurve([1..5])") > sage: e.rank() > 1 > > You can do the same with Mathematica, etc. by the way: > > sage: s = Mathematica(server="sage.math.washington.edu") > sage: s("Factorial[50]") > 30414093201713378043612608166064768844377641568960512000000000000 > > > Compare my laptop to sage.math's mathematica: > > sage: s("Timing[Factorial[10^6]][[1]]") # sage.math > 1.1099999999999999 > sage: mathematica("Timing[Factorial[10^6]][[1]]") # laptop > 0.8902620000000001 > > (I guess Mathematica 7.0 is faster at factorials than Mathematica 6.0.) > > This tests latency: > > sage: timeit('s.eval("2+2")') # over web via ssh > 5 loops, best of 3: 56.3 ms per loop > sage: timeit('mathematica.eval("2+2")') # local > 625 loops, best of 3: 209 µs per loop > > Of course latency is long over the net, since I'm in a random coffee shop. > > This remote server stuff has been in sage since 2005, and hasn't been > changed in the slightest bit since then. That's why I'm advertising > it now, since it would be cool to see some people work on it and > improve it. For example, for people without ssh keys, one could > *easily* make it so the following works: > This is truly great stuff and I am glad you are advertising it. I need to look at pexpect more because it might come in useful for some things we do with IPython. We really like Twisted, but for somethings Twisted doesn't make sense. > So there is still some potential to the pseudotty approach to > controlling processes. The main drawback in my mind is that it > works differently (and maybe not so well) on Windows (though it does > actually work, but via the "Console API"). > Question then: are you planning on continuing to use pexpect to communicate between the notebook server and worksheet or are you planning on moving to Twisted for that? Cheers, Brian --~--~---------~--~----~------------~-------~--~----~ To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://www.sagemath.org -~----------~----~----~----~------~----~------~--~---