William,

Thanks for clarifying some of the details of pexpect.  I do really want to
understand this because I am starting to use the notebook more and currently
IPython's parallel stuff works fine (there are a few things that need to be
fixed on our side to make it easier though).

Moreover, as long as the worksheet  and the notebook server are
> distinct processes (as they should be, IMHO), the difference between
> using pexpect, or xmlrpc, or anything else, for them to communicate is
> completely and totally irrelevant, since it is a black box to the
> entire rest of the program.
>

I agree with you that for the rest of the program (the notebook) this detail
is completely hidden.  But I guess I don't quite follow your statement that
the differences of using pexpect/twisted to manage this are irrelevant.  In
my mind there is a big different between pexpect and twisted:

* pexpect simply controls and observes the worksheet process (I now
understand that this can be asynchronous).  The worksheet process doesn't
have *any* custom code to enable this to work and probably doesn't even
import pexpect (unless it does so for a completely different reason - like
talking to Mathematica, etc.).

* To get Twisted to make two processes talk over TCP/IP (I am ignoring
Twisted's ability to talk to a process in the same manner as pexpect, which
I think it might be able to do - are you thinking of this?) *both* processes
must start the Twisted reactor.  Thus, if you wanted the notebook server and
a worksheet process to talk over xmlrpc or pb, the worksheet process must be
re-designed to run the Twisted reactor.  To use IPython or dsage in a
context like that, the Twisted reactor and user code must be run in
different threads.  This is a super subtle aspect of using Twisted in a
blocking manner like the dsage and IPython clients do.  I can give more
details about this aspect if needed.

Also, to correct another possible misconception, communication between
> a processes and a subprocess using pexpect is not blocking.  The
> master processes can listen for however long it wants to the
> subprocess, then stop listening.   That's why when you do
>
> for i in range(10):
>      sleep(1)
>      print(i)
>
> in the Sage notebook, you see the output as it is computed.  The
> notebook server just uses pexpect to "peak" at the output of the
> subprocess doing the actual work and look to see what has been output
> so far.
>

Not that you mention this I vaguely remember this about pexpect.


> Another misconception is that pexpect is restricted to local
> processes.  It's easy to control a process via pexpect over the
> network via ssh.   This has been in Sage since 2005, and can already
> be used for worksheet subprocesses *now* as long as you have a shared
> filesystem (just use the server_pool option).  Here is an example on
> the command line.  I have ssh keys setup so I can do "ssh
> sage.math.washington.edu" and login without typing a password.   I
> start Sage on my laptop in a coffee shop, and make a connection to a
> remote Sage that gets started running on sage.math, and I run a
> calculation.
>

Wow, I had no idea that this was possible!  I will definitely keep this in
mind as it is very nice.


> flat:sageuse wstein$ sage
> ----------------------------------------------------------------------
> | Sage Version 4.1.1, Release Date: 2009-08-14                       |
> | Type notebook() for the GUI, and license() for information.        |
> ----------------------------------------------------------------------
> sage: s = Sage(server="sage.math.washington.edu")
> No remote temporary directory (option server_tmpdir) specified, using
> /tmp/ on sage.math.washington.edu
> sage: s.eval("2+2")
> '4'
> sage: s.eval("os.system('uname -a')")
> 'Linux sage.math.washington.edu 2.6.24-23-server #1 SMP Wed Apr 1
> 22:14:30 UTC 2009 x86_64 GNU/Linux\n0'
> sage:
>
>
> The above used pexpect.  You can even interact with remote objects:
>
> sage: e = s("EllipticCurve([1..5])")
> sage: e.rank()
> 1
>
> You can do the same with Mathematica, etc. by the way:
>
> sage: s = Mathematica(server="sage.math.washington.edu")
> sage: s("Factorial[50]")
> 30414093201713378043612608166064768844377641568960512000000000000
>
>
> Compare my laptop to sage.math's mathematica:
>
> sage: s("Timing[Factorial[10^6]][[1]]")                     # sage.math
> 1.1099999999999999
> sage: mathematica("Timing[Factorial[10^6]][[1]]")  # laptop
> 0.8902620000000001
>
> (I guess Mathematica 7.0 is faster at factorials than Mathematica 6.0.)
>
> This tests latency:
>
> sage: timeit('s.eval("2+2")')        # over web via ssh
> 5 loops, best of 3: 56.3 ms per loop
> sage: timeit('mathematica.eval("2+2")')  # local
> 625 loops, best of 3: 209 µs per loop
>
> Of course latency is long over the net, since I'm in a random coffee shop.
>
> This remote server stuff has been in sage since 2005, and hasn't been
> changed in the slightest bit since then.  That's why I'm advertising
> it now, since it would be cool to see some people work on it and
> improve it.  For example, for people without ssh keys, one could
> *easily* make it so the following works:
>

This is truly great stuff and I am glad you are advertising it.  I need to
look at pexpect more because it might come in useful for some things we do
with IPython.  We really like Twisted, but for somethings Twisted doesn't
make sense.


> So there is still some potential to the pseudotty approach to
> controlling processes.     The main drawback in my mind is that it
> works differently (and maybe not so well) on Windows (though it does
> actually work, but via the "Console API").
>

Question then:  are you planning on continuing to use pexpect to communicate
between the notebook server and worksheet or are you planning on moving to
Twisted for that?

Cheers,

Brian

--~--~---------~--~----~------------~-------~--~----~
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel-unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://www.sagemath.org
-~----------~----~----~----~------~----~------~--~---

Reply via email to