On Sat, Feb 23, 2008 at 11:59 PM, alex clemesha <[EMAIL PROTECTED]> wrote:
>
>
>
>
> On Sat, Feb 23, 2008 at 10:24 AM, William Stein <[EMAIL PROTECTED]> wrote:
> >
> >
> > On Sat, Feb 23, 2008 at 9:39 AM, alex clemesha <[EMAIL PROTECTED]> wrote:
> > >
> > >
> > >
> > > On Sat, Feb 23, 2008 at 12:02 AM, didier deshommes <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > >
> > > > On Fri, Feb 22, 2008 at 5:57 PM, alex clemesha <[EMAIL PROTECTED]>
> wrote:
> > > > > In Knoboo we *decouple* the idea of a kernel, it could be another
> > > > > Python (Sage) process, with communication through Pexpect
> > > > >
> > > > > ... but it also couple be another Python (Sage) process running a
> very
> > > > > minimal XML-RPC server, and all communication occurs through
> > > > > *** HTTP instead of Pexpect ***.
> > > >
> > > > I personally am not too familiar with web development, so it's always
> > > > great to hear from someone who has (which is exactly why this
> > > > discussion was started). Regarding XML-RPC vs Pexpect:
> > > > - how slow is one compared to the other? I expect xml-rpc to be
> > > > slower, but not so slow to render it unusable.
> > >
> > > I would say that it is even faster than how the current Sage notebook
> > > does its process of compiling scripts to be passed back and forth with
> > > Pexpect.
> > >
> > > But in reality, both methods of communication, in most cases,
> > > are sub-second and constant, so this is mostly a complete non-issue.
> >
> > I'm actually pretty curious about how pexpect and XMLRPC both
> > done locally compare speedwise. I've done some simple benchmarks
> > below. The short answer is that pexpect is between several hundred
> > to several thousand times faster than XMLRPC, depending on the
> > platform.
> >
> > Here's a good pexpect benchmark to do in sage-2.10.2:
> >
> > sage: gp('2+2')
> > 4
> > sage: timeit("gp.eval('2+2')")
> > 625 loops, best of 3: 136 µs per loop
> >
> > This benchmarks adding 2 and 2 over the pexpect interface
> > to pari and getting back the result. It takes 136 microseconds
> > to do that on my OS X 10.5 intel 2.6Ghz computer. On sage.math
> > it's about 500 times faster, only 303 nanoseconds:
> >
> > sage: timeit(gp.eval('2+2'))
> > 625 loops, best of 3: 303 ns per loop
> >
> > Pexpect may have a reputation for being "dog slow", but on small
> > transactions it's actually surprisingly fast. Seriously. It's only bad
> > when the input is large.
> >
> >
> > Now let's try xmlrpc (with Yi's help). Copy in the setup from
> > http://docs.python.org/lib/simple-xmlrpc-servers.html
> > then time adding two integers (we put an r after them below
> > so that they are *Python* integers, which avoid preparsing):
> >
> > On my 2.6Ghz Intel OS X computer:
> >
> > sage: timeit('s.add(2r,3r)')
> > 25 loops, best of 3: 43.7 ms per loop
> >
> > On sage.math:
> > sage: timeit('s.add(2r,3r)')
> > 625 loops, best of 3: 1.38 ms per loop
> >
> > So let's compare:
> >
> > On OS X 2.6Ghz machine pexpect is 321 times faster
> > than XMLRPC:
> > sage: 43.7 / (136 * 10^(-3))
> > 321.323529411765
> >
> > On sage.math pexpect is 4554 times faster than XMLRPC:
> > sage: 1.38 / (303 * 10^(-6))
> > 4554.45544554455
> >
> > Obviously there may be tricks for speeding up XMLRPC (and
> > for speeding up pexpect), and I would really love for an expert
> > in XMLRPC to retry the above benchmarks but with their tricks.
> > However, XMLRPC is using the TCP/IP networking stack, and
> > probably will have to use encryption if we're to do this seriously,
> > whereas pexpect is just all happening in RAM (vis unix named
> > pipes), so it's maybe not surprising that pexpect would
> > have a big advantage speedwise.
> >
> >
>
> First off, thanks for doing those benchmarks,
> they are pretty interesting to see!
>
> I want to stress that I think that this API discussion
> should really *not* have anything to do with speed,
> and this is precisely because of the use case of the API:
>
> Users evalute cells one at a time, not 625 times in 1 second ;)
Agreed. The benchmarks are just supposed to clarify which
is the right tool for the job in each case.
I noticed I made one mistake in the benchmarks. On sage.math I did
sage: timeit(gp.eval('2+2'))
625 loops, best of 3: 303 ns per loop
but should have done
sage: timeit("gp.eval('2+2')") # note the extra quotes
625 loops, best of 3: 147 µs per loop
so the time on OS X and Linux is very similar. It's still over 300
times faster to use pexpect than XMLRPC for this benchmark.
By the way, on sage.math using pexpect one can do
10,000 pexpect transactions (so more than 625)
in just over a second:
sage: time for _ in xrange(10^4): a = gp.eval('2+2')
CPU times: user 0.99 s, sys: 0.46 s, total: 1.45 s
Wall time: 1.49
As a consistency check note that
sage: 1.49/10^4 # thing above
0.000149000000000000
sage: 147.0*10^(-6) # timeit
0.000147000000000000
> The above is not meant to be precise in any way,
> just to kind of express the issue we are dealing with here.
> If both methods allow a cell's code to be transfered in
> sub-second time, let's aim for functionality, not speed.
That is very sensible. I'm definitely excited to us XMLRPC
somehow when it is the best solution. It should be fun.
-- William
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send email to sage-devel@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://www.sagemath.org
-~----------~----~----~----~------~----~------~--~---
