On Mar 7, 8:45 pm, William Stein <wst...@gmail.com> wrote:
> On Sun, Mar 7, 2010 at 5:06 PM, Gokhan Sever <gokhanse...@gmail.com> wrote:
> > Parallel Python is a separate module. IPython is the interactive
> > Python interpreter that comes with Sage. However to use the IPython
> > parallel features you need to install some additional packages. I am
> > not sure if you could get it work from within Sage. More recommended
> > way is to have an external installation.
>
> > I don't know about @parallel yet. Please do tell me and give an
> > example of its usage. I look at the Parallel Python examples and they
> > are all lengthy. multiprocessing provides much simpler solution at a
> > first sight.
>
> I wrote @parallel. Type
>
> sage: parallel?
>
> for some documentation and look at the source code in Sage.
> You just do
>
> @parallel(4)
> def f(n):
> # do some computation
>
> then
>
> for X in f([1..10]):
> print X
>
> and 4 copies of X will run at once.
>
> The actual implementation is *very* simple compared to multiprocessing
> or Parallel Python. It's about 2 pages of custom code I wrote from
> scratch just using pure Python and the fork system call (which is part
> of the os module). However, it has special support for forking Sage
> itself -- there are a number of issues having to do with pexpect
> interfaces, etc., which @parallel takes care of, but multiprocessing
> or Parallel python wouldn't know about.
>
> Another nice (imho) thing about @parallel is that it fork's a new
> process for each evaluation of the function f. This is good because
> (1) forking is cheap, (2) the entire *state* of your process is copied
> to the forked processes as is (e.g., you can have @parallel's inside
> of functions you define on the command line or notebook, etc.), (3) if
> the computation of f(n) leaks memory or segfaults, the calling program
> doesn't die.
>
> I use @parallel constantly for my research.
>
> -- William
I see that @parallel provides a very quick solution for the problem
and it is also the easiest to use. However consider my comparison (I
am running on my local Sage v4.3.3 build):
from timeit import default_timer as clock
@parallel(2)
def f(n):
return n**3 + n**2 + n
t1 = clock()
map(f, range(1000000))
t2 = clock()
print "Elapsed time:", t2-t1
#Elapsed time: 7.36224603653
#Without @parallel; Elapsed time: 4.58867001534
Seems like something wrong. Could that be due to the map usage?
from timeit import default_timer as clock
def f(x):
return x**3 + x**2 + x
t1 = clock()
map(f, range(1000000))
t2 = clock()
print "Elapsed time using single process:", t2-t1
#Elapsed time using single process: 4.68043208122
from timeit import default_timer as clock
from multiprocessing import Pool
def f(x):
return x**3 + x**2 + x
if __name__ == '__main__':
t1 = clock()
pool = Pool(processes=2) # start 2 worker processes
dummy = pool.map(f, range(1000000))
t2 = clock()
print "Elapsed time using two processes:", t2-t1
# Elapsed time using two processes: 3.65069890022
With mp module it gets faster and threading works correctly --more
processes more speed
--
To post to this group, send email to sage-support@googlegroups.com
To unsubscribe from this group, send email to
sage-support+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sage-support
URL: http://www.sagemath.org
