Steven D'Aprano <ste...@remove.this.cybersource.com.au> wrote:
> On Tue, 19 May 2009 05:52:04 -0500, Grant Edwards wrote:
>
> > On 2009-05-19, Steven D'Aprano <ste...@remove.this.cybersource.com.au>
> > wrote:
> >> On Mon, 18 May 2009 02:27:06 -0700, jeremy wrote:
> >>
> >>> Let me clarify what I think par, pmap, pfilter and preduce would mean
> >>> and how they would be implemented.
> >> [...]
> >>
> >> Just for fun, I've implemented a parallel-map function, and done a
> >> couple of tests. Comments, criticism and improvements welcome!
> >
> > My only comment would be that your "slow function" might not be a very
> > simulation for the general-case, since it uses time.sleep() which
> > releases the GIL:
>
>
> I didn't expect my code to magically overcome fundamental limitations of
> the CPython interpreter :)
>
>
>
> >> def f(arg): # Simulate a slow function.
> >> time.sleep(0.5)
> >> return 3*arg-2
> >
> > Any Python function that isn't calling a library function written in C
> > that releases the GIL won't show any speedup will it?
>
> Not necessarily. Here's another function, that uses a loop instead of
> sleep.
>
> def g(arg, SIZE=8*10**6):
> # Default SIZE is chosen so that on my machine, the loop
> # takes approximately 0.5 second.
> for x in xrange(SIZE):
> pass
> return 3*arg-2
>
>
> >>> setup = 'from __main__ import pmap, g; data = range(50)'
> >>> min(Timer('map(g, data)', setup).repeat(repeat=5, number=3))
> 65.093590974807739
> >>> min(Timer('pmap(g, data)', setup).repeat(repeat=5, number=3))
> 20.268381118774414
I don't think that can be right - that shows python working without
the GIL contention. So unless you ran it under IronPython?
Here is what happens when I run it under CPython 2.5 on my dual core
laptop. I made SIZE=10**6 because I got bored of waiting ;-)
map
9.85280895233
pmap
28.4256689548
So the pmap took nearly 3 times as long. I expect this is because the
task was divided into 5 sections each competing madly for the GIL.
I ran the same script under the latest jython beta which was very
interesting! pmap showing a slight improvement, and faster than
cPython!
$ jython2.5rc2/jython pmap.py
map
6.242000103
pmap
5.88800001144
--
Nick Craig-Wood <n...@craig-wood.com> -- http://www.craig-wood.com/nick
--
http://mail.python.org/mailman/listinfo/python-list
