[Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

2007-09-18 Thread Krishna Sankar 
Folks,
   As a follow-up to the py3k discussions started by Bruce and Guido, I 
pinged Brett and he suggested I submit an exploratory proposal. Would 
appreciate insights, wisdom, the good, the bad and the ugly.

A)Does it make sense ?
B)Which application sets should we consider in designing the 
interfaces and implementations
C)In this proposal, parallelism and concurrency are used in an 
interchangeable fashion. Thoughts ?
D)Please suggest pertinent links, discussions and insights.
E)I have kept the proposal to a minimum to start the discussions and 
to explore if this is the right thing to do. Collaboratively, as we 
zero-in on one or two approaches, the idea is to expand it to a crisp 
and clear PEP. Need to do some more formatting as well.
Cheers

P.S : I had sent this to python-ideas couple of days ago and received 
two comments (Thanks Leonardo, Thanks Adam) I haven't incorporated their 
comments yet. Folks who are on both lists, pardon me for the spam.

 

PEP: 
Title: Concurrency for moderately massive (4 to 32 cores) multi-core 
architectures
Version: $Revision$
Last-Modified: $Date$
Author: Krishna Sankar ,
Status: Wandering ! (as in "Not all those who wander are lost ..." 
-J.R.R.Tolkien)
Type: Process
Content-Type: text/x-rst
Created: 15-Sep-2007

Abstract

This proposal aims at leveraging the multi-core capability as an 
embedded mechanism in python. It is not whether python is slow or fast, 
but of performance and control of parallelism/concurrency in a 
moderately massive parallelism world. The aim is 4 to 32 cores. The 
proposal advocates two mechanisms - one for task parallelism and another 
for data intensive parallelism. Scientific computing and web 2.0 
frameworks are the forefront users for this proposal. Other applications 
would benefit as well.

Rationale
-
Multicore architectures need no introductions and their ubiquity is 
evident. It is imperative that Python has one or more standard ways of 
leveraging multi-core architectures. OTOH, traditional thread based 
concurrency and lock based exclusions are becoming more and more 
difficult to program correctly.

First of all, the question is not whether py is slow or fast but 
performance of a system written in py. Which means, ability to leverage 
multi-core architectures as well as control. Control in term of things 
like ability to pin one process/task to a core, ability to pin one or 
more homogeneous tasks to specific cores et al, as well as not wait for 
a global lock and similar primitives. (Before anybody jumps into a 
conclusion, this is not about GIL by any means ;o))

Second, it is clear that we need a good solution (not THE solution) for 
moderately massive parallelism in multi-core architectures (i.e. 8-32 
cores). Share nothing might not be optimal; we need some form of memory 
sharing, not just copy all data via messages. May be functional 
programming based on the blackboard pattern would work, who knows.

I have seen systems saturated still having only ~25% of CPU utilization 
(in a 4 core system!). It is because we didn't leverage multi-cores and 
parallelism. So while py3k will not be slow, lack of a cohesive 
multi-core strategy will show up in system performance and byte us 
later(pun intended!).

At least, in my mind, this is not an exercise about exposing locks and 
mutexes or threads in Python. I do believe that the GIL will be 
refactored to more granularity in the coming months (similar to the 
Global Locks in Linux) and most probably we will get microThreads et al. 
As we all know, architecture is constraining as well as liberating. The 
language primitives influence greatly how we think about a problem.

In the discussions, Guido is right in insisting on speed, and Bruce is 
right in asking for language constructs. Without pragmatic speed, folks 
won't use it; same is the case without the required constructs. Both are 
barriers to adoption. We have an opportunity to offer a solution for 
multi-core architectures and let us seize it - we will rush in where 
angels fear to tread!

Programming Models
--
There are at least 3 possible paradigms

A. conventional threading model
B. Functional model, Erlang being the most appropriate C. Some form of 
limited shared memory model (message passing but pass pointers, 
blackboard model) D. Others, like Transactional Memory [2]

There is enough literature out there, so do not plan to explain these 
here. ( Do we need more explanation? )

Pragmatic proposal
--
May I suggest we embed two primitives in Python 3K:
A)A functional style share-nothing set of interfaces (and 
implementations thereof) - provides  the task parallelism/concurrency 
capability, "small messages, big computations" as Joe Armstrong calls it[3]
B)A limited shared memory based model for data intensive parallelism

Re: [Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

2007-09-18 Thread Guido van Rossum 
On 9/18/07, Krishna Sankar <[EMAIL PROTECTED]> wrote:
> Folks,
>As a follow-up to the py3k discussions started by Bruce and Guido, I
> pinged Brett and he suggested I submit an exploratory proposal. Would
> appreciate insights, wisdom, the good, the bad and the ugly.
>
> A)Does it make sense ?
> B)Which application sets should we consider in designing the
> interfaces and implementations
> C)In this proposal, parallelism and concurrency are used in an
> interchangeable fashion. Thoughts ?
> D)Please suggest pertinent links, discussions and insights.
> E)I have kept the proposal to a minimum to start the discussions and
> to explore if this is the right thing to do. Collaboratively, as we
> zero-in on one or two approaches, the idea is to expand it to a crisp
> and clear PEP. Need to do some more formatting as well.

I'd say it is a little light on specific proposals. The only section
that actually proposes anything is this:

> Pragmatic proposal
> --
> May I suggest we embed two primitives in Python 3K:
> A)A functional style share-nothing set of interfaces (and
> implementations thereof) - provides  the task parallelism/concurrency
> capability, "small messages, big computations" as Joe Armstrong calls it[3]
> B)A limited shared memory based model for data intensive parallelism
>
> Most probably this would be part of stdlib. While Guido is almost right
> in saying that this is a (std)library problem, it is not fully so. We
> would need a few primitives from the underlying PVM substrate. Possibly
> one reason for Guido's position is the lack of clarity as to what needs
> to be changed and why. IMHO, just saying take GIL off does not solve the
> problem either.

Before I can meaningfully comment I think I'd like to hear more about
what specifically you are thinking of.

I don't mind the necessary changes to the PVM. I do like to see how
this affects existing C extensions though.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

2007-09-18 Thread Aahz 
On Tue, Sep 18, 2007, Krishna Sankar wrote:
>
>As a follow-up to the py3k discussions started by Bruce and Guido, I 
> pinged Brett and he suggested I submit an exploratory proposal. Would 
> appreciate insights, wisdom, the good, the bad and the ugly.

This should probably start in python-ideas.
-- 
Aahz ([EMAIL PROTECTED])   <*> http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

2007-09-18 Thread Justin Tulloss 
On 9/18/07, Krishna Sankar <[EMAIL PROTECTED]> wrote:
>
> Folks,
>As a follow-up to the py3k discussions started by Bruce and Guido, I
> pinged Brett and he suggested I submit an exploratory proposal. Would
> appreciate insights, wisdom, the good, the bad and the ugly.


I am currently working on parallelizing python as an undergraduate
independent study. I plan on first removing the GIL with as little overall
effect as possible and then implementing a task-oriented threading API on
top, probably based on Stackless (since they already do a great job with
concurrency in a single thread).

If you're interested in all the details, I'd be happy to share. I haven't
gotten far yet (the semester just started!), but I feel that actually
implementing these things would be the best way to get a PEP through.

Justin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Extending Python 3000

2007-09-18 Thread Rob Crowther 
I'm attempting to wrap the GNU MP mpf_t type and related functions, as I
need the extra precision and speed for a package I'm writing. The available
wrappers are either unmaintained or undocumented and incompatible with
Python 3000. Following the docs in the Python 3000 section of the website,
I've started off with this:

*mpfmodule.c*


#include 
#include 
#include 

typedef struct {
PyObject_HEAD
mpf_t ob_val;
} MPFObject;

static PyTypeObject MPFType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
"mpf.MPF", /*tp_name*/
sizeof(MPFObject), /*tp_basicsize*/
0, /*tp_itemsize*/
0, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
0, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash */
0, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
Py_TPFLAGS_DEFAULT,/*tp_flags*/
"GNU MP mpf_t objects",/* tp_doc */
};

static PyMethodDef mpf_methods[] = {
{NULL}
};

#ifndef PyMODINIT_FUNC
#define PyMODINIT_FUNC void
#endif

PyMODINIT_FUNC initmpf(void) {
PyObject* m;

MPFType.tp_new = PyType_GenericNew;
if (PyType_Ready(&MPFType) < 0)
return;

m = Py_InitModule3("mpf", mpf_methods,
   "Wrapper around GNU MP mpf_t and related methods");

Py_INCREF(&MPFType);
PyModule_AddObject(m, "MPF", (PyObject *) &MPFType);
}



Upon running my setup.py script, it gives me numerous warnings. These do not
occur if I attempt to use Python 2.5. It also works fine under Python 2.5.


[EMAIL PROTECTED]:~/Code/mpf$ python setup.py build
running build
running build_ext
building 'mpf' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes
-fPIC -DMAJOR_VERSION=1 -DMINOR_VERSION=0 -I/usr/local/include
-I/usr/local/include/python3.0 -c mpfmodule.c -o build/temp.linux-i686-3.0
/mpfmodule.o
*mpfmodule.c:11: warning: missing braces around initializer
mpfmodule.c:11: warning: (near initialization for
‘MPFType.ob_base.ob_base’)
mpfmodule.c:13: warning: initialization makes integer from pointer without a
cast
mpfmodule.c:32: warning: initialization from incompatible pointer type*
gcc -pthread -shared build/temp.linux-i686-3.0/mpfmodule.o -L/usr/local/lib
-lgmp -o build/lib.linux-i686-3.0/mpf.so


Inside the Python interpreter:

[EMAIL PROTECTED]:~/Code/mpf/build/lib.linux-i686-3.0$ python
Python 3.0a1 (py3k, Sep 15 2007, 00:33:44)
[GCC 4.1.3 20070831 (prerelease) (Ubuntu 4.1.2-16ubuntu1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mpf
>>> test = mpf.MPF()
Traceback (most recent call last):
File "", line 1, in 
*MemoryError*
>>> mpf.MPF
*Segmentation fault*



Pointers as to what has changed and what I need to do to compile an
extension for Py3k would be very much appreciated. Thank you all in advance
for your time.

- Rob
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Extending Python 3000

2007-09-18 Thread Alexandre Vassalotti 
PyObject_HEAD was changed in Py3k to make it conform to C's strict
aliasing rules (See PEP 3123 [1]).

In your code, you need to change:

static PyTypeObject MPFType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
...
}

to this:

static PyTypeObject MPFType = {
PyVarObject_HEAD_INIT(NULL, 0)
...
}

Good luck,
-- Alexandre

[1]: http://www.python.org/dev/peps/pep-3123/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SSL certs

2007-09-18 Thread Thomas Wouters 
On 9/13/07, Bill Janssen <[EMAIL PROTECTED]> wrote:
>
> > However, there is an alternative to using multiple IP addresses:
> > one could also use multiple "subject alternative names", and create
> > a certificate that lists them all.
>
> Unfortunately, much of the client code that does the hostname
> verification is wrapped up in gullible Web browsers or Java HTTPS
> libraries that swallowed RFC 2818 whole, and not easily accessible by
> applications.  Does any of it recognize and accept "subject
> alternative name"?


For what it's worth, when I last looked at this (a year or so ago), only a
few fringe browsers on mobile phones had issues with accepting our wildcard
certificate, and some of those only because they didn't trust the root
authority.

-- 
Thomas Wouters <[EMAIL PROTECTED]>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Decimal news

2007-09-18 Thread Thomas Wouters 
On 9/13/07, Facundo Batista <[EMAIL PROTECTED]> wrote:
>
> Hi people!
>
> After some months, Decimal is now in the trunk again.
>
> It's fully updated to the latest Cowlishaw specification, and
> complying with the latest test cases (from a few days ago, which even
> take in consideration some feedback from ours).
>
> I want to thank so much to Mark Dickinson, who made *a lot* of this
> work, not only the math part (he's a mathematician himself), but also
> a lot of cleaning and speeding up.
>
> Now we will put our hands in the documentation, for it to be 100% OK
> way before 2.6 arrives.
>
> Py3 will come after that.


Unfortunately, that's not how it works :-) If you check something into the
trunk, it will be merged into Py3k sooner or later. I may ask the original
submitter for assistance if it's incredibly hard to figure out the changes,
but so far, I only had to do that with the SSL changes. The decimal changes
are being merged as I write this (tests running now.) Is there anything in
particular that needs to be done for decimal in Py3k, besides renaming
__div__ to __truediv__?

If you re-eally need to check something into the trunk that re-eally must
not be merged into py3k, but you're afraid it's not going to be obvious to
the merger, please record the change as 'merged' using "svnmerge merge -M
-r". Please take care when picking the revision ;) You can also
just email me or someone else you see doing merges, as I doubt this will be
a common occurance.

-- 
Thomas Wouters <[EMAIL PROTECTED]>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] IPv6 hostname resolution using socket.getaddrinfo()

2007-09-18 Thread O.R.Senthil Kumaran 
* [EMAIL PROTECTED] <[EMAIL PROTECTED]> :

> On python2.4.1
> 
> >>> socket.getaddrinfo('www.python.org', None, socket.AF_INET,
> socket.SOCK_DGRAM, socket.IPPROTO_IP, socket.AI_CANONNAME)
> [(2, 2, 17, 'dinsdale.python.org', ('82.94.237.218', 0))]
> >>>
> 
> Blinston.


Thanks a lot, Blinston. That helped.
I just have to take care of socket.AF_INET6 flag for IPv6 now.

>>>socket.getaddrinfo('localhost', None, socket.AF_INET6, socket.SOCK_DGRAM, 
>>>socket.IPPROTO_IP, socket.AI_CANONNAME)
[(10, 2, 17, 'localhost', ('fe80::219:5bff:fefd:6270', 0, 0, 0))]

Shall do a little more research on flags and see if documentation needs any 
update. Because current one speaks only about AI_CANONNAME being set.


Thank you. :)
Senthil



> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On
> Behalf Of O.R.Senthil Kumaran
> Sent: Monday, September 17, 2007 10:08 PM
> To: [email protected]
> Subject: [Python-Dev] IPv6 hostname resolution using
> socket.getaddrinfo()
> 
> To get the hostname, I can use socket.gethostbyname() but that has an
> inherent limitation wherein does it not support IPv6 name resolution,
> and
> getaddrinfo() should be used instead.
> 
> Looking up the socket.getaddrinfo() documentation, I come to know that
> 
> The getaddrinfo() function returns a list of 5-tuples with the following
> structure:
> 
> (family, socktype, proto, canonname, sockaddr)
> 
> family, socktype, proto are all integer and are meant to be passed to
> the socket() function. canonname is a string representing the canonical
> name of the host. It can be a numeric IPv4/v6 address when AI_CANONNAME
> is specified for a numeric host.
> 
> With this information, if I try something like this:
> 
> >>> for res in socket.getaddrinfo('goofy.goofy.com', None,
>   socket.AI_CANONNAME):
> 
> print res
> 
> (2, 1, 6, '', ('10.98.1.6', 0))
> (2, 2, 17, '', ('10.98.1.6', 0))
> (2, 3, 0, '', ('10.98.1.6', 0))
> 
> In the output, I see the cannoname to be always blank ''. I am not
> getting the IPv4 or IPv6 address as a result of using getaddrinfo().
> 
> Am I making any mistake?
> 
> What i am trying is a replacement function for
> socket.gethostbyname(hostname) which will work for both IPv4 and IPv6
> (and make changes in urllib2 to support that)
> 
> # return hostbyname for either IPv4 or IPv6 address. Common function.
> 
> def ipv6_gethostbyname(hostname):
> for res in socket.getaddrinfo(hostname,None,
>   socket.AI_CANONNAME):
> fa, socktype, proto, canonname, sa = res
> return cannoname
> 
> The above function does not seem to work. It returns blank value only.
> 
> Any help/ pointers? 
> 
> --
> O.R.Senthil Kumaran
> http://uthcode.sarovar.org
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/blinston_fernandes%40d
> ell.com

-- 
O.R.Senthil Kumaran
http://uthcode.sarovar.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SSL certs

2007-09-18 Thread Bill Janssen 
I guess something we should think about is whether to introduce RFC
2818 hostname checking into urllib.urlopen() and similar utilities.
Presumably one would add an optional arg specifying a file full of
root certs to validate against, and if that arg was present, would
retrieve the hostname info from the validated cert, and do the
client-side check.

Bill
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

2007-09-18 Thread Krishna Sankar 
Guido,
The vagueness is deliberate, to  keep the options open until we have 
some form o convergence. Parallelism/concurrency is a vast and important 
domain that I do not want to develop a hasty proposal. But I did want to 
start thinking in terms of concrete proposals, not pontifying, hence the 
"pragmatic" section.
Happy to hear that you are open to PVM changes. It will not be asked 
unless and until we all are crisp about it.
Cheers

   
Guido van Rossum wrote:
> On 9/18/07, Krishna Sankar <[EMAIL PROTECTED]> wrote:
>   
>> Folks,
>>As a follow-up to the py3k discussions started by Bruce and Guido, I
>> pinged Brett and he suggested I submit an exploratory proposal. Would
>> appreciate insights, wisdom, the good, the bad and the ugly.
>>
>> A)Does it make sense ?
>> B)Which application sets should we consider in designing the
>> interfaces and implementations
>> C)In this proposal, parallelism and concurrency are used in an
>> interchangeable fashion. Thoughts ?
>> D)Please suggest pertinent links, discussions and insights.
>> E)I have kept the proposal to a minimum to start the discussions and
>> to explore if this is the right thing to do. Collaboratively, as we
>> zero-in on one or two approaches, the idea is to expand it to a crisp
>> and clear PEP. Need to do some more formatting as well.
>> 
>
> I'd say it is a little light on specific proposals. The only section
> that actually proposes anything is this:
>
>   
>> Pragmatic proposal
>> --
>> May I suggest we embed two primitives in Python 3K:
>> A)A functional style share-nothing set of interfaces (and
>> implementations thereof) - provides  the task parallelism/concurrency
>> capability, "small messages, big computations" as Joe Armstrong calls it[3]
>> B)A limited shared memory based model for data intensive parallelism
>>
>> Most probably this would be part of stdlib. While Guido is almost right
>> in saying that this is a (std)library problem, it is not fully so. We
>> would need a few primitives from the underlying PVM substrate. Possibly
>> one reason for Guido's position is the lack of clarity as to what needs
>> to be changed and why. IMHO, just saying take GIL off does not solve the
>> problem either.
>> 
>
> Before I can meaningfully comment I think I'd like to hear more about
> what specifically you are thinking of.
>
> I don't mind the necessary changes to the PVM. I do like to see how
> this affects existing C extensions though.
>
>   

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

2007-09-18 Thread Krishna Sankar 
Justin,
Yep, trying out an implementation is a good way. Please share your 
thoughts as and when you are ready.
Cheers & good luck

Justin Tulloss wrote:
>
>
> On 9/18/07, *Krishna Sankar* <[EMAIL PROTECTED] 
> > wrote:
>
> Folks,
>As a follow-up to the py3k discussions started by Bruce and
> Guido, I
> pinged Brett and he suggested I submit an exploratory proposal. Would
> appreciate insights, wisdom, the good, the bad and the ugly.
>
>
> I am currently working on parallelizing python as an undergraduate 
> independent study. I plan on first removing the GIL with as little 
> overall effect as possible and then implementing a task-oriented 
> threading API on top, probably based on Stackless (since they already 
> do a great job with concurrency in a single thread).
>
> If you're interested in all the details, I'd be happy to share. I 
> haven't gotten far yet (the semester just started!), but I feel that 
> actually implementing these things would be the best way to get a PEP 
> through.
>
> Justin
>
>

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

2007-09-18 Thread Guido van Rossum 
On 9/18/07, Krishna Sankar <[EMAIL PROTECTED]> wrote:
> The vagueness is deliberate, to  keep the options open until we have
> some form o convergence. Parallelism/concurrency is a vast and important
> domain that I do not want to develop a hasty proposal. But I did want to
> start thinking in terms of concrete proposals, not pontifying, hence the
> "pragmatic" section.

As long as it's this vague it doesn't deserve to be called a PEP
though. PEPs can't be vague, they must make specific proposals. As
long as this is intentionally half-baked it belongs back in
python-ideas and there's no point in pretending to be writing a "PEP".

> Happy to hear that you are open to PVM changes. It will not be asked
> unless and until we all are crisp about it.
> Cheers
> 
>
> Guido van Rossum wrote:
> > On 9/18/07, Krishna Sankar <[EMAIL PROTECTED]> wrote:
> >
> >> Folks,
> >>As a follow-up to the py3k discussions started by Bruce and Guido, I
> >> pinged Brett and he suggested I submit an exploratory proposal. Would
> >> appreciate insights, wisdom, the good, the bad and the ugly.
> >>
> >> A)Does it make sense ?
> >> B)Which application sets should we consider in designing the
> >> interfaces and implementations
> >> C)In this proposal, parallelism and concurrency are used in an
> >> interchangeable fashion. Thoughts ?
> >> D)Please suggest pertinent links, discussions and insights.
> >> E)I have kept the proposal to a minimum to start the discussions and
> >> to explore if this is the right thing to do. Collaboratively, as we
> >> zero-in on one or two approaches, the idea is to expand it to a crisp
> >> and clear PEP. Need to do some more formatting as well.
> >>
> >
> > I'd say it is a little light on specific proposals. The only section
> > that actually proposes anything is this:
> >
> >
> >> Pragmatic proposal
> >> --
> >> May I suggest we embed two primitives in Python 3K:
> >> A)A functional style share-nothing set of interfaces (and
> >> implementations thereof) - provides  the task parallelism/concurrency
> >> capability, "small messages, big computations" as Joe Armstrong calls it[3]
> >> B)A limited shared memory based model for data intensive parallelism
> >>
> >> Most probably this would be part of stdlib. While Guido is almost right
> >> in saying that this is a (std)library problem, it is not fully so. We
> >> would need a few primitives from the underlying PVM substrate. Possibly
> >> one reason for Guido's position is the lack of clarity as to what needs
> >> to be changed and why. IMHO, just saying take GIL off does not solve the
> >> problem either.
> >>
> >
> > Before I can meaningfully comment I think I'd like to hear more about
> > what specifically you are thinking of.
> >
> > I don't mind the necessary changes to the PVM. I do like to see how
> > this affects existing C extensions though.
> >
> >
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

2007-09-18 Thread Krishna Sankar 
Agreed it is not a PEP yet. Hence the word "Exploration" in front of it. 
This ia domain which needs some discussions before developing a good 
PEP. May be I should call it a PEPlet ;o)
Cheers

Guido van Rossum wrote:
> On 9/18/07, Krishna Sankar <[EMAIL PROTECTED]> wrote:
>   
>> The vagueness is deliberate, to  keep the options open until we have
>> some form o convergence. Parallelism/concurrency is a vast and important
>> domain that I do not want to develop a hasty proposal. But I did want to
>> start thinking in terms of concrete proposals, not pontifying, hence the
>> "pragmatic" section.
>> 
>
> As long as it's this vague it doesn't deserve to be called a PEP
> though. PEPs can't be vague, they must make specific proposals. As
> long as this is intentionally half-baked it belongs back in
> python-ideas and there's no point in pretending to be writing a "PEP".
>
>   
>> Happy to hear that you are open to PVM changes. It will not be asked
>> unless and until we all are crisp about it.
>> Cheers
>> 
>>
>> Guido van Rossum wrote:
>> 
>>> On 9/18/07, Krishna Sankar <[EMAIL PROTECTED]> wrote:
>>>
>>>   
 Folks,
As a follow-up to the py3k discussions started by Bruce and Guido, I
 pinged Brett and he suggested I submit an exploratory proposal. Would
 appreciate insights, wisdom, the good, the bad and the ugly.

 A)Does it make sense ?
 B)Which application sets should we consider in designing the
 interfaces and implementations
 C)In this proposal, parallelism and concurrency are used in an
 interchangeable fashion. Thoughts ?
 D)Please suggest pertinent links, discussions and insights.
 E)I have kept the proposal to a minimum to start the discussions and
 to explore if this is the right thing to do. Collaboratively, as we
 zero-in on one or two approaches, the idea is to expand it to a crisp
 and clear PEP. Need to do some more formatting as well.

 
>>> I'd say it is a little light on specific proposals. The only section
>>> that actually proposes anything is this:
>>>
>>>
>>>   
 Pragmatic proposal
 --
 May I suggest we embed two primitives in Python 3K:
 A)A functional style share-nothing set of interfaces (and
 implementations thereof) - provides  the task parallelism/concurrency
 capability, "small messages, big computations" as Joe Armstrong calls it[3]
 B)A limited shared memory based model for data intensive parallelism

 Most probably this would be part of stdlib. While Guido is almost right
 in saying that this is a (std)library problem, it is not fully so. We
 would need a few primitives from the underlying PVM substrate. Possibly
 one reason for Guido's position is the lack of clarity as to what needs
 to be changed and why. IMHO, just saying take GIL off does not solve the
 problem either.

 
>>> Before I can meaningfully comment I think I'd like to hear more about
>>> what specifically you are thinking of.
>>>
>>> I don't mind the necessary changes to the PVM. I do like to see how
>>> this affects existing C extensions though.
>>>
>>>
>>>   
>> 
>
>
>   

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com