Simon Wittber wrote: > Michael wrote: > > Also, Paul Boddie posted a module for parallel systems a while back as well > > which might be useful (at least for ideas): > > * http://cheeseshop.python.org/pypi/parallel > > > > I've checked this out, it looks like a good idea which I could build > further on.
Feel free to expand on what I've done. The tricky part was making some kind of communications mechanism with asynchronous properties, and although I suppose I could have used asyncore, Medusa or Twisted, I was interested in the exercise of learning more about the poll system call (and without adding dependencies which dwarf the pprocess module itself). > I've just noticed that os.fork is not available on Win32. Ouch. Sorry! My motivation was to support operating systems that I personally care about and which also have solutions for transparent process migration, which I believe could be used in conjunction with the pprocess module for better-than-thread parallelisation. In other words, you get processes running independently on potentially many CPUs with only the communications as overhead, and with access to shared data either through the communications channels or via global variables which are read only due to the nature of the fork semantics. This doesn't necessarily play well with threaded-style programs wanting to modify huge numbers of shared objects, but I'd argue that the benefits of parallelisation (for performance) are somewhat reduced in such programs anyway. > Does that mean there is _no_ way for a single Python program to use > multiple CPU/core systems on Windows? (other than writing an extension > module in C (which is completely unacceptable for me!)) Rumour has it that recent versions of Windows provide fork-like semantics through a system call. Your mission is to integrate this transparently into the standard library's os.fork function. ;-) Paul -- http://mail.python.org/mailman/listinfo/python-list
