On Aug 24, 4:32 pm, Thomas Jollans <tho...@jollybox.de> wrote: > On Tuesday 24 August 2010, it occurred to Darren Dale to exclaim: > > > > > > > On Aug 23, 9:58 am, Darren Dale <dsdal...@gmail.com> wrote: > > > The following script runs without problems on Ubuntu and Windows 7. > > > h5py is a package wrapping the hdf5 library (http://code.google.com/p/ > > > h5py/): > > > > from multiprocessing import Pool > > > import h5py > > > > def update(i): > > > print i > > > > def f(i): > > > "hello foo" > > > return i*i > > > > if __name__ == '__main__': > > > pool = Pool() > > > for i in range(10): > > > pool.apply_async(f, [i], callback=update) > > > pool.close() > > > pool.join() > > > > On OS X 10.6 (tested using python-2.6.5 from MacPorts), I have to > > > comment out the as-yet unused h5py import, otherwise I get a > > > traceback: > > What on earth is h5py doing there? If what you're telling us is actually > happening, and the code works 1:1 on Linux and Windows, but fails on OSX, and > you're using the same versions of h5py and Python, then the h5py > initialization code is not only enticing multiprocessing to try to pickle > something other than usual, but it is also doing that due to some platform- > dependent witchcraft, and I doubt there's very much separating the OSX > versions from the Linux versions of anything involved.
I can't find anything in the source to suggest that h5py is doing any platform-specific magic. Do you have an idea of how it would be possible for initialization code to cause multiprocessing to try to pickle something it normally would not? > > This is a really critical bug for me, but I'm not sure how to proceed. > > Can I file a bug report on the python bugtracker if the only code I > > can come up with to illustrate the problem requires a lame import of a > > third party module? > > I doubt this is an issue with Python. File a bug on the h5py tracker and see > what they say. The people there might at least have some vague inkling of what > may be going on. Thanks for the suggestion. I was in touch with the h5py maintainer before my original post. We don't have any leads. -- http://mail.python.org/mailman/listinfo/python-list
