Depending on your exact usage and the data contained in the CDF-5 files I guess netcdf4-python would work for reading the files (if the underlying netcdf library is compiled against pnetcdf). However, this will not immediately yield mpi features. Yet, reading different segments of files could be made embarrassingly parallel which might be OK, but defeat the purpose of your code.
Yet, why do you use python on-top of C++ for data comparison? If you need the speed (you mentioned speed) why not do it in simple C++, C or fortran? Data comparison can be made extremely easy in fortran. Sorry for the blurp ;) 2015-09-17 18:20 GMT+00:00 Joel Hermanns <joel.herma...@gmail.com>: > > > FYI, you can also see what they have done in mpi4py to by-pass this > problem. > > Could you elaborate on this or give me some pointer to other resources? > > > I would actually highly recommend you to use mpi4py rather than > implementing this from scratch your-self ;) > > I fully agree that it is a bad idea to implement something like mpi4py > from scratch. However, I don’t plan to do this and > I’m not sure if mpi4py will work for us. This problem initially came up > when > working on a thin layer around some parallel netcdf functionality to > request and compare data from NetCDF (especially CDF-5) files. > It is written in C++ due to performance reasons. Additionally, I’m not > sure if there is any up-to-date python library for parallel netcdf that > could help here. > As you can see, we don’t need full blown MPI features in python, and so > I’m not really sure if mpi4py can help us. > > Please correct me if I’m wrong! > > Best, > Joel > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/09/27613.php > -- Kind regards Nick
