On Apr 22, 2008, at 9:03 AM, Tomas Ukkonen wrote:
I read from somewhere that OpenMPI supports
some kind of data compression but I couldn't find
any information about it.
Is this true and how it can be used?
Nope, sorry -- not true.
This just came up in a different context, actually. We added some
preliminary compression on our startup/mpirun messages and found that
it really had no effect; any savings that you get in bandwidth (and
therefore overall wall clock time) are eaten up by the time necessary
to compress/uncompress the messages. There were a few more things we
could have tried, but frankly we had some higher priority items to
finish for the upcoming v1.3 series. :-(
Does anyone have any experiences about using it?
Is it possible to use compression in just some
subset of communications (communicator
specific compression settings)?
In our MPI application we are transferring large
amounts of sparse/redundant data that compresses
very well. Also my initial tests showed significant
improvements in performance.
If your particular data is well-suited for fast compression, you might
want to compress it before calling MPI_SEND / after calling MPI_RECV.
Use the MPI_BYTE datatype to send/receive the messages, and then MPI
won't do anything additional for datatype conversions, etc.
--
Jeff Squyres
Cisco Systems
