Lately I have been doing a great deal of MPI debugging. I have, on an
occasion or two, fallen into the trap of "Well, that error MUST be
coming from rank X. There is no way it could be coming from any other
rank..." Then proceeding to debug what's happening at rank X, only to
find out a few frustrating hours later that rank Y is throwing the
output (I'm sure no one else out there has fallen into this trap). It
was at that point, I decided to write up some code to automatically
(sort of) output the rank and size of my domain with every output. I
write mostly in C++, and this is what I came up with:
#include <iostream>
#include <mpi.h>
std::ostream &mpi_info(std::ostream &s) {
int rank, size;
rank = MPI::COMM_WORLD.Get_rank();
size = MPI::COMM_WORLD.Get_size();
s << "[rk:" << rank << ",sz:" << size << "]: ";
return s;
}
Then in my code, I have changed:
std::cerr << "blah" << std::endl;
to:
std::cerr << mpi_info << "blah" << std::endl;
(or cout, or file stream, etc...)
where "blah" is some amazingly informative error message.
Are there other ways people do this? Simpler ways perhaps?
-Mark
