Do you know the maximum message size in OpenMPI 1.6.5? I am sending large messages, and I find I receive run-time errors such as
[[23821,1],0][btl_openib_component.c:3496:handle_wc] from shelob001 to: shelob0\ 02 error polling LP CQ with status LOCAL LENGTH ERROR status number 1 for wr_id\ c628048 opcode 1 vendor error 105 qp_idx 3 [(null):15986] Attempt to free memory that is still in use by an ongoing MPI co\ mmunication (buffer 0x2b78b5a2e000, size 1106219008). MPI job will now abort. These problems go away when I reduce the message size. I check explicitly that the message size is less than INT_MAX doubles, but I may be getting close to that limit. Can I assume that sending INT_MAX doubles is fine, or will there be integer operations using int as a type that occur internally in OpenMPI? If I wanted to add a safety check before calling MPI_Isend or MPI_Irecv, what limits should I use there? For example, would checking that the message size in bytes is less than INT_MAX be more reasonable? I realize that this is a bit of a historic question (version 1.6.5). -erik -- Erik Schnetter <schnet...@gmail.com> http://www.perimeterinstitute.ca/personal/eschnetter/
