Hi Jeff,
Thanks. Did you see my followup?
The code is all written the same, e.g. SPMD.
And it is just when I use a binary built at my institution
and run it at another.
Thx....John
On 2/16/16 6:35 AM, Jeff Squyres (jsquyres) wrote:
John --
+1 on what Gilles said.
The initial error says that a broadcast message was truncated. This likely
indicates that someone is calling MPI_Bcast with a different size than its
peers (it *could* indicate what Giles mentioned about
different-but-supposed-to-be-compatible-datatypes, but more often than not,
it's a simple accounting error in message lengths).
Also, as a sidenote: I notice you're running Open MPI 1.6.5. That's pretty
ancient. Any chance you can upgrade to something more modern, like Open MPI
1.10.x?
On February 15, 2016 at 7:04:15 PM, Gilles Gouaillardet (gil...@rist.or.jp)
wrote:
John,
the readv error is likely a consequence of the abort, and not the root
cause of the issue.
an obvious user error is if not all MPI tasks MPI_Bcast with non
compatible signatures.
coll/tuned module is known to be broken when using different but
compatible signatures.
for example, one process MPI_Bcast one vector of N MPI_DOUBLE, and one
other process MPI_Bcast N MPI_DOUBLE.
you can try to
mpirun --mca coll ^tuned ...
and see if it helps
fwiw, OpenMPI 1.6.5 is quite old nowadays...
Cheers,
Gilles
On 2/16/2016 7:28 AM, JR Cary wrote:
We have distributed a binary to a person with a Linux cluster. When
he runs our binary, he gets
[server1:10978] *** An error occurred in MPI_Bcast
[server1:10978] *** on communicator MPI COMMUNICATOR 8 DUP FROM 7
[server1:10978] *** MPI_ERR_TRUNCATE: message truncated
[server1:10978] *** MPI_ERRORS_ARE_FATAL: your MPI job will now abort
[server2][[14125,1],2][/..../openmpi-1.6.5/ompi/mca/btl/tcp/btl_tcp_frag.c:215:mca_btl_tcp_frag_recv]
mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104)
Anyone have any ideas on how to debug this?
Thanks......John Cary
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/02/28534.php
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/02/28535.php
--
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/02/28538.php
