Re: [OMPI users] MX replacement?
Rob Latham writes: >> Assuming MPI-IO, to be on-topic, is MX known to have any real advantage >> over TCP filesystem drivers, e.g. Lustre and PVFS? > > Both lustre (portals) and PVFS (BMI) have transport abstraction > layers. File systems benefit from native transport (compared to > ip-over-whatever) both in terms of latency (how fast metadata > operations take) and in bandwidth (moving lots of data to/from file > system). > > not sure if that's what you were asking... Yes. I just haven't seen numbers to know how significant the effect is in that case, apart from some for NFS RDMA. I'd rather expect MX support to be broken now anyway; apparently even IB is in released orange PVFS which I'd intended to measure.
Re: [OMPI users] PVFS/OrangeFS
Rob Latham writes: >> [I wonder why ANL apparently dumped PVFS but I probably shouldn't ask.] > > No harm in asking. I wrongly guessed it was political! > This is getting a bit off topic for an OpenMPI list > but OMPIO and ROMIO both support PVFS so perhaps it's interesting to > some. At least to me, and I think not too off-topic from an MPI-IO angle. Thanks for taking the time to record it. PVFS seems valuable particularly for the ability to spin up a decent MPI-IO filesystem on node local disks in userland. > PVFS still lives, now branded "OrangeFS". The folks at Clemson are > working towards getting PVFS support into the linux > kernel. (http://lwn.net/Articles/657333/ : Linus says "the code > doesn't look obviously horrible" -- how's *that* for an endorsement) Yes, but perhaps he doesn't realize it's basically a userland filesystem, and thus a "toy" (quoted somewhere by Jeff Darcy, if I recall correctly).
Re: [OMPI users] cleaning up old ROMIO (MPI-IO) drivers
Rob Latham writes: > PLFS folks seem to think it's not worth the effort? > > https://github.com/plfs/plfs-core/issues/361 I guess they'd know, but I'm a bit puzzled as EMC seemed to be relying on it (re-named?) fairly recently. It's in the latest ompio anyhow.
Re: [OMPI users] Open MPI backwards incompatibility issue in statically linked program
Jeff Hammond writes: > You can rely upon e.g. https://www.mpich.org/abi/ when redistributing MPI > binaries built with MPICH, but a better option would be to wrap all of your > MPI code in an implementation-agnostic wrapper and then ship a binary that > can dlopen a different version wrapper depending on which MPI > implementation the user has. That would allow you to ship a single binary > that could use both MPICH and OpenMPI. I don't know how it works with MPICH, but in OMPI don't you need all of the submitting orterun, the linked libraries, and the target ORTE to be compatible, which a wrapper would need to know? [How is the Fortran ABI fixed anyhow?] I don't think there's a check on that currently, which would be useful if the info can be propagated. There have been issues on the list due to host and target ORTE being different, and I've seen them locally. Producing a clear error with incompatible components (to the extent it's possible) could save confusion.
Re: [OMPI users] Open MPI backwards incompatibility issue in statically linked program
Yes, that is correct: the Open MPI libraries used by all components (to include mpirun and all the individual MPI processes) must be compatible. We usually advise users to use the same version across the board, and that will make things work. More specifically: it's not enough to statically link your MPI application; you would also need to statically link mpirun and the orteds, too (and/or distribute binaries for the entire Open MPI install, even if they're dynamically linked). On February 15, 2016 at 11:51:39 AM, Dave Love (d.l...@liverpool.ac.uk) wrote: > Jeff Hammond writes: > > > You can rely upon e.g. https://www.mpich.org/abi/ when redistributing MPI > > binaries built with MPICH, but a better option would be to wrap all of your > > MPI code in an implementation-agnostic wrapper and then ship a binary that > > can dlopen a different version wrapper depending on which MPI > > implementation the user has. That would allow you to ship a single binary > > that could use both MPICH and OpenMPI. > > I don't know how it works with MPICH, but in OMPI don't you need all of > the submitting orterun, the linked libraries, and the target ORTE to be > compatible, which a wrapper would need to know? [How is the Fortran ABI > fixed anyhow?] > > I don't think there's a check on that currently, which would be useful > if the info can be propagated. There have been issues on the list due > to host and target ORTE being different, and I've seen them locally. > Producing a clear error with incompatible components (to the extent it's > possible) could save confusion. > ___ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2016/02/28532.php > -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
[OMPI users] readv failed How to debug?
We have distributed a binary to a person with a Linux cluster. When he runs our binary, he gets [server1:10978] *** An error occurred in MPI_Bcast [server1:10978] *** on communicator MPI COMMUNICATOR 8 DUP FROM 7 [server1:10978] *** MPI_ERR_TRUNCATE: message truncated [server1:10978] *** MPI_ERRORS_ARE_FATAL: your MPI job will now abort [server2][[14125,1],2][//openmpi-1.6.5/ompi/mca/btl/tcp/btl_tcp_frag.c:215:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104) Anyone have any ideas on how to debug this? Thanks..John Cary
Re: [OMPI users] readv failed How to debug?
John, the readv error is likely a consequence of the abort, and not the root cause of the issue. an obvious user error is if not all MPI tasks MPI_Bcast with non compatible signatures. coll/tuned module is known to be broken when using different but compatible signatures. for example, one process MPI_Bcast one vector of N MPI_DOUBLE, and one other process MPI_Bcast N MPI_DOUBLE. you can try to mpirun --mca coll ^tuned ... and see if it helps fwiw, OpenMPI 1.6.5 is quite old nowadays... Cheers, Gilles On 2/16/2016 7:28 AM, JR Cary wrote: We have distributed a binary to a person with a Linux cluster. When he runs our binary, he gets [server1:10978] *** An error occurred in MPI_Bcast [server1:10978] *** on communicator MPI COMMUNICATOR 8 DUP FROM 7 [server1:10978] *** MPI_ERR_TRUNCATE: message truncated [server1:10978] *** MPI_ERRORS_ARE_FATAL: your MPI job will now abort [server2][[14125,1],2][//openmpi-1.6.5/ompi/mca/btl/tcp/btl_tcp_frag.c:215:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104) Anyone have any ideas on how to debug this? Thanks..John Cary ___ users mailing list us...@open-mpi.org Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post: http://www.open-mpi.org/community/lists/users/2016/02/28534.php
