Dear Edgar,
On 12/09/15 16:16, Edgar Gabriel wrote:
I tested your code in master and v1.10 ( on my local machine), and I get for both version of ompio exactly the same (correct) output that you had with romio.
I've tested it at local hard disk.. pk224850@cluster:/tmp/pk224850/cluster_15384/TMP[529]$ df -h . Filesystem Size Used Avail Use% Mounted on /dev/sda3 1.1T 16G 1.1T 2% /w0pk224850@cluster:/tmp/pk224850/cluster_15384/TMP[530]$ echo hell-o > out.txt; ./a.out
fileOffset, fileSize 7 7 fileOffset, fileSize 23 23 ierr 0 MPI_MODE_WRONLY, MPI_MODE_APPEND 4 128 pk224850@cluster:/tmp/pk224850/cluster_15384/TMP[531]$ export OMPI_MCA_io=ompiopk224850@cluster:/tmp/pk224850/cluster_15384/TMP[532]$ echo hell-o > out.txt; ./a.out
fileOffset, fileSize 0 7 fileOffset, fileSize 0 16 ierr 0 MPI_MODE_WRONLY, MPI_MODE_APPEND 4 128
However, I also noticed that in the ompio version that is in the v1.10 branch, the MPI_File_get_size function is not implemented on lustre.
Yes we have Lustre in the cluster.I believe that was one of 'another' issues mentioned, yes some users tend to use Lustre as HPC file system =)
Thanks Edgar On 12/9/2015 8:06 AM, Edgar Gabriel wrote:I will look at your test case and see what is going on in ompio. That being said, the vast number of fixes and improvements that went into ompio over the last two years were not back ported to the 1.8 (and thus 1.10) series, since it would have required changes to the interfaces of the frameworks involved (and thus would have violated one of rules of Open MPI release series) . Anyway, if there is a simple fix for your test case for the 1.10 series, I am happy to provide a patch. It might take me a day or two however. Edgar On 12/9/2015 6:24 AM, Paul Kapinos wrote:Sorry, forgot to mention: 1.10.1 Open MPI: 1.10.1 Open MPI repo revision: v1.10.0-178-gb80f802 Open MPI release date: Nov 03, 2015 Open RTE: 1.10.1 Open RTE repo revision: v1.10.0-178-gb80f802 Open RTE release date: Nov 03, 2015 OPAL: 1.10.1 OPAL repo revision: v1.10.0-178-gb80f802 OPAL release date: Nov 03, 2015 MPI API: 3.0.0 Ident string: 1.10.1 On 12/09/15 11:26, Gilles Gouaillardet wrote:Paul, which OpenMPI version are you using ? thanks for providing a simple reproducer, that will make things much easier from now. (and at first glance, that might not be a very tricky bug) Cheers, Gilles On Wednesday, December 9, 2015, Paul Kapinos <kapi...@itc.rwth-aachen.de <mailto:kapi...@itc.rwth-aachen.de>> wrote: Dear Open MPI developers, did OMPIO (1) reached 'usable-stable' state? As we reported in (2) we had some trouble in building Open MPI with ROMIO, which fact was hidden by OMPIO implementation stepping into the MPI_IO breach. The fact 'ROMIO isn't AVBL' was detected after users complained 'MPI_IO don't work as expected with version XYZ of OpenMPI' and further investigations. Take a look at the attached example. It deliver different result in case of using ROMIO and OMPIO even with 1 MPI rank on local hard disk, cf. (3). We've seen more examples of divergent behaviour but this one is quite handy. Is that a bug in OMPIO or did we miss something? Best Paul Kapinos 1) http://www.open-mpi.org/faq/?category=ompio 2) http://www.open-mpi.org/community/lists/devel/2015/12/18405.php 3) (ROMIO is default; on local hard drive at node 'cluster') $ ompi_info | grep romio MCA io: romio (MCA v2.0.0, API v2.0.0, Component v1.10.1) $ ompi_info | grep ompio MCA io: ompio (MCA v2.0.0, API v2.0.0, Component v1.10.1) $ mpif90 main.f90 $ echo hello1234 > out.txt; $MPIEXEC -np 1 -H cluster ./a.out; fileOffset, fileSize 10 10 fileOffset, fileSize 26 26 ierr 0 MPI_MODE_WRONLY, MPI_MODE_APPEND 4 128 $ export OMPI_MCA_io=ompio $ echo hello1234 > out.txt; $MPIEXEC -np 1 -H cluster ./a.out; fileOffset, fileSize 0 10 fileOffset, fileSize 0 16 ierr 0 MPI_MODE_WRONLY, MPI_MODE_APPEND 4 128 -- Dipl.-Inform. Paul Kapinos - High Performance Computing, RWTH Aachen University, IT Center Seffenter Weg 23, D 52074 Aachen (Germany) Tel: +49 241/80-24915 _______________________________________________ users mailing list us...@open-mpi.org Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users Link to this post: http://www.open-mpi.org/community/lists/users/2015/12/28145.php
-- Dipl.-Inform. Paul Kapinos - High Performance Computing, RWTH Aachen University, IT Center Seffenter Weg 23, D 52074 Aachen (Germany) Tel: +49 241/80-24915
smime.p7s
Description: S/MIME Cryptographic Signature
