-----Original Message----- From: users <users-boun...@lists.open-mpi.org> On Behalf Of Dave Love via users Sent: Friday, January 15, 2021 4:48 AM To: Gabriel, Edgar via users <users@lists.open-mpi.org> Cc: Dave Love <dave.l...@manchester.ac.uk> Subject: Re: [OMPI users] 4.1 mpi-io test failures on lustre
> How should we know that's expected to fail? It at least shouldn't fail like > that; set_atomicity doesn't return an error (which the test is prepared for > on a filesystem like pvfs2). > I assume doing nothing, but appearing to, can lead to corrupt data, and I'm > surprised that isn't being seen already. > HDF5 requires atomicity -- at least to pass its tests -- so presumably anyone > like us who needs it should use something mpich-based with recent or old > romio, and that sounds like most general HPC systems. > Am I missing something? > With the current romio everything I tried worked, but we don't get that > option with openmpi. First of all, it is mentioned on the FAQ sites of Open MPI, although admittedly it is not entirely update (it lists external32 support also as missing, which is however now available since 4.1). You don't need atomicity for the HDF5 tests, we are passing all of them to the best my knowledge, and this is one of the testsuites that we do run regularly as part of our standard testing process. I am aware that they have an atomicity test - which we pass for whatever reason. This highlight also btw the issue(s) that I am having with the atomicity option in MPI I/O. The entire infrastructure to enforce atomicity is actually in place in ompio, and I can give you the option on how to enforce strict atomic behavior for all files in ompio (just not on a per file basis), just be aware that the performance will nose-dive. This is not just the case with ompio, but also in romio, you can read up on that various discussion boards on that topic, look at NFS related posts (where you need the atomicity for correctness in basically all scenarios). Just as another data point, in the 8+ years that ompio has been available, there was not one issue reported related to correctness due to missing the atomicity option. That being said, if you feel more comfortable using romio, it is completely up to you. Open MPI offers this option, and it is incredibly easy to set the default parameters on a platform for all users such that romio is being used. We are doing with our limited resources the best we can, and while ompio is by no means perfect, we try to be responsive to issues reported by users and value constructive feedback and discussion. Thanks Edgar
