On 12/17/2014 07:04 PM, Eric Chamberland wrote:
Hi!
Here is a "poor man's fix" that works for me (the idea is not from me,
thanks to Thomas H.):
#1- char* lCwd = getcwd(0,0);
#2- chdir(lPathToFile);
#3- MPI_File_open(...,lFileNameWithoutTooLongPath,...);
#4- chdir(lCwd);
#5- ...
I think there are some limitations but it works very well for our
uses... and until a "real" fix is proposed...
A bit of a delay on my part due to the winter break but I have returned
to this topic.
I have an approach that will at least tell you something went wrong in
processing the shared file pointer name: the string is so long it
truncates the error message, but it leaves enough to tell you what went
wrong.
ERROR Returned by MPI: 1006695702
ERROR_string Returned by MPI: Invalid file name, error stack:
ADIOI_Shfp_fname(60): Pathname
this/is/a_very/long/path/that/contains/a/not/so/long/filename
/but/trying/to/collectively/mpi_file_open/it/you/will/have/a/memory/corruption/resulting/of/
invalide/writing/or/reading/past/the/end/of/one/or/some/hidden/strings/in/mpio/Simpimple/use
r
At least you get "invalid file name"
Furthermore, I'm changing that code to use PATH_MAX, not 256, which
would have fixed the specific problem you encountered (and might have
been sufficient to get us 10 more years, at which point someone might
try to create a file with 1000 characters in it)
==rob
Thanks for helping!
Eric
On 12/15/2014 11:42 PM, Gilles Gouaillardet wrote:
Eric and all,
That is clearly a limitation in romio, and this is being tracked at
https://trac.mpich.org/projects/mpich/ticket/2212
in the mean time, what we can do in OpenMPI is update
mca_io_romio_file_open() and fails with a user friendly error message
if strlen(filename) is larger that 225.
Cheers,
Gilles
On 2014/12/16 12:43, Gilles Gouaillardet wrote:
Eric,
thanks for the simple test program.
i think i see what is going wrong and i will make some changes to avoid
the memory overflow.
that being said, there is a hard coded limit of 256 characters, and your
path is bigger than 300 characters.
bottom line, and even if there is no more memory overflow, that cannot
work as expected.
i will report this to the mpich folks, since romio is currently imported
from mpich.
Cheers,
Gilles
On 2014/12/16 0:16, Eric Chamberland wrote:
Hi Gilles,
just created a very simple test case!
with this setup, you will see the bug with valgrind:
export
too_long=./this/is/a_very/long/path/that/contains/a/not/so/long/filename/but/trying/to/collectively/mpi_file_open/it/you/will/have/a/memory/corruption/resulting/of/invalide/writing/or/reading/past/the/end/of/one/or/some/hidden/strings/in/mpio/Simple/user/would/like/to/have/the/parameter/checked/and/an/error/returned/or/this/limit/removed
mpicc -o bug_MPI_File_open_path_too_long
bug_MPI_File_open_path_too_long.c
mkdir -p $too_long
echo "header of a text file" > $too_long/toto.txt
mpirun -np 2 valgrind ./bug_MPI_File_open_path_too_long
$too_long/toto.txt
and watch the errors!
unfortunately, the memory corruptions here doesn't seem to segfault
this simple test case, but in my case, it is fatal and with valgrind,
it is reported...
OpenMPI 1.6.5, 1.8.3rc3 are affected
MPICH-3.1.3 also have the error!
thanks,
Eric
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2014/12/26005.php
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2014/12/26006.php
--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA
