Two separate comments.
1. I do not know the precise status of the PVFS2 support in 1.8 series
of Open MPI for ROMIO, I haven't tested it in a while. On master, I know
that there is a compilation problem with PVFS2 and ROMIO on Open MPI and
I am about to submit a report/question to ROMIO about that.
2. for OMPIO, we use PVFS2 as our main development platform. However, we
have honestly not tried to use PVFS2 without the file system being
mounted (i.e. we do rely on the kernel component to some extent). Yes,
internally we use the library interfaces of PVFS2, but we use the file
system information to determine the type of the file system, and my
guess is that if that information is not available, the pvfs2 fs (and
fbtl for that matter) components disable themselves, and that's the
error that you see. I can look into how to make that scenario work in
OMPIO, but its definitely not in the 1.8 series.
Thanks
Edgar
On 2/25/2015 2:01 AM, vithanousek wrote:
Thanks for your repaly!
I checked my configuration parametrs and it seem, that everything is correct:
./configure --prefix=/opt/modules/openmpi-1.8.4 --with-sge --with-psm
--with-pvfs2=/opt/orangefs
--with-io-romio-flags='--with-file-system=pvfs2+ufs+nfs
--with-pvfs2=/opt/orangefs'
I have added error chceking code to my app, and I was getting multiple errors,
like en MPI_ERR_AMODE, MPI_ERR_UNKNOWN, MPI_ERR_NO_SUCH_FILE,MPI_ERR_IO.
(depend on permisions of mount point of pvfs2, and --mca io romio/ompio --mca
fs pvfs2)
But it seems that error is in sourcecode of my application, because I cant find
any more complex documentation about using ROMIO and OMPIO.
I found here https://surfsara.nl/systems/lisa/software/pvfs2, that I should use as filename
"pvfs2:/pvfs_mount_point/name_of_file" instead of
"/pvfs_mount_point/name_of_file". This is working with ROMIO.
Do you know how to use OMPIO without mounting pvfs2? if I tryed the same filename format
as in ROMIO I got "MPI_ERR_FILE: invalid file".
If I use normal filename format ("/mountpoint/filename") and force use of pvfs2
by using --mca io ompio --mca fs pvfs2, then my app fails with
mca_fs_base_file_select() failed (and backtrace).
At OrangeFS documentation (http://docs.orangefs.com/v_2_8_8/index.htm) is
chapter about using ROMIO, and it says, that i shoud compile apps with -lpvfs2.
I have tryed it, but nothing change (ROMIO works with special filename format,
OMPIO doesnt work)
Thanks for your help. If you point me to some usefull documentation, I will be
happy.
Hanousek Vít
---------- Původní zpráva ----------
Od: Rob Latham
Komu: us...@open-mpi.org, vithanou...@seznam.cz
Datum: 24. 2. 2015 22:10:08
Předmět: Re: [OMPI users] MPIIO and OrangeFS
On 02/24/2015 02:00 PM, vithanousek wrote:
Hello,
Im not sure if I have my OrangeFS (2.8.8) and OpenMPI (1.8.4) set up corectly.
One short questin?
Is it needed to have OrangeFS mounted through kernel module, if I want use
MPIIO?
nope!
My simple MPIIO hello world program doesnt work, If i havent mounted OrangeFS.
When I mount OrangeFS, it works. So I'm not sure if OMPIO (or ROMIO) is using
pvfs2 servers directly or if it is using kernel module.
Sorry for stupid question, but I didnt find any documentation about it.
http://www.pvfs.org/cvs/pvfs-2-8-branch-docs/doc/pvfs2-quickstart/pvfs2-quickstart.php#sec:romio
It sounds like you have not configured your MPI implementation with
PVFS2 support (OrangeFS is a re-branding of PVFS2, but as far as MPI-IO
is concerned, they are the same).
OpenMPI passes flags to romio like this at configure time:
--with-io-romio-flags="--with-file-system=pvfs2+ufs+nfs"
I'm not sure how OMPIO takes flags.
If pvfs2-ping and pvfs2-cp and pvfs2-ls work, then you can bypass the
kernel.
also, please check return codes:
http://stackoverflow.com/questions/22859269/what-do-mpi-io-error-codes-mean/26373193#26373193
==rob
Thanks for replays
Hanousek Vít
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2015/02/26382.php
--
Edgar Gabriel
Associate Professor
Parallel Software Technologies Lab http://pstl.cs.uh.edu
Department of Computer Science University of Houston
Philip G. Hoffman Hall, Room 524 Houston, TX-77204, USA
Tel: +1 (713) 743-3857 Fax: +1 (713) 743-3335
