[OMPI users] IRC channel
Hello Do people looking at this list ever join the #openmpi IRC channel. The channel seems to point to the website already. The medium could be useful to spread even more the use of openmpi. More specific channels could also be created. -beginner, -platform specific , -compilation issues, -performance ... regards,
[OMPI users] change between openmpi 1.4.1 and 1.5.1 about MPI2 publish name
Hello,
What are the changes between openMPI 1.4.1 and 1.5.1 about MPI2 service
of publishing name.
I have 2 programs which connect them via MPI_Publish_name and
MPI_Lookup_name subroutines and ompi-server.
That's OK with 1.4.1 version , but I have a deadlock with 1.5.1 version
inside the subroutine MPI_Publish_name and MPI_Lookup_name.
best
Bernard
That's my connection subroutine:
MPI_Comm remoteConnect(int myrank, int *srv, char *port_name, char* service)
{
int clt=0;
MPI_Request request; /* requete pour communication non bloquante */
MPI_Comm gcom;
MPI_Status status;
char port_name_clt[MPI_MAX_PORT_NAME];
if( service == NULL ) service = defaultService;
/* only process of rank null can publish name */
MPI_Barrier(MPI_COMM_WORLD);
/* A lookup for an unpublished service generate an error */
MPI_Errhandler_set(MPI_COMM_WORLD, MPI_ERRORS_RETURN);
if( myrank == 0 ){
/* Try to be a server. If there service is already published, try to be a
cient */
MPI_Open_port(MPI_INFO_NULL, port_name);
printf("[%d] Publish name\n",myrank);
if ( MPI_Publish_name(service, MPI_INFO_NULL, port_name) == MPI_SUCCESS ) {
*srv = 1;
printf("[%d] service %s available at %s\n",myrank,service,port_name);
}
else if ( MPI_Lookup_name(service, MPI_INFO_NULL, port_name_clt) ==
MPI_SUCCESS ){
MPI_Close_port( port_name );
clt = 1;
}
else
/* Throw exception */
printf("[%d] Error\n",myrank);
}
else{
/* Waiting rank 0 publish name */
sleep(1);
printf("[%d] Lookup name\n",myrank);
if ( MPI_Lookup_name(service, MPI_INFO_NULL, port_name_clt) == MPI_SUCCESS ){
clt = 1;
}
else
/* Throw exception */
;
}
MPI_Errhandler_set(MPI_COMM_WORLD, MPI_ERRORS_ARE_FATAL);
MPI_Bcast(srv,1,MPI_INT,0,MPI_COMM_WORLD);
if ( *srv )
/* I am the Master */
MPI_Comm_accept( port_name, MPI_INFO_NULL, 0, MPI_COMM_WORLD, &gcom );
else{
/* Connect to service SERVER, get the inter-communicator server*/
MPI_Errhandler_set(MPI_COMM_WORLD, MPI_ERRORS_RETURN);
if ( MPI_Comm_connect(port_name_clt, MPI_INFO_NULL, 0, MPI_COMM_WORLD, &gcom
) == MPI_SUCCESS )
printf("[%d] I get the connection with %s at %s !\n",myrank, service,
port_name_clt);
MPI_Errhandler_set(MPI_COMM_WORLD, MPI_ERRORS_ARE_FATAL);
}
if(myrank != 0) *srv = 0;
return gcom;
}
Re: [OMPI users] Using MPI_Put/Get correctly?
My bad, I found bug in my call to MPI_Win_create, the variable I passed was unallocated at the time of the call. Interestingly, no errors were returned from that call when I used the unallocated array... Anyway, everything is working since I corrected that, thanks for the help. Matt -Original Message- From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Grismer,Matthew J Civ USAF AFMC AFRL/RBAT Sent: Monday, January 03, 2011 11:18 AM To: Open MPI Users Subject: Re: [OMPI users] Using MPI_Put/Get correctly? Unfortunately correcting the integer type for the displacement does not fix the problem in my code, argh! So, thinking this might have something to do with the large arrays and amount of data being passed in the actual code, I modified my example (attached putbothways2.f90) so that the array sizes and amount of data swapped are nearly identical to the code giving me the issue. I also filled the array that is shared with random data, instead of 0's and 1's, to ensure nothing special was happening due to the simple, uniform data. Unfortunately, the example works great, but my actual code still seg faults. So, the summary is the example code that uses MPI_Put calls with indexed datatypes to swap data between 2 processors works without issue, while the actual code that communicates in the same manner fails. The only difference is the actual code allocates many other arrays, which are communicated in various ways (sends, puts, broadcasts, etc). I checked and re-checked all the argument lists associated with the indexed data, window, and puts; everything looks correct. Any thoughts or suggestions on how to proceed? Matt -Original Message- From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Grismer,Matthew J Civ USAF AFMC AFRL/RBAT Sent: Wednesday, December 29, 2010 1:42 PM To: Open MPI Users Subject: Re: [OMPI users] Using MPI_Put/Get correctly? Someone correctly pointed out the bug in my examples. In the MPI_Put I pass a 0 as the displacement, however, the argument must be of type integer (kind=MPI_ADDRESS_KIND), which is NOT the default integer type. Replacing the 0 with the correct integer type fixes both examples. Now to see if it fixes the actual code I am having difficulty with... -Original Message- From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Grismer,Matthew J Civ USAF AFMC AFRL/RBAT Sent: Monday, December 27, 2010 5:33 PM To: Open MPI Users Subject: Re: [OMPI users] Using MPI_Put/Get correctly? I decided to try and isolate the issue, and created two example test programs that appear to highlight an issue with Open MPI; both die when I run them on 2 processors. I am pretty certain the first (putoneway.f90) should work, as I am only doing a single put from one processor to a second processor; the target processor is doing nothing with the window'ed array that is receiving the data. My guess is the problem lies in the indexed datatypes that I am using for both the origin and target. The second case (putbothways.f90) closely mirrors what I am actually trying to do in my code, that is have each processor put into the other processors window'ed array at the same time. So, each process is sending from and receiving into the same array at the same time, with no overlap in the sent and received data. Once again I'm using indexed data types for both the origin and target. To build: mpif90 putoneway.f90 To run: mpiexec -np 2 a.out Matt -Original Message- From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of James Dinan Sent: Thursday, December 16, 2010 10:09 AM To: Open MPI Users Subject: Re: [OMPI users] Using MPI_Put/Get correctly? On 12/16/2010 08:34 AM, Jeff Squyres wrote: > Additionally, since MPI-3 is updating the semantics of the one-sided > stuff, it might be worth waiting for all those clarifications before > venturing into the MPI one-sided realm. One-sided semantics are much > more subtle and complex than two-sided semantics. Hi Jeff, I don't think we should give users the hope that MPI-3 RMA will be out tomorrow. The RMA revisions are still in proposal form and need work. Realistically speaking, we might be able to get this accepted into the standard within a year and it will be another year before implementations catch up. If users need one-sided now, they should use the MPI-2 one-sided API. MPI-3 RMA extends MPI-2 RMA and will be backward compatible, so anything you write now will still work. It's still unclear to me whether MPI-3's RMA semantics will be the leap forward in usability we have hoped for. We are trying to make it more flexible, but there will likely still be tricky parts due to portability and performance concerns. So, my advice: don't be scared of MPI-2. I agree, it's complicated, but once you get acclimated it's not that bad. Really. :) Best, ~Jim.
Re: [OMPI users] Granular locks?
Hi Gijsbert, Thank you for this proposal, I think it could be useful for our LQCD application, at least for further evaluations. How could I get to the code, please ? Thanks in advance for your help, Best, G. Le 03/01/2011 22:36, Gijsbert Wiesenekker a écrit : On Oct 2, 2010, at 10:54 , Gijsbert Wiesenekker wrote: On Oct 1, 2010, at 23:24 , Gijsbert Wiesenekker wrote: I have a large array that is shared between two processes. One process updates array elements randomly, the other process reads array elements randomly. Most of the time these writes and reads do not overlap. The current version of the code uses Linux shared memory with NSEMS semaphores. When array element i has to be read or updated semaphore (i % NSEMS) is used. if NSEMS = 1 the entire array will be locked which leads to unnecessary waits because reads and writes do not overlap most of the time. Performance increases as NSEMS increases, and flattens out at NSEMS = 32, at which point the code runs twice as fast when compared to NSEMS = 1. I want to change the code to use OpenMPI RMA, but MPI_Win_lock locks the entire array, which is similar to NSEMS = 1. Is there a way to have more granular locks? Gijsbert Also, is there an MPI_Win_lock equavalent for IPC_NOWAIT? Gijsbert FYI, as in my case the performance penalty by using OpenMPI RMA instead of shared memory was too large I have written a couple of wrapper functions that use OpenMPI to gracefully allocate and release shared memory: //mpi_alloc_shm is a collective operation that allocates arg_nrecords of arg_record_size each in the shared memory segment identified by arg_key with arg_nsems semaphores to control access. //arg_key is the shared memory key. //arg_nrecords is the number of records. //arg_record_size is the size of a record. //arg_default is the default record value. If not equal to NULL all arg_nrecord records will be initialized to *arg_default. //arg_nsems is the number of semaphores that will be used to control access. If record irecord has to be updated or read, semaphore (irecord % arg_nsems) will be used for exclusive access. //arg_mpi_id is the mpi_id of the process that will create the shared memory segment. If the mpi_id of the calling process is not equal to arg_mpi_id the process will not create but try to open it. void mpi_alloc_shm(key_t arg_key, i64_t arg_nrecords, i64_t arg_record_size, void *arg_default, int arg_nsems, int arg_mpi_id, MPI_Comm comm); //mpi_shm_put updates record irecord in the shared memory segment identified by shm_key with value *source. void mpi_shm_put(key_t shm_key, void *source, i64_t irecord); //mpi_shm_get tries to read record irecord in the shared memory segment identified by shm_key using IPC_NO_WAIT to request a lock. //FALSE is returned if the lock could not be obtained, else TRUE and the record in *dest. //as in my case only the creator of the shared memory segment will update it, a lock is not used if the creator tries to read record irecord. int mpi_shm_get(key_t shm_key, i64_t irecord, void *dest); //mpi_free_shm is a collective operation that deallocates the shared memory segment identified by shm_key void mpi_free_shm(key_t shm_key, MPI_Comm comm); Please feel free to contact me if you would like to have a copy of the source code of these routines. Regards, Gijsbert
Re: [OMPI users] Granular locks?
Hi Gilbert, why not to use architecture-specific atomic updates writing to the array? In this case, you wouldn't need anything special reading from array at all. Moreover, this model looks like a good candidate to be implemented as multithreaded application, rather than two separate processes sharing segment of memory. regards, Alex Granovsky - Original Message - From: "Gilbert Grosdidier" To: "Open MPI Users" Sent: Wednesday, January 05, 2011 11:47 PM Subject: Re: [OMPI users] Granular locks? Hi Gijsbert, Thank you for this proposal, I think it could be useful for our LQCD application, at least for further evaluations. How could I get to the code, please ? Thanks in advance for your help, Best, G. Le 03/01/2011 22:36, Gijsbert Wiesenekker a écrit : > On Oct 2, 2010, at 10:54 , Gijsbert Wiesenekker wrote: > >> On Oct 1, 2010, at 23:24 , Gijsbert Wiesenekker wrote: >> >>> I have a large array that is shared between two processes. One process >>> updates array elements randomly, the other process reads array elements randomly. Most of the time these writes and reads do not overlap. >>> The current version of the code uses Linux shared memory with NSEMS >>> semaphores. When array element i has to be read or updated semaphore (i % NSEMS) is used. if NSEMS = 1 the entire array will be locked which leads to unnecessary waits because reads and writes do not overlap most of the time. Performance increases as NSEMS increases, and flattens out at NSEMS = 32, at which point the code runs twice as fast when compared to NSEMS = 1. >>> I want to change the code to use OpenMPI RMA, but MPI_Win_lock locks the >>> entire array, which is similar to NSEMS = 1. Is there a way to have more granular locks? >>> >>> Gijsbert >>> >> Also, is there an MPI_Win_lock equavalent for IPC_NOWAIT? >> >> Gijsbert >> > FYI, as in my case the performance penalty by using OpenMPI RMA instead of > shared memory was too large I have written a couple of wrapper functions that use OpenMPI to gracefully allocate and release shared memory: > > file://mpi_alloc_shm is a collective operation that allocates arg_nrecords of > arg_record_size each in the shared memory segment identified by arg_key with arg_nsems semaphores to control access. > file://arg_key is the shared memory key. > file://arg_nrecords is the number of records. > file://arg_record_size is the size of a record. > file://arg_default is the default record value. If not equal to NULL all > arg_nrecord records will be initialized to *arg_default. > file://arg_nsems is the number of semaphores that will be used to control > access. If record irecord has to be updated or read, semaphore (irecord % arg_nsems) will be used for exclusive access. > file://arg_mpi_id is the mpi_id of the process that will create the shared > memory segment. If the mpi_id of the calling process is not equal to arg_mpi_id the process will not create but try to open it. > void mpi_alloc_shm(key_t arg_key, i64_t arg_nrecords, i64_t arg_record_size, >void *arg_default, int arg_nsems, int arg_mpi_id, MPI_Comm comm); > > file://mpi_shm_put updates record irecord in the shared memory segment > identified by shm_key with value *source. > void mpi_shm_put(key_t shm_key, void *source, i64_t irecord); > > file://mpi_shm_get tries to read record irecord in the shared memory segment > identified by shm_key using IPC_NO_WAIT to request a lock. > file://FALSE is returned if the lock could not be obtained, else TRUE and the > record in *dest. > file://as in my case only the creator of the shared memory segment will > update it, a lock is not used if the creator tries to read record irecord. > int mpi_shm_get(key_t shm_key, i64_t irecord, void *dest); > > file://mpi_free_shm is a collective operation that deallocates the shared > memory segment identified by shm_key > void mpi_free_shm(key_t shm_key, MPI_Comm comm); > > Please feel free to contact me if you would like to have a copy of the source > code of these routines. > > Regards, > Gijsbert ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
[OMPI users] Duplicate independent processes
Any hints on what to look for to get the second machine to behave without duplicate independent processes? "mpirun -np 2 ..." Provides as expected: "Using MPI version 2.1, 2 processes .1. .2." While on another similar machine: "Using MPI version 2.2, 1 processes .1. Using MPI version 2.2, 1 processes .1. .2. .2." On both: /usr/lib64/openmpi/1.4-gcc/bin/mpirun --version Returns: mpirun (Open MPI) 1.4 I looked in the FAQ and searched the list but did not find it yet. When I worked with MPICH2 I also had this happen from time to time. I can provide additional information should it not be as common here.
Re: [OMPI users] Duplicate independent processes
I'm afraid I don't understand your example - are you saying you provide "-np 1" and get two processes instead of 1? If so, would you please provide info on the type of system where this happens? I've never seen it with mpich or ompi On Jan 5, 2011, at 4:57 PM, Kristian Medri wrote: > Any hints on what to look for to get the second machine to behave without > duplicate independent processes? > > "mpirun -np 2 ..." > > Provides as expected: > > "Using MPI version 2.1, 2 processes > .1. > .2." > > While on another similar machine: > > "Using MPI version 2.2, 1 processes > .1. > Using MPI version 2.2, 1 processes > .1. > .2. > .2." > > On both: > > /usr/lib64/openmpi/1.4-gcc/bin/mpirun --version > > Returns: > > mpirun (Open MPI) 1.4 > > I looked in the FAQ and searched the list but did not find it yet. When I > worked with MPICH2 I also had this happen from time to time. I can provide > additional information should it not be as common here. > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
