Dear all, Dear George,
now it seems getting better:
* CALL MPI_GROUP_INCL(GROUP_WORLD, nPSObranch, MRANKS, MASTER_GROUP,ierr)*
* !!Create a new communicator based on the group*
* CALL
MPI_COMM_CREATE_GROUP(MPI_COMM_WORLD,MASTER_GROUP,0,MASTER_COMM,iErr)*
* IF(MPI_COMM_NULL .NE. MASTER_COMM)THEN*
* CALL MPI_COMM_RANK(MASTER_COMM, MPImaster%rank,MPIlocal%iErr)*
* CALL MPI_COMM_SIZE(MASTER_COMM, MPImaster%nCPU,MPIlocal%iErr)*
* ELSE*
* MPImaster%rank = MPI_PROC_NULL*
* ENDIF*
I have a further question.
How Can I automatically choose a communicator. I mean, now I have three
group and a three communicator: WORLD, LOCAL and MASTER
If I do this:
CALL MPI_SCATTER(PP, npart, MPI_DOUBLE, PPL, 10,MPI_DOUBLE, 0, *MASTER_COMM*,
iErr)
I get an error. This because some CPU does not belong to MATER_COMM. The
alternative should be:
IF(rank.LT.0)THEN
CALL MPI_SCATTER(PP, npart, MPI_DOUBLE, PPL, 10,MPI_DOUBLE, 0,
*MASTER_COMM*, iErr)
ENDIF
Is there any smarter way do to this?
Thanks again
Diego
On 28 July 2017 at 20:07, Diego Avesani <diego.aves...@gmail.com> wrote:
> Dear George, Dear all,
>
> I have just rewritten the code to make it more clear:
>
> * INTEGER :: colorl,colorglobal*
> * INTEGER :: LOCAL_COMM,MASTER_COMM*
>
> * !++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*
> * !-----------------------------------------------*
> * ! create WORLD communicator*
> * !-----------------------------------------------*
> * CALL MPI_INIT(MPIworld%iErr)*
> * CALL MPI_COMM_SIZE(MPI_COMM_WORLD, MPIworld%nCPU, MPIworld%iErr) !get
> the nCPU*
> * CALL MPI_COMM_RANK(MPI_COMM_WORLD, MPIworld%rank, MPIworld%iErr) !get
> the rank*
> * !*
> * colorl = MPIworld%rank/4*
> * !*
> * !-----------------------------------------------*
> * ! create LOCAL communicator*
> * !-----------------------------------------------*
> * CALL
> MPI_COMM_SPLIT(MPI_COMM_WORLD,colorl,MPIworld%rank,LOCAL_COMM,MPIworld%iErr)*
> * CALL MPI_COMM_RANK(LOCAL_COMM, MPIlocal%rank,MPIlocal%iErr)*
> * CALL MPI_COMM_SIZE(LOCAL_COMM, MPIlocal%nCPU,MPIlocal%iErr)*
> * !*
> * !*
> * !WRITE(*,'(A15,I3,A15,I3)') 'WORLD RANK ',MPIworld%rank,'LOCAL RANK
> ',MPIlocal%rank*
> * !*
> * !-----------------------------------------------*
> * ! create MASTER communicator*
> * !-----------------------------------------------*
> * IF(MOD(MPIworld%rank,4).EQ.0)THEN*
> * colorglobal = MOD(MPIworld%rank,4)*
> * ELSE*
> * colorglobal = MPI_COMM_NULL*
> * ENDIF*
> * !*
> * CALL
> MPI_COMM_SPLIT(MPI_COMM_WORLD,colorglobal,MPIworld%rank,MASTER_COMM,MPIworld%iErr)*
> * CALL MPI_COMM_RANK(MASTER_COMM, MPImaster%rank,MPImaster%iErr)*
> * CALL MPI_COMM_SIZE(MASTER_COMM, MPImaster%nCPU,MPImaster%iErr)*
> * !*
> * !*
> * WRITE(*,'(A15,I3,A15,I3,A15,I3)') 'WORLD RANK ',MPIworld%rank,'LOCAL
> RANK ',MPIlocal%rank,'MASTER ',MPImaster%nCPU *
>
> This is the result:
>
> WORLD RANK 2 LOCAL RANK 2 MASTER 12
> WORLD RANK 3 LOCAL RANK 3 MASTER 12
> WORLD RANK 10 LOCAL RANK 2 MASTER 12
> *WORLD RANK 12 LOCAL RANK 0 MASTER 4*
> WORLD RANK 1 LOCAL RANK 1 MASTER 12
> WORLD RANK 5 LOCAL RANK 1 MASTER 12
> *WORLD RANK 4 LOCAL RANK 0 MASTER 4*
> WORLD RANK 6 LOCAL RANK 2 MASTER 12
> WORLD RANK 13 LOCAL RANK 1 MASTER 12
> WORLD RANK 14 LOCAL RANK 2 MASTER 12
> WORLD RANK 7 LOCAL RANK 3 MASTER 12
> WORLD RANK 11 LOCAL RANK 3 MASTER 12
> *WORLD RANK 0 LOCAL RANK 0 MASTER 4*
> *WORLD RANK 8 LOCAL RANK 0 MASTER 4*
> WORLD RANK 9 LOCAL RANK 1 MASTER 12
> WORLD RANK 15 LOCAL RANK 3 MASTER 12
>
> I am expecting only a new communicator only for the master but it seems
> that I get two new groups despite I set "*colorglobal = MPI_COMM_NULL*"
>
> What do think? Is there something that I haven't understood properly?
>
> Thanks again, I am trying to learn better MPI_Comm_create_group.
>
> Thanks
>
>
> Diego
>
>
> On 28 July 2017 at 16:59, Diego Avesani <diego.aves...@gmail.com> wrote:
>
>> Dear George, Dear all,
>>
>> here the code:
>>
>> PROGRAM TEST
>> USE MPI
>> IMPLICIT NONE
>> ! mpif90 -r8 *.f90
>> !+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++++++++++++++
>> INTEGER :: rank
>> INTEGER :: subrank,leader_rank
>> INTEGER :: nCPU
>> INTEGER :: subnCPU
>> INTEGER :: ierror
>> INTEGER :: tag
>> INTEGER :: status(MPI_STATUS_SIZE)
>> INTEGER :: colorloc,colorglobal
>> INTEGER :: NEW_COMM,LEADER_COMM
>> !+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++++++++++++++
>> CALL MPI_INIT(ierror)
>> CALL MPI_COMM_SIZE(MPI_COMM_WORLD, nCPU, ierror)
>> CALL MPI_COMM_RANK(MPI_COMM_WORLD, rank, ierror)
>> !
>> colorloc = rank/4
>> !
>> !
>> CALL MPI_COMM_SPLIT(MPI_COMM_WORLD,colorloc,rank,NEW_COMM,ierror)
>> CALL MPI_COMM_RANK(NEW_COMM, subrank,ierror);
>> CALL MPI_COMM_SIZE(NEW_COMM, subnCPU,ierror);
>> !
>> IF(MOD(rank,4).EQ.0)THEN
>> colorglobal = MOD(rank,4)
>> ELSE
>> colorglobal = *MPI_COMM_NULL*
>> ENDIF
>> !
>> CALL MPI_COMM_SPLIT(MPI_COMM_WORLD,colorglobal,rank,LEADER_COMM,ierror)
>> CALL MPI_COMM_RANK(LEADER_COMM, leader_rank,ierror);
>> !
>> CALL MPI_FINALIZE(ierror)
>> ENDPROGRAM
>>
>> Now, it works.
>>
>> Could you please explain me "*MPI_Comm_create_group*". I am trying buy
>> my self, but it seems quite different from MPI_SPLIT_COMM.
>>
>> Again, really, really thanks
>>
>> Diego
>>
>>
>> On 28 July 2017 at 16:02, George Bosilca <bosi...@icl.utk.edu> wrote:
>>
>>> I guess the second comm_rank call is invalid on all non-leader
>>> processes, as their LEADER_COMM communicator is MPI_COMM_NULL.
>>>
>>> george
>>>
>>> On Fri, Jul 28, 2017 at 05:06 Diego Avesani <diego.aves...@gmail.com>
>>> wrote:
>>>
>>>> Dear George, Dear all,
>>>>
>>>> thanks, thanks a lot. I will tell you everything.
>>>> I will try also to implement your suggestion.
>>>>
>>>> Unfortunately, the program that I have show to you is not working. I
>>>> get the following error:
>>>>
>>>> [] *** An error occurred in MPI_Comm_rank
>>>> [] *** reported by process [643497985,7]
>>>> [] *** on communicator MPI_COMM_WORLD
>>>> [] *** MPI_ERR_COMM: invalid communicator
>>>> [] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now
>>>> abort,
>>>> [] *** and potentially your MPI job)
>>>> [warn] Epoll ADD(4) on fd 47 failed. Old events were 0; read change
>>>> was 0 (none); write change was 1 (add): Bad file descriptor
>>>> [warn] Epoll ADD(4) on fd 65 failed. Old events were 0; read change
>>>> was 0 (none); write change was 1 (add): Bad file descriptor
>>>> [] 8 more processes have sent help message help-mpi-errors.txt /
>>>> mpi_errors_are_fatal
>>>> [] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help /
>>>> error messages
>>>>
>>>> What do you think could be the error?
>>>>
>>>> Really, Really thanks again
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Diego
>>>>
>>>>
>>>> On 27 July 2017 at 15:57, George Bosilca <bosi...@icl.utk.edu> wrote:
>>>>
>>>>> This looks good. If performance is critical you can speed up the
>>>>> entire process by using MPI_Comm_create_group instead of the second
>>>>> MPI_COMM_SPLIT. The MPI_Comm_create_group is collective only over the
>>>>> resulting communicator and not over the source communicator, so its cost
>>>>> is
>>>>> only dependent of the number of groups and not on the total number of
>>>>> processes.
>>>>>
>>>>> You can also try to replace the first MPI_COMM_SPLIT by the same
>>>>> approach. I would be curious to see the outcome.
>>>>>
>>>>> George.
>>>>>
>>>>>
>>>>> On Thu, Jul 27, 2017 at 9:44 AM, Diego Avesani <
>>>>> diego.aves...@gmail.com> wrote:
>>>>>
>>>>>> Dear George, Dear all,
>>>>>>
>>>>>> I have tried to create a simple example. In particular, I would like
>>>>>> to use 16 CPUs and to create four groups according to rank is and then a
>>>>>> communicator between masters of each group.I have tried to follow the
>>>>>> first
>>>>>> part of this example
>>>>>> <http://mpitutorial.com/tutorials/introduction-to-groups-and-communicators/>.
>>>>>> In the last part I have tried to create a communicator be masters as
>>>>>> suggested by George.
>>>>>>
>>>>>> Here my example:
>>>>>>
>>>>>> CALL MPI_INIT(ierror)
>>>>>> CALL MPI_COMM_SIZE(MPI_COMM_WORLD, nCPU, ierror)
>>>>>> CALL MPI_COMM_RANK(MPI_COMM_WORLD, rank, ierror)
>>>>>> !
>>>>>> colorloc = rank/4
>>>>>> !
>>>>>> CALL MPI_COMM_SPLIT(MPI_COMM_WORLD,colorloc,rank,*NEW_COMM*,ierror)
>>>>>> CALL MPI_COMM_RANK(*NEW_COMM*, subrank,ierror);
>>>>>> CALL MPI_COMM_SIZE(*NEW_COMM*, subnCPU,ierror);
>>>>>> !
>>>>>> IF(MOD(rank,4).EQ.0)THEN
>>>>>> *! where I set color for the masters*
>>>>>> colorglobal = MOD(rank,4)
>>>>>> ELSE
>>>>>> colorglobal = MPI_UNDEFINED
>>>>>> ENDIF
>>>>>> !
>>>>>> CALL MPI_COMM_SPLIT(MPI_COMM_WORLD,colorglobal,rank,LEADER_COMM,i
>>>>>> error)
>>>>>> CALL MPI_COMM_RANK(*LEADER_COMM*, leader_rank,ierror);
>>>>>> CALL MPI_FINALIZE(ierror)
>>>>>>
>>>>>> I would like to know if this could be correct. I mean if I have
>>>>>> understood correctly what George told me about the code design. Now, this
>>>>>> example does not work, but probably there is some coding error.
>>>>>>
>>>>>> Really, Really thanks
>>>>>> Diego
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Diego
>>>>>>
>>>>>>
>>>>>> On 27 July 2017 at 10:42, Diego Avesani <diego.aves...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Dear George, Dear all,
>>>>>>>
>>>>>>> A question regarding program design:
>>>>>>> The draft that I have sent to you has to be done many and many
>>>>>>> times. Does the splitting procedure ensure efficiency?
>>>>>>>
>>>>>>> I will try, at a lest to create groups and split them. I am a
>>>>>>> beginner in the MPI groups environment.
>>>>>>> really, really thanks.
>>>>>>>
>>>>>>> You are my lifesaver.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Diego
>>>>>>>
>>>>>>>
>>>>>>> On 26 July 2017 at 15:09, George Bosilca <bosi...@icl.utk.edu>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Diego,
>>>>>>>>
>>>>>>>> As all your processes are started under the umbrella of a single
>>>>>>>> mpirun, they have a communicator in common, the MPI_COMM_WORLD.
>>>>>>>>
>>>>>>>> One possible implementation, using MPI_Comm_split, will be the
>>>>>>>> following:
>>>>>>>>
>>>>>>>> MPI_Comm small_comm, leader_comm;
>>>>>>>>
>>>>>>>> /* Create small_comm on all processes */
>>>>>>>>
>>>>>>>> /* Now use MPI_Comm_split on MPI_COMM_WORLD to select the leaders */
>>>>>>>> MPI_Comm_split( MPI_COMM_WORLD,
>>>>>>>> i_am_leader(small_comm) ? 1 :
>>>>>>>> MPI_UNDEFINED,
>>>>>>>> rank_in_comm_world,
>>>>>>>> &leader_Comm);
>>>>>>>>
>>>>>>>> The leader_comm will be a valid communicator on all leaders
>>>>>>>> processes, and MPI_COMM_NULL on all others.
>>>>>>>>
>>>>>>>> George.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jul 26, 2017 at 4:29 AM, Diego Avesani <
>>>>>>>> diego.aves...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Dear George, Dear all,
>>>>>>>>>
>>>>>>>>> I use "mpirun -np xx ./a.out"
>>>>>>>>>
>>>>>>>>> I do not know if I have some common grounds. I mean, I have to
>>>>>>>>> design everything from the begging. You can find what I would like to
>>>>>>>>> do in
>>>>>>>>> the attachment. Basically, an MPI cast in another MPI. Consequently,
>>>>>>>>> I am
>>>>>>>>> thinking to MPI groups or MPI virtual topology with a 2D cart, using
>>>>>>>>> the
>>>>>>>>> columns as "groups" and the first rows as the external groups to
>>>>>>>>> handle the
>>>>>>>>> columns.
>>>>>>>>>
>>>>>>>>> What do think? What do you suggest?
>>>>>>>>> Really Really thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Diego
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 25 July 2017 at 19:26, George Bosilca <bosi...@icl.utk.edu>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Diego,
>>>>>>>>>>
>>>>>>>>>> Assuming you have some common grounds between the 4 initial
>>>>>>>>>> groups (otherwise you will have to connect them via
>>>>>>>>>> MPI_Comm_connect/MPI_Comm_accept) you can merge the 4 groups
>>>>>>>>>> together and then use any MPI mechanism to create a partial group of
>>>>>>>>>> leaders (such as MPI_Comm_split).
>>>>>>>>>>
>>>>>>>>>> If you spawn the groups via MPI_Comm_spawn then the answer is
>>>>>>>>>> slightly more complicated, you need to use MPI_Intercomm_create,
>>>>>>>>>> with the
>>>>>>>>>> spawner as the bridge between the different communicators (and then
>>>>>>>>>> MPI_Intercomm_merge to create your intracomm). You can find a good
>>>>>>>>>> answer
>>>>>>>>>> on stackoverflow on this at https://stackoverflow.com/ques
>>>>>>>>>> tions/24806782/mpi-merge-multiple-intercoms-into-a-single-in
>>>>>>>>>> tracomm
>>>>>>>>>>
>>>>>>>>>> How is your MPI environment started (single mpirun or
>>>>>>>>>> mpi_comm_spawn) ?
>>>>>>>>>>
>>>>>>>>>> George.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jul 25, 2017 at 10:44 AM, Diego Avesani <
>>>>>>>>>> diego.aves...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Dear All,
>>>>>>>>>>>
>>>>>>>>>>> I am studying Groups and Communicators, but before start going
>>>>>>>>>>> in detail, I have a question about groups.
>>>>>>>>>>>
>>>>>>>>>>> I would like to know if is it possible to create a group of
>>>>>>>>>>> masters of the other groups and then a intra-communication in the
>>>>>>>>>>> new
>>>>>>>>>>> group. I have spent sometime reading different tutorial and
>>>>>>>>>>> presentation,
>>>>>>>>>>> but it is difficult, at least for me, to understand if is it
>>>>>>>>>>> possible to
>>>>>>>>>>> create this sort of MPI cast in another MPI.
>>>>>>>>>>>
>>>>>>>>>>> In the attachment you can find a pictures that summarize what I
>>>>>>>>>>> would like to do.
>>>>>>>>>>>
>>>>>>>>>>> Another strategies could be use virtual topology.
>>>>>>>>>>>
>>>>>>>>>>> What do you think?
>>>>>>>>>>>
>>>>>>>>>>> I really, really, appreciate any kind of help, suggestions or
>>>>>>>>>>> link where I can study this topics.
>>>>>>>>>>>
>>>>>>>>>>> Again, thanks
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>>
>>>>>>>>>>> Diego
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> users mailing list
>>>>>>>>>>> users@lists.open-mpi.org
>>>>>>>>>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> users mailing list
>>>>>>>>>> users@lists.open-mpi.org
>>>>>>>>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> users mailing list
>>>>>>>>> users@lists.open-mpi.org
>>>>>>>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> users mailing list
>>>>>>>> users@lists.open-mpi.org
>>>>>>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> users mailing list
>>>>>> users@lists.open-mpi.org
>>>>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> users mailing list
>>>>> users@lists.open-mpi.org
>>>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>>>>
>>>>
>>>> _______________________________________________
>>>> users mailing list
>>>> users@lists.open-mpi.org
>>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> users@lists.open-mpi.org
>>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users
>>>
>>
>>
>
_______________________________________________
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users
