Re: [OMPI users] MPI_IN_PLACE

Gilles Gouaillardet Thu, 27 Jul 2017 01:56:41 -0700

Volker,


since you are only using

include 'mpif.h'


a workaround is you edit your /.../share/openmpi/mpifort-wrapper-data.txt
and simply remove '-lmpi_usempif08 -lmpi_usempi_ignore_tkr'


Cheers,

Gilles

On 7/27/2017 3:28 PM, Volker Blum wrote:

Thanks!

If you wish, please also keep me posted.

Best wishes
Volker

On Jul 27, 2017, at 7:50 AM, Gilles Gouaillardet 
<gilles.gouaillar...@gmail.com> wrote:

Thanks Jeff for your offer, i will contact you off-list later


i tried a gcc+gfortran and gcc+ifort on both linux and OS X
so far, only gcc+ifort on OS X is failing
i will try icc+ifort on OS X from now

short story, MPI_IN_PLACE is not recognized as such by the ompi
fortran wrapper, and i do not know why.

the attached program can be used to evidence the issue.


Cheers,

Gilles

On Thu, Jul 27, 2017 at 2:15 PM, Volker Blum <volker.b...@duke.edu> wrote:

Thanks! That’s great. Sounds like the exact combination I have here.

Thanks also to George. Sorry that the test did not trigger on a more standard 
platform - that would have simplified things.

Best wishes
Volker

On Jul 27, 2017, at 3:56 AM, Gilles Gouaillardet <gil...@rist.or.jp> wrote:

Folks,


I am able to reproduce the issue on OS X (Sierra) with stock gcc (aka clang) 
and ifort 17.0.4


i will investigate this from now


Cheers,

Gilles

On 7/27/2017 9:28 AM, George Bosilca wrote:

Volker,

Unfortunately, I can't replicate with icc. I tried on a x86_64 box with Intel 
compiler chain 17.0.4 20170411 to no avail. I also tested the 3.0.0-rc1 tarball 
and the current master, and you test completes without errors on all cases.

Once you figure out an environment where you can consistently replicate the 
issue, I would suggest to attach to the processes and:
- make sure the MPI_IN_PLACE as seen through the Fortran layer matches what the 
C layer expects
- what is the collective algorithm used by Open MPI

I have a "Fortran 101" level question. When you pass an array a(:) as argument, 
what exactly gets passed via the Fortran interface to the corresponding C function ?

George.

On Wed, Jul 26, 2017 at 1:55 PM, Volker Blum <volker.b...@duke.edu 
<mailto:volker.b...@duke.edu>> wrote:

   Thanks! Yes, trying with Intel 2017 would be very nice.

On Jul 26, 2017, at 6:12 PM, George Bosilca <bosi...@icl.utk.edu

   <mailto:bosi...@icl.utk.edu>> wrote:

No, I don't have (or used where they were available) the Intel

   compiler. I used clang and gfortran. I can try on a Linux box with
   the Intel 2017 compilers.

  George.



On Wed, Jul 26, 2017 at 11:59 AM, Volker Blum

   <volker.b...@duke.edu <mailto:volker.b...@duke.edu>> wrote:

Did you use Intel Fortran 2017 as well?

(I’m asking because I did see the same issue with a combination

   of an earlier Intel Fortran 2017 version and OpenMPI on an
   Intel/Infiniband Linux HPC machine … but not Intel Fortran 2016 on
   the same machine. Perhaps I can revive my access to that
   combination somehow.)

Best wishes
Volker

On Jul 26, 2017, at 5:55 PM, George Bosilca

   <bosi...@icl.utk.edu <mailto:bosi...@icl.utk.edu>> wrote:

I thought that maybe the underlying allreduce algorithm fails

   to support MPI_IN_PLACE correctly, but I can't replicate on any
   machine (including OSX) with any number of processes.

  George.



On Wed, Jul 26, 2017 at 10:59 AM, Volker Blum

   <volker.b...@duke.edu <mailto:volker.b...@duke.edu>> wrote:

Thanks!

I tried ‘use mpi’, which compiles fine.

Same result as with ‘include mpif.h', in that the output is

* MPI_IN_PLACE does not appear to work as intended.
* Checking whether MPI_ALLREDUCE works at all.
* Without MPI_IN_PLACE, MPI_ALLREDUCE appears to work.

Hm. Any other thoughts?

Thanks again!
Best wishes
Volker

On Jul 26, 2017, at 4:06 PM, Gilles Gouaillardet

   <gilles.gouaillar...@gmail.com
   <mailto:gilles.gouaillar...@gmail.com>> wrote:

Volker,

With mpi_f08, you have to declare

Type(MPI_Comm) :: mpi_comm_global

(I am afk and not 100% sure of the syntax)

A simpler option is to

use mpi

Cheers,

Gilles

Volker Blum <volker.b...@duke.edu

   <mailto:volker.b...@duke.edu>> wrote:

Hi Gilles,

Thank you very much for the response!

Unfortunately, I don’t have access to a different system

   with the issue right now. As I said, it’s not new; it just keeps
   creeping up unexpectedly again on different platforms. What
   puzzles me is that I’ve encountered the same problem with low but
   reasonable frequency over a period of now over five years.

We can’t require F’08 in our application, unfortunately,

   since this standard is too new. Since we maintain a large
   application that has to run on a broad range of platforms, Fortran
   2008 would not work for many of our users. In a few years, this
   will be different, but not yet.

On gfortran: In our own tests, unfortunately, Intel Fortran

   consistently produced much faster executable code in the past. The
   latter observation may also change someday, but for us, the
   performance difference was an important constraint.

I did suspect mpif.h, too. Not sure how to best test this

   hypothesis, however.

Just replacing

include 'mpif.h'
with
use mpi_f08

did not work, for me.

This produces a number of compilation errors:

blum:/Users/blum/codes/fhi-aims/openmpi_test> mpif90

   check_mpi_in_place_08.f90 -o check_mpi_in_place_08.x

check_mpi_in_place_08.f90(55): error #6303: The assignment

   operation or the binary expression operation is invalid for the
   data types of the two operands.   [MPI_COMM_WORLD]

  mpi_comm_global = MPI_COMM_WORLD
----------------------^
check_mpi_in_place_08.f90(57): error #6285: There is no

   matching specific subroutine for this generic subroutine call.      
[MPI_COMM_SIZE]

  call MPI_COMM_SIZE(mpi_comm_global, n_tasks, mpierr)
---------^
check_mpi_in_place_08.f90(58): error #6285: There is no

   matching specific subroutine for this generic subroutine call.      
[MPI_COMM_RANK]

  call MPI_COMM_RANK(mpi_comm_global, myid, mpierr)
---------^
check_mpi_in_place_08.f90(75): error #6285: There is no

   matching specific subroutine for this generic subroutine call.      
[MPI_ALLREDUCE]

  call MPI_ALLREDUCE(MPI_IN_PLACE, &
---------^
check_mpi_in_place_08.f90(94): error #6285: There is no

   matching specific subroutine for this generic subroutine call.      
[MPI_ALLREDUCE]

  call MPI_ALLREDUCE(check_success, aux_check_success, 1,

   MPI_LOGICAL, &

---------^
check_mpi_in_place_08.f90(119): error #6285: There is no

   matching specific subroutine for this generic subroutine call.      
[MPI_ALLREDUCE]

     call MPI_ALLREDUCE(test_data(:), &
------------^
check_mpi_in_place_08.f90(140): error #6285: There is no

   matching specific subroutine for this generic subroutine call.      
[MPI_ALLREDUCE]

     call MPI_ALLREDUCE(check_conventional_mpi,

   aux_check_success, 1, MPI_LOGICAL, &

------------^
compilation aborted for check_mpi_in_place_08.f90 (code 1)

This is an interesting result, however … what might I be

   missing? Another use statement?

Best wishes
Volker

On Jul 26, 2017, at 2:53 PM, Gilles Gouaillardet

   <gilles.gouaillar...@gmail.com
   <mailto:gilles.gouaillar...@gmail.com>> wrote:

Volker,

thanks, i will have a look at it

meanwhile, if you can reproduce this issue on a more

   mainstream

platform (e.g. linux + gfortran) please let me know.

since you are using ifort, Open MPI was built with Fortran

bindings, so you can replace
include 'mpif.h'
with
use mpi_f08
and who knows, that might solve your issue


Cheers,

Gilles

On Wed, Jul 26, 2017 at 5:22 PM, Volker Blum

   <volker.b...@duke.edu <mailto:volker.b...@duke.edu>> wrote:

Dear Gilles,

Thank you very much for the fast answer.

Darn. I feared it might not occur on all platforms, since

   my former Macbook

(with an older OpenMPI version) no longer exhibited the

   problem, a different

Linux/Intel Machine did last December, etc.

On this specific machine, the configure line is

./configure CC=gcc FC=ifort F77=ifort

ifort version 17.0.4

blum:/Users/blum/software/openmpi-3.0.0rc1> gcc -v
Configured with:

   --prefix=/Applications/Xcode.app/Contents/Developer/usr

--with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir:

   
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

The full test program is appended.

Compilation:

mpif90 check_mpi_in_place.f90

blum:/Users/blum/codes/fhi-aims/openmpi_test> which mpif90
/usr/local/openmpi-3.0.0rc1/bin/mpif90

blum:/Users/blum/codes/fhi-aims/openmpi_test> which mpirun
/usr/local/openmpi-3.0.0rc1/bin/mpirun

blum:/Users/blum/codes/fhi-aims/openmpi_test> mpirun -np

   2 a.out

* MPI_IN_PLACE does not appear to work as intended.
* Checking whether MPI_ALLREDUCE works at all.
* Without MPI_IN_PLACE, MPI_ALLREDUCE appears to work.

blum:/Users/blum/codes/fhi-aims/openmpi_test> mpirun -np

   1 a.out

* MPI_IN_PLACE does not appear to work as intended.
* Checking whether MPI_ALLREDUCE works at all.
* Without MPI_IN_PLACE, MPI_ALLREDUCE appears to work.

Hopefully, no trivial mistakes in the testcase. I just

   spent a few days

tracing this issue through a fairly large code, which is

   where the issue

originally arose (and leads to wrong numbers).

Best wishes
Volker

On Jul 26, 2017, at 9:46 AM, Gilles Gouaillardet
<gilles.gouaillar...@gmail.com

   <mailto:gilles.gouaillar...@gmail.com>> wrote:

Volker,

i was unable to reproduce this issue on linux

can you please post your full configure command line,

   your gnu

compiler version and the full test program ?

also, how many mpi tasks are you running ?

Cheers,

Gilles

On Wed, Jul 26, 2017 at 4:25 PM, Volker Blum

   <volker.b...@duke.edu <mailto:volker.b...@duke.edu>> wrote:

Hi,

I tried openmpi-3.0.0rc1.tar.gz using Intel Fortran

   2017 and gcc on a

current MacOS system. For this version, it seems to me

   that MPI_IN_PLACE

returns incorrect results (while other MPI

   implementations, including some

past OpenMPI versions, work fine).

This can be seen with a simple Fortran example code,

   shown below. In the

test, the values of all entries of an array “test_data”

   should be 1.0d0 if

the behavior were as intended. However, the version of

   OpenMPI I have

returns 0.d0 instead.

I’ve seen this behavior on some other compute platforms

   too, in the past,

so it wasn’t new to me. Still, I thought that this

   time, I’d ask. Any

thoughts?

Thank you,
Best wishes
Volker

! size of test data array
integer :: n_data

! array that contains test data for MPI_IN_PLACE
real*8, allocatable :: test_data(:)

     integer :: mpierr

n_data = 10

allocate(test_data(n_data),stat=mpierr)

! seed test data array for allreduce call below
if (myid.eq.0) then
    test_data(:) = 1.d0
else
    test_data(:) = 0.d0
end if

! Sum the test_data array over all MPI tasks
call MPI_ALLREDUCE(MPI_IN_PLACE, &
      test_data(:), &
      n_data, &
MPI_DOUBLE_PRECISION, &
      MPI_SUM, &
mpi_comm_global, &
      mpierr )

! The value of all entries of test_data should now be

   1.d0 on all MPI

tasks.
! If that is not the case, then the MPI_IN_PLACE flag

   may be broken.






Volker Blum
Associate Professor
Ab Initio Materials Simulations
Duke University, MEMS Department
144 Hudson Hall, Box 90300, Duke University, Durham, NC

   27708, USA

volker.b...@duke.edu <mailto:volker.b...@duke.edu>
https://aims.pratt.duke.edu
+1 (919) 660 5279 <tel:%2B1%20%28919%29%20660%205279>
Twitter: Aimsduke

Office: 1111 Hudson Hall




_______________________________________________
users mailing list
users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>

   
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=QLtQXqnGSkgQnmgI4RxZXa9R6FhMmgj2FLN452Q0Wis&s=BeracGkSHhIyI_bjKJqPHCqMuP-Se2pRmbiNfugkdK8&e=
   
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=QLtQXqnGSkgQnmgI4RxZXa9R6FhMmgj2FLN452Q0Wis&s=BeracGkSHhIyI_bjKJqPHCqMuP-Se2pRmbiNfugkdK8&e=>

_______________________________________________
users mailing list
users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>

   
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=QLtQXqnGSkgQnmgI4RxZXa9R6FhMmgj2FLN452Q0Wis&s=BeracGkSHhIyI_bjKJqPHCqMuP-Se2pRmbiNfugkdK8&e=
   
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=QLtQXqnGSkgQnmgI4RxZXa9R6FhMmgj2FLN452Q0Wis&s=BeracGkSHhIyI_bjKJqPHCqMuP-Se2pRmbiNfugkdK8&e=>

Volker Blum
Associate Professor
Ab Initio Materials Simulations
Duke University, MEMS Department
144 Hudson Hall, Box 90300, Duke University, Durham, NC

   27708, USA

volker.b...@duke.edu <mailto:volker.b...@duke.edu>
https://aims.pratt.duke.edu
+1 (919) 660 5279 <tel:%2B1%20%28919%29%20660%205279>
Twitter: Aimsduke

Office: 1111 Hudson Hall





_______________________________________________
users mailing list
users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>

   
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=e9pjil1vV3SDa40dQJww0p-d0LhgyQzX_kPNhmz-oUE&s=Y4hrMiRzNuObkpm0vPojCqr6Cx6uS_wLxNyAfUaBz70&e=
   
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=e9pjil1vV3SDa40dQJww0p-d0LhgyQzX_kPNhmz-oUE&s=Y4hrMiRzNuObkpm0vPojCqr6Cx6uS_wLxNyAfUaBz70&e=>

_______________________________________________
users mailing list
users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>

   
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=e9pjil1vV3SDa40dQJww0p-d0LhgyQzX_kPNhmz-oUE&s=Y4hrMiRzNuObkpm0vPojCqr6Cx6uS_wLxNyAfUaBz70&e=
   
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=e9pjil1vV3SDa40dQJww0p-d0LhgyQzX_kPNhmz-oUE&s=Y4hrMiRzNuObkpm0vPojCqr6Cx6uS_wLxNyAfUaBz70&e=>

Volker Blum
Associate Professor
Ab Initio Materials Simulations
Duke University, MEMS Department
144 Hudson Hall, Box 90300, Duke University, Durham, NC

   27708, USA

volker.b...@duke.edu <mailto:volker.b...@duke.edu>
https://aims.pratt.duke.edu
+1 (919) 660 5279 <tel:%2B1%20%28919%29%20660%205279>
Twitter: Aimsduke

Office: 1111 Hudson Hall




_______________________________________________
users mailing list
users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>

   
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=HDubxzHgm3hz7-NQfgobz7rkGf0LWBTlGGqdgSoPCC4&s=2D1Arirt92pKR6i2-4KQKZ8YhSnZ2TPkouQQePHvNf0&e=
   
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=HDubxzHgm3hz7-NQfgobz7rkGf0LWBTlGGqdgSoPCC4&s=2D1Arirt92pKR6i2-4KQKZ8YhSnZ2TPkouQQePHvNf0&e=>

_______________________________________________
users mailing list
users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>

   
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=HDubxzHgm3hz7-NQfgobz7rkGf0LWBTlGGqdgSoPCC4&s=2D1Arirt92pKR6i2-4KQKZ8YhSnZ2TPkouQQePHvNf0&e=
   
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=HDubxzHgm3hz7-NQfgobz7rkGf0LWBTlGGqdgSoPCC4&s=2D1Arirt92pKR6i2-4KQKZ8YhSnZ2TPkouQQePHvNf0&e=>

Volker Blum
Associate Professor
Ab Initio Materials Simulations
Duke University, MEMS Department
144 Hudson Hall, Box 90300, Duke University, Durham, NC 27708, USA

volker.b...@duke.edu <mailto:volker.b...@duke.edu>
https://aims.pratt.duke.edu
+1 (919) 660 5279 <tel:%2B1%20%28919%29%20660%205279>
Twitter: Aimsduke

Office: 1111 Hudson Hall




_______________________________________________
users mailing list
users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=0X7DUjFfbjVVpI_r3F9MFV4Cs6DpbwqBCobJBfWB-DI&s=i7T7XLEvUGLBfkNjzFjixk1JmT_QB0nGMM2j_uhDWyQ&e=
     
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=0X7DUjFfbjVVpI_r3F9MFV4Cs6DpbwqBCobJBfWB-DI&s=i7T7XLEvUGLBfkNjzFjixk1JmT_QB0nGMM2j_uhDWyQ&e=>

_______________________________________________
users mailing list
users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>

   
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwICAg&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=4W_1_DI3QU9PRT2fhTXyUeXiu7HvTSNzX48E-9ifoTc&s=i3f7Olcbyor4Pu0hv6YlgO10cJ_XvOR13zZn7PPIGto&e=
   
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwICAg&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=4W_1_DI3QU9PRT2fhTXyUeXiu7HvTSNzX48E-9ifoTc&s=i3f7Olcbyor4Pu0hv6YlgO10cJ_XvOR13zZn7PPIGto&e=>

Volker Blum
Associate Professor
Ab Initio Materials Simulations
Duke University, MEMS Department
144 Hudson Hall, Box 90300, Duke University, Durham, NC 27708, USA

volker.b...@duke.edu <mailto:volker.b...@duke.edu>
https://aims.pratt.duke.edu
+1 (919) 660 5279 <tel:%2B1%20%28919%29%20660%205279>
Twitter: Aimsduke

Office: 1111 Hudson Hall




_______________________________________________
users mailing list
users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=0X7DUjFfbjVVpI_r3F9MFV4Cs6DpbwqBCobJBfWB-DI&s=i7T7XLEvUGLBfkNjzFjixk1JmT_QB0nGMM2j_uhDWyQ&e=
     
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=0X7DUjFfbjVVpI_r3F9MFV4Cs6DpbwqBCobJBfWB-DI&s=i7T7XLEvUGLBfkNjzFjixk1JmT_QB0nGMM2j_uhDWyQ&e=>

_______________________________________________
users mailing list
users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>

   
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwICAg&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=h6zBsOI45o8fovfy43A2FyCt-fL_yVpNbVSf1OA8CrQ&s=YNADyKbvRnxPmnDVmVlYmsYgZEr8m-etPBXLHPRkflw&e=
   
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwICAg&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=h6zBsOI45o8fovfy43A2FyCt-fL_yVpNbVSf1OA8CrQ&s=YNADyKbvRnxPmnDVmVlYmsYgZEr8m-etPBXLHPRkflw&e=>

   Volker Blum
   Associate Professor
   Ab Initio Materials Simulations
   Duke University, MEMS Department
   144 Hudson Hall, Box 90300, Duke University, Durham, NC 27708, USA

   volker.b...@duke.edu <mailto:volker.b...@duke.edu>
   https://aims.pratt.duke.edu
   +1 (919) 660 5279 <tel:%2B1%20%28919%29%20660%205279>
   Twitter: Aimsduke

   Office: 1111 Hudson Hall




   _______________________________________________
   users mailing list
   users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>
   
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=0X7DUjFfbjVVpI_r3F9MFV4Cs6DpbwqBCobJBfWB-DI&s=i7T7XLEvUGLBfkNjzFjixk1JmT_QB0nGMM2j_uhDWyQ&e=
     
<https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=0X7DUjFfbjVVpI_r3F9MFV4Cs6DpbwqBCobJBfWB-DI&s=i7T7XLEvUGLBfkNjzFjixk1JmT_QB0nGMM2j_uhDWyQ&e=>




_______________________________________________
users mailing list
users@lists.open-mpi.org
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=0X7DUjFfbjVVpI_r3F9MFV4Cs6DpbwqBCobJBfWB-DI&s=i7T7XLEvUGLBfkNjzFjixk1JmT_QB0nGMM2j_uhDWyQ&e=

_______________________________________________
users mailing list
users@lists.open-mpi.org
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=0X7DUjFfbjVVpI_r3F9MFV4Cs6DpbwqBCobJBfWB-DI&s=i7T7XLEvUGLBfkNjzFjixk1JmT_QB0nGMM2j_uhDWyQ&e=

Volker Blum
Associate Professor
Ab Initio Materials Simulations
Duke University, MEMS Department
144 Hudson Hall, Box 90300, Duke University, Durham, NC 27708, USA

volker.b...@duke.edu
https://aims.pratt.duke.edu
+1 (919) 660 5279
Twitter: Aimsduke

Office: 1111 Hudson Hall




_______________________________________________
users mailing list
users@lists.open-mpi.org
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwIFaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=h3HNno_BCKRMuxrZ26zNcyS_JTV7oDefvQtaN7NF3Kk&s=ENCO2vndR26kHOuem57zq1jvk1Tl1ftPBQKcQ2xIpwQ&e=

<test_constants.f90>_______________________________________________
users mailing list
users@lists.open-mpi.org
https://urldefense.proofpoint.com/v2/url?u=https-3A__rfd.newmexicoconsortium.org_mailman_listinfo_users&d=DwICAg&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=I9QLPu689VeINkpRod6EQfprr_v-FLoLsAuSXIHhDsk&m=h3HNno_BCKRMuxrZ26zNcyS_JTV7oDefvQtaN7NF3Kk&s=ENCO2vndR26kHOuem57zq1jvk1Tl1ftPBQKcQ2xIpwQ&e=

Volker Blum
Associate Professor
Ab Initio Materials Simulations
Duke University, MEMS Department
144 Hudson Hall, Box 90300, Duke University, Durham, NC 27708, USA

volker.b...@duke.edu
https://aims.pratt.duke.edu
+1 (919) 660 5279
Twitter: Aimsduke

Office: 1111 Hudson Hall




_______________________________________________
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users


_______________________________________________
users mailing list
users@lists.open-mpi.org
https://rfd.newmexicoconsortium.org/mailman/listinfo/users

Re: [OMPI users] MPI_IN_PLACE

Reply via email to