Great it is finally working !

nvml and opencl are only used by hwloc, and i do not think Open MPI is using these features,

so i suggest you go ahead, reconfigure and rebuild Open MPI and see how things go


Cheers,


Gilles


On 9/22/2017 4:59 PM, Tim Jim wrote:
Hi Gilles,

Yes, you're right. I wanted to double check the compile but didn't notice I was pointing to the exec I compiled from a previous Make.

mpicc now seems to work, running mpirun hello_c gives:

Hello, world, I am 0 of 4, (Open MPI v3.0.0, package: Open MPI tjim@DESKTOP-TA3P0PS Distribution, ident: 3.0.0, repo rev: v3.0.0, Sep 12, 2017, 115) Hello, world, I am 3 of 4, (Open MPI v3.0.0, package: Open MPI tjim@DESKTOP-TA3P0PS Distribution, ident: 3.0.0, repo rev: v3.0.0, Sep 12, 2017, 115) Hello, world, I am 1 of 4, (Open MPI v3.0.0, package: Open MPI tjim@DESKTOP-TA3P0PS Distribution, ident: 3.0.0, repo rev: v3.0.0, Sep 12, 2017, 115) Hello, world, I am 2 of 4, (Open MPI v3.0.0, package: Open MPI tjim@DESKTOP-TA3P0PS Distribution, ident: 3.0.0, repo rev: v3.0.0, Sep 12, 2017, 115)

- which I assume that means it's working?

Should I try a recompile with CUDA while declaring "export nvml_enable=no" and "export enable_opencl=no"? What effects do these declarations have on the normal functioning of mpi?

Many thanks.


On 22 September 2017 at 15:55, Gilles Gouaillardet <gilles.gouaillar...@gmail.com <mailto:gilles.gouaillar...@gmail.com>> wrote:

    Was there an error in the copy/paste ?

    The mpicc command should be
    mpicc  /opt/openmpi/openmpi-3.0.0_src/examples/hello_c.c

    Cheers,

    Gilles

    On Fri, Sep 22, 2017 at 3:33 PM, Tim Jim <timothy.m....@gmail.com
    <mailto:timothy.m....@gmail.com>> wrote:

        Thanks for the thoughts and comments. Here is the setup
        information:
        OpenMPI Ver. 3.0.0. Please see attached for the compressed
        config.log and ompi_info --all call.
        In this compile, my install steps were:
        1. declared  "export nvml_enable=no" and "export
        enable_opencl=no" in the terminal
        and the rest as seen in the logs:
        2. ./configure --without-cuda --prefix=/opt/openmpi/openmpi-3.0.0
        3. make all install

        I ultimately would like CUDA to be utilised if it can speed up
        my computation time - should I still attempt to get openMPI
        working without CUDA first?

        Thanks for the heads up about compiling the executables first
        - I tried mpicc again with the compiled version but got the
        following output:

        tjim@DESKTOP-TA3P0PS:~/Documents$ mpicc
        /opt/openmpi/openmpi-3.0.0_src/examples/hello_c
        /opt/openmpi/openmpi-3.0.0_src/examples/hello_c: In function
        `_start':
        (.text+0x0): multiple definition of `_start'
        
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o:(.text+0x0):
        first defined here
        /opt/openmpi/openmpi-3.0.0_src/examples/hello_c: In function
        `_fini':
        (.fini+0x0): multiple definition of `_fini'
        
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o:(.fini+0x0):
        first defined here
        /opt/openmpi/openmpi-3.0.0_src/examples/hello_c:(.rodata+0x0):
        multiple definition of `_IO_stdin_used'
        
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o:(.rodata.cst4+0x0):
        first defined here
        /opt/openmpi/openmpi-3.0.0_src/examples/hello_c: In function
        `data_start':
        (.data+0x0): multiple definition of `__data_start'
        
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o:(.data+0x0):
        first defined here
        /opt/openmpi/openmpi-3.0.0_src/examples/hello_c: In function
        `data_start':
        (.data+0x8): multiple definition of `__dso_handle'
        /usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o:(.data+0x0): first
        defined here
        /opt/openmpi/openmpi-3.0.0_src/examples/hello_c: In function
        `_init':
        (.init+0x0): multiple definition of `_init'
        
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o:(.init+0x0):
        first defined here
        /usr/lib/gcc/x86_64-linux-gnu/5/crtend.o:(.tm_clone_table+0x0):
        multiple definition of `__TMC_END__'
        /opt/openmpi/openmpi-3.0.0_src/examples/hello_c:(.data+0x10):
        first defined here
        /usr/bin/ld: error in
        /opt/openmpi/openmpi-3.0.0_src/examples/hello_c(.eh_frame); no
        .eh_frame_hdr table will be created.
        collect2: error: ld returned 1 exit status

        Is this due to a failed install?
        Regards,
        Tim


        On 22 September 2017 at 01:10, Sylvain Jeaugey
        <sjeau...@nvidia.com <mailto:sjeau...@nvidia.com>> wrote:

            The issue is related to openCL, not NVML.

            So the correct export would be "export enable_opencl=no"
            (you may want to "export enable_nvml=no" as well).


            On 09/21/2017 12:32 AM, Tim Jim wrote:
            Hi,

            I tried as you suggested: export nvml_enable=no, then
            reconfigured and ran make all install again, but mpicc is
            still producing the same error. What should I try next?

            Many thanks,
            Tim

            On 21 September 2017 at 16:12, Gilles Gouaillardet
            <gil...@rist.or.jp <mailto:gil...@rist.or.jp>> wrote:

                Tim,


                do that in your shell, right before invoking configure.

                export nvml_enable=no

                ./configure ...

                make && make install


                you can keep the --without-cuda flag (i think this is
                unrelated though)


                Cheers,

                Gilles

                On 9/21/2017 3:54 PM, Tim Jim wrote:

                    Dear Gilles,

                    Thanks for the mail - where should I set export
                    nvml_enable=no? Should I reconfigure with default
                    cuda support or keep the --without-cuda flag?

                    Kind regards,
                    Tim

                    On 21 September 2017 at 15:22, Gilles
                    Gouaillardet <gil...@rist.or.jp
                    <mailto:gil...@rist.or.jp>
                    <mailto:gil...@rist.or.jp
                    <mailto:gil...@rist.or.jp>>> wrote:

                        Tim,


                        i am not familiar with CUDA, but that might help

                        can you please

                        export nvml_enable=no

                        and then re-configure and rebuild Open MPI ?


                        i hope this will help you


                        Cheers,


                        Gilles



                        On 9/21/2017 3:04 PM, Tim Jim wrote:

                            Hello,

                            Apologies to bring up this old thread - I
                    finally had a chance
                            to try again with openmpi but I am still
                    have trouble getting
                            it to run. I downloaded version 3.0.0
                    hoping it would solve
                            some of the problems but on running
                    mpicc for the previous
                            test case, I am still getting an
                    undefined reference error. I
                            did as you suggested and also configured
                    it to install without
                            cuda using

                    ./configure --without-cuda
                    --prefix=/opt/openmpi/openmpi-3.0.0

                            and at the end of the summary, CUDA
                    support shows 'no'.
                    Unfortunately, the error is still the same, and
                    for some
                            reason, mpicc still seems to have
                    referenced my cuda targets.

                    tjim@DESKTOP-TA3P0PS:~/Documents$ mpicc
                    /opt/mpi4py/mpi4py_src/demo/helloworld.c -o hello.bin
                            mpicc:
                    /usr/local/cuda-8.0/targets/x86_64-linux/lib/libOpenCL.so.1:
                            no version information available (required by
                    /opt/openmpi/openmpi-3.0.0/lib/libopen-pal.so.40)
                    /opt/openmpi/openmpi-3.0.0/lib/libopen-pal.so.40:
                    undefined
                            reference to
                    `clGetPlatformInfo@OPENCL_1.0
                    <mailto:clGetPlatformInfo@OPENCL_1.0>'
                    /opt/openmpi/openmpi-3.0.0/lib/libopen-pal.so.40:
                    undefined
                            reference to `clGetPlatformIDs@OPENCL_1.0
                    <mailto:clGetPlatformIDs@OPENCL_1.0>'
                    /opt/openmpi/openmpi-3.0.0/lib/libopen-pal.so.40:
                    undefined
                            reference to `clGetDeviceInfo@OPENCL_1.0
                    <mailto:clGetDeviceInfo@OPENCL_1.0>'
                    /opt/openmpi/openmpi-3.0.0/lib/libopen-pal.so.40:
                    undefined
                            reference to `clGetDeviceIDs@OPENCL_1.0
                    <mailto:clGetDeviceIDs@OPENCL_1.0>'
                            collect2: error: ld returned 1 exit status

                            I also attempted to test mpirun, as
                    suggested in the readme,
                            however I get the following problem:

                     tjim@DESKTOP-TA3P0PS:~/Documents$ mpirun
                    /opt/openmpi/openmpi-3.0.0_src/examples/hello_c.c
                            mpirun:
                    /usr/local/cuda-8.0/targets/x86_64-linux/lib/libOpenCL.so.1:
                            no version information available (required by
                    /opt/openmpi/openmpi-3.0.0/lib/libopen-pal.so.40)
                    
--------------------------------------------------------------------------
                            Open MPI tried to fork a new process via
                    the "execve" system
                            call but
                            failed. Open MPI checks many things
                    before attempting to launch a
                            child process, but nothing is perfect.
                    This error may be
                            indicative
                            of another problem on the target host, or
                    even something as
                            silly as
                            having specified a directory for your
                    application. Your job
                            will now
                            abort.

                              Local host:  DESKTOP-TA3P0PS
                              Working dir: /home/tjim/Documents
                    Application name:
                     /opt/openmpi/openmpi-3.0.0_src/examples/hello_c.c
                              Error:           Exec format error
                    
--------------------------------------------------------------------------
                    
--------------------------------------------------------------------------
                            mpirun was unable to start the specified
                    application as it
                    encountered an
                            error:

                            Error code: 1
                            Error name: (null)
                            Node: DESKTOP-TA3P0PS

                            when attempting to start process rank 0.
                    
--------------------------------------------------------------------------
                            4 total processes failed to start
                    [DESKTOP-TA3P0PS:15231] 3 more processes have
                    sent help
                            message help-orte-odls-default.txt /
                    execve error
                    [DESKTOP-TA3P0PS:15231] Set MCA parameter
                    "orte_base_help_aggregate" to 0 to see all help /
                    error messages


                            Do you have any suggestions to what might
                    have gone wrong on
                            this install? I'm not sure if this thread
                    is still alive, so
                            if you need a refresh on the
                    situation/any more info, please
                            let me know.
                            Kind regards,
                            Tim

                            On 24 May 2017 at 09:12, Tim Jim
                    <timothy.m....@gmail.com
                    <mailto:timothy.m....@gmail.com>
                    <mailto:timothy.m....@gmail.com
                    <mailto:timothy.m....@gmail.com>>
                            <mailto:timothy.m....@gmail.com
                    <mailto:timothy.m....@gmail.com>
                            <mailto:timothy.m....@gmail.com
                    <mailto:timothy.m....@gmail.com>>>> wrote:

                                Thanks for the thoughts, I'll give it
                    a go. For reference,
                            I have
                    installed it in the opt directory, as that is
                    where I have
                            kept my
                                installs currently. Will this be a
                    problem when calling
                            mpi from
                                other packages?

                                Thanks,
                                Tim

                                On 24 May 2017 06:30, "Reuti"
                    <re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>
                            <mailto:re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>>
                    <mailto:re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>
                            <mailto:re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>>>> wrote:

                                    Hi,

                                    Am 23.05.2017 um 05:03 schrieb
                    Tim Jim:

                                    > Dear Reuti,
                                    >
                                    > Thanks for the reply. What
                    options do I have to test
                            whether
                                    it has successfully built?

                                    LIke before: can you compile and
                    run mpihello.c this
                            time –
                                    all as ordinary user in case you
                    installed the Open
                            MPI into
                    something like $HOME/local/openmpi-2.1.1 and set
                    paths
                    accordingly. There is no need to be root to install a
                            personal
                                    Open MPI version in your home
                    directory.

                                    -- Reuti


                                    >
                                    > Thanks and kind regards.
                                    > Tim
                                    >
                                    > On 22 May 2017 at 19:39, Reuti
                            <re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>
                    <mailto:re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>>
                      <mailto:re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>
                            <mailto:re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>>>> wrote:
                                    > Hi,
                                    >
                                    > > Am 22.05.2017 um 07:22
                    schrieb Tim Jim
                                    <timothy.m....@gmail.com
                    <mailto:timothy.m....@gmail.com>
                            <mailto:timothy.m....@gmail.com
                    <mailto:timothy.m....@gmail.com>>
                    <mailto:timothy.m....@gmail.com
                    <mailto:timothy.m....@gmail.com>

                    <mailto:timothy.m....@gmail.com
                    <mailto:timothy.m....@gmail.com>>>>:

                    > >
                    > > Hello,
                    > >
                    > > Thanks for your message. I'm trying to get
                    this to
                            work on
                                    a single
                    > > machine.
                    >
                    > Ok.
                    >
                    >
                    > > How might you suggest getting OpenMPIworking
                    without
                    python and
                    > > CUDA?
                    >
                    > It looks like it's detected automatically. It
                    should be
                    possible to disable it with the command line option:
                    >
                    > $ ./configure --without-cuda …
                    >
                    > At the end of the configure step out should liste
                            some lines
                    like:
                    >
                    > Miscellaneous
                    > -----------------------
                    > CUDA support: no
                    >
                    > The mpi4py seems unrelated to the compilation of
                            Open MPI
                    itself AFAICS.
                    >
                    >
                    > > I don't recall setting anything for either,
                    as the
                            only
                    command I had
                    > > run was "./configure
                    --prefix=/opt/openmpi/openmpi-2.1.0"
                                    - did it possibly
                    > > pick up the paths by accident?
                    > >
                    > > Regarding the lib directory, I checked that
                    the path
                    physically exists.
                    > > Regarding the final part of the email, is it
                    a problem
                    that 'undefined
                    > > reference' is appearing?
                    >
                    > Yes, it tries to resolve missing symbols and didn't
                            succeed.
                    >
                    > -- Reuti
                    >
                    >
                    > >
                    > > Thanks and regards,
                    > > Tim
                    > >
                    > > On 22 May 2017 at 06:54, Reuti
                            <re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>
                    <mailto:re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>>
                    <mailto:re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>

                    <mailto:re...@staff.uni-marburg.de
                    <mailto:re...@staff.uni-marburg.de>>>> wrote:
                    > >
                    > >> -----BEGIN PGP SIGNED MESSAGE-----
                    > >> Hash: SHA1
                    > >>
                    > >> Hi,
                    > >>
                    > >> Am 18.05.2017 um 07:44 schrieb Tim Jim:
                    > >>
                    > >>> Hello,
                    > >>>
                    > >>> I have been having some issues with trying
                    to get
                    OpenMPI working with
                    > >> mpi4py. I've tried to break down my
                    troubleshooting into
                                    a few chunks
                    > >> below, and I believe that there are a few,
                    distinct
                    issues that need
                    > >> solving.
                    > >>
                    > >> Are you speaking here of a single machine or a
                            cluster?
                    > >>
                    > >>
                    > >>> Following some troubleshooting in the following
                            link:
                    > >>>
                    https://bitbucket.org/mpi4py/mpi4py/issues/69/building-
                    <https://bitbucket.org/mpi4py/mpi4py/issues/69/building->
                           
                    <https://bitbucket.org/mpi4py/mpi4py/issues/69/building-
                    <https://bitbucket.org/mpi4py/mpi4py/issues/69/building->>

                           
                    <https://bitbucket.org/mpi4py/mpi4py/issues/69/building-
                    <https://bitbucket.org/mpi4py/mpi4py/issues/69/building->
                           
                    <https://bitbucket.org/mpi4py/mpi4py/issues/69/building-
                    <https://bitbucket.org/mpi4py/mpi4py/issues/69/building->>>
                    > >> mpi4py-with-openmpi-gives-error
                    > >>> -the mpi4py folks have suggested it an issue
                            that might
                                    be better
                    > >> answered here.
                    > >>
                    > >> First approach would be to get Open MPI working,
                            without
                    CUDA and Python
                    > >> being involved.
                    > >>
                    > >>
                    > >>> In summary, I have attempted to install
                    OpenMPI on
                    Ubuntu 16.04 to the
                    > >> following prefix: /opt/openmpi-openmpi-2.1.0. I
                            have also
                    manually added
                    > >> the following to my .bashrc:
                    > >>> export
                    PATH="/opt/openmpi/openmpi-2.1.0/bin:$PATH"
                    > >>> MPI_DIR=/opt/openmpi/openmpi-2.1.0
                    > >>> export
                    LD_LIBRARY_PATH=$MPI_DIR/lib:$LD_LIBRARY_PATH
                    > >>
                    > >> This looks fine, although I don't recall setting
                            MPI_DIR
                    for Open MPI
                    > >> itself. It might be a necessity for mpi4py
                    though.
                    > >>
                    > >> One pitfall might be that "lib" is sometimes
                    being
                    created as "lib64" by
                    > >> `libtool`. I forgot the details when this is
                            happening,
                    but it depends on
                    > >> the version of `libtool` being used.
                    > >>
                    > >>
                    > >>> I later became aware that Ubuntu may handle the
                    LD_LIBRARY_PATH
                    > >> differently
                    > >>
                    > >> I don't think that Ubuntu will do anything
                            different than
                    any other Linux.
                    > >>
                    > >> Did you compile Open MPI on your own, or did you
                            install
                    any repository?
                    > >>
                    > >> Are the CUDA application written by yourself
                    or any
                    freely available
                    > >> applications?
                    > >>
                    > >> - -- Reuti
                    > >>
                    > >>
                    > >>> and instead added a new file containing the
                            library path
                    > >> /opt/openmpi/openmpi-2.1.0/lib to
                    /etc/ld.so.conf.d/openmpi-2-1-0.conf,
                    > >> in the style of everything else in that
                    directory.
                    > >>>
                    > >>> I tried to run "mpicc helloworld.c -o
                    hello.bin"
                            as a
                    test on a demo
                    > >> file (as instructed in the link) to check the
                    installation but I had
                    > >> permission issues, since it was installed
                    win opt.
                    However, when I
                    > >> attempted to run the previous with sudo, or sudo
                            -E, in
                    both cases, mpicc
                    > >> could not be found. (Perhaps this is a
                    separate issue
                    with my sudo env)
                    > >>>
                    > >>> To check that mpicc actually works, I have
                    copied
                    helloworld.c to a
                    > >> directory where I could execute mpicc without
                            sudo. On
                    running the above
                    > >> command, I receive the following error:
                    > >>>
                    > >>> mpicc:

                    /usr/local/cuda-8.0/targets/x86_64-linux/lib/libOpenCL.so.1:
                    no
                    > >> version information available (required by
                    /opt/openmpi/openmpi-2.1.0/
                    > >> lib/libopen-pal.so.20)
                    > >>>
                    /opt/openmpi/openmpi-2.1.0/lib/libopen-pal.so.20:
                    undefined reference
                    > >> to `clGetPlatformInfo@OPENCL_1.0
                    <mailto:clGetPlatformInfo@OPENCL_1.0>'
                    > >>>
                    /opt/openmpi/openmpi-2.1.0/lib/libopen-pal.so.20:
                    undefined reference
                    > >> to `clGetPlatformIDs@OPENCL_1.0
                    <mailto:clGetPlatformIDs@OPENCL_1.0>'
                    > >>>
                    /opt/openmpi/openmpi-2.1.0/lib/libopen-pal.so.20:
                    undefined reference
                    > >> to `clGetDeviceInfo@OPENCL_1.0
                    <mailto:clGetDeviceInfo@OPENCL_1.0>'
                    > >>>
                    /opt/openmpi/openmpi-2.1.0/lib/libopen-pal.so.20:
                    undefined reference
                    > >> to `clGetDeviceIDs@OPENCL_1.0
                    <mailto:clGetDeviceIDs@OPENCL_1.0>'
                    > >>> collect2: error: ld returned 1 exit status
                    > >>>
                    > >>> I am unsure if I have an installation or
                    permission
                    issues, and I'd be
                    > >> grateful if anyone can shed some light based
                    on the
                    trials I've done so
                    > >> far. (I should add I also have a CUDA
                    installation, which
                    I'd like to
                    > >> leverage too, if possible). I'm still fairly new
                            to the
                    ins and outs of
                    > >> this, so I may have missed something obvious.
                            Please let
                                    me know if any
                    > >> other info is required.
                    > >>>
                    > >>> Many thanks and kind regards,
                    > >>> Tim
                    > >>>
                    > >>> --
                    > >>>
                    > >>> Timothy Jim
                    > >>> PhD Researcher in Aerospace
                    > >>> Creative Flow Research Division,
                    > >>> Institute of Fluid Science, Tohoku University
                    > >>> www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>
                            <http://www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>>
                                   
                    <http://www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>
                            <http://www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>>>
                                    > >>>
                    _______________________________________________
                                    > >>> users mailing list
                                    > >>> users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>
                    <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>>
                                    > >>>
                    https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>

                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>>
                                    > >>
                                    > >> -----BEGIN PGP SIGNATURE-----
                                    > >> Comment: GPGTools -
                    https://gpgtools.org
                                    > >>
                                    > >>

                    
iEYEARECAAYFAlkiDKIACgkQo/GbGkBRnRou3wCeNztodNKHRxXBAp5ylz041OMP
                                    > >> RIwAoLWkjzK1Qr8eWle/PdZGVUsvdN1M
                                    > >> =R6up
                                    > >> -----END PGP SIGNATURE-----
                                    > >>
                    _______________________________________________
                                    > >> users mailing list
                                    > >> users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>
                    <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>>
                                    > >>
                    https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>

                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>>
                                    > >>
                                    > >
                                    > >
                                    > >
                                    > > --
                                    > >
                                    > >
                                    > > *Timothy Jim**PhD Researcher
                    in Aerospace*
                                    > >
                                    > > Creative Flow Research Division,
                                    > > Institute of Fluid Science,
                    Tohoku University
                                    > >
                                    > > www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>
                            <http://www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>>
                      <http://www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>
                            <http://www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>>>
                                    > >
                    _______________________________________________
                                    > > users mailing list
                                    > > users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>
                    <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>>
                                    > >
                    https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>

                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>>
                                    >
                                    >
                    _______________________________________________
                                    > users mailing list
                                    > users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>
                    <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>>
                                    >
                    https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>

                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>>
                                    >
                                    >
                                    >
                                    > --
                                    >
                                    > Timothy Jim
                                    > PhD Researcher in Aerospace
                                    > Creative Flow Research Division,
                                    > Institute of Fluid Science,
                    Tohoku University
                                    > www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>
                            <http://www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>>
                            <http://www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>
                            <http://www.linkedin.com/in/timjim/
                    <http://www.linkedin.com/in/timjim/>>>
                                    >
                    _______________________________________________
                                    > users mailing list
                                    > users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>
                    <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>>
                                    >
                    https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>

                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>>

                    _______________________________________________
                    users mailing list
                    users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                    <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>
                    <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                            <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>>
                    https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>

                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
                           
                    <https://rfd.newmexicoconsortium.org/mailman/listinfo/users
                    
<https://rfd.newmexicoconsortium.org/mailman/listinfo/users>>>




                    _______________________________________________
                            users mailing list
                    users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                    <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>
                    https://lists.open-mpi.org/mailman/listinfo/users
                    <https://lists.open-mpi.org/mailman/listinfo/users>
                           
                    <https://lists.open-mpi.org/mailman/listinfo/users
                    <https://lists.open-mpi.org/mailman/listinfo/users>>


                    _______________________________________________
                        users mailing list
                    users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                    <mailto:users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>>
                    https://lists.open-mpi.org/mailman/listinfo/users
                    <https://lists.open-mpi.org/mailman/listinfo/users>
                       
                    <https://lists.open-mpi.org/mailman/listinfo/users
                    <https://lists.open-mpi.org/mailman/listinfo/users>>




                    _______________________________________________
                    users mailing list
                    users@lists.open-mpi.org
                    <mailto:users@lists.open-mpi.org>
                    https://lists.open-mpi.org/mailman/listinfo/users
                    <https://lists.open-mpi.org/mailman/listinfo/users>


                _______________________________________________
                users mailing list
                users@lists.open-mpi.org
                <mailto:users@lists.open-mpi.org>
                https://lists.open-mpi.org/mailman/listinfo/users
                <https://lists.open-mpi.org/mailman/listinfo/users>




--
                

            *Timothy Jim
            */PhD Researcher in Aerospace/

            Creative Flow Research Division,
            Institute of Fluid Science, Tohoku University

            www.linkedin.com/in/timjim/
            <http://www.linkedin.com/in/timjim/>



            _______________________________________________
            users mailing list
            users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>
            https://lists.open-mpi.org/mailman/listinfo/users
            <https://lists.open-mpi.org/mailman/listinfo/users>

            
------------------------------------------------------------------------
            This email message is for the sole use of the intended
            recipient(s) and may contain confidential information. 
            Any unauthorized review, use, disclosure or distribution
            is prohibited.  If you are not the intended recipient,
            please contact the sender by reply email and destroy all
            copies of the original message.
            
------------------------------------------------------------------------

            _______________________________________________
            users mailing list
            users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>
            https://lists.open-mpi.org/mailman/listinfo/users
            <https://lists.open-mpi.org/mailman/listinfo/users>




        _______________________________________________
        users mailing list
        users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>
        https://lists.open-mpi.org/mailman/listinfo/users
        <https://lists.open-mpi.org/mailman/listinfo/users>



    _______________________________________________
    users mailing list
    users@lists.open-mpi.org <mailto:users@lists.open-mpi.org>
    https://lists.open-mpi.org/mailman/listinfo/users
    <https://lists.open-mpi.org/mailman/listinfo/users>




--

        

*Timothy Jim
*/PhD Researcher in Aerospace/

Creative Flow Research Division,
Institute of Fluid Science, Tohoku University

www.linkedin.com/in/timjim/ <http://www.linkedin.com/in/timjim/>



_______________________________________________
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

_______________________________________________
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Reply via email to