On 10/20/2017 12:24 PM, Dave Love wrote: > Paul Kapinos <kapi...@itc.rwth-aachen.de> writes: > >> Hi all, >> sorry for the long long latency - this message was buried in my mailbox for >> months.... >> >> >> >> On 03/16/2017 10:35 AM, Alfio Lazzaro wrote: >>> Hello Dave and others, >>> we jump in the discussion as CP2K developers. >>> We would like to ask you which version of CP2K you are using in your tests >> version 4.1 (release) >> >>> and >>> if you can share with us your input file and output log. >> >> The input file is property of Mathias Schumacher (CC:) and we need a >> permission >> of him to provide it. > > I lost track of this, but the problem went away using libfabric instead > of openib, so I left it at that, though libfabric hurt IMB pingpong > latency compared with openib. > > I seem to remember there's a workaround in the cp2k development source, > but that obviously doesn't solve the general problem. >
The issue has two facing pages: - CP2K used a lot of MPI_Alloc_mem() / MPI_Free_mem() calls. This was addressed by CP2K developers, (private mail by Alfio Lazzaro): > in the new CP2K release (next week will have version 5.1), I have reduced the > amount of MPI allocations. I have also added a flag to avoid any MPI > allocations, that you can add in the CP2K input file: > &GLOBAL > &DBCSR > use_mpi_allocator F > &END > &END GLOBAL We will test the new release after it is available. (Note, the user still has to think on disabling use_mpi_allocator, if in doubt.) - in Open MPI compiled for both(1) IB and OPA, on a node with OPA, using *default* configuration (failback to 'openib' *not prohibited*), MPI_Free_mem() calls suddenly lasts 10000 or so times longer, starting to dominate the application run time. Known workaround: prohibit the failback to 'openib' BTL by '-mca btl ^tcp,openib' - that's what we implemented. It's up to Open MPI developers if they would like to follow-up this 'small' performance issue. Best, Paul Kapinos P.S. It was a hard work even to locate this issue, as only 2 tools (of 7 or 8 tried) were able to point to the evil call... (1) yes we use the same Open MPI installation on islands with InfiniBand, with OmniPath, and even on ethernet-only. -- Dipl.-Inform. Paul Kapinos - High Performance Computing, RWTH Aachen University, IT Center Seffenter Weg 23, D 52074 Aachen (Germany) Tel: +49 241/80-24915
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
