Jackson,
one more thing, how did you build openmpi ?
if you built from git (and without VPATH), then --enable-debug is
automatically set, and this is hurting performance.
if not already done, i recommend you download the latest openmpi tarball
(1.10.2) and
./configure --with-platform=contrib/platform/optimized --prefix=...
last but not least, you can
mpirun --mca mpi_leave_pinned 1 <your benchmark>
(that being said, i am not sure this is useful with TCP networks ...)
Cheers,
Gilles
On 3/9/2016 11:34 AM, Rayson Ho wrote:
If you are using instance types that support SR-IOV (aka. "enhanced
networking" in AWS), then turn it on. We saw huge differences when
SR-IOV is enabled
http://blogs.scalablelogic.com/2013/12/enhanced-networking-in-aws-cloud.html
http://blogs.scalablelogic.com/2014/01/enhanced-networking-in-aws-cloud-part-2.html
Make sure you start your instances with a placement group --
otherwise, the instances can be data centers apart!
And check that jumbo frames are enabled properly:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html
But still, it is interesting that Intel MPI is getting a 2X speedup
with the same setup! Can you post the raw numbers so that we can take
a deeper look??
Rayson
==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
On Tue, Mar 8, 2016 at 9:08 AM, Jackson, Gary L.
<gary.jack...@jhuapl.edu <mailto:gary.jack...@jhuapl.edu>> wrote:
I've built OpenMPI 1.10.1 on Amazon EC2. Using NetPIPE, I'm seeing
about half the performance for MPI over TCP as I do with raw TCP.
Before I start digging in to this more deeply, does anyone know
what might cause that?
For what it's worth, I see the same issues with MPICH, but I do
not see it with Intel MPI.
--
Gary Jackson
_______________________________________________
users mailing list
us...@open-mpi.org <mailto:us...@open-mpi.org>
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/03/28659.php
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/03/28665.php
