Re: [O-MPI users] Re: [Beowulf] Alternative to MPI ABI

[Mark Hahn]

> If there is an ABI then we have a fighting chance at focusing on the 
> applications and not the ever-so-slightly-strange version of whichever 
> flavor of MPI that they chose to use.

wonderful!  yes: ABI standards are good and proprietary
implementations (which inherently provide only negative 
definitions of support) are bad.

after all, the real appeal is that N MPI implementation only need to test
their own conformity to the standard, and M applications test their
conformity.  ie, N+M tests, rather than N*M without an ABI.  this 
assumes that the ABI/standard is broad enough, of course!

first, it's worth asking whether there is something to be lost
by going to an ABI?  yes, dynamic linking imposes some overhead - 
I have to wonder whether some of the higher-performing interconnects
(SGI/Cray/Quadrics/Pathscale) are low-latency enough to worry about 
indirect library calls blowing the pipeline.

Re: [OMPI users] How to replace --cpus-per-proc by --map-by

[Mark Hahn]

We're open to suggestion, really - just need some help identifying the best
way to get this info out there.


well, OpenMPI information is fragmented and sprayed all over.
In some places, there is mention of a wiki to be updated with 
an explanation; for other things, a consumer needs to wander around 
loosely-related blogs, mail archives, FAQs, usage statements, etc.


For instance, I've been trying to figure out how to do a simple thing,
launch a hybrid job.  Assume I have a scheduled, heterogenous cluster
where mpirun simply receives a normal nodefile like this:

clu357
clu357
clu357
clu354
clu354
clu354

and I want to launch a 2-rank, 3-thread-per-rank job.  forget about 
frills like hwloc or binding.


back when --cpus-per-proc was around, this was obvious and worked 
flawlessly.  I honestly can't figure out how it works now, though - 
for any definition of "now" since:


http://www.open-mpi.org/community/lists/devel/2011/12/10060.php

2011!  then there's a dribble more info in 2014 (!) that hints that 
"--map-by node:pe=3" might do the trick here:


http://comments.gmane.org/gmane.comp.clustering.open-mpi.user/21193

where did "pe" come from?  is it the same as slot, hwthread, core?
why does the documentation make snide comments about how the conventional
understanding of "rank" (~ equivalent to process) might not be true?

most of all, when was the break introduced?  at this point, I tell people
that 1.4.3 worked, and that everything after that is broken.

recent releases (I tried 1.7.3, 1.7.5 and 1.8.1) choke on this. 
I wonder whether it's having trouble with the fact that a job 
gets an arbitrary set of cores via cgroup, and perhaps hwloc 
doesn't understand that it can only work within this set...




   So please see this URL below(especially the first half part
   of it - from 1 to 20 pages):
   
http://www.slideshare.net/jsquyres/open-mpi-explorations-in-process-affinity-eurompi13-presentation

   Although these slides by Jeff are the explanation for LAMA,
   which is another mapping system installed in the openmpi-1.7
   series, I guess you can easily understand what is mapping and
   binding in general terms.


AFAIKT, the lama slide deck seemed to be only concerned with 
affinity settings, which are irrelevant here.


confused,
Mark Hahn.

Re: [OMPI users] password-less ssh

[Mark Hahn]

Okay, yes, setting SSH_AUTH_SOCK is the right thing to do, but this strikes 
me as clumsy.


normally, you should run ssh-agent only on the machine where you sit. 
it operates until you logout (you can also tell it to discard keys). 
ssh-agent is normally part if your X startup sequence, so that all

terminals/etc inherit SSH_AUTH_SOCK and can do agent-based operations.

I'm trying to understand how things should be set up so that I 
don't have to take special action each time I log in.  Do I do some 
.login/.logout magic?


no, that would be very clumsy.

Or, why not just go without a DSA passphrase?  The passphrase only protects 
me from root, before whom I am rather powerless anyhow.


unencrypted keys are the moral equivalent of putting your password
into a file, in the clear.  anyone who gains access to the file _owns_
your accounts.

Also, the OMPI FAQ says authorized_keys should have 644 protection.  Out on 
the web, it appears people advise 600, which doesn't make sense to me since 
it just has public keys in it anyhow.  (My head is starting to spin.)


it shouldn't be 644, since there's no reason for unrelated users to know 
what keys permit login to your account.  the answer is also wrong about
permissions for home and .ssh directories: there's no ssh reason for 
either to be readable by group or other.  what ssh is most picky about 
is that it doesn't want ~, .ssh or .ssh/authorized_keys to be writable,

even by group.  that is, no one but the user should be able to alter
what keys permit login.  but I can't think of any reason to permit 
anyone, even in your group, to read .ssh, either.



This is with regards to http://www.open-mpi.org/faq/?category=rsh#ssh-keys


this page is not wonderful.

a pretty common structure for clusters is that both login and compute nodes
are in the same administrative domain.  that is, the same definition of valid
users, shared home directory, etc.  as such, the most robust, least-effort 
use of ssh is to make the compute nodes trust (via hostbased shosts.equiv)

the login nodes.

Also, it appears that I do *NOT* have an ssh-agent running automatically 
for me.  How often do I have to start one up?  It appears that if I start


many desktop distos will start ssh-agent automatically - for instance,
.Xclients might do
  exec ssh-agent startkde
(this means that X starts ssh-agent as a parent to your window manager,
the main point of which is that the WM and all the apps it starts will
inherit your SSH_AUTH_SOCK setting.  you'll want to run ssh-add as well
to actually provide the agent with keys.  when you logout, the agent 
goes away as it should.



one up and log out and then log back in again, the old ssh-agent is still


ssh-agent should run on your desktop, since you fundamentally must trust
the machine you sit at.  you should avoid starting a remote ssh-agent
(and should avoid agent forwarding if possible.)

there but not usable.  I have to start up a new one.  So, do I have to 
start an ssh-agent each time I log in?


Or, I could use no DSA passphrase, but that seems to be frowned upon.


it's just hazardous.  if the unencrypted private key file never leaves 
your desktop, it's not entirely stupid, but putting an unencrypted priv key

on remote machine or fileserver is just asking to be owned.  (though it is
useful to note that the authorized_keys file format can express some pretty
powerful restrictions on how the key can be used - only from certain hosts,
only to do a certain command, etc.)

regards, mark hahn.