If a user connects to a cluster using SSH, starts an MPI program which contains
an infinite loop, and then breaks the SSH connection, the processes running the
MPI program continue to run on the compute nodes and they have to be killed
manually.
To investigate this, I found that if the user types Control-C (or sends SIGINT
to mpirun), mpirun says "killing job...", and on each compute node the orted
process and the process running the MPI program are killed.
However if SIGHUP is sent to mpirun, it says "Hangup" and exits, and on each
compute node the orted process is killed but the process running the MPI
program continues to run.
This is with Rocks 5.3 and Open MPI. There is no batch scheduler. The MPI
program is just:
#include "mpi.h"
main(int argc, char **argv) {
MPI_Init(&argc, &argv);
while (1) ;
MPI_Finalize();
}
I have two questions. Is this the behaviour I should expect? Is there an easy
way to kill the processes?
Thanks, Jon Hitchcock
