Hi, I am running my jobs on openmpi linux platform. I am getting the following errors:
[compute-12-23.local:10203] *** An error occurred in MPI_Comm_rank [compute-12-23.local:10203] *** on communicator MPI_COMM_WORLD [compute-12-23.local:10203] *** MPI_ERR_COMM: invalid communicator [compute-12-23.local:10203] *** MPI_ERRORS_ARE_FATAL: your MPI job will now abort -------------------------------------------------------------------------- mpirun has exited due to process rank 11 with PID 10211 on node compute-12-23.local exiting improperly. There are two reasons this could occur: 1. this process did not call "init" before exiting, but others in the job did. This can cause a job to hang indefinitely while it waits for all processes to call "init". By rule, if one process calls "init", then ALL processes must call "init" prior to termination. 2. this process called "init", but exited without calling "finalize". By rule, all processes that call "init" MUST call "finalize" prior to exiting or it will be considered an "abnormal termination" This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here). -------------------------------------------------------------------------- [compute-12-23.local:10199] 11 more processes have sent help message help-mpi-errors.txt / mpi_errors_are_fatal [compute-12-23.local:10199] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages rm: cannot remove `/tmp/145890.1.normal/rsh': No such file or directory Any help or suggestion will be appreciated. Thanks, Praveen
