That is interesting. It is disabled in 17.11.10: static bool _enable_pack_steps(void) { bool enabled = false; char *sched_params = slurm_get_sched_params();
if (sched_params && strstr(sched_params, "disable_hetero_steps")) enabled = false; else if (sched_params && strstr(sched_params, "enable_hetero_steps")) enabled = true; else if (mpi_type && strstr(mpi_type, "none")) enabled = true; xfree(sched_params); return enabled; } I wonder if it is ill advised to enable it!? Suppose I could try it. Thanks Chris! Best, Chris — Christopher Coffey High-Performance Computing Northern Arizona University 928-523-1167 On 10/10/18, 12:11 AM, "slurm-users on behalf of Chris Samuel" <slurm-users-boun...@lists.schedmd.com on behalf of ch...@csamuel.org> wrote: On 10/10/18 05:07, Christopher Benjamin Coffey wrote: > Yet, we get an error: " srun: fatal: Job steps that span multiple > components of a heterogeneous job are not currently supported". But > the docs seem to indicate it should work? Which version of Slurm are you on? It was disabled by default in 17.11.x (and I'm not even sure it works if you enable it there) and seems to be enabled by default in 18.08.x. To see check the _enable_pack_steps() function src/srun/srun.c All the best, Chris (currently away in the UK) -- Chris Samuel : https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.csamuel.org%2F&data=02%7C01%7Cchris.coffey%40nau.edu%7Cd8554994428d40e9902c08d62e7f8b5c%7C27d49e9f89e14aa099a3d35b57b2ba03%7C0%7C0%7C636747522680686112&sdata=DwgI40F74yX90rTHa4U4MtR2nPuSYqqlL5XV9XqSOXk%3D&reserved=0 : Melbourne, VIC