[slurm-dev] Re: Tasks distribution

2017-10-09 Thread Sysadmin CAOS
Hello, Between lines.. El 06/10/2017 a las 16:22, Jeffrey T Frey escribió: OK, I have added "CR_Pack_Nodes"... This will produce the 4/4/4/4/1 allocation pattern. AFAIK there's no way to alter which CPU allocation pattern gets used on a per-job basis. Nop, result is not 4/4/4/4/1... submit

[slurm-dev] Re: Tasks distribution

2017-10-09 Thread Chris Samuel
On Monday, 9 October 2017 7:11:06 PM AEDT Sysadmin CAOS wrote: > I have compiled OpenMPI 1.8.1 with --with-pmi=/usr/lib64 (where is > located libpmix.so file) Do you mean --with-pmix=${PATH_TO_PMIX} instead? -- Christopher SamuelSenior Systems Administrator Melbourne Bioinformatics -

[slurm-dev] Re: Tasks distribution

2017-10-09 Thread Sysadmin CAOS
In my CentOS-7, pmix support is installed by a RPM called "pmix" and "pmix-devel". First one contains libraries (/usr/lib64) and second one contains all include files (*.h --> pmi.h, pmi2.h, pmix.h, etc) El 09/10/2017 a las 11:11, Chris Samuel escribió: On Monday, 9 October 2017 7:11:06 PM A

[slurm-dev] Re: Tasks distribution

2017-10-09 Thread Chris Samuel
On Monday, 9 October 2017 8:11:29 PM AEDT Chris Samuel wrote: > Do you mean --with-pmix=${PATH_TO_PMIX} instead? Sorry, I thought you were configuring Slurm with PMIx support there! -- Christopher SamuelSenior Systems Administrator Melbourne Bioinformatics - The University of Melbour

[slurm-dev] Re: Tasks distribution

2017-10-09 Thread Sysadmin CAOS
Mmmm, yes... CentOS only offers PMIX packages and I don't know where I can find PMI{1,2} packages... How should I compile SLURM? El 09/10/2017 a las 11:18, Chris Samuel escribió: On Monday, 9 October 2017 8:11:29 PM AEDT Chris Samuel wrote: Do you mean --with-pmix=${PATH_TO_PMIX} instead?

[slurm-dev] Re: Tasks distribution

2017-10-09 Thread Chris Samuel
On Monday, 9 October 2017 8:46:21 PM AEDT Sysadmin CAOS wrote: > Mmmm, yes... CentOS only offers PMIX packages and I don't know where I > can find PMI{1,2} packages... How should I compile SLURM? To compile Slurm to support PMIx you need to have this in your configure: --with-pmix=${PATH_TO_PMI

[slurm-dev] GPU accounting (WAS Re: sreport not reporting gpu info...)

2017-10-09 Thread Merlin Hartley
That’s what I’ve been looking for too! Though now I see that my configuration must be wrong - I am trying to make use of a GPU cost the same as 160 CPUs - so I have this config: PartitionName=DEFAULT DefaultTime=24:0:0 MaxTime=14-0:0:0 MaxNodes=4 TRESBillingWeights="CPU=1.0,Mem=0.25G,GRES/gpu

[slurm-dev] peculiar resources configuration in SLURM

2017-10-09 Thread cyberseawolf .
Hello everybody, I'm a young system administrator that is moving from Torque/MAUI to Slurm. I set up a pretty peculiar resource management in the previous queue system and I would like to port it in the new one. - I have the following two partitions that are totally independent to each others (lik

[slurm-dev] Re: Tasks distribution

2017-10-09 Thread Sysadmin CAOS
After a lot of changes, I have recompiled all. I have executed these steps: 1. First of all, I have compiled contrib "pmi" package allocate inside SLURM 17.02.7 folder: cd contribs/pmi2 && make && make install --> pmi2 libs and includes created have been copied to my

[slurm-dev] Preemtion and signals

2017-10-09 Thread tegner
Have been trying to get preemtion to work for some time now. Goal is to have a specific partition for "cheap" jobs. It is allowed to use this freely, but as soon as a higher priority job enters the queue, these "cheap" jobs should be canceled. However, we would like to use the gracetime to kill

[slurm-dev] Re: Preemtion and signals

2017-10-09 Thread Bjørn-Helge Mevik
writes: > trap 'sig_cont' SIGCONT > trap 'sig_term' SIGTERM > trap 'sig_kill' SIGKILL > sleep 400 Note that this test might be affected by the fact that bash will _not_ abort the "sleep 400"; it will wait until it has finished before executing the trap function(!) From man bash: If bas

[slurm-dev] Camacho Barranco, Roberto ssirimu...@utep.edu

2017-10-09 Thread Suman Sirimulla
Hi, We have installed and configured slurm on our cluster, but unable to start the slurmctld daemon. We followed the instructions ( https://slurm.schedmd.com/troubleshoot.html) and tried to stop and restart it multiple times but still not working. Please see the error below. [111] ssirimulla@

[slurm-dev] Re: peculiar resources configuration in SLURM

2017-10-09 Thread Lachlan Musicman
On 9 October 2017 at 22:06, cyberseawolf . wrote: > Hello everybody, > I'm a young system administrator that is moving from Torque/MAUI to Slurm. > I set up a pretty peculiar resource management in the previous queue system > and I would like to port it in the new one. > > - I have the following

[slurm-dev] Re: Camacho Barranco, Roberto ssirimu...@utep.edu

2017-10-09 Thread Christopher Samuel
On 10/10/17 07:21, Suman Sirimulla wrote: > We have installed and configured slurm on our cluster, but unable to > start the slurmctld daemon. We followed the instructions > (https://slurm.schedmd.com/troubleshoot.html) >  and tried to stop and > re

[slurm-dev] Re: Tasks distribution

2017-10-09 Thread Christopher Samuel
On 09/10/17 22:11, Sysadmin CAOS wrote: > Now, after that, should srun distribute correctly my tasks as mpirun > does right? No, srun will distribute the tasks as how Slurm wants to, remember it's the MPI implementations job to listen to what the resource manager tells it to do, not the other wa

[slurm-dev] Re: Tasks distribution

2017-10-09 Thread r...@open-mpi.org
Just to clarify something here: OMPI 1.8 does not support PMIx. You need at least OMPI 2.0 for that. > On Oct 9, 2017, at 4:11 AM, Sysadmin CAOS wrote: > > After a lot of changes, I have recompiled all. I have executed these steps: > First of all, I have compiled contrib "pmi" package allocate

[slurm-dev] Re: Slurm 17.02.7 and PMIx

2017-10-09 Thread Christopher Samuel
On 05/10/17 11:27, Christopher Samuel wrote: > PMIX v1.2.2: Slurm complains and tells me it wants v2. I think that was due to a config issue on the system I was helping out with, after having to install some extra packages (like a C++ compiler) to get other things working I can no longer reprodu

[slurm-dev] Re: Slurm 17.02.7 and PMIx

2017-10-09 Thread r...@open-mpi.org
> On Oct 9, 2017, at 5:32 PM, Christopher Samuel wrote: > > > On 05/10/17 11:27, Christopher Samuel wrote: > >> PMIX v1.2.2: Slurm complains and tells me it wants v2. > > I think that was due to a config issue on the system I was helping out > with, after having to install some extra package

[slurm-dev] Re: Camacho Barranco, Roberto ssirimu...@utep.edu

2017-10-09 Thread Marcus Wagner
For a quick view, manually starting the controller slurmctld -D -vvv might also help. Best Marcus On 10/10/2017 01:41 AM, Christopher Samuel wrote: On 10/10/17 07:21, Suman Sirimulla wrote: We have installed and configured slurm on our cluster, but unable to start the slurmctld daemon. We

[slurm-dev] Re: peculiar resources configuration in SLURM

2017-10-09 Thread cyberseawolf .
Dear Lachlan, thank you very much for your suggestions. I will try to experiment a little bit with those settings. Have you a nice day, Emanuele 2017-10-10 0:02 GMT+02:00 Lachlan Musicman : > On 9 October 2017 at 22:06, cyberseawolf . wrote: > >> Hello everybody, >> I'm a young system administr