[slurm-dev] Re: Tasks distribution

2017-10-06 Thread sysadmin.caos
Hello, Writing between lines... El 04/10/2017 a las 18:52, Jeffrey Frey escribió: I didn't realize prior to this that the "--distribution" flag to "sbatch" only affects how an "srun" within the batch script will make CPU allocations. Prior to that happening, SLURM must allocate

[slurm-dev] Re: Disable accounting/qos policy enforcement for single job

2017-10-06 Thread Jacob Chappell
Lachlan, Thanks! __ *Jacob D. Chappell* *Research Computing Associate* Research Computing | Research Computing Infrastructure Information Technology Services | University of Kentucky 301 Rose Street | 102 James F. Hardymon Building Lexington, KY 405

[slurm-dev] srun vs mpirun

2017-10-06 Thread tegner
New to slurm, so this might be a stupid question. Using slurm-17.02.7-1.el7 on a small centos-7.3 based cluster. On it we are running an in house mpi-application (OpenMPI-1.10.7, built against intel compilers), and it is all running nicely, no problem submitting the jobs. There are two partitio

[slurm-dev] sreport not reporting gpu info, but sacct does

2017-10-06 Thread Tim Carlson
Background: Recently installed new cluster which I started with 14.03 but then upgraded to 17.02 to get better/more gres/tres information. In my other clusters I use sreport heavily to do billing and this new cluster is GPU based and I want to bill off of GPU time consumed. My assumption was I co

[slurm-dev] Re: srun vs mpirun

2017-10-06 Thread r...@open-mpi.org
Not stupid at all. I suspect the problem is that OMPI was not configured --with-pmi=. As a result, when you srun the application, each processes thinks it is a singleton and nothing works correctly. OMPI does not pickup the slurm pmi support by default due to license issues, so you have to ma

[slurm-dev] Re: sreport not reporting gpu info, but sacct does

2017-10-06 Thread Daniel Barker
Tim, I believe you have to refer to the gpu as gres/gpu. [root@slurm-login ~]# sreport -T CPU,mem,gres/gpu cluster AccountUtilizationByUser Start=2017-01-01 End=2017-12-31 Cluster/Account/User Utilization 2017-01-01

[slurm-dev] Re: sreport not reporting gpu info, but sacct does

2017-10-06 Thread Tim Carlson
Perfect! Thanks! # sreport cluster AccountUtilizationByUser -T cpu,gres/gpu Start=2017-10-06 End=2017-10-08 Cluster/Account/User Utilization 2017-10-06T00:00:00 - 2017-10-06T11:59:59 (43200 secs) Use reported in TRES