Re: [slurm-users] [Slurm 18.08.4] sacct/seff Inaccurate usercpu values

2019-01-15 Thread Henkel, Andreas
Bad news Dir the cgroup-Users, seems like the bug is „resolved“ by the site switching to task/Linux instead :-( > Am 09.01.2019 um 22:06 schrieb Christopher Benjamin Coffey > : > > Thanks... looks like the bug should get some attention now that a paying site > is complaining: > > https://bugs

Re: [slurm-users] Accounting configuration

2019-01-15 Thread Thomas M. Payerle
Generally, the add, modify, etc sacctmgr commands want an "user" or "account" entity, but can modify associations though this. E.g., if user baduser should have GrpTRESmin of cpu=1000 set on partition special, use something like sacctmgr add user name=baduser partition=special account=testacct grp

[slurm-users] Accounting configuration

2019-01-15 Thread Skouson, Gary
Slurm accounting info is stored based on user, cluster, partition and account. I'd like to be able to enforce limits for an account based on the partition it's running in. Sadly, I'm not seeing how to use sacctmgr to change the partition as part of the association. The add, modify and delet

[slurm-users] Report on gres usage

2019-01-15 Thread Matthew BETTINGER
Hello, We are trying to find a way to gather information about jobs assigned to GPU's. I'm not really finding anything we want using sreport for some reason. We would like to find cpu hours for our GPUSs. They are defined in gres and need to be passed to srun when users request gpu's. It lo

Re: [slurm-users] Re : Two jobs ends up on one GPU?

2019-01-15 Thread Magnus Jonsson
CUDA_VISIBLE_DEVICES=0 on both jobs. This is also true for the any other jobs that work just fine. There are only one device visible in the job. /Magnus On 2019-01-15 13:28, arnaud.ren...@univ-reims.fr wrote: What is the value of env. variable CUDA_VISIBLE_DEVICES ? Envoyé depuis mon mobil

[slurm-users] Re : Two jobs ends up on one GPU?

2019-01-15 Thread arnaud.ren...@univ-reims.fr
What is the value of env. variableCUDA_VISIBLE_DEVICES ? Envoyé depuis mon mobile Huawei Message original Objet : [slurm-users] Two jobs ends up on one GPU?De : Magnus Jonsson À : slurm-users@lists.schedmd.comCc : Hi!We have machines with multiple GPUs (Nvidia V100).We allow multipl

[slurm-users] Two jobs ends up on one GPU?

2019-01-15 Thread Magnus Jonsson
Hi! We have machines with multiple GPUs (Nvidia V100). We allow multiple (two) jobs on the nodes. We have a user that in some way have managed to get both jobs to end up on the same GPU (verified via nvidia-smi). We are using cgroups and the nvidia-smi command only shows one of the GPUs (if

Re: [slurm-users] Checking memory requirements in job_submit.lua

2019-01-15 Thread Bjørn-Helge Mevik
Prentice Bisbal writes: > Sorry for the delayed reply. I didn't see this earlier (6 months > ago!). I'm just seeing it now as I clean up my inbox. That happens for me as well from time to time. :) > 1. Yes, slurm.user_msg does actually print out a message to the user > in this case. > > 2. I wa