Re: [slurm-users] GPU utilization of running jobs

2022-10-19 Thread Relu Patrascu
Look up CUDA_​DEVICE_​ORDER and PCI_BUS_ID Sent from my iPhone > On Oct 19, 2022, at 04:33, Vecerka Daniel wrote: >

Re: [slurm-users] slurm-users Digest, Vol 60, Issue 20

2022-10-19 Thread Hemanta Sahu
I set the "billing" parameter instead of "cpu" parameter for "GrpTRESMins" and it is working as expected. >> [testfac3@login04 ~]$ sacctmgr modify user name=testuser100 Account=testfac3_imf set GrpTRESMins=billing=0 [testuser100@login04 testuser100]$ scontrol show assoc users=testuser100 acc

Re: [slurm-users] slurm-users Digest, Vol 60, Issue 20

2022-10-19 Thread Hemanta Sahu
Thanks Ole for pointing to necessary documents. So Do you mean to say I need to configure the "billing" parameter instead of "GrpTRESMins" . >> [testuser100@login04 3072]$ scontrol show assoc users=testuser100 accounts=testfac3_imf flags=assoc|grep GrpTRESMins GrpTRESMins=cpu=0(88),mem=N(380538)

[slurm-users] GPU utilization of running jobs

2022-10-19 Thread Vecerka Daniel
Hi,  we want to push our users to run jobs with high GPU utilization. Because it's difficult for users to get GPU utilization of their jobs, I have decided to write script, which prints utilization of running jobs. The idea is simple:  1. get list of running jobs in GPU partitions  2. get ID