Hi Gary, We wrote some wrapper scripts to pull the limits from the db and the usage from the *_usage files, and give you a ``balance''. For example:
[user@headnode01 scripts]$ sbank balance statement -u myuser01 User Usage | Account Usage | Account Limit Available (CPU hrs) ---------- --------- + -------------- --------- + ------------- --------- myuser02 20 | HPC_11_00204 441 | 400,000 399,559 myuser01 * 0 | HPC_11_00204 441 | 400,000 399,559 myuser03 421 | HPC_11_00204 441 | 400,000 399,559 myuser01 * 713 | HPC_13_00778 813 | 300,000 299,187 myuser04 100 | HPC_13_00778 813 | 300,000 299,187 It's available here: https://github.com/paddydoyle/slurm-bank (the docs need a bit of a tidy-up) We use 'AccountingStorageEnforce=safe', and have 'PriorityDecayHalfLife=14-0' and 'PriorityUsageResetPeriod=NONE'; so we can set limits but the usage decays over time and allows people to run again when they have enough balance. Thanks, Paddy On Wed, Feb 08, 2017 at 04:14:50PM -0800, Skouson, Gary B wrote: > > Using sacctmgr you can set limits like GrpCPUMins and other GrpTRESMins. > It's pretty easy to see what the limit is, but I'm not sure how to see how > close to the limit someone is. Is there a normal slurm command that can get > you the internal GrpTRESMins info slurmctld is using to enforce limits? > > Looking at the code, the share request from sshare gets sent all the data, > but there's no option to print it out. > > I made a simple patch below to add a GrpTRESRaw option to sshare to print > this info. It has to convert the usage from double to int, which isn't > ideal, but it was simple and is close enough for what I was looking for. > > Is there a better way to get this information? > > ----- > Gary Skouson > > diff -Naru slurm-16.05.9/src/sshare/process.c > slurm-16.05.9.change/src/sshare/process.c > --- slurm-16.05.9/src/sshare/process.c 2017-01-31 11:55:41.000000000 -0800 > +++ slurm-16.05.9.change/src/sshare/process.c 2017-02-08 15:45:08.019494347 > -0800 > @@ -63,6 +63,7 @@ > {10, "User", print_fields_str, PRINT_USER}, > {30, "GrpTRESMins", _print_tres, PRINT_TRESMINS}, > {30, "TRESRunMins", _print_tres, PRINT_RUNMINS}, > + {30, "GrpTRESRaw", _print_tres, PRINT_GRPTRESRAW}, > {0, NULL, NULL, 0} > }; > > @@ -226,6 +227,7 @@ > char *tmp_char = NULL; > char *local_acct = NULL; > print_field_t *field = NULL; > + uint64_t tres_raw[tres_cnt]; > > if ((options & PRINT_USERS_ONLY) && share->user == 0) > continue; > @@ -342,6 +344,14 @@ > share->tres_grp_mins, > (curr_inx == > field_count)); > break; > + case PRINT_GRPTRESRAW: > + /* convert to ints and minutes */ > + for (i=0; i<tres_cnt; i++) > + tres_raw[i] = (uint64_t) > share->usage_tres_raw[i]/60; > + field->print_routine(field, > + tres_raw, > + (curr_inx == > field_count)); > + break; > case PRINT_RUNMINS: > /* convert to minutes */ > for (i=0; i<tres_cnt; i++) > diff -Naru slurm-16.05.9/src/sshare/sshare.h > slurm-16.05.9.change/src/sshare/sshare.h > --- slurm-16.05.9/src/sshare/sshare.h 2017-01-31 11:55:41.000000000 -0800 > +++ slurm-16.05.9.change/src/sshare/sshare.h 2017-02-08 15:45:08.023488846 > -0800 > @@ -115,7 +115,8 @@ > PRINT_RAWU, > PRINT_RUNMINS, > PRINT_USER, > - PRINT_LEVELFS > + PRINT_LEVELFS, > + PRINT_GRPTRESRAW > }; > > extern int exit_code; /* sshare's exit code, =1 on any error at any time */ -- Paddy Doyle Trinity Centre for High Performance Computing, Lloyd Building, Trinity College Dublin, Dublin 2, Ireland. Phone: +353-1-896-3725 http://www.tchpc.tcd.ie/
