Hi Miguel, OK, I did'nt know this command.
I'm not sure to understand how it works regarding to my goal. I use the following command inspired by the command you gave me and I obtain a UsageRaw for each QOS. scontrol -o show assoc_mgr -accounts=myaccount Users=" " Do I have to sumup all QOS RawUsage to obtain the RawUsage of myaccount with NoDecay ? If I set GrpTRESMins for an Account and not for a QOS, does SLURM handle to sumpup these QOS RawUsage to control if the GrpTRESMins account limit is reach ? Thanks again for your precious help. Gérard [ http://www.cines.fr/ ] > De: "Miguel Oliveira" <miguel.olive...@uc.pt> > À: "Slurm-users" <slurm-users@lists.schedmd.com> > Envoyé: Mardi 28 Juin 2022 17:23:18 > Objet: Re: [slurm-users] GrpTRESMins and GrpTRESRaw usage > Hi Gérard, > The way you are checking is against the association and as such it ought to be > decreasing in order to be used by fair share appropriately. > The counter used that does not decrease is on the QoS, not the association. > You > can check that with: > scontrol -o show assoc_mgr | grep "^QOS='+account+’ ” > That ought to give you two numbers. The first is the limit, or N for not > limit, > and the second in parenthesis the usage. > Hope that helps. > Best, > Miguel Afonso Oliveira >> On 28 Jun 2022, at 08:58, [ mailto:gerard....@cines.fr | gerard....@cines.fr >> ] >> wrote: >> Hi Miguel, >> I modified my test configuration to evaluate the effect of NoDecay. >> I modified all QOS adding NoDecay Flag. >> toto@login1:~/TEST$ sacctmgr show QOS >> Name Priority GraceTime Preempt PreemptExemptTime PreemptMode Flags >> UsageThres >> UsageFactor GrpTRES GrpTRESMins GrpTRESRunMin GrpJobs GrpSubmit GrpWall >> MaxTRES >> MaxTRESPerNode MaxTRESMins MaxWall MaxTRESPU MaxJobsPU MaxSubmitPU MaxTRESPA >> MaxJobsPA MaxSubmitPA MinTRES >> ---------- ---------- ---------- ---------- ------------------- ----------- >> ---------------------------------------- ---------- ----------- ------------- >> ------------- ------------- ------- --------- ----------- ------------- >> -------------- ------------- ----------- ------------- --------- ----------- >> ------------- --------- ----------- ------------- >> normal 0 00:00:00 cluster NoDecay 1.000000 >> interactif 10 00:00:00 cluster NoDecay 1.000000 node=50 node=22 1-00:00:00 >> node=50 >> petit 4 00:00:00 cluster NoDecay 1.000000 node=1500 node=22 1-00:00:00 >> node=300 >> gros 6 00:00:00 cluster NoDecay 1.000000 node=2106 node=700 1-00:00:00 >> node=700 >> court 8 00:00:00 cluster NoDecay 1.000000 node=1100 node=100 02:00:00 >> node=300 >> long 4 00:00:00 cluster NoDecay 1.000000 node=500 node=200 5-00:00:00 >> node=200 >> special 10 00:00:00 cluster NoDecay 1.000000 node=2106 node=2106 5-00:00:00 >> node=2106 >> support 10 00:00:00 cluster NoDecay 1.000000 node=2106 node=700 1-00:00:00 >> node=2106 >> visu 10 00:00:00 cluster NoDecay 1.000000 node=4 node=700 06:00:00 node=4 >> I submitted a bunch of jobs to control the NoDecay efficiency and I noticed >> RawUsage as well as GrpTRESRaw cpu is still decreasing. >> toto@login1:~/TEST$ sshare -A dci -u " " -o account,user,GrpTRESRaw%80, >> GrpTRESMins ,RawUsage >> Account User GrpTRESRaw GrpTRESMins RawUsage >> -------------------- ---------- >> ----------------------------------------------------- >> ------------------------------ ----------- >> dci cpu=6932 >> ,mem=12998963,energy=0,node=216,billing=6932,fs/disk=0,vmem=0,pages=0 >> cpu=17150 >> 415966 >> toto@login1:~/TEST$ sshare -A dci -u " " -o account,user,GrpTRESRaw%80, >> GrpTRESMins , RawUsage >> Account User GrpTRESRaw GrpTRESMins RawUsage >> -------------------- ---------- >> ----------------------------------------------------- >> ------------------------------ ----------- >> dci cpu=6931 >> ,mem=12995835,energy=0,node=216,billing=6931,fs/disk=0,vmem=0,pages=0 >> cpu=17150 >> 415866 >> toto@login1:~/TEST$ sshare -A dci -u " " -o >> account,user,GrpTRESRaw%80,GrpTRESMins,RawUsage >> Account User GrpTRESRaw GrpTRESMins RawUsage >> -------------------- ---------- >> ----------------------------------------------------- >> ------------------------------ ----------- >> dci cpu=6929 >> ,mem=12992708,energy=0,node=216,billing=6929,fs/disk=0,vmem=0,pages=0 >> cpu=17150 >> 415766 >> Something I forgot to do ? >> Best, >> Gérard >> Cordialement, >> Gérard Gil >> Département Calcul Intensif >> Centre Informatique National de l'Enseignement Superieur >> 950, rue de Saint Priest >> 34097 Montpellier CEDEX 5 >> FRANCE >> tel : (334) 67 14 14 14 >> fax : (334) 67 52 37 63 >> web : [ http://www.cines.fr/ | http://www.cines.fr ] >>> De: "Gérard Gil" < [ mailto:gerard....@cines.fr | gerard....@cines.fr ] > >>> À: "Slurm-users" < [ mailto:slurm-users@lists.schedmd.com | >>> slurm-users@lists.schedmd.com ] > >>> Cc: "slurm-users" < [ mailto:slurm-us...@schedmd.com | >>> slurm-us...@schedmd.com ] >>> > >>> Envoyé: Vendredi 24 Juin 2022 14:52:12 >>> Objet: Re: [slurm-users] GrpTRESMins and GrpTRESRaw usage >>> Hi Miguel, >>> Good !! >>> I'll try this options on all existing QOS and see if everything works as >>> expected. >>> I'll inform you on the results. >>> Thanks a lot >>> Best, >>> Gérard >>> ----- Mail original ----- >>>> De: "Miguel Oliveira" < [ mailto:miguel.olive...@uc.pt | >>>> miguel.olive...@uc.pt ] >>>> > >>>> À: "Slurm-users" < [ mailto:slurm-users@lists.schedmd.com | >>>> slurm-users@lists.schedmd.com ] > >>>> Cc: "slurm-users" < [ mailto:slurm-us...@schedmd.com | >>>> slurm-us...@schedmd.com ] >>>> > >>>> Envoyé: Vendredi 24 Juin 2022 14:07:16 >>>> Objet: Re: [slurm-users] GrpTRESMins and GrpTRESRaw usage >>>> Hi Gérard, >>>> I believe so. All our accounts correspond to one project and all have an >>>> associated QoS with NoDecay and DenyOnLimit. This is enough to restrict >>>> usage >>>> on each individual project. >>>> You only need these flags on the QoS. The association will carry on as >>>> usual and >>>> fairshare will not be impacted. >>>> Hope that helps, >>>> Miguel Oliveira >>>>> On 24 Jun 2022, at 12:56, [ mailto:gerard....@cines.fr | >>>>> gerard....@cines.fr ] >>>>> wrote: >>>>> Hi Miguel, >>>>>> Why not? You can have multiple QoSs and you have other techniques to >>>>>> change >>>>>> priorities according to your policies. >>>>> Is this answer my question ? >>>>> "If all configured QOS use NoDecay, we can take advantage of the FairShare >>>>> priority with Decay and all jobs GrpTRESRaw with NoDecay ?" >>>>> Thanks >>>>> Best, >>> > > Gérard
smime.p7s
Description: S/MIME Cryptographic Signature
