Hi Loris, Thank you for clarification Will figure out if we can apply this approach
Thanks Alexey -----Original Message----- From: slurm-users <slurm-users-boun...@lists.schedmd.com> On Behalf Of Loris Bennett Sent: Tuesday, September 29, 2020 9:37 AM To: Slurm User Community List <slurm-users@lists.schedmd.com> Subject: RE: [EXTERNAL] [slurm-users] Working with local licenses CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi Alexey, "Tager, Alexey" <alex...@amazon.com> writes: > Hi Loris, > > I just need better understanding details of your solution Are you > talking about dynamic licenses, right? I am talking about a limited pool of licences for which one licence is consumed for each user-node pair. > Should I add license ttt to database by sacctmgr with "Count" and "% > Allocated", and also create unlimited reservation of same license ttt and > update it with "free licenses"? We didn't add anything to the database. We added the total number of licences to the slurm.conf using the 'Licenses' option. We then just calculated the number of free licences by parsing the output of the licence manager and then updated the reservation with 'scontrol' with a cron job. > How I define resource as "unlimited" ? This approach does not use a 'resource' in the sense of sacctmgr. Instead it relies on licences within a reservation created with duration=infinite > Will sacctmgr see "Count" updated and scontrol see "Total" and "Used" > accordingly ? Probably not, since this approach, as I mention above, does not use a sacctmrg resource. Cheers, Loris > Thank you > Alexey > > > > -----Original Message----- > From: slurm-users <slurm-users-boun...@lists.schedmd.com> On Behalf Of > Loris Bennett > Sent: Monday, September 28, 2020 10:23 AM > To: Slurm User Community List <slurm-users@lists.schedmd.com> > Subject: RE: [EXTERNAL] [slurm-users] Working with local licenses > > CAUTION: This email originated from outside of the organization. Do not click > links or open attachments unless you can confirm the sender and know the > content is safe. > > > > "Tager, Alexey" <alex...@amazon.com> writes: > >> Hi Ahmet, >> >> I agree, it isn't so good solution > > We used to use a solution which also wasn't very good, but perhaps > slightly >> better than restarting slurmctld every minute. We created an >> unlimited reservation and then updated the reservation via a cron job >> with the current number of free licences. Obviously the users then >> had to specify the reservation as well as the licence. > > Cheers, > > Loris > >> Thanks >> Alexey >> >> -----Original Message----- >> From: mercan <ahmet.mer...@uhem.itu.edu.tr> >> Sent: Friday, September 25, 2020 11:50 AM >> To: Tager, Alexey <alex...@amazon.com>; Slurm User Community List >> <slurm-users@lists.schedmd.com> >> Subject: RE: [EXTERNAL] [slurm-users] Working with local licenses >> >> CAUTION: This email originated from outside of the organization. Do not >> click links or open attachments unless you can confirm the sender and know >> the content is safe. >> >> >> >> Hi; >> >> But, restarting in every minute, is not a good solution. >> >> Ahmet M. >> >> >> On 25.09.2020 11:25, Tager, Alexey wrote: >>> Hi Ahmet, >>> >>> We tried remote licenses, but encountered following issues, which lead us >>> to using of local licenses. >>> - only low case while inserting by sacctmgr >>> - dead locks and duplicate records >>> - direct insert is working and case sensitive, but scontrol doesn't >>> see change until slurmctld restart >>> >>> >>> Regards >>> Alexey >>> >>> -----Original Message----- >>> From: mercan <ahmet.mer...@uhem.itu.edu.tr> >>> Sent: Friday, September 25, 2020 11:16 AM >>> To: Slurm User Community List <slurm-users@lists.schedmd.com>; >>> Tager, Alexey <alex...@amazon.com> >>> Subject: RE: [EXTERNAL] [slurm-users] Working with local licenses >>> >>> CAUTION: This email originated from outside of the organization. Do not >>> click links or open attachments unless you can confirm the sender and know >>> the content is safe. >>> >>> >>> >>> Hi; >>> >>> You don't need to modify slurm.conf and reconfigure. There is a >>> remote/dynamic licenses feature: >>> >>> https://slurm.schedmd.com/licenses.html >>> >>> You can add licenses using scontrol command such as: >>> >>> sacctmgr add resource name=matlab count=50 server=rlm_host \ >>> servertype=rlm type=license >>> >>> Regards; >>> >>> Ahmet M. >>> >>> >>> On 25.09.2020 11:09, Tager, Alexey wrote: >>>> Hello, >>>> >>>> We use local licenses by updating slurm.conf with script, >>>> >>>> which talking with FlexLM >>>> >>>> We execute “scontrol reconfigure” every 1 min after file change in >>>> crontab >>>> >>>> Users get following error when run slurm commands (srun,scontrol >>>> etc.) during >>>> >>>> crontab execution, as result srun failed on license request >>>> >>>> Any option to prevent such behavior ? >>>> >>>> [September 24, 2020, 7:18 PM] Tager, Alexey: scontrol: error: Parse >>>> error in file /tools/slurm/etc/slurm_licenses.conf line 10: "Licenses=" >>>> >>>> scontrol: error: "Include" failed in file >>>> /tools/slurm/etc/slurm.conf line 13 >>>> >>>> scontrol: fatal: Unable to process configuration file >>>> >>>> /Alexey Tager/ >>>> > -- > Dr. Loris Bennett (Mr.) > ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
