[slurm-users] Re: Slurm Job Sched Priority

2024-10-29 Thread Paul Edmon via slurm-users 
If you are looking to use the C API for this then showq may be a good 
guide: https://github.com/fasrc/slurm_showq  The -o option orders the 
pending queue in priority order.


If you are looking at native slurm commands, sprio can print out the 
current priority breakdown of any job and filter by partition, then you 
can reorder based on that.  squeue also can print out current priority. 
You might also look at the --priority option: 
https://slurm.schedmd.com/squeue.html#OPT_priority


-Paul Edmon-

On 10/29/24 9:33 AM, Bhaskar Chakraborty via slurm-users wrote:

Hello,

Is there any DS in slurmctld which portrays the dynamic relative 
priority of pending jobs?


We are trying to use slurm for developing a scheduling solution and 1 
of the problems we face at the outset is how to determine

the order of scheduling for pending jobs.

One option is to find scheduling iteration window begin & close 
pointers & cache the job ids as seen in order & then make them the 
priority order at that point of time.


( This means for 500 pending jobs, say, if we can find which are the 
slurmctld calls which mark the beginning & end of a sched iteration 
then we can use the scheduling order
of jobs as the relative priority order for that period of time, of 
course it may change depending on fairshare, user initiated priority 
modification etc. )


A concrete existing data structure showing the dynamic priority itself 
from slurmctld would be handy.


Help appreciated.

Thanks!

Bhaskar.


-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com

[slurm-users] Slurm Job Sched Priority

2024-10-29 Thread Bhaskar Chakraborty via slurm-users 
Hello,
Is there any DS in slurmctld which portrays the dynamic relative priority of 
pending jobs?
We are trying to use slurm for developing a scheduling solution and 1 of the 
problems we face at the outset is how to determinethe order of scheduling for 
pending jobs.
One option is to find scheduling iteration window begin & close pointers & 
cache the job ids as seen in order & then make them the priority order at that 
point of time.
( This means for 500 pending jobs, say, if we can find which are the slurmctld 
calls which mark the beginning & end of a sched iteration then we can use the 
scheduling orderof jobs as the relative priority order for that period of time, 
of course it may change depending on fairshare, user initiated priority 
modification etc. )
A concrete existing data structure showing the dynamic priority itself from 
slurmctld would be handy.
Help appreciated.
Thanks!
Bhaskar.

-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com

[slurm-users] Re: Job pre / post submit scripts

2024-10-29 Thread Di Bernardini, Fabio via slurm-users 
Have you already seen if SrunProlog or TaskProlog could be useful for your case?
The documentation is here: https://slurm.schedmd.com/prolog_epilog.html

--
Fabio

From: Bhaskar Chakraborty via slurm-users 
Sent: Monday, October 28, 2024 6:56 PM
To: slurm-us...@schedmd.com
Subject: [EXTERNAL] [slurm-users] Job pre / post submit scripts


CAUTION: This email originated from outside of the organization. Do not click 
links or open attachments unless you can confirm the sender and know the 
content is safe.


Hi,

Is there an option in slurm to launch a custom script at the time of job 
submission through sbatch
or salloc? The script should run with submit user permission in submit area.

The idea is that we need to enquire something which characterises our job’s 
requirement like CPU
slots, memory etc from a central server and we do need read access to user area 
prior to that.

In our use case the user doesn’t necessarily know beforehand what kind of 
resource his job needs.
(Hence, the need for such a script which will contact the server with user area 
info.)

Based on it we can modify the job a little later. A post submit script, if 
available, would inform us the slurm job id as well, it would get called just 
after the job has entered the system and prior to its scheduling.

Thanks,
Bhaskar.
Sent from Yahoo Mail for 
iPad



NICE SRL, viale Monte Grappa 3/5, 20124 Milano, Italia, Registro delle Imprese 
di Milano Monza Brianza Lodi REA n. 2096882, Capitale Sociale: 10.329,14 EUR 
i.v., Cod. Fisc. e P.IVA 01133050052, Societa con Socio Unico



-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com

[slurm-users] Re: Job pre / post submit scripts

2024-10-29 Thread Bhaskar Chakraborty via slurm-users 
 Thanks for the help.
Shall try to check the client plugin option 1st else have to revert to the 
other alternative which, though more of a workaround, also looks good & 
promising.
-Bhaskar.
On Tuesday 29 October, 2024 at 06:49:12 am IST, Christopher Samuel via 
slurm-users  wrote:  
 
 On 10/28/24 10:56 am, Bhaskar Chakraborty via slurm-users wrote:

> Is there an option in slurm to launch a custom script at the time of job 
> submission through sbatch
> or salloc? The script should run with submit user permission in submit area.

I think you are after the cli_filter functionality which can run plugins 
in that environment. There is a Lua plugin for that which will allow you 
to write your code in something a little less fraught than C.

https://slurm.schedmd.com/cli_filter_plugins.html

There is example Lua code for this here:

https://github.com/SchedMD/slurm/blob/master/etc/cli_filter.lua.example

All the best,
Chris
-- 
Chris Samuel  :  http://www.csamuel.org/  :  Berkeley, CA, USA

-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
  
-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com

[slurm-users] Re: 转发: What is the safe upgrade path when upgrade from slurm21.08 and mariadb5.5?

2024-10-29 Thread Andrej Sec via slurm-users 


Hi, 

we are facing a similar task. We have a Slurm 22.05 / MariaDB 5.5.68 
environment and want to upgrade to a newer version. According to the 
documentation, it’s recommended to upgrade from 22.05 to a maximum of 23.11 in 
one step. With the MariaDB upgrade, there’s a challenge between 10.1 and 10.2+ 
due to incompatible changes ( [ 
https://mariadb.com/kb/en/changes-improvements-in-mariadb-10-2 | 
https://mariadb.com/kb/en/changes-improvements-in-mariadb-10-2 ] ). This 
upgrade, as I understand from the documentation, requires at least slurm 22.05, 
where it is automatically handled by the slurmdbd service. 

In the test lab, we performed the following tests: 

a. Incremental upgrade - according to MariaDB recommendations: 

1. Upgrade MariaDB 5.5.68 -> 10.1.48 -> 10.2.44 
2. Start the Slurm suite 22.05, checking content after each MariaDB upgrade 
step. During the 10.1 -> 10.2 upgrade, the slurmdbd service automatically 
converted the database to the required format. We had enabled general.log in 
MariaDB, allowing detailed inspection of database changes during conversion. 
3. Upgrade slurmdbd to version 23.11 
4. Upgrade slurmctld to version 23.11 
5. Upgrade slurmd to version 23.11 
6. Check the database content and compare tests before and after the 
upgrade (we used various reports with scontrol, sreport, sacct, sacctmgr for 
verification). 



b. Direct MariaDB upgrade from 5.5.68 to 10.2.44 using the same approach. 
According to the tests, this resulted in the same state as the incremental 
approach. 




PS: If you proceed with the upgrade, I would appreciate it if you could let us 
know about any potential challenges you encountered. 




Andrej Sec 
nscc, Bratislava, Slovakia 


Od: "hermes via slurm-users"  
Komu: slurm-users@lists.schedmd.com 
Odoslané: pondelok, 28. október 2024 8:48:19 
Predmet: [slurm-users] 
=?eucgb2312_cn?q?=D7=AA=B7=A2=3A_What_is_the_safe_upgrade_path_when_upgrade_from_slurm21=2E08_and_mariadb5=2E5=3F?=
 



Hi everyone: 



We are currently running business on SLURM21.08 and mariadb5.5. 

When talking about the upgrade, we need to keep all the users and jobs history 
data. And we see the official document wrote: 

“When upgrading an existing accounting database to MariaDB 10.2.1 or later from 
an older version of MariaDB or any version of MySQL, ensure you are running 
slurmdbd 22.05.7 or later. These versions will gracefully handle changes to 
MariaDB default values that can cause problems for slurmdbd.” 



So is this mean we have to firstly build SLURM>22.05 over mariadb5.5, and do 
the SLURM upgrade. Then upgrade the mariadb to newer version, and rebuild the 
same version of SLURM over new mariadb-devel? 

And is it safe to jump directly from mariadb5.5 to latest version? How can we 
check whether the slurm have correctly inherited the historical data? 



Thanks, 

Tianyang Zhang 

SJTU Network Information Center 


-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com 
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com 

-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com

[slurm-users] Re: Scheduling oddity with multiple GPU types in same partition

2024-10-29 Thread Markus Köberl via slurm-users 
On Friday, 25 October 2024 22:49:16 CET Kevin M. Hildebrand via slurm-users 
wrote:
> We have a 'gpu' partition with 30 or so nodes, some with A100s, some with
> H100s, and a few others.
> It appears that when (for example) all of the A100 GPUs are in use, if
> there are additional jobs requesting A100 GPUs pending, and those jobs have
> the highest priority in the partition, then jobs submitted for H100s won't
> run even if there are idle H100s.  This is a small subset of our present
> pending queue- the four bottom jobs should be running, but aren't.  The top
> pending job shows reason 'Resources' while the rest all show 'Priority'.
> Any thoughts on why this might be happening?
> 
> JOBID   PRIORITYTRES_ALLOC
> 
> 8317749 501490
>  cpu=4,mem=8M,node=1,billing=48,gres/gpu=1,gres/gpu:a100=1
> 
> 8317750 501490
>  cpu=4,mem=8M,node=1,billing=48,gres/gpu=1,gres/gpu:a100=1
> 
> 8317745 501490
>  cpu=4,mem=8M,node=1,billing=48,gres/gpu=1,gres/gpu:a100=1
> 
> 8317746 501490
>  cpu=4,mem=8M,node=1,billing=48,gres/gpu=1,gres/gpu:a100=1
> 
> 8338679 500060
>  cpu=4,mem=64G,node=1,billing=144,gres/gpu=1,gres/gpu:h100=1
> 
> 8338678 500060
>  cpu=4,mem=64G,node=1,billing=144,gres/gpu=1,gres/gpu:h100=1
> 
> 8338677 500060
>  cpu=4,mem=64G,node=1,billing=144,gres/gpu=1,gres/gpu:h100=1
> 
> 8338676 500060
>  cpu=4,mem=64G,node=1,billing=144,gres/gpu=1,gres/gpu:h100=1


Do you have Backfill Scheduling configured with bf_continue?


regards
Markus Köberl
-- 
Markus Koeberl
Graz University of Technology
Signal Processing and Speech Communication Laboratory
E-mail: markus.koeb...@tugraz.at

signature.asc
Description: This is a digitally signed message part.

-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com

[slurm-users] Re: 转发: What is the safe upgrade path when upgrade from slurm21.08 and mariadb5.5?

2024-10-29 Thread hermes via slurm-users 
Thanks for all your help. So it seems we can skip the trouble of compiling 
SLURM over different mariadb versions.

 

Tianyang Zhang

SJTU Network Information Center

 

发件人: Sid Young  
发送时间: 2024年10月30日 7:19
收件人: Andrej Sec 
抄送: taleinterve...@sjtu.edu.cn; slurm-users@lists.schedmd.com
主题: Re: [slurm-users] Re: 转发: What is the safe upgrade path when upgrade from 
slurm21.08 and mariadb5.5?

 

 

I recently upgraded from 20.11 to 24.05.2, before moving the cluster from 
CentOS 7.9 to Oracle Linux 8.10

 

The DB upgrade should be pretty simple, do a mysqldump first, then uninstall 
the old DB, change the repo's and install the new DB version. It should 
recognise the DB files on disk and access them. Do another DB backup on the new 
DB version. then roll through the Slurm upgrades.

 

I picked the first and last version of each release, and systematically went 
through each node till it was done. First the slurm controller node, then the 
compute nodes. To avoid Job loss, drain the nodes or you end up with a 
situation where the slurmd can't talk to the running slurmstepd and the job(s) 
gets lost. (Shows as a "Protocol Error").

 

Ole sent me a link to this guide which mostly worked.

https://wiki.fysik.dtu.dk/Niflheim_system/Slurm_installation/#upgrade-slurmd-on-nodes

 

 

 

Sid Young

W: https://off-grid-engineering.com

 

 

 

On Tue, Oct 29, 2024 at 6:33 PM Andrej Sec via slurm-users 
mailto:slurm-users@lists.schedmd.com> > wrote:

Hi,

we are facing a similar task. We have a Slurm 22.05 / MariaDB 5.5.68 
environment and want to upgrade to a newer version. According to the 
documentation, it’s recommended to upgrade from 22.05 to a maximum of 23.11 in 
one step. With the MariaDB upgrade, there’s a challenge between 10.1 and 10.2+ 
due to incompatible changes 
(https://mariadb.com/kb/en/changes-improvements-in-mariadb-10-2). This upgrade, 
as I understand from the documentation, requires at least slurm 22.05, where it 
is automatically handled by the slurmdbd service.

In the test lab, we performed the following tests:

a. Incremental upgrade - according to MariaDB recommendations:

1.  Upgrade MariaDB 5.5.68 -> 10.1.48 -> 10.2.44
2.  Start the Slurm suite 22.05, checking content after each MariaDB 
upgrade step. During the 10.1 -> 10.2 upgrade, the slurmdbd service 
automatically converted the database to the required format. We had enabled 
general.log in MariaDB, allowing detailed inspection of database changes during 
conversion.
3.  Upgrade slurmdbd to version 23.11
4.  Upgrade slurmctld to version 23.11
5.  Upgrade slurmd to version 23.11
6.  Check the database content and compare tests before and after the 
upgrade (we used various reports with scontrol, sreport, sacct, sacctmgr for 
verification).

b. Direct MariaDB upgrade from 5.5.68 to 10.2.44 using the same approach. 
According to the tests, this resulted in the same state as the incremental 
approach.

 

PS: If you proceed with the upgrade, I would appreciate it if you could let us 
know about any potential challenges you encountered. 

 

Andrej Sec
nscc, Bratislava, Slovakia

 


  _  


Od: "hermes via slurm-users" mailto:slurm-users@lists.schedmd.com> >
Komu: slurm-users@lists.schedmd.com  
Odoslané: pondelok, 28. október 2024 8:48:19
Predmet: [slurm-users] 
=?eucgb2312_cn?q?=D7=AA=B7=A2=3A_What_is_the_safe_upgrade_path_when_upgrade_from_slurm21=2E08_and_mariadb5=2E5=3F?=

 

Hi everyone:

 

We are currently running business on SLURM21.08 and mariadb5.5. 

When talking about the upgrade, we need to keep all the users and jobs history 
data. And we see the official document wrote:

“When upgrading an existing accounting database to MariaDB 10.2.1 or later from 
an older version of MariaDB or any version of MySQL, ensure you are running 
slurmdbd 22.05.7 or later. These versions will gracefully handle changes to 
MariaDB default values that can cause problems for slurmdbd.”

 

So is this mean we have to firstly build SLURM>22.05 over mariadb5.5, and do 
the SLURM upgrade. Then upgrade the mariadb to newer version, and rebuild the 
same version of SLURM over new mariadb-devel?

And is it safe to jump directly from mariadb5.5 to latest version? How can we 
check whether the slurm have correctly inherited the historical data?

 

Thanks,

Tianyang Zhang

SJTU Network Information Center



-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com 
 
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com 
 


-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com 
 
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com 
 


-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com

[slurm-users] Re: Why AllowAccounts not work in slurm-23.11.6

2024-10-29 Thread shaobo liu via slurm-users 
My slurm is 23.02.6，used EnforcePartLimits=ALL，Accounts outside of
AllowAccounts can still submit。

Laura Hild  于2024年10月19日周六 21:59写道：

> What do you have for `sacctmgr list account`?  If "root" is your top-level
> (Slurm) (bank) account, AllowAccounts=root may just end up meaning any
> account.  To have AllowAccounts limit what users can submit, you'd need to
> name a lower-level Slurm (bank) account that only some users have an
> Association with.
>
>

-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com

[slurm-users] Re: 转发: What is the safe upgrade path when upgrade from slurm21.08 and mariadb5.5?

2024-10-29 Thread Sid Young via slurm-users 
I recently upgraded from 20.11 to 24.05.2, before moving the cluster from
CentOS 7.9 to Oracle Linux 8.10

The DB upgrade should be pretty simple, do a mysqldump first, then
uninstall the old DB, change the repo's and install the new DB version. It
should recognise the DB files on disk and access them. Do another DB backup
on the new DB version. then roll through the Slurm upgrades.

I picked the first and last version of each release, and systematically
went through each node till it was done. First the slurm controller node,
then the compute nodes. To avoid Job loss, drain the nodes or you end up
with a situation where the slurmd can't talk to the running slurmstepd and
the job(s) gets lost. (Shows as a "Protocol Error").

Ole sent me a link to this guide which mostly worked.
https://wiki.fysik.dtu.dk/Niflheim_system/Slurm_installation/#upgrade-slurmd-on-nodes



Sid Young
W: https://off-grid-engineering.com



On Tue, Oct 29, 2024 at 6:33 PM Andrej Sec via slurm-users <
slurm-users@lists.schedmd.com> wrote:

> Hi,
>
> we are facing a similar task. We have a Slurm 22.05 / MariaDB 5.5.68
> environment and want to upgrade to a newer version. According to the
> documentation, it’s recommended to upgrade from 22.05 to a maximum of 23.11
> in one step. With the MariaDB upgrade, there’s a challenge between 10.1 and
> 10.2+ due to incompatible changes (https://mariadb.com/kb/en/changes
> -improvements-in-mariadb-10-2). This upgrade, as I understand from the
> documentation, requires at least slurm 22.05, where it is automatically
> handled by the slurmdbd service.
>
> In the test lab, we performed the following tests:
>
> a. Incremental upgrade - according to MariaDB recommendations:
>
>1. Upgrade MariaDB 5.5.68 -> 10.1.48 -> 10.2.44
>2. Start the Slurm suite 22.05, checking content after each MariaDB
>upgrade step. During the 10.1 -> 10.2 upgrade, the slurmdbd service
>automatically converted the database to the required format. We had enabled
>general.log in MariaDB, allowing detailed inspection of database changes
>during conversion.
>3. Upgrade slurmdbd to version 23.11
>4. Upgrade slurmctld to version 23.11
>5. Upgrade slurmd to version 23.11
>6. Check the database content and compare tests before and after the
>upgrade (we used various reports with scontrol, sreport, sacct, sacctmgr
>for verification).
>
> b. Direct MariaDB upgrade from 5.5.68 to 10.2.44 using the same approach.
> According to the tests, this resulted in the same state as the incremental
> approach.
>
>
> PS: If you proceed with the upgrade, I would appreciate it if you could
> let us know about any potential challenges you encountered.
>
>
> Andrej Sec
> nscc, Bratislava, Slovakia
>
> --
> *Od: *"hermes via slurm-users" 
> *Komu: *slurm-users@lists.schedmd.com
> *Odoslané: *pondelok, 28. október 2024 8:48:19
> *Predmet: *[slurm-users]
> =?eucgb2312_cn?q?=D7=AA=B7=A2=3A_What_is_the_safe_upgrade_path_when_upgrade_from_slurm21=2E08_and_mariadb5=2E5=3F?=
>
> Hi everyone:
>
>
>
> We are currently running business on SLURM21.08 and mariadb5.5.
>
> When talking about the upgrade, we need to keep all the users and jobs
> history data. And we see the official document wrote:
>
> “When upgrading an existing accounting database to *MariaDB 10.2.1* or
> later from an older version of MariaDB or any version of MySQL, ensure you
> are running *slurmdbd 22.05.7* or later. These versions will gracefully
> handle changes to MariaDB default values that can cause problems for
> slurmdbd.”
>
>
>
> So is this mean we have to firstly build SLURM>22.05 over mariadb5.5, and
> do the SLURM upgrade. Then upgrade the mariadb to newer version, and
> rebuild the same version of SLURM over new mariadb-devel?
>
> And is it safe to jump directly from mariadb5.5 to latest version? How can
> we check whether the slurm have correctly inherited the historical data?
>
>
>
> Thanks,
>
> Tianyang Zhang
>
> SJTU Network Information Center
>
>
> --
> slurm-users mailing list -- slurm-users@lists.schedmd.com
> To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
>
> --
> slurm-users mailing list -- slurm-users@lists.schedmd.com
> To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
>

-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com

[slurm-users] Re: Slurm Job Sched Priority

2024-10-29 Thread Cutts, Tim via slurm-users 
I use the following horrible little shell function to give me a reasonable list 
of currently pending jobs in a partition, and the order they’re likely to run 
in (as well as individual users’ priorities):

_users_waiting() {
  squeue "$@" -t PD -h -O UserName | sort -u | tr -d ' ' | tr '\012' ',' | sed 
's/,$//'
}

priority_in_queue() {
  local users=$(_users_waiting "$@")
  echo "User Fairshare"
  echo "=="
  sshare -U -u "$users"
  echo
  echo "Pending Job Priorities"
  echo "=="
  sprio "$@" -S "-y" -u "$users" -o "%.8i %9r %.8u %.8o %.8Y %.6A %.10F %.10J 
%.9P %.7n %.6Q %.30T"
}


--
Tim Cutts
Scientific Computing Platform Lead
AstraZeneca

Find out more about R&D IT Data, Analytics & AI and how we can support you by 
visiting our Service 
Catalogue |


From: Paul Edmon via slurm-users 
Date: Tuesday, 29 October 2024 at 2:09 PM
To: slurm-users@lists.schedmd.com 
Subject: [slurm-users] Re: Slurm Job Sched Priority

If you are looking to use the C API for this then showq may be a good guide: 
https://github.com/fasrc/slurm_showq  The 
-o option orders the pending queue in priority order.

If you are looking at native slurm commands, sprio can print out the current 
priority breakdown of any job and filter by partition, then you can reorder 
based on that.  squeue also can print out current priority. You might also look 
at the --priority option: 
https://slurm.schedmd.com/squeue.html#OPT_priority

-Paul Edmon-
On 10/29/24 9:33 AM, Bhaskar Chakraborty via slurm-users wrote:
Hello,

Is there any DS in slurmctld which portrays the dynamic relative priority of 
pending jobs?

We are trying to use slurm for developing a scheduling solution and 1 of the 
problems we face at the outset is how to determine
the order of scheduling for pending jobs.

One option is to find scheduling iteration window begin & close pointers & 
cache the job ids as seen in order & then make them the priority order at that 
point of time.

( This means for 500 pending jobs, say, if we can find which are the slurmctld 
calls which mark the beginning & end of a sched iteration then we can use the 
scheduling order
of jobs as the relative priority order for that period of time, of course it 
may change depending on fairshare, user initiated priority modification etc. )

A concrete existing data structure showing the dynamic priority itself from 
slurmctld would be handy.

Help appreciated.

Thanks!

Bhaskar.








AstraZeneca UK Limited is a company incorporated in England and Wales with 
registered number:03674842 and its registered office at 1 Francis Crick Avenue, 
Cambridge Biomedical Campus, Cambridge, CB2 0AA.

This e-mail and its attachments are intended for the above named recipient only 
and may contain confidential and privileged information. If they have come to 
you in error, you must not copy or show them to anyone; instead, please reply 
to this e-mail, highlighting the error to the sender and then immediately 
delete the message. For information about how AstraZeneca UK Limited and its 
affiliates may process information, personal data and monitor communications, 
please see our privacy notice at 
www.astrazeneca.com

-- 
slurm-users mailing list -- slurm-users@lists.schedmd.com
To unsubscribe send an email to slurm-users-le...@lists.schedmd.com