Thanks. I've made that fix.
-Paul Edmon-
On 8/28/24 5:42 PM, Davide DelVento wrote:
Thanks everybody once again and especially Paul: your job_summary
script was exactly what I needed, served on a golden plate. I just had
to modify/customize the date range and change the following line (I
can
Thanks everybody once again and especially Paul: your job_summary script
was exactly what I needed, served on a golden plate. I just had to
modify/customize the date range and change the following line (I can make a
PR if you want, but it's such a small change that it'd take more time to
deal with
You can also check https://github.com/prod-feng/slurm_tools
slurm_job_perf_show.py may be helpful.
I used to try to use slurm_job_perf_show_email.py to send emails to
users to summarize their usage, like monthly. While some users seemed
to get confused, so stopped.
Best,
Feng
On Fri, Aug 9, 20
This thread when a bunch of different directions. However I ran with
Jeffrey's suggestion and wrote up a profile.d script along with other
supporting scripts to pull the data. The setup I put together is here
for the community to use as they see fit:
https://github.com/fasrc/puppet-slurm_stats
] Re: Print Slurm Stats on Login
CAUTION: This email originated outside of the Organisation. Please help to keep
us safe and do not click links or open attachments unless you recognise the
sender and know the content is safe.
Thanks Kevin and Simon,
The full thing that you do is indeed overkill
Hi Ole,
On Wed, Aug 21, 2024 at 1:06 PM Ole Holm Nielsen via slurm-users <
slurm-users@lists.schedmd.com> wrote:
> The slurmacct script can actually break down statistics by partition,
> which I guess is what you're asking for? The usage of the command is:
>
Yes, this is almost what I was askin
Hi Davide,
Thanks, I appreciate your positive feedback! Some comments are below:
On 21-08-2024 15:07, Davide DelVento wrote:
Thanks, Ole! Your tools and what you do for the community is fantastic,
we all appreciate you!
Of course, I did look (and use) your script. But I need more info.
And
Thanks, Ole! Your tools and what you do for the community is fantastic, we
all appreciate you!
Of course, I did look (and use) your script. But I need more info.
And no, this is not something that users would run *ever* (let alone at
every login). This is something I *myself* (the cluster adminis
Hi,
what Ole wrote is exactly what crossed my mind. I had an episode with stats at
login too, I put reportseff to motd script and it was a bad idea. It turned out
that if for any reason slurm controler took longer time to respond, it delayed
user login which annoyed them more than they apprecia
Hi Davide,
Did you already check out what the slurmacct script can do for you? See
https://github.com/OleHolmNielsen/Slurm_tools/blob/master/slurmacct/slurmacct
What you're asking for seems like a pretty heavy task regarding system
resources and Slurm database requests. You don't imagine th
Thanks Kevin and Simon,
The full thing that you do is indeed overkill, however I was able to learn
how to collect/parse some of the information I need.
What I am still unable to get is:
- utilization by queue (or list of node names), to track actual use of
expensive resources such as GPUs, high
Heavyweight solution (although if you have grafana and prometheus going
already a little less so):
https://github.com/rivosinc/prometheus-slurm-exporter
On Tue, Aug 20, 2024 at 12:40 AM Simon Andrews via slurm-users <
slurm-users@lists.schedmd.com> wrote:
> Possibly a bit more elaborate than you
Possibly a bit more elaborate than you want but I wrote a web based monitoring
system for our cluster. It mostly uses standard slurm commands for job
monitoring, but I've also added storage monitoring which requires a separate
cron job to run every night. It was written for our cluster, but pr
g text output of squeue command)
>
> cheers
>
> josef
>
> --
> *From:* Davide DelVento via slurm-users
> *Sent:* Wednesday, 14 August 2024 01:52
> *To:* Paul Edmon
> *Cc:* Reid, Andrew C.E. (Fed) ; Jeffrey T Frey <
> f...@udel.edu>; slurm-users@lists.schedm
s
josef
From: Davide DelVento via slurm-users
Sent: Wednesday, 14 August 2024 01:52
To: Paul Edmon
Cc: Reid, Andrew C.E. (Fed) ; Jeffrey T Frey
; slurm-users@lists.schedmd.com
Subject: [slurm-users] Re: Print Slurm Stats on Login
I too would be interested in some lightweight
I too would be interested in some lightweight scripts. XDMOD in my
experience has been very intense in workload to install, maintain and
learn. It's great if one needs that level of interactivity, granularity and
detail, but for some "quick and dirty" summary in a small dept it's not
only overkill,
Yup, we have that installed already. It's been very beneficial for over
all monitoring.
-Paul Edmon-
On 8/9/2024 12:27 PM, Reid, Andrew C.E. (Fed) wrote:
Maybe a heavier lift than you had in mind, but check
out xdmod, open.xdmod.org.
It was developed by the NSF as part of the now-shutte
Maybe a heavier lift than you had in mind, but check
out xdmod, open.xdmod.org.
It was developed by the NSF as part of the now-shuttered
XSEDE program, and is useful for both system and user monitoring.
-- A.
On Fri, Aug 09, 2024 at 12:12:08PM -0400, Paul Edmon via slurm-u
Yeah, I was contemplating doing that so I didn't have a dependency on
the scheduler being up or down or busy.
What I was more curious about is if any one had an prebaked scripts for
that.
-Paul Edmon-
On 8/9/2024 12:04 PM, Jeffrey T Frey wrote:
You'd have to do this within e.g. the system's
You'd have to do this within e.g. the system's bashrc infrastructure. The
simplest idea would be to add to e.g. /etc/profile.d/zzz-slurmstats.sh and have
some canned commands/scripts running. That does introduce load to the system
and Slurm on every login, though, and slows the startup of logi
20 matches
Mail list logo
