On 10/24/22 06:12, Richard Chang wrote:
Is there a thumb rule for the size of the directory that is NFS exported,
and to be used as StateSaveLocation.
I have a two node Slurmctld setup and both will mount an NFS exported
directory as the state save location.
It is definitely a BAD idea to store Slurm StateSaveLocation on a slow NFS
directory! SchedMD recommends to use local NVME or SSD disks because
there will be many IOPS to this file system!
I recommend you to read "Field Notes 6: From The Frontlines of Slurm
Support", Jason Booth, SchedMD available from
https://slurm.schedmd.com/publications.html. Read the Hardware pages
18-20 which recommend:
Fast path to the StateSaveLocation
■ IOPS this filesystem can sustain is a major bottleneck to job throughput
● At least 2 directories and two files created per job
● The corresponding unlink() calls will add to the load
/Ole
