Hi Paul, I submitted the poll - thanks! For bug #7609, while I'd be happier with a built in slurm solution, you may find that our jobscript archiver implementation would work nicely for you. It is very high-performing and has no effect on the scheduler, or db performance.
The solution is a multithreaded c++ program which starts 1 thread for each /var/spool/slurm/hash.N directory. Each thread subscribes to inotify filesystem change events and when a new job directory shows up under hash.N, the program copies the jobscript file, and environment file to a local archive directory, at the same time creating user based ACLs on the files/dirs for security. We then have a cron that moves the jobscripts to a NFS share from which users can grab their jobscripts if desired. For our model, we wanted only the admins, and the user that submitted the script to have access to the jobscripts. Thus the reason for the ACLs on the files/dirs. We tried a slurmctld_prolog solution initially to archive jobs, but impacted scheduler performance dramatically. We have been very happy with it. Check it out, if you find it useful let me know! https://github.com/nauhpc/job_archive If you have any questions, please let me know! Best, Chris — Christopher Coffey High-Performance Computing Northern Arizona University 928-523-1167 On 8/28/19, 7:25 AM, "slurm-users on behalf of Paul Edmon" <slurm-users-boun...@lists.schedmd.com on behalf of ped...@cfa.harvard.edu> wrote: We have several pending feature requests to SchedMD regarding different features we would like to see, as I am sure many other groups have. We were curious if anyone else in the community is interested in these features and if your group would be interested in talking with us (Harvard FAS Research Computing) about getting these implemented (possibly through some coalition or collaboration). Please let us know which features you are interested in the poll below and then please list your University or organization and contact person. If you don't want to send your preferred contact info just post your organization and we will reach out to you. Thanks in advance. https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fforms.gle%2FLGWLuu9b3bRcihHs7&data=02%7C01%7Cchris.coffey%40nau.edu%7C53f4b79669d1455b7a5b08d72bc3a322%7C27d49e9f89e14aa099a3d35b57b2ba03%7C0%7C0%7C637025991574652242&sdata=8yOILmYvqUwCVvZNpiHmZdSdAMgVyduGRqbuStXvANQ%3D&reserved=0 -Paul Edmon- FAS Research Computing Harvard University
