Hello Our slurmdbd database is getting rather large and affecting performance, but we want to keep usage data around for a few years for metric purposes in order to figure out how our users work. I read a suggestion to have a backup DB which has all the usage data synced to it for metric purposes and a main slurmdbd setup for the cluster to use that cleans out old data based on your user working needs.
Is there any documentation suggesting how to set up a second slurmdbd server that will receive a copy of all the main slurmdbd entries without purging so we can start purging on the in use slurmdbd service to keep short term performance snappy? Presumably the upgrade process will be complicated by this as well since we have to keep the archive slurmdbd setup in sync with the cluster slurmdbd. Thanks. *EDIT before hitting send* I was re-reading the slurmdbd.conf man page and just saw the Archive* options and this sounds like it would work to implement something like this. Are archive files readable by sacct and sreport, or easily manually parseable? I am going to turn these on in my test cluster, but hearing about other peoples experiences with this would probably be helpful.
