I'm trying to setup a system where, when a job from a certain account is submitted, if no nodes are available that have a specific feature, then the job will be paused/held/pending and a node will be dynamically created with that feature.
I can now dynamically bring up the node with the feature, and it shows in the sinfo output as having the feature. But I can't yet figure out how to intercept the job submission request and put it on hold so that I can bring up the node. If I don't do anything, then the job just instantly fails because there are no nodes with that feature. Could I maybe create a "dummy" node that has the feature, but no resources? So the job would be set to pending for resources and would stay that way until I brought up a new node with the feature and with resources. I've tried using slurm_job_submit.lua, and I detected the requested feature, and the tried to set the job to hold...but it still error'd out because of "invalid feature specification". Thanks for the help. Rob
