Thanks, Ralph. I'm not sure I explained the problem clearly. Salt and JupyterHub are distractions, sorry.
I have code which "wires up" a cluster for MPI. What I need is scheduler that allows users to: * Select which Docker image they'd like to wire up * Request a number of nodes/cores * Understands that clusters can be dynamically created * Invokes an external command to create the cluster Here's what I'd like the user be able to do: $ queue-job --image=radiasoft/beamsim --cores=5000 my-script.sh queue-job would then have to be able to call a 3rd-party module to get the user: # 3rd-party-environ-collector This command would return a logical user and a network-accessible directory. This info would be added to the queue, and then when the scheduler decided to start the job, it would call: # 3rd-party-start --image=radiasoft/beamsim --cores=5000 --user=robnagler --mount=nfs:// intra.server.com/var/nfs/bla/robnagler/foo/bar:/home/docker-user/run my-script.sh The bit that I need is the scheduling. For example, the scheduler would have give the user a maximum number of core hours. It would maybe give the job a unique group id (a la Grid Engine) to manage disk quotas. These constraints would need to be passed to the 3rd-party programs so they could constrain the doctor container. What I have now is 3rd-party-environ-collector (JupyterHub) and 3rd-party-start (Salt). What I need is a scheduler that has an architecture that supports dynamic clusters and users who have no local credentials (non-Unix user -- just a name and a home directory). TIA, Rob
