Hi John, Thanks for your thoughts. Lots of new technology out there!
have you looked at Singularity > https://github.com/gmkurtzer/singularity/releases/tag/2.0 > Looks very cool, but it doesn't address our problem. We already have the images built with our codes. Our users don't have Unix user ids. We know their execution environment. It doesn't handle queueing, which the problem I have. > I was gobsmacked to see how easy it was to install Juia ClusterManagers > and get Slurm integration. > This is very nice, and something many Docker-oriented tools have. What they don't have is good multi-user support. Remember, you can't let people run Docker directly, because it gives them root access to the machine. The queue manager has to control that part. You don't even want to start the container as root, because you might be running an arbitrary container. > ps. Also have you looked at Bright Cluster Manager? > http://www.brightcomputing.com/whats-new-in-7.2 > We want both bare metal and commercial VPCs. Provisioning bare metal is not a problem we have right now. Our cluster is small and already provisioned. For VPCs, we can use StarCluster to launch the cluster in the cloud, but that cluster is standalone. The queue manager needs to know it was created and push the user's environment to it. The interesting times we are living in are at odds with our infrastructure-oriented past. Clusters can come and go, and users can package their code portably. The "module load" systems like Bright Cluster offers are irrelevant. Let users build their images as they like with only a few requirements, and they can run them with JupyterHub AND in an HPC environment, which eliminates the need for Singularity. Rob
