Re: [slurm-users] Python and R installation in a SLURM cluster

Just for completion. I can recommend using singularity /docker containers. You can copy them to the worker nodes during sbatch. In the long run it eases up a lot. Viele Grüße / kind regards Georg 2018-05-10 17:48 GMT+02:00 Simon Flood : > On 10/05/18 16:35, Eric F.

Re: [slurm-users] Python and R installation in a SLURM cluster

Completely as an aside, the next question then is 'Aha - but what happens when you have new users on the cluster' I am currently working with sssd authentication and with the pam_mkhomedir plugin. I guess if an MPI job is launched using ssh then pam_mkhomedir would automatically create the home dir

Re: [slurm-users] Python and R installation in a SLURM cluster

Well I DID say that you need 'what looks like a home directory'. So yes indeed you prove, correctly, that this works just fine! On 12 May 2018 at 20:17, Eric F. Alemany wrote: > > Hi John, > > No worries at all. I take all ideas, comments and advice with the greatest > respect. > I know that my

Re: [slurm-users] Python and R installation in a SLURM cluster

Hi, Home directory are also shared among all nodes (and head one as well). The base system (that is, local drive, including os, system files, etc) are identical and cloned based among all the hosts. If I need to install some light package (dev version of a library for example or similar) I install

Re: [slurm-users] Python and R installation in a SLURM cluster

._ Eric F. Alemany System Administrator for Research Division of Radiation & Cancer Biology Department of Radiation Oncology Stanford University School of Medicine Stanford, California 94305

Re: [slurm-users] Python and R installation in a SLURM cluster

Eric, I'm sorry to be a little prickly here. Each node has an independent home directory for the user? How then do applications update dot files? How then would as a for instance do the users edit the .bashrc file to bring Anaconda into their paths? Beofre anyone says it, a proper Modules system i

Re: [slurm-users] Python and R installation in a SLURM cluster

Hi Chris, Thank you for your comments. I will look at Easybuild. There are quite a few options to automate the creation of software modules. I will be doing lots of reading this week-end. By the way, i signed up to the Beowulf mailing list. Thank you, Eric

Re: [slurm-users] Python and R installation in a SLURM cluster

Hi John, Regarding NFS shares and Python, and plenty of other packages too, pay attention to where the NFS server is located on your network. The NFS server should be part of your cluster, or at least have a network interface on your cluster fabric. If you perhaps have a home directory server wh

Re: [slurm-users] Python and R installation in a SLURM cluster

HI Miguel, Thank you for your comment. That sounds pretty straight forward. you never had issues with programs relying on the system files or relying on the home directory location? Thanks Eric _

Re: [slurm-users] Python and R installation in a SLURM cluster

On Friday, 11 May 2018 5:11:38 PM AEST John Hearns wrote: > Eric, my advice would be to definitely learn the Modules system and > implement modules for your users. I will echo that, and the suggestion of shared storage (we use our Lustre filesystem for that). I would also suggest looking at a s

Re: [slurm-users] Python and R installation in a SLURM cluster

Regarding NFS shares and Python, and plenty of other packages too, pay attention to where the NFS server is located on your network. The NFS server should be part of your cluster, or at least have a network interface on your cluster fabric. If you perhaps have a home directory server which is a ca

Re: [slurm-users] Python and R installation in a SLURM cluster

Hi, I install all my apps in a shared storage, and change environment variables (path, vars, etc.) with lmod. It's very useful. Regards. El vie., 11 may. 2018 a las 6:19, Eric F. Alemany () escribió: > Hi Lachlan, > > Thank you for sharing your environment. Everyone has their own set of > rules

Re: [slurm-users] Python and R installation in a SLURM cluster

Hi Lachlan, Thank you for sharing your environment. Everyone has their own set of rules and i appreciate everyone’s input. It seems as if the NFS share is a great place to start. Best, Eric _ Eri

Re: [slurm-users] Python and R installation in a SLURM cluster

Thank you Thomas for your suggestion. I take note of all the people’s comment and hope to come with a good solution. _ Eric F. Alemany System Administrator for Research Division of Radiation & C

Re: [slurm-users] Python and R installation in a SLURM cluster

On 11 May 2018 at 01:35, Eric F. Alemany wrote: > Hi All, > > I know this might sounds as a very basic question: where in the cluster > should I install Python and R? > Headnode? > Execute nodes ? > > And is there a particular directory (path) I need to install Python and R. > > Background: > SLU

Re: [slurm-users] Python and R installation in a SLURM cluster

Assuming you plan for users to use R in jobs, it will need to be accessible to the execute/compute nodes. I would usually suggest on a shared drive. Although it should be OK if locally installed on each compute node (probably want at same exact path and with same R packages installed). Presumabl

Re: [slurm-users] Python and R installation in a SLURM cluster

Thank you Simon for your quick reply. I liked the "(N)either" touch - makes sense ._ Eric F. Alemany System Administrator for Research Division of Radiation & Cancer Biology Department of Radia

Re: [slurm-users] Python and R installation in a SLURM cluster

Hi Eric, On 10/05/18 23:35, Eric F. Alemany wrote: > I know this might sounds as a very basic question: where in > the cluster should I install Python and R? > Headnode? > Execute nodes ? I don't think there is a fixed rule for a question like this and it depends on the compromise between what

Re: [slurm-users] Python and R installation in a SLURM cluster

On 10/05/18 16:35, Eric F. Alemany wrote: I know this might sounds as a very basic question: where in the cluster should I install Python and R? Headnode? Execute nodes ? And is there a particular directory (path) I need to install Python and R. Background: SLURM on Ubuntu 18.04 1 headnode 4

[slurm-users] Python and R installation in a SLURM cluster

Hi All, I know this might sounds as a very basic question: where in the cluster should I install Python and R? Headnode? Execute nodes ? And is there a particular directory (path) I need to install Python and R. Background: SLURM on Ubuntu 18.04 1 headnode 4 execute nodes NFS shared drive among