Hi Ralph, thanks for your help!
Ralph Castain writes:
> It would have to be done via MPI_Info arguments, and we never had a
> request to do so (and hence, don't define such an argument). It would
> be easy enough to do so (look in the ompi/mca/dpm/orte/dpm_orte.c
> code).
Well, I wanted to just report success, but I've only got the easy
side of it: saving the arguments from the MPI_Info arguments into
the orte_job_t struct. See attached "0003" patch (against trunk).
However, I couldn't figure out how to get the other side: reading out
the environment variables and setting them at fork. Maybe you could
help with (or do :-) that?
Or just guide me as to where again: I threw abort()s in 'spawn'
functions I found under plm/, but my programs didn't abort and so I'm
not sure where they went.
> MPI implementations generally don't forcibly propagate envars because
> it is so hard to know which ones to handle - it is easy to propagate
> a system envar that causes bad things to happen on the remote end.
I understand. Though in this case, I'm /trying/ to make Bad Things
(tm) happen ;-).
> One thing you could do, of course, is add that envar to your default
> shell setup (.bashrc or whatever). This would set the variable by
> default on your remote locations (assuming you are using rsh/ssh
> for your launcher), and then any process you start would get
> it. However, that won't help if this is an envar intended only for
> the comm_spawned process.
Unfortunately what I want to play with at the moment are LD_*
variables, and fiddling with these in my .bashrc will mess up a lot
more than just the simulation I am presently hacking.
> I can add this capability to the OMPI trunk, and port it to the 1.7
> release - but we don't go all the way back to the 1.4 series any
> more.
Yes, having this in a 1.7 release would be great!
BTW, I encountered a couple other small things while grepping through
source/waiting for trunk to build, so there are two other small patches
attached. One gets rid of warnings about unused functions in generated
lexing code. I believe the second fixes resource leaks on error paths.
However, it turned out none of my user-level code hit that function at
all, so I haven't been able to test it. Take from it what you will...
-tom
> On Wed, Dec 11, 2013 at 2:10 PM, tom fogal <tfo...@sci.utah.edu> wrote:
>
> > Hi all,
> >
> > I'm developing on Open MPI 1.4.5-ubuntu2 on Ubuntu 13.10 (so, Ubuntu's
> > packaged Open MPI) at the moment.
> >
> > I'd like to pass environment variables to processes started via
> > MPI_Comm_spawn. Unfortunately, the MPI 3.0 standard (at least) does
> > not seem to specify a way to do this; thus I have been searching for
> > implementation-specific ways to accomplish my task.
> >
> > I have tried setting the environment variable using the POSIX setenv(3)
> > call, but it seems that Open MPI comm-spawn'd processes do not inherit
> > environment variables. See the attached 2 C99 programs; one prints
> > out the environment it receives, and one sets the MEANING_OF_LIFE
> > environment variable, spawns the previous 'env printing' program, and
> > exits. I run via:
> >
> > $ env -i HOME=/home/tfogal \
> > PATH=/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin \
> > mpirun -x TJFVAR=testing -n 5 ./mpienv ./envpar
> >
> > and expect (well, hope) to find the MEANING_OF_LIFE in 'envpar's
> > output. I do see TJFVAR, but the MEANING_OF_LIFE sadly does not
> > propagate. Perhaps I am asking the wrong question...
> >
> > I found another MPI implementation which allowed passing such
> > information via the MPI_Info argument, however I could find no
> > documentation of similar functionality in Open MPI.
> >
> > Is there a way to accomplish what I'm looking for? I could even be
> > convinced to hack source, but a starting pointer would be appreciated.
> >
> > Thanks,
> >
> > -tom
From 8285a7625e5ea014b9d4df5dd65a7642fd4bc322 Mon Sep 17 00:00:00 2001
From: Tom Fogal <tfo...@alumni.unh.edu>
List-Post: users@lists.open-mpi.org
Date: Fri, 13 Dec 2013 12:03:56 +0100
Subject: [PATCH 1/3] btl: Remove warnings about unused lexing functions.
---
ompi/mca/btl/openib/btl_openib_lex.l | 2 ++
1 file changed, 2 insertions(+)
diff --git a/ompi/mca/btl/openib/btl_openib_lex.l b/ompi/mca/btl/openib/btl_openib_lex.l
index 2aa6059..7455b78 100644
--- a/ompi/mca/btl/openib/btl_openib_lex.l
+++ b/ompi/mca/btl/openib/btl_openib_lex.l
@@ -1,3 +1,5 @@
+%option nounput
+%option noinput
%{ /* -*- C -*- */
/*
* Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
--
1.8.3.2
From dff9fd5ef69f09de6d0fee2236c39a79e8674f92 Mon Sep 17 00:00:00 2001
From: Tom Fogal <tfo...@alumni.unh.edu>
List-Post: users@lists.open-mpi.org
Date: Fri, 13 Dec 2013 13:06:41 +0100
Subject: [PATCH 2/3] mca: cleanup buf, ps when errors occur.
---
orte/mca/plm/base/plm_base_proxy.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/orte/mca/plm/base/plm_base_proxy.c b/orte/mca/plm/base/plm_base_proxy.c
index 5d2b100..275cb3a 100644
--- a/orte/mca/plm/base/plm_base_proxy.c
+++ b/orte/mca/plm/base/plm_base_proxy.c
@@ -128,14 +128,15 @@ int orte_plm_proxy_spawn(orte_job_t *jdata)
command = ORTE_PLM_LAUNCH_JOB_CMD;
if (ORTE_SUCCESS != (rc = opal_dss.pack(buf, &command, 1, ORTE_PLM_CMD))) {
ORTE_ERROR_LOG(rc);
+ OBJ_RELEASE(buf);
goto CLEANUP;
}
/* pack the jdata object */
if (ORTE_SUCCESS != (rc = opal_dss.pack(buf, &jdata, 1, ORTE_JOB))) {
ORTE_ERROR_LOG(rc);
+ OBJ_RELEASE(buf);
goto CLEANUP;
-
}
/* create the proxy spawn object */
@@ -153,6 +154,7 @@ int orte_plm_proxy_spawn(orte_job_t *jdata)
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
+ OBJ_RELEASE(ps);
goto CLEANUP;
}
--
1.8.3.2
From a90f1fb49df1ff9442476b5e4294353ebb94498b Mon Sep 17 00:00:00 2001
From: Tom Fogal <tfo...@alumni.unh.edu>
List-Post: users@lists.open-mpi.org
Date: Fri, 13 Dec 2013 15:09:10 +0100
Subject: [PATCH 3/3] info: accept env vars desired in child processes
This looks for "env" keys in MPI_Info structures, which should be
then used to forward environment variables from parent to child
when spawning jobs. However, note this doesn't (yet) change the
spawn machinery.
---
ompi/mca/dpm/orte/dpm_orte.c | 12 ++++++++++++
orte/runtime/orte_globals.c | 2 ++
orte/runtime/orte_globals.h | 2 ++
3 files changed, 16 insertions(+)
diff --git a/ompi/mca/dpm/orte/dpm_orte.c b/ompi/mca/dpm/orte/dpm_orte.c
index 65099a5..b61d6f2 100644
--- a/ompi/mca/dpm/orte/dpm_orte.c
+++ b/ompi/mca/dpm/orte/dpm_orte.c
@@ -680,6 +680,7 @@ static int spawn(int count, const char *array_of_commands[],
char mapper[OPAL_PATH_MAX];
int npernode;
char slot_list[OPAL_PATH_MAX];
+ char envvar[1024]; /* better magic number? */
orte_job_t *jdata;
orte_app_context_t *app;
@@ -705,6 +706,7 @@ static int spawn(int count, const char *array_of_commands[],
- "path": list of directories where to look for the executable
- "file": filename, where additional information is provided.
- "soft": see page 92 of MPI-2.
+ - "env": environment variables desired in the children.
*/
/* setup the job object */
@@ -1358,6 +1360,16 @@ static int spawn(int count, const char *array_of_commands[],
jdata->stdin_target = strtoul(stdin_target, NULL, 10);
}
}
+
+ /* did the user want us to forward any environment variables? */
+ ompi_info_get (array_of_info[i], "env", sizeof(envvar)-1, envvar,
+ &flag);
+ if ( flag ) {
+ jdata->nenv_vars++;
+ jdata->env_vars = realloc(jdata->env_vars,
+ jdata->nenv_vars*sizeof(char*));
+ jdata->env_vars[jdata->nenv_vars-1] = strdup(envvar);
+ }
}
/* default value: If the user did not tell us where to look for the
diff --git a/orte/runtime/orte_globals.c b/orte/runtime/orte_globals.c
index f3e3029..e4ba975 100644
--- a/orte/runtime/orte_globals.c
+++ b/orte/runtime/orte_globals.c
@@ -742,6 +742,8 @@ static void orte_job_construct(orte_job_t* job)
job->ckpt_snapshot_ref = NULL;
job->ckpt_snapshot_loc = NULL;
#endif
+ job->env_vars = NULL;
+ job->nenv_vars = 0;
}
static void orte_job_destruct(orte_job_t* job)
diff --git a/orte/runtime/orte_globals.h b/orte/runtime/orte_globals.h
index f284045..d12296b 100644
--- a/orte/runtime/orte_globals.h
+++ b/orte/runtime/orte_globals.h
@@ -463,6 +463,8 @@ typedef struct {
/* snapshot location */
char *ckpt_snapshot_loc;
#endif
+ char** env_vars;
+ size_t nenv_vars;
} orte_job_t;
ORTE_DECLSPEC OBJ_CLASS_DECLARATION(orte_job_t);
--
1.8.3.2
