Hi Sam,
thanks for sharing. If i get it right you are enumerating variables after
allocation (slurm_spank_task_init) on the running node, but I am really
curious if it even possible to get those vars (or some vars) before nodes
will be allocated:
int slurm_spank_init (spank_t sp, int ac, char **av) {
if (spank_context () != S_CTX_ALLOCATOR) {
return (0);
}
...
}
I believe at this step srun command line options and #SBATCH instructions
should be parsed already or, at least, available.
Kind regards,
Dmitry
Kind regards,
Dmitry Chirikov
On Mon, Jan 30, 2017 at 4:41 PM, Sam Gallop (NBI) <[email protected]>
wrote:
> Hi Dmitry,
>
>
>
> I just recently posted this for similar query about SPANK plugins (see
> below for the example code). It helps dump some of the S_JOB_ARGV or
> S_JOB_ENV variables you are interested in. It's very basic and there is
> lots that can be done with SPANK. Example output looks like this …
>
> <snip>
>
> slurm_spank_task_init (ARGV): /bin/sleep
>
> slurm_spank_task_init (ARGV): 30
>
> …
>
> slurm_spank_task_init (ENV): SLURM_JOB_NAME=slurm1.sh
>
> slurm_spank_task_init (ENV): SLURMD_NODENAME=centos1
>
> slurm_spank_task_init (ENV): SLURM_JOBID=266
>
> slurm_spank_task_init (ENV): SLURM_JOB_PARTITION=debug
>
> slurm_spank_task_init (ENV): SLURM_JOB_NUM_NODES=1
>
> slurm_spank_task_init (ENV): SLURM_MEM_PER_NODE=1024
>
> <snip>
>
>
>
> # cat userenv.c
>
> /*
>
> * To compile:
>
> * gcc -shared -o userenv.so userenv.c
>
> *
>
> */
>
> #include <sys/types.h>
>
> #include <stdio.h>
>
> #include <stdlib.h>
>
> #include <unistd.h>
>
> #include <string.h>
>
> #include <libgen.h>
>
> #include <signal.h>
>
> #include <sys/resource.h>
>
>
>
> #include <slurm/spank.h>
>
> #include <slurm/slurm.h>
>
>
>
> SPANK_PLUGIN(userenv, 1);
>
>
>
> static int _userenv_opt_process (int val,
>
> const char *optarg,
>
> int remote);
>
> void get_env(spank_t sp, char *function);
>
>
>
> struct spank_option spank_options[] =
>
> {
>
> {
>
> "userenv",
>
> "",
>
> "helper function",
>
> 0,
>
> 0,
>
> (spank_opt_cb_f) _userenv_opt_process
>
> },
>
> SPANK_OPTIONS_TABLE_END
>
> };
>
>
>
> int slurm_spank_task_init(spank_t sp, int ac, char **av)
>
> {
>
> FILE *f;
>
> char filename[256];
>
> char function[50];
>
> int i, argc;
>
> char **argv;
>
> char **env;
>
> uint32_t s_id;
>
>
>
> sprintf(function, "slurm_spank_task_init");
>
>
>
> if (spank_get_item(sp, S_JOB_ID, &s_id) != ESPANK_SUCCESS) {
>
> slurm_info("userenv: Failed to get id");
>
> return (0);
>
> }
>
>
>
> sprintf(filename, "/tmp/%s_%d_%d", function, spank_context(),
> s_id);
>
> f = fopen(filename, "w");
>
>
>
> if (spank_get_item(sp, S_JOB_ARGV, &argc, &argv) ==
> ESPANK_SUCCESS) {
>
> for (i = 0; i < argc; i++) {
>
> fprintf(f, "%s (ARGV): %s\n", function, argv[i]);
>
> }
>
> }
>
>
>
> if (spank_get_item(sp, S_JOB_ENV, &env) == ESPANK_SUCCESS) {
>
> while (*env != NULL) {
>
> fprintf(f, "%s (ENV): %s\n", function,
> *env);
>
> ++env;
>
> }
>
> }
>
>
>
> fclose(f);
>
> }
>
>
>
> static int _userenv_opt_process(int val, const char *optarg, int remote)
>
> {
>
> slurm_info("userenv: userenv plugin has been registered");
>
> return (0);
>
> }
>
>
>
> ---
>
> Sam Gallop
>
>
>
> [image: Description: Macintosh HD:Users:fretter:Documents:SugarSync Shared
> Folders:NBI:pics for hpc wiki:CiSlogo578x293.png]
>
> Have you tried looking through our *Documentation Portal*
> <http://docs.cis.nbi.ac.uk/display/CIS/CiSSupport+Home>
>
> Our documentation isn’t all text, check out our *Video Tutorials*
> <http://docs.cis.nbi.ac.uk/display/CIS/Video+tutorials>
>
> Have you tried looking through *CiS Service Desk*
> <http://support.cis.nbi.ac.uk/>
>
> Keep up to date with availability at *CiS Service Status*
> <http://status.cis.nbi.ac.uk/>
>
> More information on our HPC, Linux, Storage at *HPC Support*
> <http://docs.cis.nbi.ac.uk/display/CIS/CiSSupport+Home>* Site*
>
>
>
> To speak to us about technical issues feel free to call the *Computing
> infrastructure for Science* team on *group phone extension * *2003**.*
>
> If your request is urgent, please contact the *NBIP Computing Helpdesk*
> at [email protected] or call *phone extension **1234**.*
>
>
>
> *From:* Dmitry Chirikov [mailto:[email protected]]
> *Sent:* 26 January 2017 12:04
> *To:* slurm-dev <[email protected]>
> *Subject:* [slurm-dev] SPANK and job variables/options
>
>
>
> Hi all,
>
>
>
> Playing with SPANK I faced up with an issue
> Seems I can't get any useful bits about job which user is about to submit.
>
> I am trying to add a hook to allocator context in slurm_spank_init
> or slurm_spank_init_post_opt functions.
>
>
>
> Idea was to check some option of the committed job file (or srun options)
> and make a decision about should it be accepted to be queued or not:
>
> But seems like it is not possible to get very basic variables
> like S_JOB_ARGV or S_JOB_ENV
>
>
> If i get it right this check limits the variables spank able to use to,
> literally, slurm version.
> https://github.com/SchedMD/slurm/blob/d7770b9b68d992f4f38417d5ef7204
> 895954eff8/src/common/plugstack.c#L1765
>
>
>
> Prolog facilies were looking promised, but they cannot drop the job - it
> can be re-queued only.
>
>
>
> So the question is, how to do it properly, when variables are available to
> use, and it is not too late to deny user to submit a job?
>
>
>
> Thank you in advance.
>
>
> Kind regards,
>
> Dmitry Chirikov
>
