I'm guess it's because they took a shotgun approach to that and have not figured out how to let the user specify the -v can be used, and how. But that's just a guess.
I don't follow your suggestion, though... Juan ________________________________________ From: Reuti [re...@staff.uni-marburg.de] Sent: Sunday, April 09, 2017 17:09 To: Jimenez, Juan Esteban Cc: Jesse Becker; SGE-discuss@liv.ac.uk Subject: Re: [SGE-discuss] Sizing the qmaster -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Am 09.04.2017 um 12:38 schrieb juanesteban.jime...@mdc-berlin.de: > Update. > > Weeks later the overkill of setting memory to 12gb for the VM that runs > qmaster has resulted in relative stability. I could investigate how much I > can reduce that without cause problems but not in the mood for tempting > Murphy, > > FYI, those of you using SnakeMake will find out that it does not support > using -v. Because it makes the user's lives "easier" they don't want to go > through the trouble of using -v manually. I don't know why SnakeMake insists > on not following best practice. I've just told the users that if they use > SnakeMake to create thousands of jobs using -V and they crash the scheduler > as a result, the blame will come right back to them... I never used SnakeMake, but why is -v not supported? In the plain download I can't spot -V in the demo files. In case a long list needs to be exported (with or w/o an assigned value), it could be put in each job specific directory or the users' home directories in a .sge_request file - whether SnakeMake like it or not. - -- Reuti > Mfg, > Juan Jimenez > System Administrator, HPC > MDC Berlin / IT-Dept. > Tel.: +49 30 9406 2800 > > > ________________________________________ > From: Reuti [re...@staff.uni-marburg.de] > Sent: Tuesday, March 21, 2017 17:19 > To: Jimenez, Juan Esteban > Cc: Jesse Becker; SGE-discuss@liv.ac.uk > Subject: Re: [SGE-discuss] Sizing the qmaster > >> Am 21.03.2017 um 16:15 schrieb juanesteban.jime...@mdc-berlin.de: >> >>> The "size" of job metadata (scripts, ENV, etc) doesn't really affect >>> the RAM usage appreciably that I've seen. We routinely have jobs >>> ENVs of almost 4k or more, and it's never been a problem. The >>> "data" processed by jobs isn't a factor in qmaster RAM usage, so far as >>> I know. >> >> I’ve been reading otherwise, with specific recommendations to use –v rather >> than –V, but many tools out there are lazy and just use –V. passing the >> entire environment. > > Although it falls out of the main discussion here: > > I second to use only -v and only for environment variables you need in the > job script, preferably with an assignment and not only broadcasting any value > of the current bash session to the job. The best is to have self contained > scripts, so that you can reproduce the behavior of a job also in a month for > sure. Whether you have -v in the job script as a #$ line for SGE, or > set/export it directly in the script, might be a matter of taste. One real > application of -v with an assignment on the command line, is to use a > different program flow in the job script (and print this case of execution in > the output). > > I avoided the word "path" in the last sentence above by intention. When I > started with queuing systems, I saw many users using -V as it looks appealing > to execute in the queuing system what you would otherwise start right now on > the command line. But on the command line you notice instantly whether any > $PATH was set in the wrong way and target the wrong binary or a wrong value > for any application was assigned which changes its behavior. Recalling in one > month why a different version of program XY was used than it was intended, or > why the job crashed using e.g. a different parameter file can be hard. So I > convinced my users to stop this procedure, as it was impossible for me as an > admin to explain to them why their job crashed, when I'm not aware of the > actual settings inside their shell session was at submission time. Even > worse: having more than one terminal window open to the head node can result > in the effect that in one session it's working, but not in the other because > the shell's environment was changed by the user. > > -- Reuti > -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iEYEARECAAYFAljqTqMACgkQo/GbGkBRnRr+cQCguRgTKdENd6paZgdlBJI068jN gvAAnRrKoeUTGtW8pAmL0iXL8wNFIonr =iHmT -----END PGP SIGNATURE----- _______________________________________________ SGE-discuss mailing list SGE-discuss@liv.ac.uk https://arc.liv.ac.uk/mailman/listinfo/sge-discuss
