Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-redhat-linux-gnu'
-DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/l\
ocale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib
-D_GNU_SOURCE -DRECYCLES_PIDS -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2
-fexceptions -fstack-prot\
ector --param=ssp-buffer-size=4 -m64 -mtune=generic -fwrapv
uname output: Linux njblxpx39d 2.6.32-358.23.2.el6.x86_64 #1 SMP Sat Sep 14
05:32:37 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-redhat-linux-gnu
Bash Version: 4.1
Patch Level: 2
Release Status: release
Description:
*Sometimes* a redirected file is not generated, even though directory
permissions are fine.
Repeat-By:
THE RELEVANT CODE SECTION
log_notice "DANDEBUG Storing pidfile... ${PIDFILE}"
ps -p ${NEWPID} -o pid= -o args= >${PIDFILE}
PSRET=$?
log_notice "DANDEBUG PSRET: ${PSRET}"
if [[ -f "${PIDFILE}" ]]
then
log_notice "DANDEBUG PIDFILE exists."
log_notice "DANDEBUG Contents of pidfile..."
cat ${PIDFILE}
else
log_error "PIDFILE doesn't exist."
fi
Each morning I have two cronjobs (dscache_copsmaster, dscache_copsslave) which
each launch an application through a common "appbase.sh" application script.
The way this works is that each defines a PROC_COMMAND_START and then sources
appbase.sh which will give commands like "start", "stop", "status", such that
things like the following can be called: "./dscache_copsmaster.sh start",
"./dscache_copsmaster.sh stop". Starting an application in this way will
create a requisite run directory and put a pidfile there for reference by a
later call to stop. Every morning, when the instance called dscache_copsmaster
is started, it complains that the PIDFILE doesn't exist. However, for
dscache_copsslave, the correct behavior occurs and there is no complaint. The
failure output is as follows:
20141909 03:00:01.037 NOTICE DANDEBUG Storing pidfile...
/apps/home/nye_pxd/logs/today/dscache/dscache_copsmaster/dscache_copsmaster.pid
20141909 03:00:01.074 NOTICE DANDEBUG PSRET: 0
20141909 03:00:01.075 ERROR PIDFILE doesn't exist.
20141909 03:00:06.077 NOTICE dscache_copsmaster started (pid: 46465).
However, dscache_copsslave (and all other applications managed through the same
script) always succeeds and here is its successful output:
20141909 03:00:01.037 NOTICE DANDEBUG Storing pidfile...
/apps/home/nye_pxd/logs/today/dscache/dscache_copsslave/dscache_copsslave.pid
20141909 03:00:01.070 NOTICE DANDEBUG PSRET: 0
20141909 03:00:01.070 NOTICE DANDEBUG PIDFILE exists.
20141909 03:00:01.071 NOTICE DANDEBUG Contents of pidfile...
46464 /apps/home/nye_pxd/dscache/bin/dscache
/apps/home/nye_pxd/dscache/config/dscache_copsslave.shadow.amer.xml
20141909 03:00:06.074 NOTICE dscache_copsslave started (pid: 46464).
Other relevant bits:
1) This section of the script runs with "set -o nounset" and "set -o pipefail"
2) The directories in which the pidfiles are to be written are created by the
script in an earlier section, and are writable (confirmed, because the
application itself outputs a file there immediately on starting). Here is how
they are created:
[[ -d "${RUNDIR}" ]] || mkdir -p ${RUNDIR}
Where RUNDIR for dscache_copsmaster is
"/apps/home/nye_pxd/logs/today/dscache/dscache_copsmaster" and for
dscache_copsslave is "/apps/home/nye_pxd/logs/today/dscache/dscache_copsslave".
3) The only differences between the two application launch scripts is as
follows:
$ diff dscache_copsmaster.sh dscache_copsslave.sh
17c17
< PROC_COMMAND_START="${APPROOT}/bin/dscache
${APPROOT}/config/dscache_copsmaster.${PX_ENV}.${PX_REGION}.xml"
---
> PROC_COMMAND_START="${APPROOT}/bin/dscache
> ${APPROOT}/config/dscache_copsslave.${PX_ENV}.${PX_REGION}.xml"
4) The $? of the ps command is confirmed to be 0 through the above log output,
though even if it failed I would have expected ${PIDFILE} to have been created.
5) The file does not exist even after the script exits, causing problems with
later requests for "status" and "stop" which depend on its contents. I could
see that perhaps the creation had a race with the test for file, but what is
very odd to me is that the file is not created at all even by when the script
completes. I could deal with first touching the file before redirecting to it
if redirects were known to race with the file test, but the above example seems
to indicate that redirects sometimes silently entirely fails to create the file.
_______________________________________________
This message is for information purposes only, it is not a recommendation,
advice, offer or solicitation to buy or sell a product or service nor an
official confirmation of any transaction. It is directed at persons who are
professionals and is not intended for retail customer use. Intended for
recipient only. This message is subject to the terms at:
www.barclays.com/emaildisclaimer.
For important disclosures, please see:
www.barclays.com/salesandtradingdisclaimer regarding market commentary from
Barclays Sales and/or Trading, who are active market participants; and in
respect of Barclays Research, including disclosures relating to specific
issuers, please see http://publicresearch.barclays.com.
_______________________________________________
