HI Simon, selinux is disabled, I have just double checked.
BTW, this is what I can see in the cloudstack-agent.err log - seems like some CEPH related issues, but not sure why would agent die... If I recall correclty, this might be happening since the CEPH update from 0.80.3? to 0.87 - and this seesm like some crash in librados.... libust[1907/2046]: Warning: HOME environment variable not set. Disabling LTTng-UST per-user tracing. (in setup_local_apps() at lttng-ust-comm.c:305) libvirt: error : name in virDomainLookupByName must not be NULL libvirt: error : name in virDomainLookupByName must not be NULL libvirt: error : name in virDomainLookupByName must not be NULL libvirt: error : name in virDomainLookupByName must not be NULL libvirt: Storage Driver error : failed to remove volume 'cloudstack/bd751250-de35-4d2e-a4e3-3ee4b636c2a7': Device or resource busy ./log/SubsystemMap.h: In function 'bool ceph::log::SubsystemMap::should_gather(unsigned int, int)' thread 7f04427fc700 time 2015-02-21 06:39:38.839210 ./log/SubsystemMap.h: 62: FAILED assert(sub < m_subsys.size()) ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578) 1: (()+0x1fe223) [0x7f060c932223] 2: (ObjectCacher::flusher_entry()+0x155) [0x7f060c9866e5] 3: (ObjectCacher::FlusherThread::entry()+0xd) [0x7f060c9976cd] 4: (()+0x79d1) [0x7f06605ee9d1] 5: (clone()+0x6d) [0x7f066033bb5d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. terminate called after throwing an instance of 'ceph::FailedAssertion' 21/02/2015 06:39:38 1905 jsvc.exec error: Service did not exit cleanly On 20 February 2015 at 21:56, Simon Weller <swel...@ena.com> wrote: > Andrija, > > What is SELinux set to on this host? > > > - SI > > > ________________________________________ > From: Andrija Panic <andrija.pa...@gmail.com> > Sent: Friday, February 20, 2015 6:06 AM > To: dev@cloudstack.apache.org; us...@cloudstack.apache.org > Subject: Agent dies every night/morning.... memory violation > > Hi, > > I have crazy agent on one of the hosts, that is being killed each morning > and I found this in /var/log/audit.log: > > type=ANOM_ABEND msg=audit(1424321463.930:430678): auid=0 uid=0 gid=0 > ses=68891 pid=10831 comm="jsvc" reason="memory violation" sig=6 > > I dont remember changing anything on the system, but this keeps happening > each morning arrond same time 5.20am-5.40am. > > I'm wondering what the hack is happening, any suggestions where to > troubleshoot ? > Will check logs in details anyway... > > -- > > Andrija Panić > -- Andrija Panić
