Would¹ve been nice to raise a bug about it :) On 3/8/14, 10:32 AM, "Marcus" <shadow...@gmail.com> wrote:
>Yeah, I've just seen on busy systems where even with log rotation working >properly the little space left in var after OS files is barely enough, for >example the conntrackd log on a busy VPC. We actually ended up rolling our >own system vm, the existing image has plenty of space, its just locked up >in other partitions. >On Mar 8, 2014 8:58 AM, "Rajesh Battala" <rajesh.batt...@citrix.com> >wrote: > >> Yes, only 435MB is available for /var . we can increase the space also. >> But we need to find out the root cause which services are causing the >>/var >> to fill up. >> Can you please find out and post which log files are taking up more >>space >> in /var >> >> Thanks >> Rajesh Battala >> >> -----Original Message----- >> From: Marcus [mailto:shadow...@gmail.com] >> Sent: Saturday, March 8, 2014 8:19 PM >> To: dev@cloudstack.apache.org >> Subject: RE: system vm disk space issue in ACS 4.3 >> >> Perhaps there's a new service. I know in the past we've seen issues with >> this , specifically the conntrackd log. I think the cloud logs weren't >> getting rolled either, but I thought it was all fixed. >> >> There's also simply not a ton of space on /var, I wish we would go back >>to >> just having one partition because it orphans lots of free space in other >> filesystems. >> On Mar 8, 2014 12:37 AM, "Rajesh Battala" <rajesh.batt...@citrix.com> >> wrote: >> >> > AFAIK, log roation is enabled in the systemvm. >> > Can you check whether the logs are getting zipped .? >> > >> > -----Original Message----- >> > From: Anirban Chakraborty [mailto:abc...@juniper.net] >> > Sent: Saturday, March 8, 2014 12:46 PM >> > To: dev@cloudstack.apache.org >> > Subject: system vm disk space issue in ACS 4.3 >> > >> > Hi All, >> > >> > I am seeing system vm disk has no space left after running for few >>days. >> > Cloudstack UI shows the agent in v-2-VM in alert state, while agent >> > state of s-1-VM shows blank (hyphen in the UI). >> > Both the system vms are running and ssh-able from the host. The log in >> > s-1-Vm shows following errors: >> > >> > root@s-1-VM:~# grep 'Exception' /var/log/cloud/*.* >> > /var/log/cloud/cloud.out.2:java.io.IOException: No space left on >> > device >> > /var/log/cloud/cloud.out.2:java.io.IOException: No space left on >> > device >> > >> > whereas logs in v-1-VM shows >> > /var/log/cloud/cloud.out.3:java.io.IOException: No space left on >> > device >> > /var/log/cloud/cloud.out.3:java.io.IOException: No space left on >> > device >> > /var/log/cloud/cloud.out.3:07:18:00,547 INFO CSExceptionErrorCode:87 >> > - Could not find exception: >> > com.cloud.exception.AgentControlChannelException >> > in error code list for exceptions >> > >> >>/var/log/cloud/cloud.out.3:com.cloud.exception.AgentControlChannelExcepti >>on: >> > Unable to post agent control request as link is not available >> > >> > Looks like cloud agent is filling up the log, which is leading to the >> > disk full state. >> > >> > Is this a known issue? Thanks. >> > >> > Anirban >> > >>
