Yeah, I've just seen on busy systems where even with log rotation working properly the little space left in var after OS files is barely enough, for example the conntrackd log on a busy VPC. We actually ended up rolling our own system vm, the existing image has plenty of space, its just locked up in other partitions. On Mar 8, 2014 8:58 AM, "Rajesh Battala" <rajesh.batt...@citrix.com> wrote:
> Yes, only 435MB is available for /var . we can increase the space also. > But we need to find out the root cause which services are causing the /var > to fill up. > Can you please find out and post which log files are taking up more space > in /var > > Thanks > Rajesh Battala > > -----Original Message----- > From: Marcus [mailto:shadow...@gmail.com] > Sent: Saturday, March 8, 2014 8:19 PM > To: dev@cloudstack.apache.org > Subject: RE: system vm disk space issue in ACS 4.3 > > Perhaps there's a new service. I know in the past we've seen issues with > this , specifically the conntrackd log. I think the cloud logs weren't > getting rolled either, but I thought it was all fixed. > > There's also simply not a ton of space on /var, I wish we would go back to > just having one partition because it orphans lots of free space in other > filesystems. > On Mar 8, 2014 12:37 AM, "Rajesh Battala" <rajesh.batt...@citrix.com> > wrote: > > > AFAIK, log roation is enabled in the systemvm. > > Can you check whether the logs are getting zipped .? > > > > -----Original Message----- > > From: Anirban Chakraborty [mailto:abc...@juniper.net] > > Sent: Saturday, March 8, 2014 12:46 PM > > To: dev@cloudstack.apache.org > > Subject: system vm disk space issue in ACS 4.3 > > > > Hi All, > > > > I am seeing system vm disk has no space left after running for few days. > > Cloudstack UI shows the agent in v-2-VM in alert state, while agent > > state of s-1-VM shows blank (hyphen in the UI). > > Both the system vms are running and ssh-able from the host. The log in > > s-1-Vm shows following errors: > > > > root@s-1-VM:~# grep 'Exception' /var/log/cloud/*.* > > /var/log/cloud/cloud.out.2:java.io.IOException: No space left on > > device > > /var/log/cloud/cloud.out.2:java.io.IOException: No space left on > > device > > > > whereas logs in v-1-VM shows > > /var/log/cloud/cloud.out.3:java.io.IOException: No space left on > > device > > /var/log/cloud/cloud.out.3:java.io.IOException: No space left on > > device > > /var/log/cloud/cloud.out.3:07:18:00,547 INFO CSExceptionErrorCode:87 > > - Could not find exception: > > com.cloud.exception.AgentControlChannelException > > in error code list for exceptions > > > /var/log/cloud/cloud.out.3:com.cloud.exception.AgentControlChannelException: > > Unable to post agent control request as link is not available > > > > Looks like cloud agent is filling up the log, which is leading to the > > disk full state. > > > > Is this a known issue? Thanks. > > > > Anirban > > >
