Not sure if it's related but I do see the "SSL Fail to find the generated
keystore" error which means your cloud.keystore file is not in the
/etc/cloudstack/management folder..
> Date: Tue, 26 Aug 2014 19:18:51 -0400
> From: fgaudrea...@cloudops.com
> To: dev@cloudstack.apache.org
> Subject: Re: VMs stuck in expunging
>
> Oh man, we have four in this state, and you don't want to see the other
> names lol
>
> I'll pump the log to trace and see what's going on.
>
> Thanks!
>
> FG
>
> On 2014-08-26, 2:41 PM, ilya musayev wrote:
> > Its apparent that expunge failed, because VM name "ketchup-web" was to
> > good to be deleted :)
> >
> > Try increasing the verbosity level on log4j so we can get a better
> > picture.
> >
> >
> > On 8/26/14, 7:29 AM, Francois Gaudreault wrote:
> >> Hi,
> >>
> >> We have couple VMs stuck in Expunging state in our setup, and I am
> >> not sure why. This is a 4.3.1-snapshot. This is what the log says:
> >>
> >> 2014-08-25 23:30:31,789 DEBUG [c.c.v.VirtualMachineManagerImpl]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Stopped called on
> >> VM[User|ketchup-web] but the state is Expunging
> >> 2014-08-25 23:30:31,808 WARN [c.c.u.n.Link]
> >> (AgentManager-Selector:null) SSL: Fail to find the generated
> >> keystore. Loading fail-safe one to continue.
> >> 2014-08-25 23:30:31,810 DEBUG [c.c.c.CapacityManagerImpl]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) VM state transitted from :Expunging
> >> to Expunging with event: ExpungeOperationvm's original host id: 1 new
> >> host id: null host id before state transition: null
> >> 2014-08-25 23:30:31,811 DEBUG [c.c.v.VirtualMachineManagerImpl]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Destroying vm VM[User|ketchup-web]
> >> 2014-08-25 23:30:31,811 DEBUG [c.c.v.VirtualMachineManagerImpl]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Cleaning up NICS
> >> 2014-08-25 23:30:31,811 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Cleaning network for vm: 61
> >> 2014-08-25 23:30:31,814 DEBUG [c.c.v.VirtualMachineManagerImpl]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Cleaning up hypervisor data
> >> structures (ex. SRs in XenServer) for managed storage
> >> 2014-08-25 23:30:31,821 DEBUG [o.a.c.e.o.VolumeOrchestrator]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Cleaning storage for vm: 61
> >> 2014-08-25 23:30:31,825 DEBUG [c.c.v.VirtualMachineManagerImpl]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Expunged VM[User|ketchup-web]
> >> 2014-08-25 23:30:31,825 DEBUG [c.c.v.UserVmManagerImpl]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Starting cleaning up vm
> >> VM[User|ketchup-web] resources...
> >> 2014-08-25 23:30:31,856 DEBUG [c.c.n.f.FirewallManagerImpl]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) No firewall rules are found for vm
> >> id=61
> >> 2014-08-25 23:30:31,872 DEBUG [c.c.v.UserVmManagerImpl]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Firewall rules are removed
> >> successfully as a part of vm id=61 expunge
> >> 2014-08-25 23:30:31,884 DEBUG [c.c.u.d.T.Transaction]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Rolling back the transaction: Time
> >> = 5 Name = UserVm-Scavenger-1; called by
> >> -TransactionLegacy.rollback:896-TransactionLegacy.removeUpTo:839-TransactionLegacy.close:663-Transaction.execute:41-Transaction.execute:46-FirewallManagerImpl.revokeRule:732-GeneratedMethodAccessor278.invoke:-1-DelegatingMethodAccessorImpl.invoke:43-Method.invoke:606-AopUtils.invokeJoinpointUsingReflection:317-ReflectiveMethodInvocation.invokeJoinpoint:183-ReflectiveMethodInvocation.proceed:150
> >> 2014-08-25 23:30:31,887 WARN [c.c.v.UserVmManagerImpl]
> >> (UserVm-Scavenger-1:ctx-bfdf6aa3) Unable to expunge VM[User|ketchup-web]
> >>
> >> Any idea what's going on? Bug?
> >>
> >
> >
> >
>
>
> --
> Francois Gaudreault
> Gestionnaire de Produit | Product Manager - Cloud Platform & Services
> t:514-629-6775
>
> CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
> 420 rue Guy | Montreal | Quebec | H3J 1S6
> w: cloudops.com | tw: @CloudOps_
>
