I did nothing unsual to the nic. What it is true is that:
I had two sessions opened:
- 1 session as user. This user owns two specific networks. And this VM
had two nics on those networks.
- 1 session ad Admin
The user belongs to a specific domain. Therefore the admin cannot do all
operations the user or de domain-admin can do.
I cannot say if I stopped the VM as user or as admin. But for any reason
something strange happened to the nic.
I will check the database...
El 14/02/2017 a las 18:17, Sergey Levitskiy escribió:
I saw something like this when VM ‘lost’ a NIC somehow. Can you check if there
is at least one NIC assigned to the VM? Fix was to restore NIC record for that
VM in the DB. Another option (if VM is slated for destruction) is just to
change VM state to ‘Destroyed’ and garbage collector thread will cleanup
volumes and other artifacts.
On 2/14/17, 9:09 AM, "Rafael Weingärtner" <[email protected]> wrote:
Hi Natalia,
This problem is caused at class method
“org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.releaseNic(VirtualMachineProfile,
long)” line 1631 as you can see by the stack trace.
Did you change something else in the database or ACS while these VMs were
stuck in "stopping"?
The code at that line is the following:
updateNic(nic, network.getId(), -1);
For some reason the network object is null. The line that creates the
“network” object is 1609. The code is the following:
final NetworkVO network = _networksDao.findById(nic.getNetworkId());
It seems that the network that was used to assign the IPs for the VMs NIC
was deleted/removed? Could you check that?
The network interface card table is “nics”, you can look for the NICs of
your VM by filtering for “instance_id”.
Also, if you want to check the networks table is “networks”.
On Tue, Feb 14, 2017 at 11:52 AM, Natalia Costas Lago <[email protected]>
wrote:
> Dear Rafael,
>
> Thre requested information:
>
> ACS version: 4.9.0
>
> hypervisors used: kvm
> storage systems used: gluster as primary
> networking: advanced
>
> And this is the error...
>
>
>
> 2017-02-14 17:48:56,112 DEBUG [c.c.u.d.T.Transaction]
> (API-Job-Executor-2:ctx-4eca9fd0 job-1329 ctx-39cd3fd6) (logid:2eed47c7)
> Rolling back the transaction: Time = 1 Name = API-Job-Executor-2; called
by
> -TransactionLegacy.rollback:879-TransactionLegacy.removeUpTo
> :822-TransactionLegacy.close:646-Transaction.execute:43-Tra
> nsaction.execute:47-NetworkOrchestrator.releaseNic:1600-Netw
> orkOrchestrator.release:1588-UserVmManagerImpl.releaseNetwo
> rkResourcesOnExpunge:2102-UserVmManagerImpl.expunge:
> 2050-UserVmManagerImpl.destroyVm:2636-NativeMethodAccessorImpl.invok
> e0:-2-NativeMethodAccessorImpl.invoke:62
> 2017-02-14 17:48:56,156 ERROR [c.c.a.ApiAsyncJobDispatcher]
> (API-Job-Executor-2:ctx-4eca9fd0 job-1329) (logid:2eed47c7) Unexpected
> exception while executing org.apache.cloudstack.api.comm
> and.user.vm.DestroyVMCmd
> java.lang.NullPointerException
> at org.apache.cloudstack.engine.orchestration.NetworkOrchestrat
> or$6.doInTransaction(NetworkOrchestrator.java:1631)
> at org.apache.cloudstack.engine.orchestration.NetworkOrchestrat
> or$6.doInTransaction(NetworkOrchestrator.java:1600)
> at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction
> .java:50)
> at com.cloud.utils.db.Transaction.execute(Transaction.java:40)
> at com.cloud.utils.db.Transaction.execute(Transaction.java:47)
> at org.apache.cloudstack.engine.orchestration.NetworkOrchestrat
> or.releaseNic(NetworkOrchestrator.java:1600)
> at org.apache.cloudstack.engine.orchestration.NetworkOrchestrat
> or.release(NetworkOrchestrator.java:1588)
> at com.cloud.vm.UserVmManagerImpl.releaseNetworkResourcesOnExpu
> nge(UserVmManagerImpl.java:2102)
> at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.
> java:2050)
> at com.cloud.vm.UserVmManagerImpl.destroyVm(UserVmManagerImpl.
> java:2636)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> ssorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> thodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsin
> gReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation
> .invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation
> .proceed(ReflectiveMethodInvocation.java:150)
> at org.apache.cloudstack.network.contrail.management.EventUtils
> $EventInterceptor.invoke(EventUtils.java:106)
> at org.springframework.aop.framework.ReflectiveMethodInvocation
> .proceed(ReflectiveMethodInvocation.java:161)
> at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInt
> erceptor.java:51)
> at org.springframework.aop.framework.ReflectiveMethodInvocation
> .proceed(ReflectiveMethodInvocation.java:161)
> at org.springframework.aop.interceptor.ExposeInvocationIntercep
> tor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation
> .proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(
> JdkDynamicAopProxy.java:204)
> at com.sun.proxy.$Proxy200.destroyVm(Unknown Source)
> at org.apache.cloudstack.api.command.user.vm.DestroyVMCmd.execu
> te(DestroyVMCmd.java:123)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
> at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispat
> cher.java:108)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImp
> l$5.runInContext(AsyncJobManagerImpl.java:554)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable
> $1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedCon
> text$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedCon
> text.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedCon
> text.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable
> .run(ManagedContextRunnable.java:46)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImp
> l$5.run(AsyncJobManagerImpl.java:502)
> at java.util.concurrent.Executors$RunnableAdapter.call(
> Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Executor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2017-02-14 17:48:56,157 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-2:ctx-4eca9fd0 job-1329) (logid:2eed47c7) Complete
> async job-1329, jobStatus: FAILED, resultCode: 530, result:
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"
> uuidList":[],"errorcode":530}
> 2017-02-14 17:48:56,158 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-2:ctx-4eca9fd0 job-1329) (logid:2eed47c7) Publish async
> job-1329 complete on message bus
> 2017-02-14 17:48:56,158 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-2:ctx-4eca9fd0 job-1329) (logid:2eed47c7) Wake up jobs
> related to job-1329
> 2017-02-14 17:48:56,158 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-2:ctx-4eca9fd0 job-1329) (logid:2eed47c7) Update db
> status for job-1329
> 2017-02-14 17:48:56,159 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-2:ctx-4eca9fd0 job-1329) (logid:2eed47c7) Wake up jobs
> joined with job-1329 and disjoin all subjobs created from job- 1329
> 2017-02-14 17:48:56,186 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-2:ctx-4eca9fd0 job-1329) (logid:2eed47c7) Done
> executing org.apache.cloudstack.api.command.user.vm.DestroyVMCmd for
> job-1329
> 2017-02-14 17:48:56,186 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (API-Job-Executor-2:ctx-4eca9fd0 job-1329) (logid:2eed47c7) Remove
> job-1329 from job monitoring
> 2017-02-14 17:48:59,082 DEBUG [c.c.a.ApiServlet]
> (http-nio-6443-exec-14:ctx-dc7db42c) (logid:078a1edc) ===START===
> 193.144.34.125 -- GET command=queryAsyncJobResult&jo
> bId=2eed47c7-4e14-49fb-830e-b24eb785f678&response=json&_=1487090924322
>
>
>
> El 13/02/2017 a las 20:34, Rafael Weingärtner escribió:
>
>> Could you share the error here?
>> Could you also post some more information (stack trace of the erro, ACS
>> version, hypervisors used, storage systems used, networking
>> configuration[advanced or basic])?
>>
>> On Mon, Feb 13, 2017 at 2:31 PM, Natalia Costas Lago <[email protected]>
>> wrote:
>>
>> I did that. Now I am able to Destroy the VM... but I get an error when I
>>> try to expunge. I can recover the VM.. destroy.. .but not expunge.
>>>
>>>
>>>
>>> El 13/02/2017 a las 20:22, Rafael Weingärtner escribió:
>>>
>>> I have already had this experience of VMs getting stuck in “Stopping”
>>>> state. It is quite trivial the task, you just have to make sure you
are
>>>> changing the states of the right VMs. You just have to change the
state
>>>> to
>>>> “Stopped” on the database.
>>>>
>>>> On Mon, Feb 13, 2017 at 2:07 PM, Natalia Costas Lago
<[email protected]>
>>>> wrote:
>>>>
>>>> Dear all,
>>>>
>>>>> I stopped two VMs and it seems they where stopped but they appear
>>>>> "Stopping" in the user interface. I have seen workarounds to this
issue
>>>>> that update the status of the VM directly in the database.. but this
>>>>> seems
>>>>> risky to me.
>>>>>
>>>>> How should I proceed? Is that the only way?
>>>>>
>>>>> Thanx,
>>>>>
>>>>> Kind regards,
>>>>>
>>>>>
>>>>> --
>>>>> ====================================================
>>>>> Natalia Costas Lago
>>>>> Senior Communications Technician
>>>>> Galicia Supercomputing Centre (CESGA)
>>>>> (CESGA on Twitter <https://twitter.com/#%21/CESGA_> | CESGA on
>>>>> Facebook <
>>>>> https://www.facebook.com/profile.php?id=100000707537407>)
>>>>>
>>>>> Avenida de Vigo, s/n (Campus Vida)
>>>>> 15705 Santiago de Compostela - SPAIN
>>>>>
>>>>> E-mail: [email protected]
>>>>> Cell: +34 981 56 98 10 (ext. 237)
>>>>> Fax: +34 981 59 46 16
>>>>> Web: https://www.cesga.es/
>>>>> ====================================================
>>>>> [IMPORTANTE] La información contenida en este mensaje y
>>>>> sus posibles documentos adjuntos es privada y confidencial
>>>>> y está dirigida únicamente a su destinatario/a. Si usted no
>>>>> es el/la destinatario/a original de este mensaje, por favor
>>>>> elimínelo. La distribución o copia de este mensaje no está
>>>>> autorizada.
>>>>>
>>>>>
>>>>>
>>>> --
>>> ====================================================
>>> Natalia Costas Lago
>>> Senior Communications Technician
>>> Galicia Supercomputing Centre (CESGA)
>>> (CESGA on Twitter <https://twitter.com/#%21/CESGA_> | CESGA on
>>> Facebook <
>>> https://www.facebook.com/profile.php?id=100000707537407>)
>>>
>>> Avenida de Vigo, s/n (Campus Vida)
>>> 15705 Santiago de Compostela - SPAIN
>>>
>>> E-mail: [email protected]
>>> Cell: +34 981 56 98 10 (ext. 237)
>>> Fax: +34 981 59 46 16
>>> Web: https://www.cesga.es/
>>> ====================================================
>>> [IMPORTANTE] La información contenida en este mensaje y
>>> sus posibles documentos adjuntos es privada y confidencial
>>> y está dirigida únicamente a su destinatario/a. Si usted no
>>> es el/la destinatario/a original de este mensaje, por favor
>>> elimínelo. La distribución o copia de este mensaje no está
>>> autorizada.
>>>
>>>
>>
>>
>
> --
> ====================================================
> Natalia Costas Lago
> Senior Communications Technician
> Galicia Supercomputing Centre (CESGA)
> (CESGA on Twitter <https://twitter.com/#%21/CESGA_> | CESGA on Facebook
<
> https://www.facebook.com/profile.php?id=100000707537407>)
>
> Avenida de Vigo, s/n (Campus Vida)
> 15705 Santiago de Compostela - SPAIN
>
> E-mail: [email protected]
> Cell: +34 981 56 98 10 (ext. 237)
> Fax: +34 981 59 46 16
> Web: https://www.cesga.es/
> ====================================================
> [IMPORTANTE] La información contenida en este mensaje y
> sus posibles documentos adjuntos es privada y confidencial
> y está dirigida únicamente a su destinatario/a. Si usted no
> es el/la destinatario/a original de este mensaje, por favor
> elimínelo. La distribución o copia de este mensaje no está
> autorizada.
>
--
Rafael Weingärtner
--
====================================================
Natalia Costas Lago
Senior Communications Technician
Galicia Supercomputing Centre (CESGA)
(CESGA on Twitter <https://twitter.com/#%21/CESGA_> | CESGA on Facebook
<https://www.facebook.com/profile.php?id=100000707537407>)
Avenida de Vigo, s/n (Campus Vida)
15705 Santiago de Compostela - SPAIN
E-mail: [email protected]
Cell: +34 981 56 98 10 (ext. 237)
Fax: +34 981 59 46 16
Web: https://www.cesga.es/
====================================================
[IMPORTANTE] La información contenida en este mensaje y
sus posibles documentos adjuntos es privada y confidencial
y está dirigida únicamente a su destinatario/a. Si usted no
es el/la destinatario/a original de este mensaje, por favor
elimínelo. La distribución o copia de este mensaje no está
autorizada.
