It looks like a LXC resource agent bug. LXC resource agent needs to respond VNC command correctly in order to get console work on LXC.
Could you file a bug against LXC resource agent component? Kelven On 10/29/13, 7:13 AM, "Francois Gaudreault" <fgaudrea...@cloudops.com> wrote: >Kelven, > >This is what I get in the LXC host agent log when I try to check the >console from CS: > >2013-10-29 10:07:42,079 WARN [cloud.agent.Agent] >(agentRequest-Handler-4:null) Caught: >java.lang.NullPointerException > at >com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(Libvirt >ComputingResource.java:2711) > at >com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest( >LibvirtComputingResource.java:1216) > at com.cloud.agent.Agent.processRequest(Agent.java:525) > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) > at com.cloud.utils.nio.Task.run(Task.java:83) > at >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: >1145) > at >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java >:615) > at java.lang.Thread.run(Thread.java:744) > >This is the error on the CS side: > >2013-10-29 10:12:58,215 DEBUG [agent.manager.AgentManagerImpl] >(catalina-exec-16:null) Details from executing class >com.cloud.agent.api.GetVncPortCommand: java.lang.NullPointerException > at >com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(Libvirt >ComputingResource.java:2711) > at >com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest( >LibvirtComputingResource.java:1216) > at com.cloud.agent.Agent.processRequest(Agent.java:525) > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) > at com.cloud.utils.nio.Task.run(Task.java:83) > at >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: >1145) > at >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java >:615) > at java.lang.Thread.run(Thread.java:744) > >2013-10-29 10:12:58,215 ERROR [cloud.servlet.ConsoleProxyServlet] >(catalina-exec-16:null) Unexepected exception in ConsoleProxyServlet >java.lang.ClassCastException: com.cloud.agent.api.Answer cannot be cast >to com.cloud.agent.api.GetVncPortAnswer > at >com.cloud.server.ManagementServerImpl.getVncPort(ManagementServerImpl.java >:2193) > at >com.cloud.servlet.ConsoleProxyServlet.composeConsoleAccessUrl(ConsoleProxy >Servlet.java:381) > at >com.cloud.servlet.ConsoleProxyServlet.handleAccessRequest(ConsoleProxyServ >let.java:269) > at >com.cloud.servlet.ConsoleProxyServlet.doGet(ConsoleProxyServlet.java:171) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) > at >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicati >onFilterChain.java:290) > at >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilter >Chain.java:206) > at >org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve. >java:233) > at >org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve. >java:191) > at >org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:1 >27) > at >org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:1 >02) > at >org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) > at >org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.ja >va:109) > at >org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298 >) > at >org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.jav >a:889) > at >org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process >(Http11NioProtocol.java:721) > at >org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.jav >a:2268) > at >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: >1146) > at >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java >:615) > at java.lang.Thread.run(Thread.java:679) > > >On 10/28/2013, 9:05 PM, Kelven Yang wrote: >> As long as the container VNC console is reachable from console proxy >> agent, I don't see a technical issue here. Need some log details to >> understand the situation. For example, what information does LXC >> hypervisor resource agent returns on GetVncPort command? >> >> Kelven >> >> >> On 10/28/13 10:59 AM, "Francois Gaudreault" <fgaudrea...@cloudops.com> >> wrote: >> >>> Thanks! Is the console proxy issue will be fixed at some point in the >>> future or this won't be possible? >>> >>> Francois >>> >>> On 10/28/2013, 1:57 PM, Phong Nguyen wrote: >>>> Yes, the console proxy VM is unable to connect to the LXC console. Use >>>> virsh to connect from the host. >>>> >>>> virsh -c lxc:/// console i-2-20-VM >>>> >>>> -Phong >>>> >>>> >>>> On Mon, Oct 28, 2013 at 1:22 PM, Francois Gaudreault >>>> <fgaudrea...@cloudops.com <mailto:fgaudrea...@cloudops.com>> wrote: >>>> >>>> Ok, I followed Chiradeep's comment here, and it now works fine. I >>>> have CentOS 6 containers running. >>>> >>>> My only other questions would be: >>>> - Is it normal that the console proxy VM cannot proxy the >>>> container's console? >>>> - How can I access the container console from the host? (I am on >>>> CentOS) I can ssh to the container tho. >>>> >>>> Thanks! >>>> >>>> Francois >>>> >>>> >>>> On 10/24/2013, 7:00 PM, Chiradeep Vittal wrote: >>>> >>>> So here's what should work >>>> Create zone >>>> Add a KVM cluster -> add a KVM host -> wait for systemvms to >>>> start >>>> Add a LXC cluster -> add a LXC host >>>> >>>> On 10/24/13 9:55 AM, "Francois Gaudreault" >>>> <fgaudrea...@cloudops.com <mailto:fgaudrea...@cloudops.com>> >>>> wrote: >>>> >>>> If it's designed to do that, then something is wrong with >>>> how CS deals >>>> with it. >>>> >>>> When I was trying to get the KVM images to work, they >>>>were >>>> starting, >>>> getting IPs, but then something was killing the VM. I >>>> though for >>>> sometime that libvirt was the issue, so I tried Ubuntu >>>> 13.10, 12.04 and >>>> CentOS with the same results. I then switched the >>>> hypervisor type in CS >>>> >>>> >from LXC to KVM (rebuilt the zone), keep the same settings >>>>on >>>> my host, >>>> >>>> and the System VMs are running fine since then. >>>> >>>> Anyone have time to help me troubleshoot? I mean, this is >>>> not a blocker, >>>> but I can't get standalone LXC cluster to work... >>>> >>>> Francois >>>> >>>> On 10/24/2013, 11:53 AM, Phong Nguyen wrote: >>>> >>>> So we need a KVM cluster to run the VMs? (Added >>>> the author of the >>>> >>>> feature) >>>> >>>> As it was originally discussed and implemented, the >>>> decision was to >>>> use KVM as the system VM for LXC clusters instead of >>>> creating an LXC >>>> system VM. A zone with only LXC clusters will deploy >>>>a >>>> KVM system VM >>>> on a host running an LXC agent. Behind the scenes, >>>> this is possible >>>> because both KVM and LXC agents use libvirt for >>>> provisioning (and that >>>> the setup of an LXC agent is almost identical to KVM >>>> and perfectly >>>> capable of running KVM VMs). >>>> >>>> -Phong >>>> >>>> >>>> On Thu, Oct 24, 2013 at 8:57 AM, Francois Gaudreault >>>> <fgaudrea...@cloudops.com >>>> <mailto:fgaudrea...@cloudops.com> >>>> <mailto:fgaudrea...@cloudops.com >>>> <mailto:fgaudrea...@cloudops.com>>> wrote: >>>> >>>> If this is the case, then you should remove the >>>> ability to create >>>> LXC zones or clarify the documentation about >>>>that. >>>> >>>> According to the wiki page: >>>> >>>> Each of the different hypervisors currently have >>>> their own System >>>> VMs. These system VM images are used to run a >>>> console proxy, >>>> secondary storage, and router VMs. >>>> >>>> We discussed the possibility of creating System >>>> VMs for LXC. There >>>> was concern with the complexity and potential >>>> issues involving >>>> iptables for the router inside an LXC container. >>>> As an >>>> intermediate solution we are going to use KVM >>>> System VMs inside >>>> the LXC Cluster. >>>> >>>> So we need a KVM cluster to run the VMs? (Added >>>> the author of the >>>> feature) >>>> >>>> Francois >>>> >>>> On 10/22/2013, 1:24 AM, Chiradeep Vittal wrote: >>>> >>>> As far as I understand, in an LXC scenario, >>>> the system vms are >>>> expected to >>>> run on real hypervisors. >>>> You can always use the QuickCloud way to not >>>> use system vms at >>>> all. >>>> >>>> On 10/21/13 1:45 PM, "Francois Gaudreault" >>>> <fgaudrea...@cloudops.com >>>> <mailto:fgaudrea...@cloudops.com> >>>> <mailto:fgaudrea...@cloudops.com >>>> <mailto:fgaudrea...@cloudops.com>>> >>>> wrote: >>>> >>>> Ok I think we have to look at this >>>> further. I'll stop >>>> hijacking other >>>> threads. >>>> >>>> I am trying to get the SSVM/CPVM to run >>>> on a LXC host. The >>>> SSVM/CPVM >>>> starts, get IPs, but then CloudStack >>>>kill >>>> them for some >>>> reason. Yes, I >>>> use the 4.2 images : >>>> >>>> 2013-10-21 16:19:21,605 DEBUG >>>> [agent.manager.AgentManagerImpl] >>>> (AgentManager-Handler-9:null) SeqA >>>>73--1: >>>> Processing Seq >>>> 73--1: { Cmd , >>>> MgmtId: -1, via: 73, Ver: v1, Flags: >>>>111, >>>> >>>> >>>> >>>>[{"com.cloud.agent.api.ShutdownCommand":{"reason":"sig.kill","wait":0}} >>>>] >>>> } >>>> 2013-10-21 16:19:21,605 INFO >>>> [agent.manager.AgentManagerImpl] >>>> (AgentManager-Handler-9:null) Host 73 >>>>has >>>> informed us that >>>> it is >>>> shutting down with reason sig.kill and >>>> detail null >>>> 2013-10-21 16:19:21,606 INFO >>>> [agent.manager.AgentManagerImpl] >>>> (AgentTaskPool-11:null) Host 73 is >>>> disconnecting with event >>>> ShutdownRequested >>>> 2013-10-21 16:19:21,609 DEBUG >>>> [agent.manager.AgentManagerImpl] >>>> (AgentTaskPool-11:null) The next status >>>> of agent 73is >>>> Disconnected, >>>> current status is Up >>>> 2013-10-21 16:19:21,609 DEBUG >>>> [agent.manager.AgentManagerImpl] >>>> (AgentTaskPool-11:null) Deregistering >>>> link for 73 with >>>> state Disconnected >>>> 2013-10-21 16:19:21,609 DEBUG >>>> [agent.manager.AgentManagerImpl] >>>> (AgentTaskPool-11:null) Remove Agent : >>>>73 >>>> 2013-10-21 16:19:21,609 DEBUG >>>> [agent.manager.ConnectedAgentAttache] >>>> (AgentTaskPool-11:null) Processing >>>> Disconnect. >>>> >>>> I transferred the host to KVM, and now >>>> the same SSVM/CPVM >>>> images are >>>> running fine for the last 30min ( so I >>>> assume it works >>>> fine...). >>>> Something seems to be wrong with the LXC >>>> side :S >>>> >>>> Anyone wants to invest some time to >>>> troubleshoot? I'll >>>> open a ticket also. >>>> >>>> -- >>>> Francois Gaudreault >>>> Architecte de Solution Cloud | Cloud >>>> Solutions Architect >>>> fgaudrea...@cloudops.com >>>> <mailto:fgaudrea...@cloudops.com> >>>> <mailto:fgaudrea...@cloudops.com >>>> <mailto:fgaudrea...@cloudops.com>> >>>> 514-629-6775 <tel:514-629-6775> <tel:514-629-6775 >>>> <tel:514-629-6775>> >>>> - - - >>>> CloudOps >>>> 420 rue Guy >>>> Montréal QC H3J 1S6 >>>> www.cloudops.com <http://www.cloudops.com> >>>> <http://www.cloudops.com> >>>> @CloudOps_ >>>> >>>> >>>> >>>> >>>> -- >>>> Francois Gaudreault >>>> Architecte de Solution Cloud | Cloud Solutions >>>> Architect >>>> fgaudrea...@cloudops.com >>>> <mailto:fgaudrea...@cloudops.com> >>>> <mailto:fgaudrea...@cloudops.com >>>> <mailto:fgaudrea...@cloudops.com>> >>>> 514-629-6775 <tel:514-629-6775> <tel:514-629-6775 >>>> <tel:514-629-6775>> >>>> - - - >>>> CloudOps >>>> 420 rue Guy >>>> Montréal QC H3J 1S6 >>>> www.cloudops.com <http://www.cloudops.com> >>>> <http://www.cloudops.com> >>>> @CloudOps_ >>>> >>>> >>>> >>>> -- >>>> Francois Gaudreault >>>> Architecte de Solution Cloud | Cloud Solutions Architect >>>> fgaudrea...@cloudops.com >>>><mailto:fgaudrea...@cloudops.com> >>>> 514-629-6775 <tel:514-629-6775> >>>> - - - >>>> CloudOps >>>> 420 rue Guy >>>> Montréal QC H3J 1S6 >>>> www.cloudops.com <http://www.cloudops.com> >>>> @CloudOps_ >>>> >>>> >>>> >>>> >>>> -- >>>> Francois Gaudreault >>>> Architecte de Solution Cloud | Cloud Solutions Architect >>>> fgaudrea...@cloudops.com <mailto:fgaudrea...@cloudops.com> >>>> 514-629-6775 <tel:514-629-6775> >>>> - - - >>>> CloudOps >>>> 420 rue Guy >>>> Montréal QC H3J 1S6 >>>> www.cloudops.com <http://www.cloudops.com> >>>> @CloudOps_ >>>> >>>> >>> >>> -- >>> Francois Gaudreault >>> Architecte de Solution Cloud | Cloud Solutions Architect >>> fgaudrea...@cloudops.com >>> 514-629-6775 >>> - - - >>> CloudOps >>> 420 rue Guy >>> Montréal QC H3J 1S6 >>> www.cloudops.com >>> @CloudOps_ >>> >> > > >-- >Francois Gaudreault >Architecte de Solution Cloud | Cloud Solutions Architect >fgaudrea...@cloudops.com >514-629-6775 >- - - >CloudOps >420 rue Guy >Montréal QC H3J 1S6 >www.cloudops.com >@CloudOps_ >
