Thanks Will! yes the repo is pointing to 4.9 release for all KVMs and for the management server: /cloudstack:~# cat /etc/apt/sources.list.d/cloudstack.list // //deb http://cloudstack.apt-get.eu/ubuntu trusty 4.9/
All KVM agents and the mgmt server are upgraded to release 4.9 based on the documentation.We have restarted all the cloudstack-agents and the cloudstack-management service as well. Network traces are showing packets from KVM <-> Mgmt on port 8250. there is no security device in between. thanks fanfarlo On 08/24/2016 04:13 PM, Will Stevens wrote: > @rohit, I am guessing they should be installing the cloudstack-agent using > the following repo right? That is what is described in the upgrade (trusty > instead of precise though). > > http://cloudstack.apt-get.eu/ubuntu/dists/trusty/4.9/ > > @fanfarlo, are your repo's setup to point to the new 4.9 version? > > cheers, > > will > > On Wed, Aug 24, 2016 at 9:46 AM, Rohit Yadav <rohit.ya...@shapeblue.com> > wrote: > >> The PR and fix already exists in 4.9.0 release. Please make sure to >> upgrade all of your management server(s) and KVM agents and then also >> restart them after the upgrade. >> >> >> If you are seeing SSL handshake failures, it could be due to network or >> security issue and most likely due to mismatch between CloudStack mgmt >> server and KVM agent version. >> >> >> Regards. >> >> rohit.ya...@shapeblue.com >> www.shapeblue.com >> @shapeblue >> >> >> >> ------------------------------ >> *From:* Will Stevens <williamstev...@gmail.com> >> *Sent:* 24 August 2016 18:17:17 >> *To:* dev@cloudstack.apache.org; Rohit Yadav >> *Subject:* Re: CS 4.9 NIO Selector wait time PR-1601 >> >> >> That PR is already merged, so you don't have to do anything to get that >> code, you already have it. >> >> @rohit, can you review this? I think this is a similar to the issue Simon >> reported earlier. >> >> Will >> >> On Aug 24, 2016 6:56 AM, "fanfarlo" <fanfar...@gmail.com> wrote: >> >>> hi all >>> >>> We have the following environment: >>> - OS: Debian 14.04 (hypervisors and management) >>> - 4 KVM Hosts >>> - Cloudstack Release 4.9 with local database >>> >>> Since we upgraded to Release 4.9 the KVM hosts no longer connect to the >>> management Server. Upgrade procedure was followed as described: >>> http://docs.cloudstack.apache.org/projects/cloudstack-releas >>> e-notes/en/4.9.0/upgrade/upgrade-4.8.html >>> >>> >>> On the KVM hosts we have the following error message: >>> /2016-08-24 10:42:49,678 INFO [utils.exception.CSExceptionErrorCode] >>> (main:null) (logid:) Could not find exception: >>> com.cloud.utils.exception.NioConnectionException in error code list for >>> exceptions >>> 2016-08-24 10:42:49,678 WARN [cloud.agent.Agent] (main:null) (logid:) >>> NIO Connection Exception >>> com.cloud.utils.exception.NioConnectionException: SSL Handshake failed >>> while connecting to host: 10.100.12.10 port: 8250 >>> 2016-08-24 10:42:49,678 INFO [cloud.agent.Agent] (main:null) (logid:) >>> Attempted to connect to the server, but received an unexpected >>> exception, trying again... >>> 2016-08-24 10:42:54,679 INFO [utils.nio.NioClient] (main:null) (logid:) >>> Connecting to 10.100.12.10:8250 >>> 2016-08-24 10:42:54,684 WARN [utils.nio.Link] (main:null) (logid:) This >>> SSL engine was forced to close inbound due to end of stream. >>> 2016-08-24 10:42:54,684 ERROR [utils.nio.NioClient] (main:null) (logid:) >>> SSL Handshake failed while connecting to host: 10.100.12.10 port: 8250 >>> 2016-08-24 10:42:54,685 ERROR [utils.nio.NioConnection] (main:null) >>> (logid:) Unable to initialize the threads. >>> java.io.IOException: SSL Handshake failed while connecting to host: >>> 10.100.12.10 port: 8250 >>> at com.cloud.utils.nio.NioClient.init(NioClient.java:67) >>> at com.cloud.utils.nio.NioConnection.start(NioConnection.java:88) >>> at com.cloud.agent.Agent.start(Agent.java:237) >>> at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:399) >>> at >>> com.cloud.agent.AgentShell.launchAgentFromClassInfo(AgentShell.java:367) >>> at com.cloud.agent.AgentShell.launchAgent(AgentShell.java:351) >>> at com.cloud.agent.AgentShell.start(AgentShell.java:456) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >>> ssorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >>> thodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at >>> org.apache.commons.daemon.support.DaemonLoader.start(DaemonL >>> oader.java:243) >>> 2016-08-24 10:42:54,685 INFO [utils.exception.CSExceptionErrorCode] >>> (main:null) (logid:) Could not find exception: >>> com.cloud.utils.exception.NioConnectionException in error code list for >>> exceptions >>> 2016-08-24 10:42:54,685 WARN [cloud.agent.Agent] (main:null) (logid:) >>> NIO Connection Exception >>> com.cloud.utils.exception.NioConnectionException: SSL Handshake failed >>> while connecting to host: 10.100.12.10 port: 8250 >>> 2016-08-24 10:42:54,686 INFO [cloud.agent.Agent] (main:null) (logid:) >>> Attempted to connect to the server, but received an unexpected >>> exception, trying again.../ >>> >>> >>> Port is open on the management server, there is no firewall in between. >>> We found that there was a bug report here: >>> https://issues.apache.org/jira/browse/CLOUDSTACK-9348. There is a PR >>> changing the NIO Selector wait time: >>> https://github.com/apache/cloudstack/pull/1601 which was merged into >>> master branch. >>> >>> Since we installed Release 4.9 we probably need to patch the >>> NioConection.class as described in PR1601 , right? >>> >>> kvm03# unzip -v /usr/share/cloudstack-agent/lib/cloud-utils-4.9.0.jar | >>> grep NioConnection >>> 3923 Defl:N 1778 55% 2016-08-02 09:28 05aaf7d5 >>> com/cloud/utils/nio/NioConnection$1.class >>> 881 Defl:N 495 44% 2016-08-02 09:28 e378984c >>> com/cloud/utils/nio/NioConnection$ChangeRequest.class >>> 15410 Defl:N 7130 54% 2016-08-02 09:28 b3281f5a >>> com/cloud/utils/nio/NioConnection.class >>> 1134 Defl:N 584 49% 2016-08-02 09:28 8d5cb4a8 >>> com/cloud/utils/exception/NioConnectionException.class >>> >>> Due to a lack of java expertise we have some basic questions: >>> - Is there a patched jar file available ? public build server? >>> - Do we need to create the jar from sources ? procedure? >>> - How do we apply the patch ? >>> >>> many thanks! >>> fanfarlo >>>
