Hi Nicolas,
Did you deploy multiple managements at the same time? When you deploy multiple management server(s), wait for the first management server to initialize database where it sets up some default offerings, global settings and the root CA keypair and certificate. Only when you see the first management server's UI in browser, proceed with deployment of other management server(s). For your environment, you can test this workaround and let me know if that works for you: 1. Shutdown all the management server(s). 2. Delete ca keypair and cert: delete from configuration where name like "ca.plugin.root.private.key"; delete from configuration where name like "ca.plugin.root.public.key"; delete from configuration where name="ca.plugin.root.ca.certificate"; 3. Start one management server and wait for it to complete internal setup, until you see the UI. 4. Start all the other management server(s). - Rohit <https://cloudstack.apache.org> ________________________________ From: Nicolas Bouige <[email protected]> Sent: Monday, April 30, 2018 2:59:29 PM To: [email protected] Subject: certificate issue second mgmt-server Hello All, I have an issue with one of my Cloudstack mgmt-server (4.11) The second node has been deployed with the command "cloudstack-setup-databases cloud:dbpassword@dbhost" i didnt have any problem during few days and now sometimes i got an error on web GUI when i perfom some basic task, the error is "Resource [Host:1] is unreachable: Host 1: Unable to reach the peer that the agent is connected" After a quick investigation, i had to stop cloudstack-management service from second mgmt-server and i noticed a lot of messages related with ca-certificate used by cloudstack : 2018-04-27 11:18:24,076 ERROR [c.c.u.n.Link] (StatsCollector-1:ctx-82335701) (logid:95fda6d7) SSL error caught during unwrap data: Received fatal alert: certificate_unknown, for local address=/172.16.22.61:60128, remote address=/172.16.22.60:8250. The client may have invalid ca-certificates. 2018-04-27 11:18:24,076 WARN [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-82335701) (logid:95fda6d7) Unable to connect to peer management server: 130719784044197, ip: 172.16.22.60 due to SSL: Fail to init SSL! java.io.IOException: SSL: Handshake failed with peer management server '130719784044197' on 172.16.22.60:8250 java.io.IOException: SSL: Fail to init SSL! java.io.IOException: SSL: Handshake failed with peer management server '130719784044197' on 172.16.22.60:8250 at com.cloud.agent.manager.ClusteredAgentManagerImpl.connectToPeer(ClusteredAgentManagerImpl.java:529) at com.cloud.agent.manager.ClusteredAgentAttache.send(ClusteredAgentAttache.java:177) at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:398) at com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:456) at com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:362) at com.cloud.agent.manager.AgentManagerImpl.easySend(AgentManagerImpl.java:954) at com.cloud.resource.ResourceManagerImpl.getHostStatistics(ResourceManagerImpl.java:2645) at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) at com.sun.proxy.$Proxy178.getHostStatistics(Unknown Source) at com.cloud.server.StatsCollector$HostCollector.runInContext(StatsCollector.java:438) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-04-27 11:18:24,077 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-1:ctx-82335701) (logid:95fda6d7) Seq 9-9075597674081682614: Unable to forward null 2018-04-27 11:18:24,177 ERROR [c.c.u.n.Link] (StatsCollector-1:ctx-82335701) (logid:95fda6d7) SSL error caught during unwrap data: Received fatal alert: certificate_unknown, for local address=/172.16.22.61:60130, remote address=/172.16.22.60:8250. The client may have invalid ca-certificates. 2018-04-27 11:18:24,177 WARN [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-82335701) (logid:95fda6d7) Unable to connect to peer management server: 130719784044197, ip: 172.16.22.60 due to SSL: Fail to init SSL! java.io.IOException: SSL: Handshake failed with peer management server '130719784044197' on 172.16.22.60:8250 Im not familiar with the using of self-signed certificate in cloudstack, do you know where i can find out more information to investigate deeper ? or if you have any idea ? I tried to check keystore on both mgmt-server but i need a password i havnt... Thanks upfront, Have a nice day, Best regards, Nicolas Bouige DIMSI cloud.dimsi.fr<http://www.cloud.dimsi.fr> 4, avenue Laurent Cely Tour d’Asnière – 92600 Asnière sur Seine T/ +33 (0)6 28 98 53 40 [email protected] www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
