I faced the same issue, then I logged on to the xenserver host and deleted the tags, restarted the MS but even after that i could see these errors. Finally I rebooted the Xen Host and after that the error was not seen. Filed a bug for the same, https://issues.apache.org/jira/browse/CLOUDSTACK-3377
Thanks and regards, Abhinav -----Original Message----- From: Daan Hoogland [mailto:dhoogl...@schubergphilis.com] Sent: Friday, July 05, 2013 2:56 PM To: dev@cloudstack.apache.org Subject: master not perfect:( H, I get a repetition of the following sequence of logging when running master. Anybody knows what happened? INFO [agent.manager.AgentManagerImpl] (AgentManager-Handler-11:) Connection from /127.0.0.1 closed but no cleanup was done. WARN [utils.nio.Link] (AgentManager-Selector:) SSL: Fail to find the generated keystore. Loading fail-safe one to continue. ERROR [agent.transport.Request] (AgentManager-Handler-13:) Caught problem with [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"1","pod":"1","guid":"s-1-VM-NfsSecondaryStorageResource","name":"s-1-VM","version":"4.2.0-SNAPSHOT","iqn":"NoIqn","publicIpAddress":"195.66.90.212","publicNetmask":"255.255.255.224","publicMacAddress":"06:fb:ca:00:00:01","privateIpAddress":"10.200.23.62","privateMacAddress":"06:ad:58:00:00:08","privateNetmask":"255.255.255.0","storageIpAddress":"10.200.23.62","storageNetmask":"255.255.255.0","storageMacAddress":"06:ad:58:00:00:08","resourceName":"NfsSecondaryStorageResource","contextMap":{},"wait":0}}] com.google.gson.JsonParseException: The JsonDeserializer com.cloud.agent.transport.ArrayTypeAdaptor@7db106d failed to deserialize json object [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"1","pod":"1","guid":"s-1-VM-NfsSecondaryStorageResource","name":"s-1-VM","version":"4.2.0-SNAPSHOT","iqn":"NoIqn","publicIpAddress":"195.66.90.212","publicNetmask":"255.255.255.224","publicMacAddress":"06:fb:ca:00:00:01","privateIpAddress":"10.200.23.62","privateMacAddress":"06:ad:58:00:00:08","privateNetmask":"255.255.255.0","storageIpAddress":"10.200.23.62","storageNetmask":"255.255.255.0","storageMacAddress":"06:ad:58:00:00:08","resourceName":"NfsSecondaryStorageResource","contextMap":{},"wait":0}}] given the type class [Lcom.cloud.agent.api.Command; at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:64) at com.google.gson.JsonDeserializationVisitor.invokeCustomDeserializer(JsonDeserializationVisitor.java:92) at com.google.gson.JsonDeserializationVisitor.visitUsingCustomHandler(JsonDeserializationVisitor.java:80) at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:101) at com.google.gson.JsonDeserializationContextDefault.fromJsonArray(JsonDeserializationContextDefault.java:67) at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:52) at com.google.gson.Gson.fromJson(Gson.java:551) at com.google.gson.Gson.fromJson(Gson.java:498) at com.cloud.agent.transport.Request.getCommands(Request.java:235) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.processRequest(AgentManagerImpl.java:1196) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.doTask(AgentManagerImpl.java:1349) at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:659) at com.cloud.utils.nio.Task.run(Task.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: com.cloud.utils.exception.CloudRuntimeException: can't find StartupSecondaryStorageCommand at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:77) at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:37) at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:51) ... 15 more WARN [utils.nio.Task] (AgentManager-Handler-13:) Caught the following exception but pushing on com.google.gson.JsonParseException: The JsonDeserializer com.cloud.agent.transport.ArrayTypeAdaptor@7db106d failed to deserialize json object [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"1","pod":"1","guid":"s-1-VM-NfsSecondaryStorageResource","name":"s-1-VM","version":"4.2.0-SNAPSHOT","iqn":"NoIqn","publicIpAddress":"195.66.90.212","publicNetmask":"255.255.255.224","publicMacAddress":"06:fb:ca:00:00:01","privateIpAddress":"10.200.23.62","privateMacAddress":"06:ad:58:00:00:08","privateNetmask":"255.255.255.0","storageIpAddress":"10.200.23.62","storageNetmask":"255.255.255.0","storageMacAddress":"06:ad:58:00:00:08","resourceName":"NfsSecondaryStorageResource","contextMap":{},"wait":0}}] given the type class [Lcom.cloud.agent.api.Command; at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:64) at com.google.gson.JsonDeserializationVisitor.invokeCustomDeserializer(JsonDeserializationVisitor.java:92) at com.google.gson.JsonDeserializationVisitor.visitUsingCustomHandler(JsonDeserializationVisitor.java:80) at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:101) at com.google.gson.JsonDeserializationContextDefault.fromJsonArray(JsonDeserializationContextDefault.java:67) at com.google.gson.JsonDeserializationContextDefault.deserialize(JsonDeserializationContextDefault.java:52) at com.google.gson.Gson.fromJson(Gson.java:551) at com.google.gson.Gson.fromJson(Gson.java:498) at com.cloud.agent.transport.Request.getCommands(Request.java:235) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.processRequest(AgentManagerImpl.java:1196) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.doTask(AgentManagerImpl.java:1349) at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:659) at com.cloud.utils.nio.Task.run(Task.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: com.cloud.utils.exception.CloudRuntimeException: can't find StartupSecondaryStorageCommand at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:77) at com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:37) at com.google.gson.JsonDeserializerExceptionWrapper.deserialize(JsonDeserializerExceptionWrapper.java:51) ... 15 more WARN [apache.cloudstack.alerts] (CapacityChecker:) alertType:: 4 // dataCenterId:: 1 // podId:: null // clusterId:: null // message:: System Alert: Number of unallocated virtual network public IPs is low in availability zone MCCDZone