Hi,
Is there a reason you're testing 4.2 and not 4.8? 4.2 was released in mid 2013, so is very old. - Si ________________________________ From: Cloud List <cloud-l...@sg.or.id> Sent: Friday, June 17, 2016 9:27 PM To: us...@cloudstack.apache.org; dev@cloudstack.apache.org Subject: ERROR: Java process not running. Try restarting the SSVM. Dear all, I am running ACS 4.2.0 in our test environment and after restarting the KVM host, I am not able to get SSVM agent up and running, despite destroying and recreating it. The problem is similar to what Mike Tutkowsi reported on below thread, same symptoms but different cloud.out stack error message: http://comments.gmane.org/gmane.comp.apache.cloudstack.devel/28827 I can SSH into SSVM but unable to start the Java agent. SSVM health check result shows two error: - Unable to connect NFS (wrong IP detected); and - Unable to start Java. Unable to connect NFS is because wrong IP is detected for the NFS ip. I can manually mount the NFS server IP using correct IP. Result: ==== root@s-121-VM:/var/log/cloud# /usr/local/cloud/systemvm/ssvm-check.sh ================================================ First DNS server is 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=0 ttl=49 time=4.572 ms 64 bytes from 8.8.8.8: icmp_seq=1 ttl=49 time=4.365 ms --- 8.8.8.8 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 4.365/4.468/4.572/0.104 ms Good: Can ping DNS server ================================================ Good: DNS resolves download.cloud.com ================================================ ERROR: NFS is not currently mounted Try manually mounting from inside the VM NFS server is 169.254.0.127 PING 169.254.0.127 (169.254.0.127): 56 data bytes 64 bytes from 169.254.0.127: icmp_seq=0 ttl=64 time=0.106 ms 64 bytes from 169.254.0.127: icmp_seq=1 ttl=64 time=0.040 ms --- 169.254.0.127 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.040/0.073/0.106/0.033 ms Good: Can ping NFS server ================================================ Management server is 192.168.0.222. Checking connectivity. Good: Can connect to management server port 8250 ================================================ ERROR: Java process not running. Try restarting the SSVM. ==== Repeated error shown on /var/log/cloud/cloud.out: ==== + java -Djavax.net.ssl.trustStore=./certs/realhostip.keystore -Dlog.home=/var/log/cloud/ -mx-24m -cp ./:./conf:activation-1.1.jar:antisamy-1.4.3.jar:aopalliance-1.0.jar:apache-log4j-extras-1.1.jar:aspectjrt-1.7.1.jar:aspectjweaver-1.7.1.jar:aws-java-sdk-1.3.22.jar:backport-util-concurrent-3.1.jar:batik-css-1.7.jar:batik-ext-1.7.jar:batik-util-1.7.jar:bcprov-jdk16-1.46.jar:bsh-core-2.0b4.jar:c3p0-0.9.1.1.jar:cglib-nodep-2.2.2.jar:cloud-agent-4.2.0.jar:cloud-api-4.2.0.jar:cloud-console-proxy-4.2.0.jar:cloud-core-4.2.0.jar:cloud-engine-api-4.2.0.jar:cloud-engine-components-api-4.2.0.jar:cloud-engine-schema-4.2.0.jar:cloud-framework-events-4.2.0.jar:cloud-framework-ipc-4.2.0.jar:cloud-framework-jobs-4.2.0.jar:cloud-framework-rest-4.2.0.jar:cloud-secondary-storage-4.2.0.jar:cloud-server-4.2.0.jar:cloud-utils-4.2.0.jar:commons-beanutils-core-1.7.0.jar:commons-codec-1.6.jar:commons-collections-3.2.1.jar:commons-configuration-1.8.jar:commons-daemon-1.0.10.jar:commons-dbcp-1.4.jar:commons-discovery-0.5.jar:commons-fileupload-1.2.jar:commons-httpclient-3.1.jar:commons-io-1.4.jar:commons-lang-2.6.jar:commons-logging-1.1.1.jar:commons-pool-1.6.jar:cxf-bundle-jaxrs-2.7.0.jar:dom4j-1.6.1.jar:ehcache-1.5.0.jar:ejb-api-3.0.jar:esapi-2.0.1.jar:geronimo-javamail_1.4_spec-1.7.1.jar:geronimo-servlet_3.0_spec-1.0.jar:gson-1.7.1.jar:guava-11.0.2.jar:httpclient-4.2.1.jar:httpcore-4.2.1.jar:jackson-annotations-2.1.1.jar:jackson-core-2.1.1.jar:jackson-core-asl-1.8.9.jar:jackson-databind-2.1.1.jar:jackson-jaxrs-json-provider-2.1.1.jar:jackson-mapper-asl-1.8.9.jar:jackson-module-jaxb-annotations-2.1.1.jar:jasypt-1.9.0.jar:java-ipv6-0.10.jar:javassist-3.12.1.GA.jar:javax.inject-1.jar:javax.persistence-2.0.0.jar:javax.ws.rs-api-2.0-m10.jar:jaxen-1.1-beta-8.jar:jdom-1.0.jar:jetty-continuation-8.1.7.v20120910.jar:jetty-http-8.1.7.v20120910.jar:jetty-io-8.1.7.v20120910.jar:jetty-security-8.1.7.v20120910.jar:jetty-util-8.1.7.v20120910.jar:jsch-0.1.42.jar:jsr107cache-1.0.jar:jsr305-1.3.9.jar:jstl-1.2.jar:log4j-1.2.16.jar:mail-1.4.jar:nekohtml-1.9.12.jar:quartz-2.1.6.jar:reflections-0.9.8.jar:slf4j-api-1.6.2.jar:spring-aop-3.1.2.RELEASE.jar:spring-asm-3.1.2.RELEASE.jar:spring-beans-3.1.2.RELEASE.jar:spring-context-3.1.2.RELEASE.jar:spring-core-3.1.2.RELEASE.jar:spring-expression-3.1.2.RELEASE.jar:spring-web-3.1.2.RELEASE.jar:stax2-api-3.1.1.jar:trilead-ssh2-build213-svnkit-1.3-patch.jar:woodstox-core-asl-4.1.4.jar:wsdl4j-1.6.2.jar:xalan-2.7.0.jar:xercesImpl-2.6.2.jar:xml-apis-1.0.b2.jar:xml-apis-ext-1.3.04.jar:xmlParserAPIs-2.6.2.jar:xmlschema-core-2.0.3.jar:xom-1.1.jar:xpp3_min-1.1.4c.jar:xstream-1.3.1.jar com.cloud.agent.AgentShell template=domP type=secstorage host=192.168.0.222 port=8250 name=s-121-VM zone=1 pod=1 guid=s-121-VM resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource instance=SecStorage sslcopy=true role=templateProcessor mtu=1500 eth2ip=192.168.5.170 eth2mask=255.255.255.0 gateway=192.168.5.1 eth0ip=169.254.0.127 eth0mask=255.255.0.0 eth1ip=192.168.0.154 eth1mask=255.255.255.0 mgmtcidr=192.168.0.0/24 localgw=192.168.0.1 private.network.device=eth1 eth3ip=192.168.0.160 eth3mask=255.255.255.0 storageip=192.168.0.160 storagenetmask=255.255.255.0 storagegateway=192.168.0.1 internaldns1=8.8.8.8 dns1=8.8.8.8 dns2=8.8.4.4 Invalid maximum heap size: -Xmx-24m Could not create the Java virtual machine. ==== I did some research and found that this could be due to the use of mixed OSS and non-OSS package, but I installed both the management and agent from the OSS package. There's also mentioned that it could be due to systemvm.iso file, I have tried to re-register the system VM template using cloud-install-sys-tmplt, delete and restore the systemvm.iso file from /usr/share/cloudstack-common/vms folder, but none helped. The strange thing is that it was working before, I only restarted the KVM host and now the problem happened. I even tried to backup the DB and do a clean reinstallation of ACS 4.2.0 from scratch, and the problem still persists! Any pointer is highly appreciated. Thank you. -ip-
