Making some progress. I mounted the VR root and verified that it is indeed version 4.3.
I decided to stop the MS service before it killed the VR so that I could manually try the command. And the problem is the id_rsa.cloud key. So how do I find the correct key and put it on the xenhosts? I tried using the one from /usr/share/cloudstack-common/scripts/vm/systemvm/id_rsa.cloud but it did not work. thanks, _Carlos On Jul 26, 2014, at 9:44 AM, Carlos Reátegui <create...@gmail.com> wrote: > Adding dev list > > Thanks Andrei, although not what I wanted to hear. > > I decided to try upgrading to 4.4 in hopes the results would be different. > Unfortunately not. > > So here is what is happening. I did the upgrade from 4.1.1 with everything > completely stopped. I had previously registered the new 4.3 svm template > which should work for both 4.3 and 4.4. > > The problem is that the VR needs to be upgraded, but it can’t since it is > stopped. But starting it also fails because it does not respond correctly to > /opt/cloud/bin/router_proxy.sh get_template_version.sh. > > So looks like I have 2 options. > - 1) Go back to 4.1.1 and make sure the VR is running before attempting the > upgrade to 4.3/4 > - 2) Mount the VR VHD and try to fix GetDomRVersionCmd so that the VR starts > and I am then able to upgrade it from the UI (since cloudstack-sysvmadm does > not work on ubuntu — CLOUDSTACK-4778) > > Any thoughts on #2? > > thanks, > Carlos > > > > On Jul 26, 2014, at 4:31 AM, Andrei Mikhailovsky <and...@arhont.com> wrote: > >> I had a similar problem with upgrading to 4.3. My cpvm and ssvm would start >> fine, but I was not able to create new VRs. Most of my existing VRs would >> actually start, but no new VRs would get created. So, I was not able to >> create new networks and not able to restart existing networks with the clean >> up option. >> >> No one was able to help me fix the issue, so I ended up moving back to >> 4.2.1. After the downgrade the problem went away. >> >> I do hope these problems would be fixed in 4.3.1. >> >> Cheers >> >> -- >> Andrei Mikhailovsky >> Director >> Arhont Information Security >> >> Web: http://www.arhont.com >> http://www.wi-foo.com >> Tel: +44 (0)870 4431337 >> Fax: +44 (0)208 429 3111 >> PGP: Key ID - 0x2B3438DE >> PGP: Server - keyserver.pgp.com >> >> DISCLAIMER >> >> The information contained in this email is intended only for the use of the >> person(s) to whom it is addressed and may be confidential or contain legally >> privileged information. If you are not the intended recipient you are hereby >> notified that any perusal, use, distribution, copying or disclosure is >> strictly prohibited. If you have received this email in error please >> immediately advise us by return email at and...@arhont.com and delete and >> purge the email and any attachments without making a copy. >> >> >> ----- Original Message ----- >> >> From: "Carlos Reátegui" <create...@gmail.com> >> To: "CloudStack-Users" <us...@cloudstack.apache.org> >> Sent: Saturday, 26 July, 2014 6:49:34 AM >> Subject: VR will not start after upgrade from 4.1.1 to 4.3 >> >> I have tried destroying the VR and restarting it but it won’t go. The SSVM >> and Console VM are up. >> >> The MS does not like the response it is getting from the VR but I am not >> sure how to fix. >> >> Here is the template info: >> mysql> select name, display_text,url,guest_os_name,hypervisor_type,state >> from template_view where display_text='systemvm-xenserver-4.3'; >> +------------------------+------------------------+------------------------------------------------------------------------------------------+----------------------------+-----------------+-------+ >> >> | name | display_text | url | guest_os_name | hypervisor_type | state | >> +------------------------+------------------------+------------------------------------------------------------------------------------------+----------------------------+-----------------+-------+ >> >> | systemvm-xenserver-4.3 | systemvm-xenserver-4.3 | >> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-master-xen.vhd.bz2 >> | Debian GNU/Linux 6(64-bit) | XenServer | Ready | >> +------------------------+------------------------+------------------------------------------------------------------------------------------+----------------------------+-----------------+-------+ >> >> >> >> Here are the relevant logs: >> >> 2014-07-26 10:49:05,137 DEBUG [c.c.h.x.r.CitrixResourceBase] >> (DirectAgent-467:ctx-6cb72049) Trying to connect to 169.254.2.69 attempt 2 >> of 100 >> 2014-07-26 10:49:05,460 DEBUG [c.c.a.ApiServlet] >> (catalina-exec-25:ctx-b6b6006f) ===START=== 10.110.3.50 -- GET >> command=queryAsyncJobResult&jobId=0cfb6b86-dc2e-4040-a06e-f080aee83a15&response=json&sessionkey=TziV%2FqqSUFAO1eiR6bCbh0lq%2FDc%3D&_=1406351949321 >> >> 2014-07-26 10:49:05,480 DEBUG [c.c.a.ApiServlet] >> (catalina-exec-25:ctx-b6b6006f ctx-782d64ad) ===END=== 10.110.3.50 -- GET >> command=queryAsyncJobResult&jobId=0cfb6b86-dc2e-4040-a06e-f080aee83a15&response=json&sessionkey=TziV%2FqqSUFAO1eiR6bCbh0lq%2FDc%3D&_=1406351949321 >> >> 2014-07-26 10:49:07,414 DEBUG [c.c.h.x.r.CitrixResourceBase] >> (DirectAgent-467:ctx-6cb72049) Ping command port succeeded for vm r-98-VM >> 2014-07-26 10:49:08,113 DEBUG [c.c.a.m.DirectAgentAttache] >> (DirectAgent-467:ctx-6cb72049) Seq 3-1423704582: Cancelling because one of >> the answers is false and it is stop on error. >> 2014-07-26 10:49:08,114 DEBUG [c.c.a.m.DirectAgentAttache] >> (DirectAgent-467:ctx-6cb72049) Seq 3-1423704582: Response Received: >> 2014-07-26 10:49:08,116 DEBUG [c.c.a.t.Request] >> (DirectAgent-467:ctx-6cb72049) Seq 3-1423704582: Processing: { Ans: , >> MgmtId: 159090355471825, via: 3, Ver: v1, Flags: 10, >> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":98,"name":"r-98-VM","bootloader":"PyGrub","type":"DomainRouter","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"Debian >> GNU/Linux 6(64-bit)","bootArgs":" template=domP name=r-98-VM >> eth0ip=10.120.60.219 eth0mask=255.255.255.0 gateway=10.120.60.254 >> domain=cs1cloud.internal dhcprange=10.120.60.1 eth1ip=169.254.2.69 >> eth1mask=255.255.0.0 type=dhcpsrvr disable_rp_filter=true dns1=8.8.8.8 >> dns2=202.56.230.6","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"72af77840162bb9e","params":{},"uuid":"a1197b63-760e-4c5c-86d2-401cfd3ac3b4","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"b5b1a6cb-165e-4f7f-97a6-00011a5c7061","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"db74fb7a-1647-3599-b751-57658758d8d1","id":206,"poolType":"NetworkFilesystem","host":"192.168.200.1","path":"/export/primary","port":2049,"url":"NetworkFilesystem://192.168.200.1//export/primary/?ROLE=Primary&STOREUUID=db74fb7a-1647-3599-b751-57658758d8d1"}},"name":"ROOT-98","size":2621440000,"path":"213c90cc-09a7-46e7-80c9-d96dc11afa64","volumeId":160,"vmName":"r-98-VM","accountId":1,"format":"VHD","id":160,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"213c90cc-09a7-46e7-80c9-d96dc11afa64","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"192.168.200.1","volumeSize":"2621440000"}}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"c78beb1c-8a23-43bc-b65f-ebca67b5fad4","ip":"10.120.60.219","netmask":"255.255.255.0","gateway":"10.120.60.254","mac":"06:3e:ca:00:00:3c","dns1":"8.8.8.8","dns2":"202.56.230.6","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://untagged","isolationUri":"ec2://untagged","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"42408069-20d3-4b7d-8722-c328a5f7018c","ip":"169.254.2.69","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:02:45","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"_iqnToPath":{},"result":true,"wait":0}},{"com.cloud.agent.api.check.CheckSshAnswer":{"result":true,"wait":0}},{"com.cloud.agent.api.GetDomRVersionAnswer":{"result":false,"details":"getDomRVersionCmd >> failed","wait":0}}] } >> 2014-07-26 10:49:08,116 DEBUG [c.c.a.t.Request] (Job-Executor-8:ctx-a0bce0e2 >> ctx-28304b43) Seq 3-1423704582: Received: { Ans: , MgmtId: 159090355471825, >> via: 3, Ver: v1, Flags: 10, { StartAnswer, CheckSshAnswer, >> GetDomRVersionAnswer } } >> 2014-07-26 10:49:08,125 WARN [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> (Job-Executor-8:ctx-a0bce0e2 ctx-28304b43) Unable to get the >> template/scripts version of router r-98-VM due to: getDomRVersionCmd failed >> 2014-07-26 10:49:08,125 INFO [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-8:ctx-a0bce0e2 ctx-28304b43) The guru did not like the answers >> so stopping VM[DomainRouter|r-98-VM] >> 2014-07-26 10:49:08,130 DEBUG [c.c.a.t.Request] (Job-Executor-8:ctx-a0bce0e2 >> ctx-28304b43) Seq 3-1423704585: Sending { Cmd , MgmtId: 159090355471825, >> via: 3(noilabxen04), Ver: v1, Flags: 100011, >> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"r-98-VM","wait":0}}] >> } >> 2014-07-26 10:49:08,130 DEBUG [c.c.a.t.Request] (Job-Executor-8:ctx-a0bce0e2 >> ctx-28304b43) Seq 3-1423704585: Executing: { Cmd , MgmtId: 159090355471825, >> via: 3(noilabxen04), Ver: v1, Flags: 100011, >> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"r-98-VM","wait":0}}] >> } >> 2014-07-26 10:49:08,131 DEBUG [c.c.a.m.DirectAgentAttache] >> (DirectAgent-452:ctx-72bd5b30) Seq 3-1423704585: Executing request >> 2014-07-26 10:49:08,433 DEBUG [c.c.h.x.r.CitrixResourceBase] >> (DirectAgent-452:ctx-72bd5b30) 9. The VM r-98-VM is in Stopping state >> 2014-07-26 10:49:09,063 INFO [c.c.h.x.r.CitrixResourceBase] >> (DirectAgent-452:ctx-72bd5b30) Removed network rules for vm r-98-VM >> >> >> thanks, >> Carlos >> >