After 3 days we managed to find the problem. It has been fixed and pushed towards master. KVM is back on business!
Details here: https://github.com/apache/cloudstack/pull/612 Cheers, Wilder On 20 Jul 2015, at 15:51, Wilder Rodrigues <wrodrig...@schubergphilis.com<mailto:wrodrig...@schubergphilis.com>> wrote: Hi Wido, I’m doing a regression test on it in order to find out when it broke. I started getting my PR from 27th May, which introduced the big Libvirt refactor. It works fine… like a charm: [root@kvm1 ~]# ls -lart ~/.ssh/ total 16 dr-xr-x---. 3 root root 4096 Jul 20 09:35 .. drwx------. 2 root root 4096 Jul 20 09:35 . -rw-r--r--. 1 root root 389 Jul 20 09:35 id_rsa.pub.cloud -rw-------. 1 root root 1674 Jul 20 09:35 id_rsa.cloud [root@kvm1 ~]# ssh -i ~/.ssh/id_rsa.cloud -p 3922 169.254.0.191 The authenticity of host '[169.254.0.191]:3922 ([169.254.0.191]:3922)' can't be established. ECDSA key fingerprint is 0a:36:fe:c4:08:ce:2b:46:47:22:ee:f4:1a:fc:e2:88. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '[169.254.0.191]:3922' (ECDSA) to the list of known hosts. Linux s-1-VM 3.2.0-4-amd64 #1 SMP Debian 3.2.68-1+deb7u2 x86_64 The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. Last login: Sun Jul 19 23:19:41 2015 from 10.0.2.2 The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. root@s-1-VM:~# I will now move on to a PR that was merged 25 days ago. I will keep you in the loop. Btw, I’m taking my PRs first just as a sanity check, because I do remember using KVM just fine about 3 weeks ago. Hope I find the root cause today. Cheers, Wilder On 17 Jul 2015, at 17:43, Wilder Rodrigues <wrodrig...@schubergphilis.com<mailto:wrodrig...@schubergphilis.com><mailto:wrodrig...@schubergphilis.com>> wrote: Could you please ask him, Wido? I will look into it again tomorrow and get it fixed! Thanks for the reply! Cheers, Wilder Sent from my iPhone On 17 Jul 2015, at 15:19, Wido den Hollander <w...@widodh.nl<mailto:w...@widodh.nl><mailto:w...@widodh.nl>> wrote: On 17-07-15 13:53, Wilder Rodrigues wrote: Hi again, I just cleaned up the whole KVM host, also removing the .ssh/ dir contents and deployed a new DC. The private key is not created anymore, only the pub key: [root@kvm1 ~]# ls -lart .ssh/ total 8 dr-xr-x---. 4 root root 4096 Jul 17 06:08 .. drwx------. 2 root root 4096 Jul 17 07:38 . -rw-r--r--. 1 root root 0 Jul 17 07:38 id_rsa.pub.cloud [root@kvm1 ~]# ssh -i ~/.ssh/id_rsa.pub.cloud -p 3922 169.254.0.100 The authenticity of host '[169.254.0.100]:3922 ([169.254.0.100]:3922)' can't be established. ECDSA key fingerprint is 81:be:00:fe:37:8d:3f:99:63:1d:e2:ff:3f:4b:56:73. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '[169.254.0.100]:3922' (ECDSA) to the list of known hosts. @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Permissions 0644 for '/root/.ssh/id_rsa.pub.cloud' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. bad permissions: ignore key: /root/.ssh/id_rsa.pub.cloud Permission denied (publickey). [root@kvm1 ~]# Any thoughts? No, not really. I do know that my colleague Boris faced the same with deploying from master. Don't know if he actually got it fixed. Wido Cheers, Wilder On 17 Jul 2015, at 13:33, Wilder Rodrigues <wrodrig...@schubergphilis.com<mailto:wrodrig...@schubergphilis.com><mailto:wrodrig...@schubergphilis.com><mailto:wrodrig...@schubergphilis.com>> wrote: Hi all, I’m nt able to use the id_rsa.pub.cloud on KVM hosts. See snippet bellow: [root@kvm1 ~]# ssh -i ~/.ssh/id_rsa.pub.cloud -p 3922 169.254.0.136 The authenticity of host '[169.254.0.136]:3922 ([169.254.0.136]:3922)' can't be established. ECDSA key fingerprint is 81:be:00:fe:37:8d:3f:99:63:1d:e2:ff:3f:4b:56:73. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '[169.254.0.136]:3922' (ECDSA) to the list of known hosts. @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Permissions 0644 for '/root/.ssh/id_rsa.pub.cloud' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. bad permissions: ignore key: /root/.ssh/id_rsa.pub.cloud Permission denied (publickey). [root@kvm1 ~]# chmod 600 /root/.ssh/id_rsa.pub.cloud [root@kvm1 ~]# ssh -i ~/.ssh/id_rsa.pub.cloud -p 3922 169.254.0.136 Enter passphrase for key '/root/.ssh/id_rsa.pub.cloud': Permission denied (publickey). It was working fine few days ago. The injectkeys.py did not change, although it says 644, but for the private key. print ("Copying new private key file as it is not matching with old file") shutil.copyfile(newKey, currDir + pathSep + "id_rsa.cloud") os.chmod(currDir + pathSep + "id_rsa.cloud", 0644) <<<<<<============== I think we should change it to 600 On XenServer, the patch files set the key permission to 600, instead: id_rsa.cloud=../../../systemvm,0600,/root/.ssh On LibvirtModifySshKeyCommandWrapper it sets the public key to 600 final Script script = new Script("chmod", libvirtComputingResource.getTimeout(), s_logger); script.add("600", sshprvkeypath); script.execute(); Have you guys seen something like that before? Cheers, Wilder
