Now have upgraded to opennebula 4.8.0 and still struggling
with successfully launching a Ceph VM. We have worked out
all permissions issues on the client host and in the Ceph datastore
and have gotten to the point where opennebula can deploy
the VM from RBD and the virtual machine starts.. but we get "Geom Error"
on the console of the VM and that is all. Has anyone seen this error
before and have any idea how to deal with it? I presume that
it means that the virtual machine can not even find the boot sector of
the disk that it sees as /dev/vda but I can't find any information
on this error anywhere. Any help is appreciated.
We are trying to replace an old san-gfs file store with Ceph
but need better success than that if this is going to work.
Steve Timm
On Wed, 10 Sep 2014, Steven Timm wrote:
The first and most obvious problem below was that we were running an
old version of qemu-img and qemu-kvm that ships with RHEL6/Centos6/SL6
that doesn't support the "rbd" format. We were able to find
a modified version that the Ceph people had back-ported and now
we can import an image into the datastore and have
gotten as far to getting a deployment.0 written on the hypervisor.
It can't contact Ceph as yet, is getting connection refused but
we think that is an authentication issue.
http://ceph.com/packages/qemu-kvm/redhat/x86_64/
is where these packages for 6.2 live.
Hopefully this all gets easier pretty soon now that RedHat has bought Ceph
and all the right packages will be in RHEL7. Or will they be only
proprietarily available in Redhat Enterprise Virtualization? Has anyone
tried yet?
Steve
On Wed, 10 Sep 2014, Steven Timm wrote:
I have configured a Ceph datastore on one 4.6 and have gotten as
far as to get opennebula to accept the datastore. But when we
try to do the first oneimage create into the datastore we get the
following error in oned.log :
Wed Sep 10 13:10:44 2014 [ImM][I]: Command execution fail:
/var/lib/one/remotes/datastore/ceph/cp
PERTX0RSSVZFUl9BQ1RJT05fREFUQT48SU1BR0U+PElEPjE4PC9JRD48VUlEPjA8L1VJRD48R0lEPjA8L0dJRD48VU5BTUU+b25lYWRtaW48L1VOQU1FPjxHTkFNRT5vbmVhZG1pbjwvR05BTUU+PE5BTUU+Y2VwaHRlc3Q8L05BTUU+PFBFUk1JU1NJT05TPjxPV05FUl9VPjE8L09XTkVSX1U+PE9XTkVSX00+MTwvT1dORVJfTT48T1dORVJfQT4wPC9PV05FUl9BPjxHUk9VUF9VPjA8L0dST1VQX1U+PEdST1VQX00+MDwvR1JPVVBfTT48R1JPVVBfQT4wPC9HUk9VUF9BPjxPVEhFUl9VPjA8L09USEVSX1U+PE9USEVSX00+MDwvT1RIRVJfTT48T1RIRVJfQT4wPC9PVEhFUl9BPjwvUEVSTUlTU0lPTlM+PFRZUEU+MDwvVFlQRT48RElTS19UWVBFPjM8L0RJU0tfVFlQRT48UEVSU0lTVEVOVD4wPC9QRVJTSVNURU5UPjxSRUdUSU1FPjE0MTAzNzI1NTU8L1JFR1RJTUU+PFNPVVJDRT48L1NPVVJDRT48UEFUSD4vY2xvdWQvaW1hZ2VzL3RpbW0vNDBnYi5xY293MjwvUEFUSD48RlNUWVBFPjwvRlNUWVBFPjxTSVpFPjQwOTYwPC9TSVpFPjxTVEFURT40PC9TVEFURT48UlVOTklOR19WTVM+MDwvUlVOTklOR19WTVM+PENMT05JTkdfT1BTPjA8L0NMT05JTkdfT1BTPjxDTE9OSU5HX0lEPi0xPC9DTE9OSU5HX0lEPjxEQVRBU1RPUkVfSUQ+MTAzPC9EQVRBU1RPUkVfSUQ+PERBVEFTVE9SRT5jZXBoX2RhdGFzdG9yZTwvREFUQVNUT1JFPjxWTVM+PC9WTVM+PENMT05FUz48L0NMT05FUz48VEVNUExBVEU+PERFU0NSSV
BUS
U9OPjwhW0NEQVRBW3Rlc3QgY2VwaCBnb2xkZW4gaW1nXV0+PC9ERVNDUklQVElPTj48REVWX1BSRUZJWD48IVtDREFUQVtoZF1dPjwvREVWX1BSRUZJWD48RFJJVkVSPjwhW0NEQVRBW3Fjb3cyXV0+PC9EUklWRVI+PC9URU1QTEFURT48L0lNQUdFPjxEQVRBU1RPUkU+PElEPjEwMzwvSUQ+PFVJRD4wPC9VSUQ+PEdJRD4wPC9HSUQ+PFVOQU1FPm9uZWFkbWluPC9VTkFNRT48R05BTUU+b25lYWRtaW48L0dOQU1FPjxOQU1FPmNlcGhfZGF0YXN0b3JlPC9OQU1FPjxQRVJNSVNTSU9OUz48T1dORVJfVT4xPC9PV05FUl9VPjxPV05FUl9NPjE8L09XTkVSX00+PE9XTkVSX0E+MDwvT1dORVJfQT48R1JPVVBfVT4xPC9HUk9VUF9VPjxHUk9VUF9NPjA8L0dST1VQX00+PEdST1VQX0E+MDwvR1JPVVBfQT48T1RIRVJfVT4wPC9PVEhFUl9VPjxPVEhFUl9NPjA8L09USEVSX00+PE9USEVSX0E+MDwvT1RIRVJfQT48L1BFUk1JU1NJT05TPjxEU19NQUQ+Y2VwaDwvRFNfTUFEPjxUTV9NQUQ+Y2VwaDwvVE1fTUFEPjxCQVNFX1BBVEg+L3Zhci9saWIvb25lLy9kYXRhc3RvcmVzLzEwMzwvQkFTRV9QQVRIPjxUWVBFPjA8L1RZUEU+PERJU0tfVFlQRT4zPC9ESVNLX1RZUEU+PENMVVNURVJfSUQ+LTE8L0NMVVNURVJfSUQ+PENMVVNURVI+PC9DTFVTVEVSPjxUT1RBTF9NQj42MTAyNzMyODwvVE9UQUxfTUI+PEZSRUVfTUI+NjA4NTE1NTI8L0ZSRUVfTUI+PFVTRURfTUI+MTc1Nzc2PC9VU0VEX01CPjxJTUFHRVM+PC9JTUF
HRV
M+PFRFTVBMQVRFPjxCQVNFX1BBVEg+PCFbQ0RBVEFbL3Zhci9saWIvb25lLy9kYXRhc3RvcmVzL11dPjwvQkFTRV9QQVRIPjxCUklER0VfTElTVD48IVtDREFUQVtvbmU0ZGV2XV0+PC9CUklER0VfTElTVD48Q0VQSF9IT1NUPjwhW0NEQVRBW3N0a2VuZGNhMDFhIHN0a2VuZGNhMDRhIHN0a2VuZGNhMDJhXV0+PC9DRVBIX0hPU1Q+PENFUEhfU0VDUkVUPjwhW0NEQVRBWy9ldGMvY2VwaC9jZXBoLmNsaWVudC5hZG1pbi5rZXlyaW5nXV0+PC9DRVBIX1NFQ1JFVD48Q0xPTkVfVEFSR0VUPjwhW0NEQVRBW1NFTEZdXT48L0NMT05FX1RBUkdFVD48REFUQVNUT1JFX0NBUEFDSVRZX0NIRUNLPjwhW0NEQVRBW3llc11dPjwvREFUQVNUT1JFX0NBUEFDSVRZX0NIRUNLPjxESVNLX1RZUEU+PCFbQ0RBVEFbUkJEXV0+PC9ESVNLX1RZUEU+PERTX01BRD48IVtDREFUQVtjZXBoXV0+PC9EU19NQUQ+PExOX1RBUkdFVD48IVtDREFUQVtOT05FXV0+PC9MTl9UQVJHRVQ+PFBPT0xfTkFNRT48IVtDREFUQVtvbmVdXT48L1BPT0xfTkFNRT48U1RBR0lOR19ESVI+PCFbQ0RBVEFbL3Zhci9saWIvb25lL2NlcGgtdG1wXV0+PC9TVEFHSU5HX0RJUj48VE1fTUFEPjwhW0NEQVRBW2NlcGhdXT48L1RNX01BRD48L1RFTVBMQVRFPjwvREFUQVNUT1JFPjwvRFNfRFJJVkVSX0FDVElPTl9EQVRBPg==
18
Wed Sep 10 13:10:44 2014 [ImM][I]: cp: Copying local image
/cloud/images/timm/40gb.qcow2 to the image repository
Wed Sep 10 13:10:44 2014 [ImM][E]: cp: Command " set -e
Wed Sep 10 13:10:44 2014 [ImM][I]:
Wed Sep 10 13:10:44 2014 [ImM][I]: if [ "" = "2" ]; then
Wed Sep 10 13:10:44 2014 [ImM][I]: FORMAT=$(qemu-img info
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c | grep "^file
format:"
| awk '{print }')
Wed Sep 10 13:10:44 2014 [ImM][I]:
Wed Sep 10 13:10:44 2014 [ImM][I]: if [ "$FORMAT" != "raw" ]; then
Wed Sep 10 13:10:44 2014 [ImM][I]: qemu-img convert -O raw
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c.raw
Wed Sep 10 13:10:44 2014 [ImM][I]: mv
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c.raw
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c
Wed Sep 10 13:10:44 2014 [ImM][I]: fi
Wed Sep 10 13:10:44 2014 [ImM][I]:
Wed Sep 10 13:10:44 2014 [ImM][I]: rbd import --format 2
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c one/one-18
Wed Sep 10 13:10:44 2014 [ImM][I]: else
Wed Sep 10 13:10:44 2014 [ImM][I]: qemu-img convert
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c rbd:one/one-18
Wed Sep 10 13:10:44 2014 [ImM][I]: fi
Wed Sep 10 13:10:44 2014 [ImM][I]:
Wed Sep 10 13:10:44 2014 [ImM][I]: # remove original
Wed Sep 10 13:10:44 2014 [ImM][I]: rm -f
/var/lib/one/ceph-tmp/68bec7e25cb73c98a31a48117022d72c" failed: Unknown
protocol 'rbd:one/one-18'
Wed Sep 10 13:10:44 2014 [ImM][E]: Error registering one/one-18 in one4dev
Wed Sep 10 13:10:44 2014 [ImM][I]: ExitCode: 1
Wed Sep 10 13:10:44 2014 [ImM][E]: Error copying image in the datastore:
Error registering one/one-18 in one4dev
---
Clear we are afailed to register the rbd, but not clear why.. any
or clues on why we failed are helpful.
Several places in the docs refer to libvirt 1.x. Has anyone
made this work on RHEL6/Centos 6 (we are running the newer kernel so
we do have the rbd.o kernel module available, and rbd import/export
works from the command line.)
Steve Timm
------------------------------------------------------------------
Steven C. Timm, Ph.D (630) 840-8525
t...@fnal.gov http://home.fnal.gov/~timm/
Fermilab Scientific Computing Division, Scientific Computing Services
Quad.
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing
------------------------------------------------------------------
Steven C. Timm, Ph.D (630) 840-8525
t...@fnal.gov http://home.fnal.gov/~timm/
Fermilab Scientific Computing Division, Scientific Computing Services Quad.
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing
------------------------------------------------------------------
Steven C. Timm, Ph.D (630) 840-8525
t...@fnal.gov http://home.fnal.gov/~timm/
Fermilab Scientific Computing Division, Scientific Computing Services Quad.
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing
_______________________________________________
Users mailing list
Users@lists.opennebula.org
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
