Hi,
From the ones we managed to configure in our lab here. I noticed that
using image format "raw" instead of "qcow2" worked for us.
Regards,
Luke
-----Original Message-----
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
Steven Timm
Sent: Thursday, 18 September, 2014 5:01 AM
To: ceph-users@lists.ceph.com
Subject: [ceph-users] ceph issue: rbd vs. qemu-kvm
I am trying to use Ceph as a data store with OpenNebula 4.6 and have
followed the instructions in OpenNebula's documentation at
http://docs.opennebula.org/4.8/administration/storage/ceph_ds.html
and compared them against the "using libvirt with ceph"
http://ceph.com/docs/master/rbd/libvirt/
We are using the ceph-recompiled qemu-kvm and qemu-img as found at
http://ceph.com/packages/qemu-kvm/
under Scientific Linux 6.5 which is a Redhat clone. Also a kernel-lt-3.10
kernel.
[root@fgtest15 qemu]# kvm -version
QEMU PC emulator version 0.12.1 (qemu-kvm-0.12.1.2), Copyright (c)
2003-2008 Fabrice Bellard
From qemu-img
Supported formats: raw cow qcow vdi vmdk cloop dmg bochs vpc vvfat qcow2
qed parallels nbd blkdebug host_cdrom host_floppy host_device file rbd
------------------
Libvirt is trying to execute the following KVM command:
2014-09-17 19:50:12.774+0000: starting up
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none
/usr/libexec/qemu-kvm -name one-60 -S -M rhel6.3.0 -enable-kvm -m 4096
-smp 2,sockets=2,cores=1,threads=1 -uuid
572499bf-07f3-3014-8d6a-dfa1ebb99aa4 -nodefconfig -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/one-60.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
-no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=rbd:one/one-19-60-0:id=libvirt2:key=AQAV5BlU2OV7NBAApurqxG0K8UkZlQVy6hKmkA==:auth_supported=cephx\;none:mon_host=stkendca01a\:6789\;stkendca04a\:6789\;stkendca02a\:6789,if=none,id=drive-virtio-disk0,format=qcow2,cache=none
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-drive
file=/var/lib/one//datastores/102/60/disk.1,if=none,id=drive-virtio-disk1,format=raw,cache=none
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1
-drive
file=/var/lib/one//datastores/102/60/disk.2,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
-device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0
-netdev tap,fd=22,id=hostnet0,vhost=on,vhostfd=23 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=54:52:00:02:0b:04,bus=pci.0,addr=0x3
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:60 -k en-us -vga
cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
char device redirected to /dev/pts/3
qemu-kvm: -drive
file=rbd:one/one-19-60-0:id=libvirt2:key=AQAV5BlU2OV7NBAApurqxG0K8UkZlQVy6hKmkA==:auth_supported=cephx\;none:mon_host=stkendca01a\:6789\;stkendca04a\:6789\;stkendca02a\:6789,if=none,id=drive-virtio-disk0,format=qcow2,cache=none:
could not open disk image
rbd:one/one-19-60-0:id=libvirt2:key=AQAV5BlU2OV7NBAApurqxG0K8UkZlQVy6hKmkA==:auth_supported=cephx\;none:mon_host=stkendca01a\:6789\;stkendca04a\:6789\;stkendca02a\:6789:
Invalid argument
2014-09-17 19:50:12.980+0000: shutting down
-----------
just to show that from the command line I can see the rbd pool fine
[root@fgtest15 qemu]# rbd list one
foo
one-19
one-19-58-0
one-19-60-0
[root@fgtest15 qemu]# rbd info one/one-19-60-0
rbd image 'one-19-60-0':
size 40960 MB in 10240 objects
order 22 (4096 kB objects)
block_name_prefix: rb.0.3c39.238e1f29
format: 1
and even mount stuff with rbd map, etc.
It's only inside libvirt that we had the problem.
At first we were getting "permission denied" but then I upped the
permissions allowed to the libvirt user (client.libvirt2) and then
we are just getting "invalid argument"
client.libvirt2
key: AQAV5BlU2OV7NBAApurqxG0K8UkZlQVy6hKmkA==
caps: [mon] allow r
caps: [osd] allow *, allow rwx pool=one
------
Any idea why kvm doesn't like the argument I am delivering in the file=
argument? Better--does anyone have a working kvm command out
of either opennebula or openstack against which I can compare?
Thanks
Steve Timm
------------------------------------------------------------------
Steven C. Timm, Ph.D (630) 840-8525
t...@fnal.gov http://home.fnal.gov/~timm/
Fermilab Scientific Computing Division, Scientific Computing Services
Quad.
Grid and Cloud Services Dept., Associate Dept. Head for Cloud Computing
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
________________________________
DISCLAIMER:
This e-mail (including any attachments) is for the addressee(s) only and
may be confidential, especially as regards personal data. If you are not
the intended recipient, please note that any dealing, review,
distribution, printing, copying or use of this e-mail is strictly
prohibited. If you have received this email in error, please notify the
sender immediately and delete the original message (including any
attachments).
MIMOS Berhad is a research and development institution under the purview
of the Malaysian Ministry of Science, Technology and Innovation. Opinions,
conclusions and other information in this e-mail that do not relate to the
official business of MIMOS Berhad and/or its subsidiaries shall be
understood as neither given nor endorsed by MIMOS Berhad and/or its
subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts
responsibility for the same. All liability arising from or in connection
with computer viruses and/or corrupted e-mails is excluded to the fullest
extent permitted by law.
