Hi, I am setting up a test ceph cluster, on decommissioned hardware (hence : not optimal, I know). I have installed CentOS7, installed and setup ceph mons and OSD machines using puppet, and now I'm trying to add OSDs with the servers OSD disks... and I have issues (of course ;) ) I used the Ceph RHEL7 RPMs (ceph-0.80.6-0.el7.x86_64)
When I run "ceph-disk prepare" for a disk, I most of the time (but not always) get the partitions created, but not activated : [root@ceph4 ~]# ceph-disk list|grep sdh WARNING:ceph-disk:Old blkid does not support ID_PART_ENTRY_* fields, trying sgdisk; may not correctly identify ceph volumes with dmcrypt /dev/sdh : /dev/sdh1 ceph data, prepared, cluster ceph, journal /dev/sdh2 /dev/sdh2 ceph journal, for /dev/sdh1 I tried to debug udev rules thinking they were not launched to activate the OSD, but they are, and they fail on this error : + ln -sf ../../sdh2 /dev/disk/by-partuuid/5b3bde8f-ccad-4093-a8a5-ad6413ae8931 + mkdir -p /dev/disk/by-parttypeuuid + ln -sf ../../sdh2 /dev/disk/by-parttypeuuid/45b0969e-9b03-4f30-b4c6-b4b80ceff106.5b3bde8f-ccad-4093-a8a5-ad6413ae8931 + case $ID_PART_ENTRY_TYPE in + /usr/sbin/ceph-disk -v activate-journal /dev/sdh2 INFO:ceph-disk:Running command: /usr/bin/ceph-osd -i 0 --get-journal-uuid --osd-journal /dev/sdh2 SG_IO: bad/missing sense data, sb[]: 70 00 05 00 00 00 00 0b 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 DEBUG:ceph-disk:Journal /dev/sdh2 has OSD UUID 00000000-0000-0000-0000-000000000000 INFO:ceph-disk:Running command: /sbin/blkid -p -s TYPE -ovalue -- /dev/disk/by-partuuid/00000000-0000-0000-0000-000000000000 error: /dev/disk/by-partuuid/00000000-0000-0000-0000-000000000000: No such file or directory ceph-disk: Cannot discover filesystem type: device /dev/disk/by-partuuid/00000000-0000-0000-0000-000000000000: Command '/sbin/blkid' returned non-zero exit status 2 + exit + exec You'll notice the zeroed UUID... Because of this, I looked at the output of ceph-disk prepare, and saw that partx complains at the end (this is the partx -a command) : Warning: The kernel is still using the old partition table. The new table will be used at the next reboot. The operation has completed successfully. partx: /dev/sdh: error adding partitions 1-2 And indeed, running "partx -a /dev/sdh" does not change anything. But I just discovered that running "partx -u /dev/sdh" will fix everything ....???? I.e : right after I send this update command to the kernel, my debug logs show that the udev rule does everything fine and the OSD starts up. I'm therefore wondering what I did wrong ? is this CentOS 7 that is misbehaving, or the kernel, or...? Any reason why partx -a is used instead of partx -u ? I'd be glad to hear others advice on this ! Thanks && regards Frederic Schaer
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
