Hello, I have been trying to deploy bluestore OSDs in a test cluster of 2x OSDs and 3x mon (xen1,2,3) on Ubuntu Xenial and Jewel 10.2.1.
Activating the OSDs gives an error in systemd as follows. the culprit is the command "ceph-osd --get-device-fsid" which fails to get fsid. =========================================================== root@xen2:/# fdisk -l /dev/sdb Device Start End Sectors Size Type /dev/sdb1 2048 206847 204800 100M Ceph OSD /dev/sdb2 206848 67108830 66901983 31.9G unknown root@xen2:/var/log# systemctl start ceph-disk@dev-sdb2 Job for ceph-disk@dev-sdb2.service failed because the control process exited with error code. See "systemctl status ceph-disk@dev-sdb2.service" and "journalctl -xe" for details. root@xen2:/# journalctl -xe -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- The system clock has been changed to REALTIME microseconds after January 1st, 1970. Jun 02 19:00:40 xen2 systemd[1]: apt-daily.timer: Adding 4h 10min 28.620269s random time. Jun 02 19:00:41 xen2 ntpd[1294]: Soliciting pool server 41.79.80.34 Jun 02 19:00:41 xen2 ntpd[1294]: Soliciting pool server 197.157.194.21 Jun 02 19:02:32 xen2 systemd[1]: Starting Ceph disk activation: /dev/sdb2... -- Subject: Unit ceph-disk@dev-sdb2.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit ceph-disk@dev-sdb2.service has begun starting up. Jun 02 19:02:32 xen2 sh[1842]: main_trigger: main_trigger: Namespace(cluster='ceph', dev='/dev/sdb2', dmcrypt=None, dmcrypt_key_dir='/etc/ceph/dmcrypt-keys', func=<function main_trigger at Jun 02 19:02:32 xen2 sh[1842]: command: Running command: /sbin/init --version Jun 02 19:02:32 xen2 sh[1842]: command: Running command: /sbin/blkid -o udev -p /dev/sdb2 Jun 02 19:02:32 xen2 sh[1842]: command: Running command: /sbin/blkid -o udev -p /dev/sdb2 Jun 02 19:02:32 xen2 sh[1842]: main_trigger: trigger /dev/sdb2 parttype cafecafe-9b03-4f30-b4c6-b4b80ceff106 uuid 6fba12f6-edf6-4345-af26-682bad97de40 Jun 02 19:02:32 xen2 sh[1842]: command: Running command: /usr/sbin/ceph-disk --verbose activate-block /dev/sdb2 Jun 02 19:02:32 xen2 sh[1842]: main_trigger: Jun 02 19:02:32 xen2 sh[1842]: main_trigger: get_dm_uuid: get_dm_uuid /dev/sdb2 uuid path is /sys/dev/block/8:18/dm/uuid Jun 02 19:02:32 xen2 sh[1842]: command: Running command: /sbin/blkid -o udev -p /dev/sdb2 Jun 02 19:02:32 xen2 sh[1842]: command: Running command: /usr/bin/ceph-osd --get-device-fsid /dev/sdb2 Jun 02 19:02:32 xen2 sh[1842]: Traceback (most recent call last): Jun 02 19:02:32 xen2 sh[1842]: File "/usr/sbin/ceph-disk", line 9, in <module> Jun 02 19:02:32 xen2 sh[1842]: load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')() Jun 02 19:02:32 xen2 sh[1842]: File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 4964, in run Jun 02 19:02:32 xen2 sh[1842]: main(sys.argv[1:]) Jun 02 19:02:32 xen2 sh[1842]: File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 4915, in main Jun 02 19:02:32 xen2 sh[1842]: args.func(args) Jun 02 19:02:32 xen2 sh[1842]: File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 4673, in <lambda> Jun 02 19:02:32 xen2 sh[1842]: func=lambda args: main_activate_space(name, args), Jun 02 19:02:32 xen2 sh[1842]: File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 3679, in main_activate_space Jun 02 19:02:32 xen2 sh[1842]: osd_uuid = get_space_osd_uuid(name, dev) Jun 02 19:02:32 xen2 sh[1842]: File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 3654, in get_space_osd_uuid Jun 02 19:02:32 xen2 sh[1842]: e, Jun 02 19:02:32 xen2 sh[1842]: ceph_disk.main.Error: Error: failed to get osd uuid/fsid from block: Command '/usr/bin/ceph-osd' returned non-zero exit status 1 Jun 02 19:02:32 xen2 sh[1842]: Traceback (most recent call last): Jun 02 19:02:32 xen2 sh[1842]: File "/usr/sbin/ceph-disk", line 9, in <module> Jun 02 19:02:32 xen2 sh[1842]: load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')() Jun 02 19:02:32 xen2 sh[1842]: File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 4964, in run Jun 02 19:02:32 xen2 sh[1842]: main(sys.argv[1:]) Jun 02 19:02:32 xen2 sh[1842]: File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 4915, in main Jun 02 19:02:32 xen2 sh[1842]: args.func(args) Jun 02 19:02:32 xen2 sh[1842]: File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 4352, in main_trigger Jun 02 19:02:32 xen2 sh[1842]: raise Error('return code ' + str(ret)) Jun 02 19:02:32 xen2 sh[1842]: ceph_disk.main.Error: Error: return code 1 Jun 02 19:02:32 xen2 systemd[1]: ceph-disk@dev-sdb2.service: Main process exited, code=exited, status=1/FAILURE Jun 02 19:02:32 xen2 systemd[1]: Failed to start Ceph disk activation: /dev/sdb2. -- Subject: Unit ceph-disk@dev-sdb2.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit ceph-disk@dev-sdb2.service has failed. -- -- The result is failed. Jun 02 19:02:32 xen2 systemd[1]: ceph-disk@dev-sdb2.service: Unit entered failed state. Jun 02 19:02:32 xen2 systemd[1]: ceph-disk@dev-sdb2.service: Failed with result 'exit-code'. root@xen2:/# /sbin/blkid -o udev -p /dev/sdb2 ID_PART_ENTRY_SCHEME=gpt ID_PART_ENTRY_NAME=ceph\x20block ID_PART_ENTRY_UUID=6fba12f6-edf6-4345-af26-682bad97de40 ID_PART_ENTRY_TYPE=cafecafe-9b03-4f30-b4c6-b4b80ceff106 ID_PART_ENTRY_NUMBER=2 ID_PART_ENTRY_OFFSET=206848 ID_PART_ENTRY_SIZE=66901983 ID_PART_ENTRY_DISK=8:16 root@xen2:/# /usr/bin/ceph-osd --get-device-fsid /dev/sdb2 2016-06-02 19:03:50.960521 7f203b2928c0 -1 bluestore(/dev/sdb2) _read_bdev_label unable to decode label at offset 62: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) unknown encoding version > 1 2016-06-02 19:03:50.963348 7f203b2928c0 -1 journal read_header error decoding journal header failed to get device fsid for /dev/sdb2: (22) Invalid argument ============================================================= Any help on thie matter will be appreciated. Thanks, hpcre
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
