The message 'feature set mismatch' is exactly what Gregory was talking about.
This indicates that you are using a CRUSH feature in your Ceph environment that the Kernel RBD client doesn't understand. So, it's unable to communicate. In this instance, it's likely to do with the cache pool itself, but I'm not terribly familiar with the features support in 3.14 kernel... Thanks, Michael J. Kidd Sr. Storage Consultant Inktank Professional Services On Mon, Mar 24, 2014 at 2:58 PM, Ирек Фасихов <malm...@gmail.com> wrote: > Hi, Gregory! > I think that there is no interesting :). > > *dmesg:* > Key type dns_resolver registered > Key type ceph registered > libceph: loaded (mon/osd proto 15/24) > rbd: loaded (major 252) > libceph: mon1 192.168.100.202:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon1 192.168.100.202:6789 socket error on read > libceph: mon1 192.168.100.202:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon1 192.168.100.202:6789 socket error on read > libceph: mon2 192.168.100.203:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon2 192.168.100.203:6789 socket error on read > libceph: mon2 192.168.100.203:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon2 192.168.100.203:6789 socket error on read > libceph: mon1 192.168.100.202:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon1 192.168.100.202:6789 socket error on read > libceph: mon1 192.168.100.202:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon1 192.168.100.202:6789 socket error on read > libceph: mon1 192.168.100.202:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon1 192.168.100.202:6789 socket error on read > libceph: mon2 192.168.100.203:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon2 192.168.100.203:6789 socket error on read > libceph: mon2 192.168.100.203:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon2 192.168.100.203:6789 socket error on read > libceph: mon0 192.168.100.201:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon0 192.168.100.201:6789 socket error on read > libceph: mon1 192.168.100.202:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon1 192.168.100.202:6789 socket error on read > libceph: mon2 192.168.100.203:6789 feature set mismatch, my 384a042a42 < > server's 2384a042a42, missing 20000000000 > libceph: mon2 192.168.100.203:6789 socket error on read > > > > 2014-03-24 21:16 GMT+04:00 Gregory Farnum <g...@inktank.com>: > > I don't remember what features should exist where, but I expect that >> the cluster is making use of features that the kernel client doesn't >> support yet (despite the very new kernel). Have you checked to see if >> there's anything interesting in dmesg? >> -Greg >> Software Engineer #42 @ http://inktank.com | http://ceph.com >> >> >> On Mon, Mar 24, 2014 at 1:30 AM, Ирек Фасихов <malm...@gmail.com> wrote: >> > Created cache pool for documentation: >> > http://ceph.com/docs/master/dev/cache-pool/ >> > >> > ceph osd pool create cache 100 >> > ceph osd tier add rbd cache >> > ceph osd tier cache-mode cache writeback >> > ceph osd tier set-overlay rbd cache >> > ceph osd pool set cache hit_set_type bloom >> > ceph osd pool set cache hit_set_count 1 >> > ceph osd pool set cache hit_set_period 600 >> > ceph osd pool set cache target_max_bytes 10000000000 #10Gb >> > >> > ceph osd tree: >> > # id weight type name up/down reweight >> > -1 6 root default >> > -2 2 host ceph01 >> > 0 1 osd.0 up 1 >> > 3 1 osd.3 up 1 >> > -3 2 host ceph02 >> > 1 1 osd.1 up 1 >> > 4 1 osd.4 up 1 >> > -4 2 host ceph03 >> > 2 1 osd.2 up 1 >> > 5 1 osd.5 up 1 >> > >> > rbd -p rbd ls -l >> > NAME SIZE PARENT FMT PROT LOCK >> > test 10240M 1 >> > >> > rbd map rbd/test >> > rbd: add failed: (5) Input/output error >> > >> > uname -a >> > Linux ceph01.bank-hlynov.ru 3.14.0-rc7-bank-hlynov.ru #1 SMP Mon Mar 17 >> > 11:49:22 MSK 2014 x86_64 x86_64 x86_64 GNU/Linux. Linux CentOS 6.5 >> > >> > Debug mode and strace in logs. >> > >> > >> https://drive.google.com/folderview?id=0BxoNLVWxzOJWX0NLV1kzQ1l3Ymc&usp=sharing >> > >> > >> > -- >> > С уважением, Фасихов Ирек Нургаязович >> > Моб.: +79229045757 >> > >> > _______________________________________________ >> > ceph-users mailing list >> > ceph-users@lists.ceph.com >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > >> > > > > -- > С уважением, Фасихов Ирек Нургаязович > Моб.: +79229045757 > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
