On Fri, Jun 6, 2014 at 4:34 PM, Kenneth Waegeman
<kenneth.waege...@ugent.be> wrote:
>
> ----- Message from Igor Krstic <igor.z.krs...@gmail.com> ---------
> Date: Fri, 06 Jun 2014 13:23:19 +0200
> From: Igor Krstic <igor.z.krs...@gmail.com>
> Subject: Re: [ceph-users] question about feature set mismatch
> To: Ilya Dryomov <ilya.dryo...@inktank.com>
> Cc: ceph-users@lists.ceph.com
>
>
>
>> On Fri, 2014-06-06 at 11:51 +0400, Ilya Dryomov wrote:
>>>
>>> On Thu, Jun 5, 2014 at 10:38 PM, Igor Krstic <igor.z.krs...@gmail.com>
>>> wrote:
>>> > Hello,
>>> >
>>> > dmesg:
>>> > [ 690.181780] libceph: mon1 192.168.214.102:6789 feature set mismatch,
>>> > my
>>> > 4a042a42 < server's 504a042a42, missing 5000000000
>>> > [ 690.181907] libceph: mon1 192.168.214.102:6789 socket error on read
>>> > [ 700.190342] libceph: mon0 192.168.214.101:6789 feature set mismatch,
>>> > my
>>> > 4a042a42 < server's 504a042a42, missing 5000000000
>>> > [ 700.190481] libceph: mon0 192.168.214.101:6789 socket error on read
>>> > [ 710.194499] libceph: mon1 192.168.214.102:6789 feature set mismatch,
>>> > my
>>> > 4a042a42 < server's 504a042a42, missing 5000000000
>>> > [ 710.194633] libceph: mon1 192.168.214.102:6789 socket error on read
>>> > [ 720.201226] libceph: mon1 192.168.214.102:6789 feature set mismatch,
>>> > my
>>> > 4a042a42 < server's 504a042a42, missing 5000000000
>>> > [ 720.201482] libceph: mon1 192.168.214.102:6789 socket error on read
>>> >
>>> > 5000000000 should be:
>>> > CEPH_FEATURE_CRUSH_V2 36 1000000000
>>> > and
>>> > CEPH_FEATURE_OSD_ERASURE_CODES 38 4000000000
>>> > CEPH_FEATURE_OSD_TMAP2OMAP 38* 4000000000
>>> >
>>> > That is happening on two separate boxes that are just my nfs and block
>>> > gateways (they are not osd/mon/mds). So I just need on them something
>>> > like:
>>> > sudo rbd map share2
>>> > sudo mount -t xfs /dev/rbd1 /mnt/share2
>>> >
>>> > On ceph cluster and on those two separate boxes:
>>> > ~$ ceph -v
>>> > ceph version 0.80.1
>>> >
>>> > What could be the problem?
>>>
>>> Which kernel version are you running? Do you have any erasure coded
>>> pools?
>>>
>>> Thanks,
>>>
>>> Ilya
>>
>> ~$ uname -a
>> Linux ceph-gw1 3.13.0-24-generic #47~precise2-Ubuntu SMP Fri May 2
>> 23:30:46 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
>>
>> Yes, one of the pools is erasure coded pool but the only thing I use on
>> that box is rbd and rbd pool is not ec pool. It is replicated pool. I to
>> not touch ec pool from there. Or at least, I believe so :)
>
>
> Well, I saw something similar with CephFS: I didn't touch the pools in use
> by cephfs, but I created another pool with Erasure Code, and the ceph client
> (kernel 3.13, but not enough for EC) also stopped working with 'feature set
> mismatch'. Thus I guess the clients can't read the crushmap anymore when
> there is a 'erasure' mentioning in it:)
Unfortunately that's true. If there are any erasure code pools in the
cluster, kernel clients (both krbd and kcephfs) won't work. The only
way it will work is if you remove all erasure coded pools.
CRUSH_V2 is also not present in 3.13. You'll have to uprade to 3.14.
Alternatively, CRUSH_V2 can be disabled, but I can't tell you how off
the top of my head. The fundamental problem is that you are running
latest userspace, and the defaults it ships with are incompatible with
older kernels.
Thanks,
Ilya
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
