On Fri, May 29, 2020 at 1:28 PM jifeng-call <[email protected]> wrote:
> Hi All, > I have 6 servers that form a glusterfs 2x3 distributed replication volume, > the details are as follows: > > [root@node1 ~]# gluster volume info > Volume Name: ksvd_vol > Type: Distributed-Replicate > Volume ID: c9848daa-b06f-4f82-a2f8-1b425b8e869c > Status: Started > Snapshot Count: 0 > Number of Bricks: 2 x 3 = 6 > Transport-type: tcp > Bricks: > Brick1: node1:/export/kylin/ksvd > Brick2: node2:/export/kylin/ksvd > Brick3: node3:/export/kylin/ksvd > Brick4: node4:/export/kylin/ksvd > Brick5: node5:/export/kylin/ksvd > Brick6: node6:/export/kylin/ksvd > Options Reconfigured: > diagnostics.client-log-level: DEBUG > diagnostics.brick-log-level: DEBUG > performance.client-io-threads: off > nfs.disable: on > transport.address-family: inet > cluster.quorum-type: auto > > Restart 6 servers at a time, the server automatically mounts the ksvd_vol > volume to the / home / kylin-ksvd directory during the startup process： > > [root@node1 ~]# df -h | grep ksvd_vol > node1:/ksvd_vol 4.5T 362G 4.1T 9% /home/kylin-ksvd > > Failed to create a file in the glusterfs mount directory of node3, the log > shows the error as follows (the files created by other servers are normal)： > > Possibly the layout setxattr failed which should not have happened given it is a replicated volume. You can check log on client side and brick side for the parent directory for more information. Post remount, a fresh triggers which heals the layout missing before and create goes in. > [2020-05-29 05:55:03.065656] E [MSGID: 109011] > [dht-common.c:8683:dht_create] 0-ksvd_vol-dht: no subvolume in layout for > path=/.ksvd.time.405fa8a6-575d-4474-a97f-a107cbf1c673.18512 > [2020-05-29 05:55:03.065719] W [fuse-bridge.c:2122:fuse_create_cbk] > 0-glusterfs-fuse: 2454790: > /.ksvd.time.405fa8a6-575d-4474-a97f-a107cbf1c673.18512 => -1 (输入/输出错误) > [2020-05-29 05:55:03.680303] W [MSGID: 109011] > [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash > (value) = 1738969696 > [2020-05-29 05:55:04.687456] W [MSGID: 109011] > [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash > (value) = 491618322 > [2020-05-29 05:55:04.688612] W [MSGID: 109011] > [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash > (value) = 1738969696 > [2020-05-29 05:55:05.694446] W [MSGID: 109011] > [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash > (value) = 1738969696 > [2020-05-29 05:55:05.830555] W [MSGID: 109011] > [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash > (value) = 491618322 > [2020-05-29 05:55:06.700423] W [MSGID: 109011] > [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash > (value) = 1738969696 > [2020-05-29 05:55:07.706536] W [MSGID: 109011] > [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash > (value) = 1738969696 > [2020-05-29 05:55:07.833049] W [MSGID: 109011] > [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash > (value) = 491618322 > [2020-05-29 05:55:08.712128] W [MSGID: 109011] > [dht-layout.c:186:dht_layout_search] 0-ksvd_vol-dht: no subvolume for hash > (value) = 1738969696 > The message "W [MSGID: 109011] [dht-layout.c:186:dht_layout_search] > 0-ksvd_vol-dht: no subvolume for hash (value) = 1738969696" repeated 2 > times between [2020-05-29 05:55:08.712128] and [2020-05-29 05:55:09.718541] > > The pathinfo information in the / home / kylin-ksvd directory is displayed > as follows: > > [root@node3 kylin-ksvd]# getfattr -d -m . -n trusted.glusterfs.pathinfo > /home/kylin-ksvd/ > getfattr: Removing leading '/' from absolute path names > # file: home/kylin-ksvd/ > trusted.glusterfs.pathinfo="((<DISTRIBUTE:ksvd_vol-dht> > (<REPLICATE:ksvd_vol-replicate-0> > <POSIX(/export/kylin/ksvd):node2:/export/kylin/ksvd/> > <POSIX(/export/kylin/ksvd):node3:/export/kylin/ksvd/> > <POSIX(/export/kylin/ksvd):node1:/export/kylin/ksvd/>) > (<REPLICATE:ksvd_vol-replicate-1> > <POSIX(/export/kylin/ksvd):node5:/export/kylin/ksvd/> > <POSIX(/export/kylin/ksvd):node6:/export/kylin/ksvd/> > <POSIX(/export/kylin/ksvd):node4:/export/kylin/ksvd/>)) > (ksvd_vol-dht-layout (ksvd_vol-replicate-0 0 0) (ksvd_vol-replicate-1 > 3539976838 4294967295)))" > > It can be seen from the above information that ksvd_vol-dht-layout is > missing the interval 0 to 3539976837 > > After umount, remounting returned to normal ... What is the reason? > > Best regards > > > > > > > ________ > > > > Community Meeting Calendar: > > Schedule - > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > [email protected] > https://lists.gluster.org/mailman/listinfo/gluster-users >
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://bluejeans.com/441850968 Gluster-users mailing list [email protected] https://lists.gluster.org/mailman/listinfo/gluster-users
