Hey Greg, The inconsistent view is only over nfs/smb on top of our /ceph mount.
When I look directly on the /ceph mount (which is using the cephfs kernel module), everything looks fine It is possible that this issue just went unnoticed, and it only being a infernalis problem is just a red herring. With that, it is oddly coincidental that we just started seeing issues. On Wed, Jan 13, 2016 at 11:30 AM, Gregory Farnum <gfar...@redhat.com> wrote: > On Wed, Jan 13, 2016 at 11:24 AM, Mike Carlson <m...@bayphoto.com> wrote: > > Hello. > > > > Since we upgraded to Infernalis last, we have noticed a severe problem > with > > cephfs when we have it shared over Samba and NFS > > > > Directory listings are showing an inconsistent view of the files: > > > > > > $ ls /lts-mon/BD/xmlExport/ | wc -l > > 100 > > $ sudo umount /lts-mon > > $ sudo mount /lts-mon > > $ ls /lts-mon/BD/xmlExport/ | wc -l > > 3507 > > > > > > The only work around I have found is un-mounting and re-mounting the nfs > > share, that seems to clear it up > > Same with samba, I'd post it here but its thousands of lines. I can add > > additional details on request. > > > > This happened after our upgrade to infernalis. Is it possible the MDS is > in > > an inconsistent state? > > So this didn't happen to you until after you upgraded? Are you seeing > missing files when looking at cephfs directly, or only over the > NFS/Samba re-exports? Are you also sharing Samba by re-exporting the > kernel cephfs mount? > > Zheng, any ideas about kernel issues which might cause this or be more > visible under infernalis? > -Greg > > > > > We have cephfs mounted on a server using the built in cephfs kernel > module: > > > > lts-mon:6789:/ /ceph ceph > > name=admin,secretfile=/etc/ceph/admin.secret,noauto,_netdev > > > > > > We are running all of our ceph nodes on ubuntu 14.04 LTS. Samba is up to > > date, 4.1.6, and we export nfsv3 to linux and freebsd systems. All seem > to > > exhibit the same behavior. > > > > system info: > > > > # uname -a > > Linux lts-osd1 3.13.0-63-generic #103-Ubuntu SMP Fri Aug 14 21:42:59 UTC > > 2015 x86_64 x86_64 x86_64 GNU/Linux > > root@lts-osd1:~# lsb > > lsblk lsb_release > > root@lts-osd1:~# lsb_release -a > > No LSB modules are available. > > Distributor ID: Ubuntu > > Description: Ubuntu 14.04.3 LTS > > Release: 14.04 > > Codename: trusty > > > > > > package info: > > > > # dpkg -l|grep ceph > > ii ceph 9.2.0-1trusty > > amd64 distributed storage and file system > > ii ceph-common 9.2.0-1trusty > > amd64 common utilities to mount and interact with a ceph storage > > cluster > > ii ceph-fs-common 9.2.0-1trusty > > amd64 common utilities to mount and interact with a ceph file > system > > ii ceph-mds 9.2.0-1trusty > > amd64 metadata server for the ceph distributed file system > > ii libcephfs1 9.2.0-1trusty > > amd64 Ceph distributed file system client library > > ii python-ceph 9.2.0-1trusty > > amd64 Meta-package for python libraries for the Ceph libraries > > ii python-cephfs 9.2.0-1trusty > > amd64 Python libraries for the Ceph libcephfs library > > > > > > What is interesting, is a directory or file will not show up in a > listing, > > however, if we directly access the file, it shows up in that instance: > > > > > > # ls -al |grep SCHOOL > > # ls -alnd SCHOOL667055 > > drwxrwsr-x 1 21695 21183 2962751438 Jan 13 09:33 SCHOOL667055 > > > > > > Any tips are appreciated! > > > > Thanks, > > Mike C > > > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
