Hi John and Zheng,

Thanks for the quick replies!
I'm using kernel 4.2. I'll test out that fix.

Arthur

On Wed, Sep 2, 2015 at 10:29 PM, Yan, Zheng <uker...@gmail.com> wrote:

> probably caused by http://tracker.ceph.com/issues/12551
>
> On Wed, Sep 2, 2015 at 7:57 PM, Arthur Liu <arthurhs...@gmail.com> wrote:
> > Hi,
> >
> > I am experiencing an issue with CephFS with cache tiering where the
> kernel
> > clients are reading files filled entirely with 0s.
> >
> > The setup:
> > ceph 0.94.3
> > create cephfs_metadata replicated pool
> > create cephfs_data replicated pool
> > cephfs was created on the above two pools, populated with files, then:
> > create cephfs_ssd_cache replicated pool,
> > then adding the tiers:
> > ceph osd tier add cephfs_data cephfs_ssd_cache
> > ceph osd tier cache-mode cephfs_ssd_cache writeback
> > ceph osd tier set-overlay cephfs_data cephfs_ssd_cache
> >
> > While the cephfs_ssd_cache pool is empty, multiple kernel clients on
> > different hosts open the same file (the size of the file is small, <10k)
> at
> > approximately the same time. A number of the clients from the OS level
> see
> > the entire file being empty. I can do a rados -p {cache pool} ls for the
> > list of files cached, and do a rados -p {cache pool} get {object}
> /tmp/file
> > and see the complete contents of the file.
> > I can repeat this by setting cache-mode to forward, rados -p {cache pool}
> > cache-flush-evict-all, checking no more objects in cache with rados -p
> > {cache pool} ls, resetting cache-mode to writeback with an empty pool,
> and
> > doing the multiple same file opens.
> >
> > Has anyone seen this issue? It seems like what may be a race condition
> where
> > the object is not yet completely loaded into the cache pool so the cache
> > pool serves out an incomplete object.
> > If anyone can shed some light or any suggestions to help debug this
> issue,
> > that would be very helpful.
> >
> > Thanks,
> > Arthur
> >
> > _______________________________________________
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Reply via email to