Thanks for the quick confirm and fix. I reviewed the pull request. Seems that this fix is going into Jewel and Hammer. So does this mean this issue also exists in Jewel? why I test Jewel and cannot reproduce this issue??
Zhongyan On Wed, Nov 30, 2016 at 12:32 PM, Venky Shankar <vshan...@redhat.com> wrote: > On 16-11-30 12:07:33, Zhongyan Gu wrote: > > Jason, > > I test Jewel and confirmed Jewel has no such issue. > > Could you tell me what is the specific pull that can be backported to > > hammer to fix this issue?? > > PR: https://github.com/ceph/ceph/pull/12218 > > > > > Zhongyan > > > > On Tue, Nov 29, 2016 at 10:52 PM, Jason Dillaman <jdill...@redhat.com> > > wrote: > > > > > You are correct that there is an issue with the Hammer version when > > > calculating diffs. If the clone has an object that obscures an extent > > > within the parent image but didn't exist within the first snapshot of > > > the clone, the diff results from the parent image won't be included in > > > the result. I'll open a ticket against this issue, but in the meantime > > > I'd suggest trying a Jewel-based librbd client instead of Hammer. > > > > > > On Mon, Nov 28, 2016 at 10:18 PM, Zhongyan Gu <zhongyan...@gmail.com> > > > wrote: > > > > Thank you Jason. > > > > We are designing a backup system for production cluster based on > ceph's > > > > export /import diff feature. > > > > We found this issue and hopefully it can be confirmed and then fixed > > > soon. > > > > If you need any more information for debugging, Please just let me > know. > > > > > > > > Thanks, > > > > Zhongyan > > > > > > > > On Mon, Nov 28, 2016 at 10:05 PM, Jason Dillaman < > jdill...@redhat.com> > > > > wrote: > > > >> > > > >> OK, in that case, none of my previous explanation is relevant. I'll > > > >> spin up a hammer cluster and try to reproduce. > > > >> > > > >> On Wed, Nov 23, 2016 at 9:13 PM, Zhongyan Gu <zhongyan...@gmail.com > > > > > >> wrote: > > > >> > BTW, I used Hammer 0.94.5 to do the test. > > > >> > > > > >> > Zhongyan > > > >> > > > > >> > On Thu, Nov 24, 2016 at 10:07 AM, Zhongyan Gu < > zhongyan...@gmail.com> > > > >> > wrote: > > > >> >> > > > >> >> Thank you Jason. My test shows in the following case, image B > will be > > > >> >> exactly same: > > > >> >> 1. clone image A from parent: > > > >> >> #rbd clone 1124-parent@snap1 A > > > >> >> > > > >> >> 2. create snap for A > > > >> >> #rbd snap create A@snap1 > > > >> >> > > > >> >> 3. create empty image B > > > >> >> #rbd create B -s 1 > > > >> >> > > > >> >> 4. export-diff A then impor-diff B: > > > >> >> #rbd export-diff A@snap1 -|./rbd import-diff - B > > > >> >> > > > >> >> 5. check A@snap1 equals B@snap1 > > > >> >> #rbd export A@snap1 -|md5sum > > > >> >> Exporting image: 100% complete...done. > > > >> >> 880709d7352b6c9926beb1d829673366 - > > > >> >> #rbd export B@snap1 -|md5sum > > > >> >> Exporting image: 100% complete...done. > > > >> >> 880709d7352b6c9926beb1d829673366 - > > > >> >> output shows A@snap1 equals B@snap1 > > > >> >> > > > >> >> However, in the following case, image B will not be exactly same: > > > >> >> 1. clone image A from parent: > > > >> >> #rbd clone 1124-parent@snap1 A > > > >> >> > > > >> >> 2. create snap for A > > > >> >> #rbd snap create A@snap1 > > > >> >> > > > >> >> 3. use fio make some change to A > > > >> >> > > > >> >> 4. create empty image B > > > >> >> #rbd create B -s 1 > > > >> >> > > > >> >> 4. export-diff A then impor-diff B: > > > >> >> #rbd export-diff A@snap1 -|./rbd import-diff - B > > > >> >> > > > >> >> 5. check A@snap1 equals B@snap1 > > > >> >> #rbd export A@snap1 -|md5sum > > > >> >> Exporting image: 100% complete...done. > > > >> >> 880709d7352b6c9926beb1d829673366 - > > > >> >> #rbd export B@snap1 -|md5sum > > > >> >> Exporting image: 100% complete...done. > > > >> >> bbf7cf69a84f3978c66f5eb082fb91ec - > > > >> >> output shows A@snap1 DOES NOT equal B@snap1 > > > >> >> > > > >> >> The second case can always be reproduced. What is wrong with the > > > second > > > >> >> case? > > > >> >> > > > >> >> Thanks, > > > >> >> Zhongyan > > > >> >> > > > >> >> > > > >> >> On Wed, Nov 23, 2016 at 10:11 PM, Jason Dillaman < > > > jdill...@redhat.com> > > > >> >> wrote: > > > >> >>> > > > >> >>> What you are seeing sounds like a side-effect of deep-flatten > > > support. > > > >> >>> If you write to an unallocated extent within a cloned image, the > > > >> >>> associated object extent must be read from the parent image, > > > modified, > > > >> >>> and written to the clone image. > > > >> >>> > > > >> >>> Since the Infernalis release, this process has been tweaked if > the > > > >> >>> cloned image has a snapshot. In that case, the associated object > > > >> >>> extent is still read from the parent, but instead of being > modified > > > >> >>> and written to the HEAD revision, it is left unmodified and is > > > written > > > >> >>> to "pre" snapshot history followed by writing the original > > > >> >>> modification (w/o the parent's object extent data) to the HEAD > > > >> >>> revision. > > > >> >>> > > > >> >>> This change to the IO path was made to support flattening > clones and > > > >> >>> dissociating them from their parents even if the clone had > > > snapshots. > > > >> >>> > > > >> >>> Therefore, what you are seeing with export-diff is actually the > > > >> >>> backing object extent of data from the parent image written to > the > > > >> >>> clone's "pre" snapshot history. If you had two snapshots and > your > > > >> >>> export-diff'ed from the first to second snapshot, you wouldn't > see > > > >> >>> this extra data. > > > >> >>> > > > >> >>> To your question about how to prepare image B to make sure it > will > > > be > > > >> >>> exactly the same, the answer is that you don't need to do > anything. > > > In > > > >> >>> your example above, I am assuming you are manually creating an > empty > > > >> >>> Image B and using "import-diff" to populate it. The difference > in > > > the > > > >> >>> export-diff is most likely related to fact that the clone lost > its > > > >> >>> sparseness on any backing object that was written (e.g. instead > of a > > > >> >>> one or more 512 byte diffs within a backing object extent, you > will > > > >> >>> see a single, full-object extent with zeroes where the parent > image > > > >> >>> had no data). > > > >> >>> > > > >> >>> > > > >> >>> On Wed, Nov 23, 2016 at 5:06 AM, Zhongyan Gu < > zhongyan...@gmail.com > > > > > > > >> >>> wrote: > > > >> >>> > Let me make the issue more clear. > > > >> >>> > Suppose I cloned image A from a parent image and create snap1 > for > > > >> >>> > image > > > >> >>> > A > > > >> >>> > and then make some change of image A. > > > >> >>> > If I did the rbd export-diff @snap1. how should I prepare the > > > >> >>> > existing > > > >> >>> > image > > > >> >>> > B to make sure it will be exactly same with image A@snap1 > after > > > >> >>> > import-diff > > > >> >>> > against this image B. > > > >> >>> > > > > >> >>> > Thanks, > > > >> >>> > Zhongyan > > > >> >>> > > > > >> >>> > > > > >> >>> > On Wed, Nov 23, 2016 at 11:34 AM, Zhongyan Gu > > > >> >>> > <zhongyan...@gmail.com> > > > >> >>> > wrote: > > > >> >>> >> > > > >> >>> >> Thanks Jason, very clear explanation. > > > >> >>> >> However, I found some strange behavior when export-diff on a > > > cloned > > > >> >>> >> image, > > > >> >>> >> not sure it is a bug on calc_snap_set_diff(). > > > >> >>> >> The test is, > > > >> >>> >> Image A is cloned from a parent image. then create snap1 for > > > image > > > >> >>> >> A. > > > >> >>> >> The content of export-diff A@snap1 will be changed when > update > > > >> >>> >> image > > > >> >>> >> A. > > > >> >>> >> Only after image A has no overlap with parent, the content of > > > >> >>> >> export-diff > > > >> >>> >> A@snap1 is stabled, which is almost zero. > > > >> >>> >> I don't think it is a designed behavior. export-diff A@snap1 > > > should > > > >> >>> >> always > > > >> >>> >> get a stable output no matter image A is cloned or not. > > > >> >>> >> > > > >> >>> >> Please correct me if anything wrong. > > > >> >>> >> > > > >> >>> >> Thanks, > > > >> >>> >> Zhongyan > > > >> >>> >> > > > >> >>> >> > > > >> >>> >> > > > >> >>> >> > > > >> >>> >> On Tue, Nov 22, 2016 at 10:31 PM, Jason Dillaman > > > >> >>> >> <jdill...@redhat.com> > > > >> >>> >> wrote: > > > >> >>> >>> > > > >> >>> >>> On Tue, Nov 22, 2016 at 5:31 AM, Zhongyan Gu > > > >> >>> >>> <zhongyan...@gmail.com> > > > >> >>> >>> wrote: > > > >> >>> >>> > So if initial snapshot is NOT specified, then: > > > >> >>> >>> > rbd export-diff image@snap1 will diff all data to snap1. > this > > > >> >>> >>> > cmd > > > >> >>> >>> > equals to > > > >> >>> >>> > : > > > >> >>> >>> > rbd export image@snap1. Is my understand right or not?? > > > >> >>> >>> > > > >> >>> >>> > > > >> >>> >>> While they will both export all data associated w/ > image@snap1, > > > >> >>> >>> the > > > >> >>> >>> "export" command will generate a raw, non-sparse dump of the > > > full > > > >> >>> >>> image whereas "export-diff" will export only sections of the > > > image > > > >> >>> >>> that contain data. The file generated from "export" can be > used > > > >> >>> >>> with > > > >> >>> >>> the "import" command to create a new image, whereas the file > > > >> >>> >>> generated > > > >> >>> >>> from "export-diff" can only be used with "import-diff" > against > > > an > > > >> >>> >>> existing image. > > > >> >>> >>> > > > >> >>> >>> -- > > > >> >>> >>> Jason > > > >> >>> >> > > > >> >>> >> > > > >> >>> > > > > >> >>> > > > >> >>> > > > >> >>> > > > >> >>> -- > > > >> >>> Jason > > > >> >> > > > >> >> > > > >> > > > > >> > > > >> > > > >> > > > >> -- > > > >> Jason > > > > > > > > > > > > > > > > > > > > -- > > > Jason > > > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
