Re: About in-band dedupe for v4.7

2016-05-16 Thread David Sterba
On Fri, May 13, 2016 at 11:13:18AM +0800, Wang Shilong wrote: > I am commenting not because of Qu's patch, of course, Qu and Mark > Fasheh > Do a really good thing for Btrfs contributions. > > Just my two cents! > > 1) I think Currently, we should really focus on Btrfs stability,

Re: About in-band dedupe for v4.7

2016-05-16 Thread David Sterba
On Thu, May 12, 2016 at 01:54:26PM -0700, Mark Fasheh wrote: > On Wed, May 11, 2016 at 07:36:59PM +0200, David Sterba wrote: > > On Tue, May 10, 2016 at 07:52:11PM -0700, Mark Fasheh wrote: > > > Taking your history with qgroups out of this btw, my opinion does not > > > change. > > > > > > With r

Re: About in-band dedupe for v4.7

2016-05-13 Thread Zygo Blaxell
On Fri, May 13, 2016 at 08:14:10AM -0400, Austin S. Hemmelgarn wrote: > On 2016-05-12 16:54, Mark Fasheh wrote: > >Now ask yourself the question - would you accept a write cache which is > >expensive to fill and would only have a hit rate of less than 5%? > In-band deduplication is a feature that i

Re: About in-band dedupe for v4.7

2016-05-13 Thread Qu Wenruo
On 05/13/2016 08:14 PM, Austin S. Hemmelgarn wrote: On 2016-05-12 16:54, Mark Fasheh wrote: On Wed, May 11, 2016 at 07:36:59PM +0200, David Sterba wrote: On Tue, May 10, 2016 at 07:52:11PM -0700, Mark Fasheh wrote: Taking your history with qgroups out of this btw, my opinion does not change.

Re: About in-band dedupe for v4.7

2016-05-13 Thread Austin S. Hemmelgarn
On 2016-05-12 16:54, Mark Fasheh wrote: On Wed, May 11, 2016 at 07:36:59PM +0200, David Sterba wrote: On Tue, May 10, 2016 at 07:52:11PM -0700, Mark Fasheh wrote: Taking your history with qgroups out of this btw, my opinion does not change. With respect to in-memory only dedupe, it is my hones

Re: About in-band dedupe for v4.7

2016-05-13 Thread Duncan
Mark Fasheh posted on Thu, 12 May 2016 13:54:26 -0700 as excerpted: > For example, my 'large' duperemove test involves about 750 gigabytes of > general purpose data - quite literally /home off my workstation. > > After the run I'm usually seeing between 65-75 gigabytes saved for a > total of only

Re: About in-band dedupe for v4.7

2016-05-12 Thread Zygo Blaxell
On Fri, May 13, 2016 at 11:44:40AM +0800, Qu Wenruo wrote: > Although maybe out of your expectation, inband de-dedupe did exposed some > existing bugs we didn't ever found before. > And they are all reproducible without inband dedupe. > > Some examples: > [...] > 3) Slow backref walk. >Already

Re: About in-band dedupe for v4.7

2016-05-12 Thread Zygo Blaxell
On Wed, May 11, 2016 at 07:36:59PM +0200, David Sterba wrote: > I like the in-memory dedup backend. It's lightweight, only a heuristic, > does not need any IO or persistent storage. OTOH I consider it a subpart > of the in-band deduplication that does all the persistency etc. So I > treat the ioctl

Re: About in-band dedupe for v4.7

2016-05-12 Thread Qu Wenruo
Wang Shilong wrote on 2016/05/13 11:13 +0800: Hello Guys, I am commenting not because of Qu's patch, of course, Qu and Mark Fasheh Do a really good thing for Btrfs contributions. Just my two cents! 1) I think Currently, we should really focus on Btrfs stability, there are sti

Re: About in-band dedupe for v4.7

2016-05-12 Thread Wang Shilong
Hello Guys, I am commenting not because of Qu's patch, of course, Qu and Mark Fasheh Do a really good thing for Btrfs contributions. Just my two cents! 1) I think Currently, we should really focus on Btrfs stability, there are still many bugs hidden inside Btrfs, please See Filip

Re: About in-band dedupe for v4.7

2016-05-12 Thread Mark Fasheh
On Wed, May 11, 2016 at 07:36:59PM +0200, David Sterba wrote: > On Tue, May 10, 2016 at 07:52:11PM -0700, Mark Fasheh wrote: > > Taking your history with qgroups out of this btw, my opinion does not > > change. > > > > With respect to in-memory only dedupe, it is my honest opinion that such a > >

Re: About in-band dedupe for v4.7

2016-05-11 Thread David Sterba
On Tue, May 10, 2016 at 07:52:11PM -0700, Mark Fasheh wrote: > Taking your history with qgroups out of this btw, my opinion does not > change. > > With respect to in-memory only dedupe, it is my honest opinion that such a > limited feature is not worth the extra maintenance work. In particular > t

Re: About in-band dedupe for v4.7

2016-05-11 Thread David Sterba
On Tue, May 10, 2016 at 03:11:19PM -0700, Mark Fasheh wrote: > On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: > > Hi, Chris, Josef and David, > > > > As merge window for v4.7 is coming, it would be good to hear your > > ideas about the inband dedupe. > > > > We are addressing the ENOS

Re: About in-band dedupe for v4.7

2016-05-11 Thread David Sterba
On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: > As merge window for v4.7 is coming, it would be good to hear your ideas > about the inband dedupe. For me it's still in the process of review. > We are addressing the ENOSPC problem which Josef pointed out, and we > believe the final

Re: About in-band dedupe for v4.7

2016-05-11 Thread Qu Wenruo
Solve the hard problems first (dedupe with a disk backend, make in-memory perform), then come asking for inclusion of your feature. Again, this I would say about the patches regardless of whose name is on them. --Mark David has already agreed on the idea to merge in-memory backend f

Re: About in-band dedupe for v4.7

2016-05-10 Thread Mark Fasheh
On Wed, May 11, 2016 at 09:40:51AM +0800, Qu Wenruo wrote: > > > Chris Mason wrote on 2016/05/10 20:37 -0400: > >On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: > >>Hi, Chris, Josef and David, > >> > >>As merge window for v4.7 is coming, it would be good to hear your ideas > >>about th

Re: About in-band dedupe for v4.7

2016-05-10 Thread Mark Fasheh
On Wed, May 11, 2016 at 09:03:24AM +0800, Qu Wenruo wrote: > > > Mark Fasheh wrote on 2016/05/10 15:11 -0700: > >On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: > >>Hi, Chris, Josef and David, > >> > >>As merge window for v4.7 is coming, it would be good to hear your > >>ideas about th

Re: About in-band dedupe for v4.7

2016-05-10 Thread Satoru Takeuchi
On 2016/05/11 10:40, Qu Wenruo wrote: Chris Mason wrote on 2016/05/10 20:37 -0400: On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: Hi, Chris, Josef and David, As merge window for v4.7 is coming, it would be good to hear your ideas about the inband dedupe. We are addressing the EN

Re: About in-band dedupe for v4.7

2016-05-10 Thread Qu Wenruo
Chris Mason wrote on 2016/05/10 20:37 -0400: On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: Hi, Chris, Josef and David, As merge window for v4.7 is coming, it would be good to hear your ideas about the inband dedupe. We are addressing the ENOSPC problem which Josef pointed out, a

Re: About in-band dedupe for v4.7

2016-05-10 Thread Qu Wenruo
Mark Fasheh wrote on 2016/05/10 15:11 -0700: On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: Hi, Chris, Josef and David, As merge window for v4.7 is coming, it would be good to hear your ideas about the inband dedupe. We are addressing the ENOSPC problem which Josef pointed out, a

Re: About in-band dedupe for v4.7

2016-05-10 Thread Chris Mason
On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: > Hi, Chris, Josef and David, > > As merge window for v4.7 is coming, it would be good to hear your ideas > about the inband dedupe. > > We are addressing the ENOSPC problem which Josef pointed out, and we believe > the final fix patch wo

Re: About in-band dedupe for v4.7

2016-05-10 Thread Mark Fasheh
On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: > Hi, Chris, Josef and David, > > As merge window for v4.7 is coming, it would be good to hear your > ideas about the inband dedupe. > > We are addressing the ENOSPC problem which Josef pointed out, and we > believe the final fix patch wo

About in-band dedupe for v4.7

2016-05-10 Thread Qu Wenruo
Hi, Chris, Josef and David, As merge window for v4.7 is coming, it would be good to hear your ideas about the inband dedupe. We are addressing the ENOSPC problem which Josef pointed out, and we believe the final fix patch would come out at the beginning of the merge window.(Next week) If