Re: [zfs-discuss] offline dedup

2011-05-29 Thread Daniel Carosone
On Fri, May 27, 2011 at 07:28:06AM -0400, Edward Ned Harvey wrote: > > From: Daniel Carosone [mailto:d...@geek.com.au] > > Sent: Thursday, May 26, 2011 8:19 PM > > > > Once your data is dedup'ed, by whatever means, access to it is the > > same. You need enough memory+l2arc to indirect references

Re: [zfs-discuss] offline dedup

2011-05-27 Thread Frank Van Damme
2011/5/27 Edward Ned Harvey : > I don't think this is true.  The reason you need arc+l2arc to store your DDT > is because when you perform a write, the system will need to check and see > if that block is a duplicate of an already existing block.  If you dedup > once, and later disable dedup, the s

Re: [zfs-discuss] offline dedup

2011-05-27 Thread Jim Klimov
Dan> ... It would still need a complex bp_rewrite. Are you certain about that? For example, scrubbing/resilvering and fixing corrupt blocks with non-matching checksums is a post-processing operation which works on an existing pool and rewrites some blocks if needed. And it works without a bp_rew

Re: [zfs-discuss] offline dedup

2011-05-27 Thread Edward Ned Harvey
> From: Daniel Carosone [mailto:d...@geek.com.au] > Sent: Thursday, May 26, 2011 8:19 PM > > Once your data is dedup'ed, by whatever means, access to it is the > same. You need enough memory+l2arc to indirect references via > DDT. I don't think this is true. The reason you need arc+l2arc to s

Re: [zfs-discuss] offline dedup

2011-05-26 Thread Daniel Carosone
On Fri, May 27, 2011 at 04:32:03AM +0400, Jim Klimov wrote: > One more rationale in this idea is that with deferred dedup > in place, the DDT may be forced to hold only non-unique > blocks (2+ references), and would require less storage in > RAM, disk, L2ARC, etc. - in case we agree to remake the >

Re: [zfs-discuss] offline dedup

2011-05-26 Thread Jim Klimov
2011-05-26 19:37, Edward Ned Harvey ?: Hey, I got another question for ZFS developers - Given: If you enable dedup and write a bunch of data, and then disable dedup, the formerly written data will remain dedup'd. Given: The zdb -s command, which simulates dedup to provide dedup statis

Re: [zfs-discuss] offline dedup

2011-05-26 Thread Daniel Carosone
On Thu, May 26, 2011 at 09:04:04AM -0700, Brandon High wrote: > On Thu, May 26, 2011 at 8:37 AM, Edward Ned Harvey > wrote: > > Question:? Is it possible, or can it easily become possible, to periodically > > dedup a pool instead of keeping dedup running all the time?? It is easy to > > I think i

Re: [zfs-discuss] offline dedup

2011-05-26 Thread Brandon High
On Thu, May 26, 2011 at 8:37 AM, Edward Ned Harvey wrote: > Question:  Is it possible, or can it easily become possible, to periodically > dedup a pool instead of keeping dedup running all the time?  It is easy to I think it's been discussed before, and the conclusion is that it would require bp_

[zfs-discuss] offline dedup

2011-05-26 Thread Edward Ned Harvey
Hey, I got another question for ZFS developers - Given: If you enable dedup and write a bunch of data, and then disable dedup, the formerly written data will remain dedup'd. Given: The zdb -s command, which simulates dedup to provide dedup statistics without actually enabling dedup.