Re: [zfs-discuss] zfs diff performance

2012-02-28 Thread Edward Ned Harvey
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Ulrich Graef > > Do you use de-duplication? (does not directly harm the performance, but > needs memory > and slows down zfs diff through that)? Yikes. That couldn't be more wrong. Yes, dedu

Re: [zfs-discuss] zfs diff performance

2012-02-28 Thread Ian Collins
On 02/28/12 12:53 PM, Ulrich Graef wrote: Hi Ian, On 26.02.12 23:42, Ian Collins wrote: I had high hopes of significant performance gains using zfs diff in Solaris 11 compared to my home-brew stat based version in Solaris 10. However the results I have seen so far have been disappointing. Test

Re: [zfs-discuss] zfs diff performance

2012-02-27 Thread Ulrich Graef
Hi Ian, On 26.02.12 23:42, Ian Collins wrote: I had high hopes of significant performance gains using zfs diff in Solaris 11 compared to my home-brew stat based version in Solaris 10. However the results I have seen so far have been disappointing. Testing on a reasonably sized filesystem (4TB

[zfs-discuss] zfs diff performance

2012-02-26 Thread Ian Collins
I had high hopes of significant performance gains using zfs diff in Solaris 11 compared to my home-brew stat based version in Solaris 10. However the results I have seen so far have been disappointing. Testing on a reasonably sized filesystem (4TB), a diff that listed 41k changes took 77 minu

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Ian Collins
On 09/27/11 10:59 AM, Tomas Forsman wrote: On 27 September, 2011 - Ian Collins sent me these 0,8K bytes: On 09/27/11 07:55 AM, Jesus Cea wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I just upgraded to Solaris 10 Update 10, and one of the improvements is "zfs diff". Using the "birt

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Tomas Forsman
On 27 September, 2011 - Ian Collins sent me these 0,8K bytes: > On 09/27/11 07:55 AM, Jesus Cea wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> I just upgraded to Solaris 10 Update 10, and one of the improvements >> is "zfs diff". >> >> Using the "birthtime" of the sectors, I wou

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Ian Collins
On 09/27/11 07:55 AM, Jesus Cea wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I just upgraded to Solaris 10 Update 10, and one of the improvements is "zfs diff". Using the "birthtime" of the sectors, I would expect very high performance. The actual performance doesn't seems better that

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Nico Williams
Ah yes, of course. I'd misread your original post. Yes, disabling atime updates will reduce the number of superfluous transactions. It's *all* transactions that count, not just the ones the app explicitly caused, and atime implies lots of transactions. Nico -- ___

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Bill Sommerfeld
On 09/26/11 12:31, Nico Williams wrote: > On Mon, Sep 26, 2011 at 1:55 PM, Jesus Cea wrote: >> Should I disable "atime" to improve "zfs diff" performance? (most data >> doesn't change, but "atime" of most files would change). > > atime has nothing to do with it. based on my experiences with time

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Bob Friesenhahn
On Mon, 26 Sep 2011, Jesus Cea wrote: "rsync" takes a bit less than 7 minutes. So "zfs diff" is actually slower!. It is important to define what is meant by "rsync". For example, a common rsync operating mode is to simply compare whole-file timestamps and file size in order to determine tha

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/09/11 22:54, Jesus Cea wrote: > On 26/09/11 22:29, David Magda wrote: >> Talking about "7.55 GB" is mostly useless as well. If it's a >> dozen video files then stat()ing them all with be done very >> quickly by just running find(1). If however th

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/09/11 21:31, Nico Williams wrote: > atime has nothing to do with it. > > How much work zfs diff has to do depends on how much has changed > between snapshots. That is what I thought, but look at my example: less than 20 changes and more than 1

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/09/11 22:29, David Magda wrote: > Talking about "7.55 GB" is mostly useless as well. If it's a dozen > video files then stat()ing them all with be done very quickly by > just running find(1). If however the 7.55 GB is made up of > 7,550,000 files

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread David Magda
On Mon, September 26, 2011 14:55, Jesus Cea wrote: [...] > real10m0.272s > user0m0.809s > sys 2m6.693s > """ > > 10 minutes to "diff" 7.55 GB is... disappointing. > > This machine uses a 2-mirror configurations, and there is no more > activity going on in the machine. ZPOOL version 29,

Re: [zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Nico Williams
On Mon, Sep 26, 2011 at 1:55 PM, Jesus Cea wrote: > I just upgraded to Solaris 10 Update 10, and one of the improvements > is "zfs diff". > > Using the "birthtime" of the sectors, I would expect very high > performance. The actual performance doesn't seems better that an > standard "rdiff", though

[zfs-discuss] "zfs diff" performance disappointing

2011-09-26 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I just upgraded to Solaris 10 Update 10, and one of the improvements is "zfs diff". Using the "birthtime" of the sectors, I would expect very high performance. The actual performance doesn't seems better that an standard "rdiff", though. Quite disappo

Re: [zfs-discuss] zfs diff cannot stat shares

2010-10-14 Thread dirk schelfhout
I had to upgrade zfs zfs upgrade -a then pfexec zfs set sharesmb=off data pfexec zfs set sharesmb=on data after this zfs diff failed with the old snapshots. But with newly created snapshots it worked. Thanks Tim, Dirk -- This message posted from opensolaris.org ___

Re: [zfs-discuss] zfs diff cannot stat shares

2010-10-14 Thread David Blasingame Oracle
a diff to list the file differences between snapshots http://arc.opensolaris.org/caselog/PSARC/2010/105/mail Dave On 10/13/10 15:48, Edward Ned Harvey wrote: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of dirk schelfhout Wanted to test the

Re: [zfs-discuss] zfs diff cannot stat shares

2010-10-13 Thread Edward Ned Harvey
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of dirk schelfhout > > Wanted to test the zfs diff command and ran into this. What's zfs diff? I know it's been requested, but AFAIK, not implemented yet. Is that new feature being developed no

Re: [zfs-discuss] zfs diff cannot stat shares

2010-10-13 Thread dirk schelfhout
cd /data/.zfs sche...@osolpro:/data/.zfs$ ls -alt ls: cannot access shares: Operation not supported total 4 drwxr-xr-x 19 schelfd staff 25 2010-10-13 18:57 .. dr-xr-xr-x 2 rootroot 2 2010-10-13 17:44 snapshot dr-xr-xr-x 4 rootroot 4 2009-01-28 23:08 . ?? ? ? ? ?

Re: [zfs-discuss] zfs diff cannot stat shares

2010-10-13 Thread Tim Haley
On 10/13/10 10:20 AM, dirk schelfhout wrote: Wanted to test the zfs diff command and ran into this. I turned off all windows sharing. the rpool has normal permissions for .zfs/shares how do I fix this ? Dirk r...@osolpro:/data/.zfs# zfs diff d...@10aug2010 d...@13oct2010 Cannot stat /data/.zfs/

[zfs-discuss] zfs diff cannot stat shares

2010-10-13 Thread dirk schelfhout
Wanted to test the zfs diff command and ran into this. I turned off all windows sharing. the rpool has normal permissions for .zfs/shares how do I fix this ? Dirk r...@osolpro:/data/.zfs# zfs diff d...@10aug2010 d...@13oct2010 Cannot stat /data/.zfs/shares/: unable to generate diffs pwd /data/.z

Re: [zfs-discuss] zfs diff

2010-03-30 Thread Edward Ned Harvey
> On Mon, Mar 29, 2010 at 5:39 PM, Nicolas Williams > wrote: > > One really good use for zfs diff would be: as a way to index zfs send > > backups by contents. > > Or to generate the list of files for incremental backups via NetBackup > or similar. This is especially important for file systems w

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Ian Collins
On 03/30/10 12:44 PM, Mike Gerdts wrote: On Mon, Mar 29, 2010 at 5:39 PM, Nicolas Williams wrote: One really good use for zfs diff would be: as a way to index zfs send backups by contents. Or to generate the list of files for incremental backups via NetBackup or similar. This is es

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Tim Haley
On 3/29/10 8:02 PM, Daniel Carosone wrote: On Tue, Mar 30, 2010 at 12:37:15PM +1100, Daniel Carosone wrote: There will also need to be clear rules on output ordering, with respect to renames, where multiple changes have happened to renamed files. Separately, but relevant in particular to the

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Daniel Carosone
On Tue, Mar 30, 2010 at 12:37:15PM +1100, Daniel Carosone wrote: > There will also need to be clear rules on output ordering, with > respect to renames, where multiple changes have happened to renamed > files. Separately, but relevant in particular to the above due to the potential for races: wh

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Daniel Carosone
On Mon, Mar 29, 2010 at 06:38:47PM -0400, David Magda wrote: > A new ARC case: I read this earlier this morning. Welcome news indeed! I have some concerns about the output format, having worked with similar requirements in the past. In particular: as part of the monotone VCS when reporting works

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Bart Smaalders
On 03/29/10 16:44, Mike Gerdts wrote: On Mon, Mar 29, 2010 at 5:39 PM, Nicolas Williams wrote: One really good use for zfs diff would be: as a way to index zfs send backups by contents. Or to generate the list of files for incremental backups via NetBackup or similar. This is especially imp

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Mike Gerdts
On Mon, Mar 29, 2010 at 5:39 PM, Nicolas Williams wrote: > One really good use for zfs diff would be: as a way to index zfs send > backups by contents. Or to generate the list of files for incremental backups via NetBackup or similar. This is especially important for file systems will millions o

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Bruno Sousa
On 30-3-2010 0:39, Nicolas Williams wrote: > One really good use for zfs diff would be: as a way to index zfs send > backups by contents. > > Nico > Any prevision about the release target? snv_13x? Bruno smime.p7s Description: S/MIME Cryptographic Signature

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Nicolas Williams
zfs diff is incredibly cool. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] zfs diff

2010-03-29 Thread Nicolas Williams
One really good use for zfs diff would be: as a way to index zfs send backups by contents. Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[zfs-discuss] zfs diff

2010-03-29 Thread David Magda
A new ARC case: There is a long-standing RFE for zfs to be able to describe what has changed between the snapshots of a dataset. To provide this capability, we propose a new 'zfs diff' sub-command. When run with appropriate privilege the sub-command describes what file system level changes hav

[zfs-discuss] zfs diff @snap1 @snap2

2008-05-18 Thread Akhilesh Mritunjai
Hi Is it possible to see what changed between two snapshots (efficiently) ? I tried to take a look what "zfs send -i" does, and I found that it operates at very low (dmu) level and basically dumps the blocks. Any pointers on extracting inode info from this stream or otherwise ? - mritun Th