I'm using ZFS snapshots and send and receive for a proof of concept, and
I'd like to better understand how the incremental feature works.
Consider this example:
1. create a tar file using tar -cvf of 10 image files
2. ZFS snapshot the filesystem that contains this tar file
3. Use ZFS send and receive and ssh to replicate this file system on
another (remote) system
4. add 3 more image files to the tar file using tar -uvf
5. ZFS snapshot the same file system
6. Repeat step 3 above but this time do an incremental on the zfs send
7. observing the network traffic (iftop) I see that only the
incremental data is transferred between the systems. This is my
goal, to NOT have to resend the entire tar, or container file,
over the network of each incremental.
If I repeat the above experiment, but instead do a "tar cvf" at step 4,
and just add more image files each time, i.e.
step 1: tar cvf container01.tar file02 file02 file03 file04 file05
step 4: tar cvf container01.tar file02 file02 file03 file04 file05
file06 file07 file08
I see the amount of data equivalent to the entire container01.tar get
transferred over the network. This not the behavior I want.
In the second experimment above, what is it about ZFS that's catching
the fact that it is a "new" file.
I used tar in my experiments just because I'm familiar with it and it's
on my Solaris 10 VM's. Does the "tar uvf" do an open() with the append
flag, so ZFS somehow knows about that? What got changed when I did the
"tar cvf" the second time, writing to the same file name, but instead
with more files?
I feel like I understand what tar is doing, but I'm curious about what
is it that ZFS is looking at that makes it a "successful" incremental
send? That is, not send the entire file again. Does it have to do with
how the application (tar in this example) does a file open, fopen(), and
what mode is used? i.e. open for read, open for write, open for append.
Or is it looking at a file system header, or checksum? I'm just trying
to explain some observed behavior we're seeing during our testing.
My proof of concept is to remote replicate these "container files",
which are created by a 3rd party application.
Thanks in advance,
Pat
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
