Hi,
I will try to make this very simple and show the issue by example only
when possible. I use two old servers on the Internet for the tests. The
source use real example sparse file, but that have only ~1GB of usable
data in it. The size show by 'ls -al' as an example gives~17GB. That's
what we will use for the demonstration and explications of the issues as
presented at first were network bandwidth waist, much servers resources
waisted as well for way more then it should be and the issue of not be
able to even complete the sync in some cases as well in the process.
Also, a miss judgment at the start as looking at df output to know
quickly the space I would need to transfer the content, I forgot the
possible issue with sparse files. If I use 20% or 30% of a file system,
it's was fair for me to assume anyway that I would definitely be able to
copy these files on other systems that provide a minimum of the same
size of more. That's where I went wrong and had to work around it.
A simple mistakes one would say, however interesting to find why as it
snot obvious at first specially if you look at the final df look on
remote servers as well. If the remote system was much bigger then the
source, there isn't any problem for the transfer. Still waist bandwidth
in some cases, but it will work as the remote file systems will grow up,
but not fill up like I show in the example below. However, you would
never see it at first glance as doing df at the source or at the
destination, you see no differences in size, forgetting the space
required to transfer the data, but again, who would think that if you
don't use 50% of the space available, that you would be in a situation
where you couldn't transfer it? I could even have thought that may be if
I wanted to, I could even have done a full backup as I use 30%, so there
is plenty of space right?
So, that's what created the look for what the hell could be wrong
process and prompt me to look for a possible way to eliminate sparse
files in specific cases.
Again, just to be sure no one go crazy, sparse files are not a bad
thing. They have pretty useful at times. But you can get bitten by them
too, in some cases. (;>
1. ==============
Some numbers:
# df /home
Filesystem 512-blocks Used Avail Capacity Mounted on
/dev/wd0h 2096316 1973256 18248 99% /home
# ls -al /home/sparcefile
-rw-r--r-- 1 douellet douellet 17416290304 Nov 10 02:02 /home/sparcefile
You can see that partition can only suppose ~1GB of real data and the
file is showing up at ~17GB. So, definitely sparse file. Nothing wrong
there.
2. ==============
To answer the question about network data transfer usages. Yes a sparse
file will use that much waisted bandwidth, depending what you use to
transfer the file. Three example:
2.1 ==============
rsync without -S, just can't do. The remote disk fill itself up and then
transfer session crash, plain an simple. Plus it takes an insane amount
of time:
And just to see it live as well, I keep doing df /var/www/sites as it
was doing it to see it in action. It filled up, then crash the transfer.
Destination is:
Filesystem 512-blocks Used Avail Capacity Mounted on
/dev/wd1a 41280348 12668632 26547700 32% /var/www/sites
Filesystem 512-blocks Used Avail Capacity Mounted on
/dev/wd1a 41280348 39216344 -12 100% /var/www/sites
Filesystem 512-blocks Used Avail Capacity Mounted on
/dev/wd1a 41280348 12668632 26547700 32% /var/www/sites
Sat Nov 10 18:06:19 EST 2007
rsync: writefd_unbuffered failed to write 16385 bytes: phase "unknown"
[sender]: Broken pipe (32)
rsync: write failed on "/var/www/sites/testing/sparcefile": No space
left on device (28)
rsync error: error in file IO (code 11) at
/usr/obj/i386/rsync-2.6.6p0/rsync-2.6.6/receiver.c(291)
rsync: connection unexpectedly closed (1140235 bytes received so far)
[generator]
rsync error: error in rsync protocol data stream (code 12) at
/usr/obj/i386/rsync-2.6.6p0/rsync-2.6.6/io.c(434)
rsync: connection unexpectedly closed (1056064 bytes received so far)
[sender]
rsync error: error in rsync protocol data stream (code 12) at
/usr/obj/ports/rsync-2.6.6p0/rsync-2.6.6/io.c(434)
Sat Nov 10 18:39:22 EST 2007
2.2 ==============
With -S you can transfer it but still will take a lots of time and the
bandwidth usage is representing the real data of the file content, not
it's size. So rsync is doing an ok job here and will send the changes
only, witch is what we want. PF also reflect that.
# ./fullsync-test
Sat Nov 10 20:09:52 EST 2007
Sat Nov 10 20:40:51 EST 2007
# pfctl -si | grep Bytes
Bytes In 369728751 0
Bytes Out 10365589 0
Note, I did reset pf stats just before starting the test.
2.3 ==============
Now using scp as many times it's can also be use for quick sync of
changed files. Here however, we are up for a big surprise as well for
sure. Here we can't even do it as the sparse file like in rsync example
#1 will stop as it is to big in size, even if the data however is not.
And we will also waist way more bandwidth trying to do it in the process
as well. If the file was smaller in sparse size, then the copy process
would work, however the waisted bandwidth would be present anyway making
the point of trying to avoid the problem in the first place of
transferring sparse files across file systems. Or at best trying to use
something that would minimize the problem.
Source would stop with:
# scp /home/sparcefile [EMAIL PROTECTED]:/var/www/sites/
[EMAIL PROTECTED]'s password:
sparcefile
100% 16GB 5.2MB/s 52:50
scp: /var/www/sites//sparcefile: No space left on device
Destination would have:
# df /var/www/sites
Filesystem 512-blocks Used Avail Capacity Mounted on
/dev/wd1a 41280348 41203896 -1987564 105% /var/www/sites
# pfctl -si | grep Bytes
Bytes In 18202911548 0
Bytes Out 686933455 0
And a none usable file on the destination servers that is also incomplete.
# ls -al /var/www/sites/sparcefile
-rw-r--r-- 1 douellet douellet 15610658816 Nov 10 21:32
/var/www/sites/sparcefile
I hope this show it a bit better.
Thanks
Daniel.
