Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Eric Shubert
Thanks Fredrick. That certainly clarifies the matter. Now if I could only remember to use info once in a while. Oh yeah, the navigation. ;) (coming from a long time vi user) -- -Eric 'shubes' On 10/22/2011 10:41 PM, Frederick Grose wrote: On Sun, Oct 23, 2011 at 12:14 AM, Kevin Korb mailto:k..

Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yes, I am aware of info pages. The problem is that reading them is more inconvenient than reverse engineering the behavior of the program. Luckily we have Google to present actual information in a readable format. On 10/23/11 01:41, Frederick Grose w

Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Frederick Grose
On Sun, Oct 23, 2011 at 12:14 AM, Kevin Korb wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Well, I just looked at 'man du' and you are right about it. I could > swear that it used to say something more useful about the subject but I > couldn't quote it. info du "If two or mo

Re: handling of final partial block in rsync

2011-10-22 Thread Wayne Davison
On Fri, Oct 21, 2011 at 7:28 PM, Peter A. Friend wrote: > I've read the thesis and the source again, and it looks like rsync is > generating a signature on the final partial block. Would someone be able to > confirm this, and is rsync handling this as I describe? > Search for "remainder" in gener

Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Well, I just looked at 'man du' and you are right about it. I could swear that it used to say something more useful about the subject but I couldn't quote it. In general the man pages for GNU tools suck. On 10/23/11 00:05, Eric Shubert wrote: > I've

Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Eric Shubert
I've read the man page: -s, --summarize display only a total for each argument Leaves a bit to interpretation, I think. Could go either way, no? After all, the name says: du - estimate file space usage I guess du estimates better than I assumed. Thanks for setting me straight Kevi

Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 lol, I have been doing rsync based backups since sometime in 2000. I have learned a few tricks. On 10/23/11 00:00, Eric Shubert wrote: > On 10/22/2011 08:49 PM, Eric Shubert wrote: >> How sure are you of that? I guess I assumed that -s would simply >

Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Eric Shubert
On 10/22/2011 08:49 PM, Eric Shubert wrote: How sure are you of that? I guess I assumed that -s would simply summarize the total for each directory entry. Should probably do a test to see if that's indeed true or not. Thanks Kevin. You are the wise one: testdir$ ls -lr * -rw-r--r-- 3 shubes sh

Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The man page says so. My experience doesn't disagree. I am satisfied. On 10/22/11 23:49, Eric Shubert wrote: > How sure are you of that? I guess I assumed that -s would simply > summarize the total for each directory entry. Should probably do a test

Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Eric Shubert
How sure are you of that? I guess I assumed that -s would simply summarize the total for each directory entry. Should probably do a test to see if that's indeed true or not. -- -Eric 'shubes' On 10/22/2011 08:43 PM, Kevin Korb wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 du -shc back

Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 du -shc backup1 backup2 ... du -s will not count the same file twice even if it appears 50 times. On 10/22/11 23:32, Eric Shubert wrote: > On 10/22/2011 12:32 PM, Kevin Korb wrote: >> 2. I wanted to know what files were unique to a backu

Re: With rsync --link-dest, how can you tell how much space is actually used?

2011-10-22 Thread Eric Shubert
On 10/22/2011 12:32 PM, Kevin Korb wrote: 2. I wanted to know what files were unique to a backup and therefore consuming disk space and why they were different. Which begs another question that I've struggled a bit with lately. I've considered making a series of snapshots by using --link-dest

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Eric Shubert
I agree. Raid is no backup. I expect your experience is extensive to see that many dual drive RAID1 failures. Haven't seen one yet myself (knock wood). Several years ago, I recovered all but a handful of files from a 3-drive raid-5 array where 2 drives had failed. Wasn't pretty. Needless to s

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I would say that not doing backups of RAID at any level is rather unwise. Of course I have seen dual drive failures of a RAID1 array on about 5 occasions. I have also seen a dual drive failure and a controller SNAFU of a RAID5 on 2 occasions. Simply

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Eric Shubert
On 10/22/2011 07:12 AM, Andrew Gideon wrote: On Fri, 21 Oct 2011 12:10:09 -0400, Kevin Korb wrote: If you want something you can run after the fact here is a tool I wrote a while back that does a sort of diff across 2 --link-dest based backups: http://sanitarium.net/unix_stuff/rspaghetti_backup

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Eric Shubert
On 10/22/2011 04:04 PM, Ido Magal wrote: 'find' on the droboFS doesn't support '-links' so I had to get creative. And apparently perl has issues on it as well. On Sat, Oct 22, 2011 at 08:02, Eric Shubert wrote: # find /backup/dir -type f -links 1 ? Sorry, I'm not familiar with droboFS. So

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Ido Magal
'find' on the droboFS doesn't support '-links' so I had to get creative. And apparently perl has issues on it as well. On Sat, Oct 22, 2011 at 08:02, Eric Shubert wrote: > > # find /backup/dir -type f -links 1 > ? -- Please use reply-all for most replies to avoid omitting the mailing list. To u

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Well, I had 2 reasons for writing the script... 1. I wanted to know what was different between 2 backups. Capturing - --itemize-changes can certainly do that (I don't remember if my script pre-dates --itemize-changes or not but it predates my knowle

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Andrew Gideon
On Fri, 21 Oct 2011 19:14:00 -0700, Ido Magal wrote: >>error while loading shared libraries: libperl.so.5.10: cannot open >>shared object file: No such file or directory This doesn't appear to be a complaint about something the Perl script is doing, but about Perl itself not working. -

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Andrew Gideon
On Sat, 22 Oct 2011 12:07:33 -0400, Kevin Korb wrote: > If that is the only thing that is different you might be using rsync > incorrectly. And that's why you left it being reported? Interesting idea. Thanks for explaining. - Andrew -- Please use reply-all for most replies to avoid o

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Opps, forgot that one. That isn't a perl library either but another shell command. On 10/22/11 11:02, Eric Shubert wrote: > On 10/21/2011 07:14 PM, Ido Magal wrote: >> Thanks! >> >> Unfortunately, it doesn't look like the perl for DroboFS supports th

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yes, if a file is a different inode it will be reported by my script even if that is the only difference. If the inode number is different then the file is in fact different and takes up additional disk space. If that is the only thing that is differe

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Andrew Gideon
On Fri, 21 Oct 2011 12:10:09 -0400, Kevin Korb wrote: > If you want something you can run after the fact here is a tool I wrote > a while back that does a sort of diff across 2 --link-dest based > backups: > http://sanitarium.net/unix_stuff/rspaghetti_backup/diff_backup.pl.txt It > will also tell

Re: With rsync --link-dest, is it possible to determine which files are new?

2011-10-22 Thread Eric Shubert
On 10/21/2011 07:14 PM, Ido Magal wrote: Thanks! Unfortunately, it doesn't look like the perl for DroboFS supports the required libraries for your script, error while loading shared libraries: libperl.so.5.10: cannot open shared object file: No such file or directory but --itemize-changes s