Re: Batch mode creates huge diffs, bug(s)?

2012-03-20 Thread Matt Van Mater
I ran one more test on a separate VM to check and see if rsync would allow me to specify block size for a smaller file while using batch mode... it works. To me that indicates that rsync has a problem processing very large batch files, especially when you specify a particular block size. More sig

Re: Batch mode creates huge diffs, bug(s)?

2012-03-20 Thread Joachim Otahal (privat)
Matt Van Mater schrieb: Let me restate my last email regarding rdiff: All of my image files are from the same Windows XP VM, created using FOG/partimage.  Image1 is the "baseline", Image2 is Image1 + the WinSCP binary downloaded (not even installed).

Re: Batch mode creates huge diffs, bug(s)?

2012-03-20 Thread Joachim Otahal (privat)
Maybe don't sync one big file, hack the image up in small chunks, then whatever the gap size is rsync might have a bigger chance to resync with including --fuzzy. Though it might not help at all since the number of files would be large. IF it is only a "once every fe

Re: Batch mode creates huge diffs, bug(s)?

2012-03-20 Thread Matt Van Mater
Let me restate my last email regarding rdiff: All of my image files are from the same Windows XP VM, created using FOG/partimage. Image1 is the "baseline", Image2 is Image1 + the WinSCP binary downloaded (not even installed). I am not imaging an Ubuntu machine. I am using the Ubuntu machine as

Re: Batch mode creates huge diffs, bug(s)?

2012-03-20 Thread Matt Van Mater
I agree with your assessment somewhat Joachim and think you're following the same line of reasoning as I am. Some details I did not include in my first post: FOG/partimage does indeed only capture the used blocks in its images when you select "ntfs - resizable". So running a clean utility (e.g.

Re: Batch mode creates huge diffs, bug(s)?

2012-03-20 Thread Joachim Otahal (privat)
Matt Van Mater schrieb: Alternate assessment - I ran a similar comparison against the two image files using rdiff that comes with Ubuntu 10.04.4 LTS (shown up as librsync 0.9.7) and have a significantly smaller delta file (closer to what i expect). Just plain luck. If ubuntu wrote the most

Re: Batch mode creates huge diffs, bug(s)?

2012-03-20 Thread Matt Van Mater
Thanks for your response Eric but I disagree with your assessment and here is why: Functionally - I agree that Windows is bound to update multiple timestamps on log files, registry entries, pagefile, etc every time it boots. However I think it is unrealistic to assume literally half of the capaci

Re: Batch mode creates huge diffs, bug(s)?

2012-03-20 Thread Joachim Otahal (privat)
Matt Van Mater schrieb: image1 size in bytes: 17,062,442,700 image2 size in bytes: 16,993,256,652 about 70 MB of change between a boot with a small program install. That is realistic. Thi

Re: Batch mode creates huge diffs, bug(s)?

2012-03-20 Thread ericbambach1
Matt, Its probably not a rsync bug. Its likely that after booting to create the second image a large number of updates has happened at many different parts in the filesystem. You may have added only a few MB of data but a lot of little things are going on in an active system like files

Batch mode creates huge diffs, bug(s)?

2012-03-20 Thread Matt Van Mater
So the short summary of my problem is, the batch file rsync creates is HUGE for a very small change. The idea is to create workstation image with partimage, update it with some software and send the image update diff over the wire to a large number of destinations over a satellite link, but the ba

FW: killing rsync seems to wipe-out the --partial .hidden files

2012-03-20 Thread Frank Hamersley
... then reads about the very next switch ... --delay-updates ... which has some of the hallmarks expressed in the synchronised directory - but I expect iff the rsync session is uninterrupted? Cheers, Frank. -Original Message- From: Frank Hamersley [mailto:terab...@bigpond.com] Sent

RE: killing rsync seems to wipe-out the --partial .hidden files

2012-03-20 Thread Frank Hamersley
Thanks for the tip - it will be employed tonight, Looks like it has some of the specified attributes - but not ... 1. the "unified directory" which is of course not rsync's pedigree being file oriented per se, and 2. I will be interested to see how "smart" the restart is in terms of the dropped 8

Re: killing rsync seems to wipe-out the --partial .hidden files

2012-03-20 Thread Paul Slootman
On Tue 20 Mar 2012, Frank Hamersley wrote: > > Thinking quickly (as I have to go to a Mindari) the approach I would take > for --partial is to ... Perhaps you need to examine the manpage bit more thoroughly, you could e.g. use --partial-dir. Paul -- Please use reply-all for most replies to avo

RE: killing rsync seems to wipe-out the --partial .hidden files

2012-03-20 Thread Frank Hamersley
G'day Paul, Thanks for the prompt response. It is prolly just me but this behaviour seems rather counter-intuitive! To give more context to the "business" interest the subject files are database dumps ... not very large (2G per stripe) and there are 4 in total. They are uncompressed and do not c