Re: process --files-from filelist as given

2023-12-31 Thread Francis.Montagnac--- via rsync
Hi. On Sun, 31 Dec 2023 20:28:21 +0100 Roland via rsync wrote: > apparently, rsync sorts the list of files  provided to "--files-from". > how can i avoid sorting of that list ? According to the man, this is not possible. See: SORTED TRANSFER ORDER that suggest also the --dela

[Bug 15546] New: disable of sorting when files to transfer is fed via --files-from

2023-12-31 Thread just subscribed for rsync-qa from bugzilla via rsync
https://bugzilla.samba.org/show_bug.cgi?id=15546 Bug ID: 15546 Summary: disable of sorting when files to transfer is fed via --files-from Product: rsync Version: 3.2.0 Hardware: All OS: All

Re: process --files-from filelist as given

2023-12-31 Thread Roland via rsync
I want to copy a list of files in specific order Why ? because i want to serialize files on disk so they are stored on disk in the order being accessed regularly i built that list for --files-from   via output from fatrace tool. i now did use tar to transfer the files i will add an RFE to

process --files-from filelist as given

2023-12-31 Thread Roland via rsync
hello, apparently,  rsync sorts the list of files  provided to "--files-from". how can i avoid sorting of that list ? I want to copy a list of files in specific order regards Roland -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe

Re: rsync over ssh fails with --files-from

2023-12-21 Thread Alex via rsync
> The errors column is 0. The drop column is 18. The second bit number > is the number of packets which should grow. At least that is how I read > it. Column makes it more readable in a terminal but not so much in an > email. > Yes, my apologies. I even debated inserting a screenshot. errs was

Re: rsync over ssh fails with --files-from

2023-12-21 Thread Kevin Korb via rsync
<http://config1.cf/>,0) config1.cf <http://config1.cf/> is uptodate send_files(0, config1.cf <http://config1.cf/>) then it just stalls until it eventually times out. However, if I remove any one of the nine files from the filelist, it completes normally. I

Re: rsync over ssh fails with --files-from

2023-12-21 Thread Alex via rsync
nabled > recv_generator(config1.cf,0) > config1.cf is uptodate > send_files(0, config1.cf) > > then it just stalls until it eventually times out. However, if I remove > any one of the nine files from the filelist, it completes normally. > > It actually also exhibits the same proble

Re: rsync over ssh fails with --files-from

2023-12-20 Thread Alex via rsync
aving a problem with transferring multiple files > > at a time to one specific host using --files-from. I can't think of what > > might have changed to have caused this. Using rsync to transfer a single > > file to this problematic host works successfully. It appears to be >

Re: rsync over ssh fails with --files-from

2023-12-20 Thread Kevin Korb via rsync
at a time to one specific host using --files-from. I can't think of what might have changed to have caused this. Using rsync to transfer a single file to this problematic host works successfully. It appears to be related to the number of files in the --files-from filelist. More than n

rsync over ssh fails with --files-from

2023-12-20 Thread Alex via rsync
Hi, I've been using rsync on fedora over ssh to sync directories for decades, but suddenly having a problem with transferring multiple files at a time to one specific host using --files-from. I can't think of what might have changed to have caused this. Using rsync to transfer a sing

Re: prevent filter excluded files from being deleted

2023-03-07 Thread Heiko Schlittermann via rsync
Robin Lee Powell via rsync (Di 07 Mär 2023 16:20:39 CET): > First of all, I disagree that rsync handles very large files badly, > but that's not super relevant. :) Consdier --partial for large > files. I'm using --inplace (because the receiving side does snapshots of the underlying file system)

Re: prevent filter excluded files from being deleted

2023-03-07 Thread Robin Lee Powell via rsync
e it out myself). > > I did, but left with some uncertainty. > > "H" hides the files from the transfer? What does it mean? > "P" protects files from being deleted? > > I even tried '+r *', no luck. > > > Short version: per-directory ru

Re: prevent filter excluded files from being deleted

2023-03-07 Thread Heiko Schlittermann via rsync
Hello Kevin, Kevin Korb via rsync (Di 07 Mär 2023 00:01:27 CET): > I am not 100% sure I am interpreting this correctly but I think you are > complaining that the file was being deleted in the first command? If so, > instead of -F try --include='*/' --exclude='*'. Otherwise, maybe you want a > s

Re: prevent filter excluded files from being deleted

2023-03-07 Thread Heiko Schlittermann via rsync
Robin Lee Powell via rsync (Di 07 Mär 2023 07:07:01 CET): > Read the "PER-DIRECTORY RULES AND DELETE" of the man page. (And > don't feel bad, it took me a while to figure it out myself). I did, but left with some uncertainty. "H" hides the files from the

Re: prevent filter excluded files from being deleted

2023-03-06 Thread Robin Lee Powell via rsync
Read the "PER-DIRECTORY RULES AND DELETE" of the man page. (And don't feel bad, it took me a while to figure it out myself). Short version: per-directory rules only apply on the side they're *specified on*, but you need the exclusion to apply to *both* sides. The following works, for the reasons

Re: prevent filter excluded files from being deleted

2023-03-06 Thread Kevin Korb via rsync
I am not 100% sure I am interpreting this correctly but I think you are complaining that the file was being deleted in the first command? If so, instead of -F try --include='*/' --exclude='*'. Otherwise, maybe you want a second -F? On 3/6/23 16:04, Heiko Schlittermann via rsync wrote: Hello

prevent filter excluded files from being deleted

2023-03-06 Thread Heiko Schlittermann via rsync
Hello, given are 2 directories: a ├── a-file └── .rsync-filter b └── a-file I'd like to sync a/ -> b/, but I'd like to *exclude* all files. But I do not want to delete the excluded files. (The real scenario is a way more complex, the above is my reproducer.) and the follow

[Bug 3465] Option to delete unlisted files with --files-from

2020-07-27 Thread just subscribed for rsync-qa from bugzilla via rsync
https://bugzilla.samba.org/show_bug.cgi?id=3465 --- Comment #8 from Wayne Davison --- Jeff: the command you mention already works fine, since --files-from is not involved. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to

Re: sync of ordinary files from lvm snapshot fails with --sparse, 3.2.2pre2

2020-07-07 Thread Scott Mcdermott via rsync
On 2020-07-06 12:40 -0700 Perry Hutchison wrote: > we either need to read the part of the output file that > we're about to seek over -- and overwrite anything that reads back > non-zero with zeros The files that give verification errors are just small text files with no null bytes. The sparse co

Re: sync of ordinary files from lvm snapshot fails with --sparse, 3.2.2pre2

2020-07-06 Thread Perry Hutchison via rsync
Scott Mcdermott via rsync wrote: > Getting some puzzling errors doing the following backup procedure > on a root filesystem that's on LVM, when using sparse flag: > > lvcreate --snapshot --name baksnap --size 3G /dev/vg0/root > mount -o ro /dev/vg0/baksnap /var/tmp/snapmnt > rsync -izaHx \ >

sync of ordinary files from lvm snapshot fails with --sparse, 3.2.2pre2

2020-07-05 Thread Scott Mcdermott via rsync
Getting some puzzling errors doing the following backup procedure on a root filesystem that's on LVM, when using sparse flag: lvcreate --snapshot --name baksnap --size 3G /dev/vg0/root mount -o ro /dev/vg0/baksnap /var/tmp/snapmnt rsync -izaHx \ --delete --delete-excluded --delete-after \

[Bug 14395] New: Add an option to remove exclude files from the source

2020-05-26 Thread just subscribed for rsync-qa from bugzilla via rsync
https://bugzilla.samba.org/show_bug.cgi?id=14395 Bug ID: 14395 Summary: Add an option to remove exclude files from the source Product: rsync Version: 3.2.0 Hardware: All OS: All Status: NEW Severity

Re: hardlinking missing files from src to a dest: didn't work way I thought it would.

2019-11-14 Thread Pierre Bernhardt via rsync
After rsync ~/tmp/b should also have all the files linked from a which are also not modified on local ~/o/ and created new files from o to b which are new or modified: pierre@in94:~$ ls -li o tmp/a tmp/b o: insgesamt 8 257419 -rw-r--r-- 1 pierre pierre 4 Nov 14 10:53 1 257420 -rw-r--r-- 1 pier

Re: hardlinking missing files from src to a dest: didn't work way I thought it would.

2019-11-14 Thread Pierre Bernhardt via rsync
Am 14.11.19 um 15:02 schrieb Paul Slootman via rsync: > On Thu 14 Nov 2019, Pierre Bernhardt via rsync wrote: > So it's looking for b/a as the link-dest directory. > > Use a full pathname for --link-dest to remove all uncertainty. > E.g.: > > rsync -av --link-dest=$(pwd)/a a/ b/ > > In this

Re: hardlinking missing files from src to a dest: didn't work way I thought it would.

2019-11-14 Thread Paul Slootman via rsync
On Thu 14 Nov 2019, Pierre Bernhardt via rsync wrote: > Am 14.11.19 um 10:54 schrieb Paul Slootman via rsync: > > You need to specify the source directory as the link-dest directory. > > Hi, I tried it also because it's an old question which has never worked > for me. Instead it creates copies and

Re: hardlinking missing files from src to a dest: didn't work way I thought it would.

2019-11-14 Thread Pierre Bernhardt via rsync
Am 14.11.19 um 10:54 schrieb Paul Slootman via rsync: > You need to specify the source directory as the link-dest directory. Hi, I tried it also because it's an old question which has never worked for me. Instead it creates copies and not hard links: pierre@in94:~/tmp$ ls -li a b a: insgesamt 8

Re: hardlinking missing files from src to a dest: didn't work way I thought it would.

2019-11-14 Thread Paul Slootman via rsync
On Thu 14 Nov 2019, L A Walsh via rsync wrote: > Have a directory with a bunch rpms in it, mostly x86_64. > > Have another directory with a bunch, mostly 'noarch'. > > Some of the noarch files are already in the x86_64 dir > and don't want to overwrite them. They are on the same > physical disk

hardlinking missing files from src to a dest: didn't work way I thought it would.

2019-11-14 Thread L A Walsh via rsync
Have a directory with a bunch rpms in it, mostly x86_64. Have another directory with a bunch, mostly 'noarch'. Some of the noarch files are already in the x86_64 dir and don't want to overwrite them. They are on the same physical disk, so really, just want the new 'noarch' files hardlinked into

Re: rsync of big image files from virtual hosts takes lot of time

2018-10-21 Thread Fabian Cenedese via rsync
>This is the output with --stats for a 50GB image. >Appended also the output from time. >You see it's horrible slow. Please give also the source and destination parameters from your rsync command. These define whether it's a local sync or not (for rsync), the hardware setup doesn't really matter

Re: rsync of big image files from virtual hosts takes lot of time

2018-10-21 Thread Kevin Korb via rsync
Your rsync must be a bit old. The current --stats output is about twice as long as that. Anyway, there isn't that much change in the file. If you were using rsync over ssh it would probably take around 30 minutes. I would guess cp would take longer but hard to say how much longer but it would b

Re: rsync of big image files from virtual hosts takes lot of time

2018-10-21 Thread Lentes, Bernd via rsync
- Am 20. Okt 2018 um 20:05 schrieb rsync ML rsync@lists.samba.org: > I don't see any --stats output. It will tell you exactly how much data > was involved. > > A local copy means that rsync isn't networking. Rsync speeds things up > by running on 2 separate computers and communicating us

Re: rsync of big image files from virtual hosts takes lot of time

2018-10-20 Thread Kevin Korb via rsync
I don't see any --stats output. It will tell you exactly how much data was involved. A local copy means that rsync isn't networking. Rsync speeds things up by running on 2 separate computers and communicating using the rsync protocol. If you are running rsync on a single computer with the sourc

Re: rsync of big image files from virtual hosts takes lot of time

2018-10-20 Thread Lentes, Bernd via rsync
- Am 20. Okt 2018 um 18:56 schrieb rsync ML rsync@lists.samba.org: > First, add --stats to find out how much is being transferred. Second, > if these are fixed size or sparse image files then --sparse will be a > big help. Finally, if they are not fixed size or sparse then --inplace > is

Re: rsync of big image files from virtual hosts takes lot of time

2018-10-20 Thread Kevin Korb via rsync
doing a local copy. If you are doing a local copy then just use cp as it is much faster at that. On 10/20/2018 12:36 PM, Lentes, Bernd via rsync wrote: > Hi, > > i'd like to rsync some image files from virtual hosts each night. The image > files are between 50GB and 400GB big.

rsync of big image files from virtual hosts takes lot of time

2018-10-20 Thread Lentes, Bernd via rsync
Hi, i'd like to rsync some image files from virtual hosts each night. The image files are between 50GB and 400GB big. I thought rsync would be the appropriate solution because it just transfers the differences, not the whole file. But nevertheless rsync takes hours. The images aren&#

[Bug 11656] Escaping broken with --files-from

2017-10-08 Thread just subscribed for rsync-qa from bugzilla via rsync
https://bugzilla.samba.org/show_bug.cgi?id=11656 Wayne Davison changed: What|Removed |Added Status|NEW |RESOLVED Resolution|---

[Bug 12378] why i cannot exclude dir/files if using option "--files-from"

2016-10-14 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=12378 --- Comment #6 from Kevin Korb --- If you want to sync files newer than say 3 days ago that is what --files-from is for... cd /source find . -mtime -3 -print | rsync -vai --files-from=- . /target The primary purpose of --files-from is to give

[Bug 12378] why i cannot exclude dir/files if using option "--files-from"

2016-10-14 Thread samba-bugs
what happens to me. And if you remove the "--files-from" you see that "cache" folder is really excluded from the copy. I was wondering why the behavior is different applying the "--files-from" or not and unfortunately WayneD said it's by design. Thanks for y

[Bug 12378] why i cannot exclude dir/files if using option "--files-from"

2016-10-13 Thread samba-bugs
|RESOLVED --- Comment #4 from Wayne Davison --- Command args (which includes names inside a files-from file) are never excluded by an exclude directive. You told rsync to copy it, so it copies it. Excludes only affect matching of files that rsync finds inside directories that you

[Bug 12378] why i cannot exclude dir/files if using option "--files-from"

2016-10-13 Thread samba-bugs
tia[3%]> cd !$ cd test kmk@dementia[4%]> mkdir src dst kmk@dementia[5%]> cd src kmk@dementia[6%]> mkdir -p a b c d e cache/c kmk@dementia[7%]> cd .. kmk@dementia[8%]> /bin/echo -ne "a\nb\nc\nd\ne\ncache/c\n" > list kmk@dementia[9%]> cat list a b c d e cache/c kmk

[Bug 12378] why i cannot exclude dir/files if using option "--files-from"

2016-10-13 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=12378 --- Comment #2 from Alessio --- (In reply to Kevin Korb from comment #1) Excuse me, but i don't get it, what's the difference between excluding (matching) the pattern from a --files-from and a remote host? debugging the rsync transfe

[Bug 12378] why i cannot exclude dir/files if using option "--files-from"

2016-10-13 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=12378 --- Comment #1 from Kevin Korb --- It did not copy the directory you excluded it copied the files within that directory that you explicitly told it to copy and created the appropriate directories to allow that to happen. IOW, .cache is not relativ

[Bug 12378] New: why i cannot exclude dir/files if using option "--files-from"

2016-10-13 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=12378 Bug ID: 12378 Summary: why i cannot exclude dir/files if using option "--files-from" Product: rsync Version: 3.1.2 Hardware: x64 OS: Linux

[Bug 11671] New: --files-from= with nonexistant files and --delete-missing-args complains about vanished source files

2016-01-13 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11671 Bug ID: 11671 Summary: --files-from= with nonexistant files and --delete-missing-args complains about vanished source files Product: rsync Version: 3.1.1

[Bug 11656] Escaping broken with --files-from

2015-12-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11656 --- Comment #9 from Gennady Uraltsev --- I looked through the source code and it seems that whatever is happening is going bad in the function static void filtered_fwrite in log.c in particular the line #134 fprintf(f, "\\#%03o", *(uchar*)s); i

[Bug 11656] Escaping broken with --files-from

2015-12-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11656 --- Comment #8 from Kevin Korb --- I was not offended. I was just trying to establish your use case and offer possible alternative methods of accomplishing it while not actually being an rsync dev. Wayne is really the only person who can say "Yep

[Bug 11656] Escaping broken with --files-from

2015-12-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11656 --- Comment #7 from Gennady Uraltsev --- Furthermore consider this test case: in addition to what we did before create the file with the actual name aaa\#012bbb by doing touch 'src/aaa\#012bbb' then $ rsync -n --itemize-changes -a src/* dst/ >

[Bug 11656] Escaping broken with --files-from

2015-12-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11656 --- Comment #6 from Gennady Uraltsev --- I hope I am not upsetting anyone. Maybe I wasn't clear: --itemize-changes is half the problem. Maybe I should post another bug. In the situation I described $ rsync -n --itemize-changes -a src/* dst/ give

[Bug 11656] Escaping broken with --files-from

2015-12-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11656 --- Comment #5 from Kevin Korb --- I would say that if your goal is to make an editable list to be run through rsync later you would be a lot better off with an --itemize-changes list and a script to reformat it after editing. I don't know about y

[Bug 11656] Escaping broken with --files-from

2015-12-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11656 --- Comment #4 from Gennady Uraltsev --- Well, imagine a poor mans replacement for batch files. We want to generate a list of operations, maybe edit it by hand (a batch file is binary...) and then feed it back to rsync. Or maybe do a dry run, look

[Bug 11656] Escaping broken with --files-from

2015-12-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11656 --- Comment #3 from Kevin Korb --- I am not sure what exactly the point of using an rsync -n to feed an rsync --files-from would be. The --files-from option is really designed to be fed from find which has a -print0 option which will format

[Bug 11656] Escaping broken with --files-from

2015-12-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11656 --- Comment #2 from Gennady Uraltsev --- Actually this doesn't help. $ mkdir src; mkdir dst; touch src/"$(echo -e 'foo\nbar')" $ rsync -n --out-format='%n' src/* dst/| tr '\n' '\0'

[Bug 11656] Escaping broken with --files-from

2015-12-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11656 --- Comment #1 from Kevin Korb --- This is what --from0 is for. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change op

Re: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-16 Thread Simon Hobson
Andrew Gideon wrote: >> btrfs has support for this: you make a backup, then create a btrfs >> snapshot of the filesystem (or directory), then the next time you make a >> new backup with rsync, use --inplace so that just changed parts of the >> file are written to the same blocks and btrfs will ta

Re: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-16 Thread Andrew Gideon
On Tue, 14 Jul 2015 08:59:25 +0200, Paul Slootman wrote: > btrfs has support for this: you make a backup, then create a btrfs > snapshot of the filesystem (or directory), then the next time you make a > new backup with rsync, use --inplace so that just changed parts of the > file are written to th

Re: Fwd: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-16 Thread Ken Chase
yeah, i read somewhere that zfs DOES have separate tuning for metadata and data cache, but i need to read up on that more. as for heavy block duplication: daily backups of the whole system = alot of dupe. /kc On Thu, Jul 16, 2015 at 05:42:32PM +, Andrew Gideon said: >On Mon, 13 Jul 201

Re: Fwd: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-16 Thread Andrew Gideon
On Mon, 13 Jul 2015 17:38:35 -0400, Selva Nair wrote: > As with any dedup solution, performance does take a hit and its often > not worth it unless you have a lot of duplication in the data. This is so only in some volumes in our case, but it appears that zfs permits this to be enabled/disabled

Re: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-14 Thread Simon Hobson
Ken Chase wrote: > And what's performance like? I've heard lots of COW systems performance > drops through the floor when there's many snapshots. For BTRFS I'd suspect the performance penalty to be fairly small. Snapshots can be done in different ways, and the way BTRFS and (I think) ZFS do it

Re: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-14 Thread Ken Chase
And what's performance like? I've heard lots of COW systems performance drops through the floor when there's many snapshots. /kc On Tue, Jul 14, 2015 at 08:59:25AM +0200, Paul Slootman said: >On Mon 13 Jul 2015, Andrew Gideon wrote: >> >> On the other hand, I do confess that I am sometime

Re: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-14 Thread Paul Slootman
On Mon 13 Jul 2015, Andrew Gideon wrote: > > On the other hand, I do confess that I am sometimes miffed at the waste > involved in a small change to a very large file. Rsync is smart about > moving minimal data, but it still stores an entire new copy of the file. > > What's needed is a file sy

Fwd: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-13 Thread Selva Nair
On Mon, Jul 13, 2015 at 5:19 PM, Simon Hobson wrote: > > What's needed is a file system that can do what hard links do, but at the > > file page level. I imagine that this would work using the same Copy On > > Write logic used in managing memory pages after a fork(). > > Well some (all ?) enterp

Re: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-13 Thread Simon Hobson
Andrew Gideon wrote: > However, you've made be a little > apprehensive about storebackup. I like the lack of a need for a "restore > tool". This permits all the standard UNIX tools to be applied to > whatever I might want to do over the backup, which is often *very* > convenient. Well if y

Re: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-13 Thread Andrew Gideon
On Mon, 13 Jul 2015 15:40:51 +0100, Simon Hobson wrote: > The think here is that you are into "backup" tools rather than the > general purpose tool that rsync is intended to be. Yes, that is true. Rsync serves so well as a core component to backup, I can be blind about "something other than rsy

Re: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-13 Thread Ken Chase
ds the scan would be desirable. > >I should also add that I mistrust time-stamp, and even time-stamp+file- >size, mechanism for detecting changes. Checksums, on the other hand, are >prohibitively expensive for backup of large file systems. > >These both bring me to the

Re: rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-13 Thread Simon Hobson
Andrew Gideon wrote: > These both bring me to the idea of using some file system auditing > mechanism to drive - perhaps with an --include-from or --files-from - > what rsync moves. > > Where I get stuck is that I cannot envision how I can provide rsync with > a limited lis

rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

2015-07-13 Thread Andrew Gideon
kup of large file systems. These both bring me to the idea of using some file system auditing mechanism to drive - perhaps with an --include-from or --files-from - what rsync moves. Where I get stuck is that I cannot envision how I can provide rsync with a limited list of files to move that doesn

Re: Sync files-from with delete

2015-06-27 Thread Simon Hobson
john espiro wrote: > The remote location is rather remote, so that wouldn't work in this > particular case. I don't follow, weren't you planning to take the USB device to the remote location to copy some of the files ? If you are doing that, then it doesn't really matter if you have extra fil

Re: Sync files-from with delete

2015-06-26 Thread john espiro
- Original Message - From: Simon Hobson To: "rsync@lists.samba.org" Cc: Sent: Friday, June 26, 2015 8:07 AM Subject: Re: Sync files-from with delete john espiro wrote: > I have a local directory that I am trying to sync with a remote directory. > That's fi

Re: Sync files-from with delete

2015-06-26 Thread Simon Hobson
an a dry-run between the two to generate a list of files in > listOfFiles.txt > > Then I ran rsync from local to localBackup. > > > rsync -av --files-from=/tmp/listOfFiles.txt /var/MyData/ /USB/Backup/ > > > Now, while this is happening, I am still rsycning b

Sync files-from with delete

2015-06-25 Thread john espiro
f files in listOfFiles.txt Then I ran rsync from local to localBackup. rsync -av --files-from=/tmp/listOfFiles.txt /var/MyData/ /USB/Backup/ Now, while this is happening, I am still rsycning between my local and my remote. So I re-generate the listOfFiles.txt, which contains less things to

[Bug 11253] --exclude=dir doesn't work with --files-from=

2015-05-04 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11253 --- Comment #2 from Vegard Nossum --- (In reply to Wayne Davison from comment #1) Thank you for the explanation. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitt

[Bug 11253] --exclude=dir doesn't work with --files-from=

2015-05-04 Thread samba-bugs
|RESOLVED --- Comment #1 from Wayne Davison --- Excludes don't affect args, just items that are found in recursion. A files-from list is the same as specifying all the names as args on the command-line, and thus are not affected. Specifying an exclude of just the dir name ma

[Bug 11253] New: --exclude=dir doesn't work with --files-from=

2015-05-04 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=11253 Bug ID: 11253 Summary: --exclude=dir doesn't work with --files-from= Product: rsync Version: 3.1.1 Hardware: All OS: All Status: NEW Severity: n

Re: Does the --delete conflict with --files-from?

2015-04-15 Thread Hongyi Zhao
So, --delete will only work if > your files-from file is a list of directories rather that files. > > OTOH, you are probably actually looking for the --delete-missing-args > option which is a fairly new feature. Thanks a lot for your hints. Regards -- .: Hongyi Zhao [ hongyi.zhao AT g

Re: Does the --delete conflict with --files-from?

2015-04-14 Thread Kevin Korb
your files-from file is a list of directories rather that files. OTOH, you are probably actually looking for the --delete-missing-args option which is a fairly new feature. On 04/15/2015 02:29 AM, Hongyi Zhao wrote: > Hi all, > > I've tried to use the --files-from and --dele

Does the --delete conflict with --files-from?

2015-04-14 Thread Hongyi Zhao
Hi all, I've tried to use the --files-from and --delete options together. But finally find that in this case, the `--delete' won't delete the extraneous files from dest dirs. So, does the --delete conflict with --files-from? Any hints on this issue? Regards -- .: Hongyi Zha

Re: Downloading a great number of files from different rsync servers for good loadbalancing and high efficiency.

2015-04-04 Thread Karl O. Pinc
On Sat, 4 Apr 2015 15:21:21 +0800 Hongyi Zhao wrote: > I'm using Debian, I want to make a local repository which can let me > install packages more conveniently. Your solution will not work for mirroring debian since it does not do a 2-stage mirroring process. This is described in: https://www.

Downloading a great number of files from different rsync servers for good loadbalancing and high efficiency.

2015-04-04 Thread Hongyi Zhao
Hi all, I'm using Debian, I want to make a local repository which can let me install packages more conveniently. Considering that the rsync tool is the Debian official proposed tool for syncing the files among its different rsync server sites, I use the rsync client to downloading the deb package

[Bug 10681] Add an option to delete non-existent files in --files-from

2014-07-02 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10681 --- Comment #4 from Dmitry Yu Okunev 2014-07-02 09:48:25 UTC --- Hello. (In reply to comment #0) > I have a script which monitors file system events and then calls rsync using > --files-from, however that does not handle the case where

[Bug 10681] Add an option to delete non-existent files in --files-from

2014-06-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10681 --- Comment #3 from Daniel O'Connor 2014-07-01 04:49:03 UTC --- Hah thanks, I thought I had the latest but apparently not. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because:

[Bug 10681] Add an option to delete non-existent files in --files-from

2014-06-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10681 Wayne Davison changed: What|Removed |Added Status|NEW |RESOLVED Resolution|

[Bug 10681] Add an option to delete non-existent files in --files-from

2014-06-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10681 --- Comment #1 from Kevin Korb 2014-07-01 03:20:27 UTC --- Upgrade your rsync and use --delete-missing-args -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the

[Bug 10681] New: Add an option to delete non-existent files in --files-from

2014-06-30 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10681 Summary: Add an option to delete non-existent files in --files-from Product: rsync Version: 3.0.9 Platform: All OS/Version: All Status: NEW Severity

Re: excluding changed or deleted files from --backup

2014-02-06 Thread Helmut Schweinzer
option > (e.g. --backup-exclude or --backup-exclude-from)? > > You'd need to exclude the files from the transfer and then do a separate > rsync run that only included those files, this time without the --backup > option. One possibility is to use --max-size for the backup transfe

Re: excluding changed or deleted files from --backup

2014-01-30 Thread Wayne Davison
On Thu, Jan 30, 2014 at 6:06 AM, Helmut Schweinzer wrote: > Is there a way to exclude files only from the --backup option > (e.g. --backup-exclude or --backup-exclude-from)? > You'd need to exclude the files from the transfer and then do a separate rsync run that only included tho

excluding changed or deleted files from --backup

2014-01-30 Thread Helmut Schweinzer
on my destination server. I want to have these files synced, but I don't want to have a --backup copy of the previous version. Is there a way to exclude files only from the --backup option (e.g. --backup-exclude or --backup-exclude-from)? In local rsync scripts I use to delete the files

Re: Question about --files-from= and folder structure

2014-01-05 Thread Wayne Davison
On Fri, Jan 3, 2014 at 12:39 PM, Bill Dorrian wrote: > The script that I'm running works - sort of - in that it syncs the files; > but it syncs their parent directories too, which I'm trying to avoid. --files-from implies -R (--relative), which tells rsync to include the pa

Re: Question about --files-from= and folder structure

2014-01-03 Thread Kevin Korb
3 ./Against > Me!/Against Me! - I Was a Teenage Anarchist.mp3 > > > The script that I'm running works - sort of - in that it syncs the > files; but it syncs their parent directories too, which I'm trying > to avoid. > > > This is the command that I'm running n

Question about --files-from= and folder structure

2014-01-03 Thread Bill Dorrian
The script that I'm running works - sort of - in that it syncs the files; but it syncs their parent directories too, which I'm trying to avoid. This is the command that I'm running now ($f is defined as the name of the "files from" and the destination directory name). rsync

[Bug 9416] --files-from : RERR_PARTIAL vs RERR_VANISHED

2013-06-02 Thread samba-bugs
||WONTFIX --- Comment #1 from Wayne Davison 2013-06-02 23:25:59 UTC --- In looking at this, I like the RERR_PARTIAL return better for missing args (which is what files-from files are). However, you can use one of these to get rsync to treat missing args as OK: --ignore-missing

RE: Rsync not preserving file permissions of ACL files from solaris to Linux

2013-02-08 Thread Mathew, Balu
Hi, I am facing a strange rsync issue when I am syncing files from a solaris->linux. The rsync is not preserving the file permissions [ owner,group,world ] for files which has ACL's defined om it. The permissions of the file in the target Linux server is getting appended somehow in t

Re: rsync - using a --files-from list to cut out scanning. How to handle deletions? (fwd)B

2013-01-21 Thread Robert Bell
Paul Wayne, Kevin, Teodor and others, Thanks for your contributions in response to my postings. Paul: I was very imprecise if not plain wrong in my description. :-( Thanks for explaining what really happens. "Rsync will not update an existing file in-place unless you use the --inplace option.

Re: rsync - using a --files-from list to cut out scanning. How to handle deletions?

2013-01-18 Thread Wayne Davison
r files if a hard-linked older version is already in the destination. This is the case where a file exists in the destination, does not exist in > the source, but is named in the --files-from= list. > Use the --delete-missing-args option of 3.1.0. Though it is not yet released, it hopeful

Re: rsync - using a --files-from list to cut out scanning. How to handle deletions? (fwd)

2013-01-18 Thread Paul Slootman
On Fri 18 Jan 2013, Robert Bell wrote: > > > >If a file exists in the target directory when using --link-dest rsync > >modifies the link rather than replacing it which means you don't have > >history for files that have been replaced rather than added or deleted. > Thanks for your astute observatio

Re: rsync - using a --files-from list to cut out scanning. How to handle deletions? (fwd)

2013-01-17 Thread Robert Bell
To: rsync@lists.samba.org Subject: Re: rsync - using a --files-from list to cut out scanning. How to handle deletions? -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 If you are going to do it this way please be aware of: https://bugzilla.samba.org/show_bug.cgi?id=8712 and https://bugzilla.samba

Re: rsync - using a --files-from list to cut out scanning. How to handle deletions?

2013-01-15 Thread Kevin Korb
owever, there is one case that I have not been able to get to work > in a test of rsync. This is the case where a file exists in the > destination, does not exist in the source, but is named in the > --files-from= list. This would be the case if a file had been > deleted from the source.

rsync - using a --files-from list to cut out scanning. How to handle deletions?

2013-01-14 Thread Robert Bell
ere a file exists in the destination, does not exist in the source, but is named in the --files-from= list. This would be the case if a file had been deleted from the source. We would want rsync in this case to delete the file on the destination. However, with a test command like: rsync -a -i --d

[Bug 9416] New: --files-from : RERR_PARTIAL vs RERR_VANISHED

2012-11-21 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=9416 Summary: --files-from : RERR_PARTIAL vs RERR_VANISHED Product: rsync Version: 3.1.0 Platform: All OS/Version: All Status: NEW Severity: normal Priority: P5

Re: rsync files from subfolders on source to root of a folder on destination

2012-04-24 Thread James Robertson
> I think hard-links are the easiest solution. > > mkdir hardlinks > find Music -type f -print0 | xargs -0 cp -al -t hardlinks > (You get a warning from cp about doubles, if any) > > Then you can rsync the 'hardlinks'-directory as usual. > > When you want to update you should just 'rm -rf hardlinks

Re: rsync files from subfolders on source to root of a folder on destination

2012-04-23 Thread Matthias Schniedermeyer
On 23.04.2012 16:50, James Robertson wrote: > I wish to sync a bunch of flac files that reside in various subfolders > to the root of a folder on a destination. > > An example of the directory structure on the source is: ... I think hard-links are the easiest solution. mkdir hardlinks find Musi

  1   2   3   4   >