On Wed, Aug 3, 2022 at 5:41 PM Robin Lee Powell via rsync <
rsync@lists.samba.org> wrote:
> On Wed, Aug 03, 2022 at 02:04:22PM -0400, Rob Campbell via rsync wrote:
> > The problem isn't that there are many syncs because the problem happens
> on
> > the first one that runs.
>
> You didn't actually
I suspect you want a duplicate finder more than a file transfer tool.
EG: https://stromberg.dnsalias.org/~strombrg/equivalence-classes.html
On Tue, Jun 7, 2022 at 5:36 PM hput via rsync wrote:
> I want to merge 3 slightly different directories of mostly images.
>
> Not just mostly but the vast
On Sat, Mar 12, 2022 at 12:23 PM Dr. Mark Asbach via rsync <
rsync@lists.samba.org> wrote:
> Hi there, hi past me,
>
> > My (non-working) attempt:
> > […]
> > So it seems the "-l" is dropped into the void letting ssh assume USER
> was the target host? I don’t actually get what I can do.
>
> Turns
Why not rsync directly as root? Then you can use a passwordless,
passphraseless RSA (or similar) keypair.
On Fri, Mar 11, 2022 at 4:58 AM Dr. Mark Asbach via rsync <
rsync@lists.samba.org> wrote:
> Hi there,
>
> We are using ansible to deploy system configuration and web application
> source cod
On Thu, Feb 3, 2022 at 3:50 PM Andy Smith via rsync
wrote:
> I am tempted to blow away the btrfs filesystem and just do xfs to
> xfs, to rule out weird issues there. It would be a shame though as
> I was hoping to use btrfs's compression here.
>
You might be able to do a partial transfer to a sm
Can rsync back up an NTFS using a Windows 10 kernel? So far I've had good
luck backing up NTFS filesystems on a dual boot system when booted into
Linux, but not when booted into Windows.
I've been bitten in the past by /usr/bin/find (for EG) having problems with
Windows junctions over sshfs. The
rsync --link-dest is fast and simple, good stuff.
I recommend using it with some sort of wrapper script for rotations, like
Backup.rsync:
https://stromberg.dnsalias.org/~strombrg/Backup.remote.html
On Wed, Oct 6, 2021 at 7:29 AM Kevin Korb via rsync
wrote:
> See --link-dest. That is what makes
I was thinking --link-dest too.
Sometimes this can be done with cpio too; check out the -pdlv options.
On Sat, Sep 4, 2021 at 4:57 PM Kevin Korb via rsync
wrote:
> Rsync does almost everything cp does but since it is designed to network
> it never got that feature. I was thinking maybe --link-
I suppose I may as well mention:
https://stromberg.dnsalias.org/~dstromberg/Backup.remote
It just does rsync snapshotting with --link-dest, and keeps the last n
snapshots. It's smart enough to resume a previously interrupted snapshot.
It's pretty simple - both to set up and to use. I used to us
In Backup.rsync, which of course is a wrapper around rsync that can be used
for backups, I do not use --backup, but I do use --link-dest:
https://stromberg.dnsalias.org/~strombrg/Backup.remote.html
On Sun, Jul 18, 2021 at 11:13 PM Lisa via rsync
wrote:
> I would like some feedback about the --ba
This is probably more of a Cygwin question than an rsync question.
On Cygwin, E: should show up automatically as /cygdrive/e
You can test that by opening a Cygwin terminal and cd'ing to /cygdrive/e
On Tue, Apr 6, 2021 at 1:32 PM Tim Evans via rsync
wrote:
> Cygwin distribution of rsync for Win
ile it
> handles multicat, it doesn't seem to be able to handle multiple interfaces,
> if I read the docs correctly.
> Am I wrong?
> harry
>
> On Fri, Mar 26, 2021 at 12:29 PM Dan Stromberg
> wrote:
>
>>
>> Hi Harry. Are you the person I worked with at UCI a bi
Hi Harry. Are you the person I worked with at UCI a bit?
Anyway, you might consider trying mrsync; it's intended to do rsync over
multicast.
HTH.
On Fri, Mar 26, 2021 at 12:22 PM Harry Mangalam via rsync <
rsync@lists.samba.org> wrote:
> Spent an hour trying to find the answer to this on the v
On Tue, Dec 15, 2020 at 3:25 AM Laurent B via rsync
wrote:
> Dear all,
>
> I'm encountering a problem with one of my backup. For some files, the
> checksum calculation is failing leading to the following error :
>
It sounds a little like a bug, but perhaps if you share the command you're
using fo
On Sun, Dec 13, 2020 at 11:59 AM Wayne Davison via rsync <
rsync@lists.samba.org> wrote:
> I should also mention that there are totally valid reasons why the dir
> might be huge on day4. For instance, if someone changed the mode on the
> files from 664 to 644 then the files cannot be hard-linked t
Hi.
Is it possible that, if day4 is consuming too much space, that day3 was an
incomplete backup?
The rsync wrapper I wrote goes to a little trouble to make sure that
incomplete backups aren't allowed. It's called Backup.rsync, and can be
found at:
https://stromberg.dnsalias.org/~strombrg/Backup
On Tue, Jun 26, 2018 at 12:02 PM, Дугин Сергей via rsync <
rsync@lists.samba.org> wrote:
> I am launching a cron bash script that does the following:
>
> Day 1
> /usr/bin/rsync -aH --link-dest /home/backuper/.BACKUP/009/2018-06-25
> root@192.168.1.103:/home/ /home/backuper/.BACKUP/009/2018
Is there such a thing?
I saw librsync, which appears to be the right algorithm, but not the
protocol.
And I saw the acrosync-library, which appears to be the protocol, but it's
not GPL-compatible.
Are there others?
Thanks!
--
Please use reply-all for most replies to avoid omitting the mailing
e-
> From: Kevin Korb via rsync [mailto:rsync@lists.samba.org]
> Sent: zondag 25 maart 2018 22:42
> To: rsync@lists.samba.org
> Subject: Re: Rsync between 2 datacenters not working
>
> Note that if you do this you are stuck with --whole-file
>
> On 03/25/2018 04:36 PM, Dan
You could try using an automounter, like autofs, in combination with
sshfs. It'll be slower, possibly a lot slower, but it should be more
reliable over an unreliable connection.
I've been using:
remote
-fstype=fuse,allow_other,nodev,noatime,reconnect,ServerAliveInterval=15,ServerAliveCountMax=40
On Mon, Mar 5, 2018 at 3:09 PM, just subscribed for rsync-qa from
bugzilla via rsync wrote:
> https://bugzilla.samba.org/show_bug.cgi?id=13317
>
> --- Comment #6 from Rui DeSousa ---
> (In reply to Rui DeSousa from comment #5)
>
> It looks like no error is returned and result is a sparse file. I
Why not enable Jumbo Frames? http://stromberg.dnsalias.org/~strombrg/jumbo.html
For NFS, you can use
http://stromberg.dnsalias.org/~strombrg/nfs-test.html to get some fast
settings. The script could be modified to do CIFS I suppose.
HTH
--
Please use reply-all for most replies to avoid omittin
On Tue, Aug 26, 2014 at 10:01 AM, wrote:
> https://bugzilla.samba.org/show_bug.cgi?id=10785
>
>Summary: [PATCH] Add a flag to use numeric sort
>Product: rsync
>Version: 3.1.1
> Platform: All
> OS/Version: All
> Status: NEW
>
On Fri, Jul 25, 2014 at 11:35 PM, Kevin Korb wrote:
>> I just moved my home partition to a new harddisk w/more space.
>
> Home Partition? Are we in 1995? Why would you have a partition
> mounted anywhere other than /boot ?
That's a bit harsh, particularly considering that having a /home
partiti
On Tue, Jul 8, 2014 at 8:07 AM, Adam Edgar wrote:
> It seems the issue is indeed in the ssh layer. scp has the same issue and
> some work has been done in “fixing” that:
>
> http://www.psc.edu/index.php/hpn-ssh
>
> From the papers abstract:
>
> SCP and the underlying SSH2 protocol impleme
First off, thanks much for your suggestions.
On Thu, Jan 2, 2014 at 6:19 PM, Kevin Korb wrote:
> First, a new column for the old cp -al then rsync on top of it method
> that --link-dest mostly replaced. It is slower since all the hard
> links get made and then some get replaced or deleted but it
On Wed, Jan 1, 2014 at 10:09 AM, Charles Marcus
wrote:
> If I change the permissions on the source maildirs, will this cause
> everything to be transferred again? Meaning, will rsync see everything as
> 'modified', thus creating a new copy of the entire mail store on the backup
> target?
>
> Or wi
On Mon, Nov 25, 2013 at 1:41 PM, Chris Dennis wrote:
> Hello rsync people
>
> Today I was recovering data from a beginning-to-fail external USB hard
> disk.
>
> I started with my usual 'rsync -av --ignore-errors ', and
> that was fine until it got to the first I/O errors. It paused but
> continu
I wonder if rdiff would do this nicely...?
http://librsync.sourcefrog.net/doc/rdiff.html
On Thu, Sep 27, 2012 at 12:33 AM, Benjamin Ward wrote:
>
> Hi everyone,
>
> I'm trying to archive to a remote host but the size of the data to copy is
> prohibitively large to consider doing the sync over th
list that aren't really
hardlinks - the goal is chiefly to dramatically cut down on the storage
requirements, and avoid a large sort() or O(file_count) hash table. That's
where the "filter" part of the name comes from - it's filtering the count
of the objects of interest down to a mo
Ah.
Not being interactive is important for running in cron; I believe stdin
will probably immediately EOF.
But redirecting stdout and stderr is unnecessary - the output just goes to
a cron e-mail with most cron's. Sometimes it's better to redirect to a
file, but that's more of a user preferen
On Sat, Jul 28, 2012 at 3:48 AM, M. Carrasco wrote:
> 3. Cron
> It can run properly run from cron as it is demonized.
>
What's this about? I've never had problems running run of the mill scripts
from cron, once the environment is adequately replicated.
> "--hard-links" is not used and there is
On Fri, Jul 13, 2012 at 1:21 PM, Rob J. Caskey wrote:
> Howdy all,
>
> ** **
>
> I’m using rsync in conjunction with backuppc and have been backing up this
> share without incident for almost 3 years and it has decided to go and hang
> on me. Other shares on the same machine sync fine with id
On Sat, Jul 7, 2012 at 4:15 PM, Henri Shustak wrote:
>
> > Wow! Thanks for making it so easy. I will try that asap.
>
> If you do not have any luck with the patched version of rsync there are
> various projects which spring to mind which offer this kind of
> functionality.
>
For backups, not re
I've heard lots of good suggestions already - another thing that I've not
seen mentioned is, upgrading your kernel may help. Somewhere shortly
before kernel 3.0, pathname lookups got noticeably faster.
You could also try an alternative filesystem like xfs. It's supposed to be
pretty good at larg
This is perhaps more a matter of which rsync wrapper you choose, than
rsync itself. rsync provides just enough functionality to enable this
kind of behavior, if your wrapper feeds it the right directories to
work on.
I used to use my Backup.rsync wrapper to get good resumption of
interrupted back
On Sun, Nov 6, 2011 at 2:29 PM, Dan Stromberg wrote:
>
> http://stromberg.dnsalias.org/~strombrg/backshift/documentation/comparison/index.html
>
I've updated the above URL to include a comparison against Lessfs and git
wrappers.
The table has also become easier to navigate re
On Sat, Nov 19, 2011 at 5:43 AM, Andrea Gelmini wrote:
> 2011/11/3 Alex Waite :
> >Recently I learned that rsync does a checksum of every file
> > transferred. I thought it might be interesting to record the path and
> > checksum of each file in a table. On future backups, the checksum of
>
On Thu, Nov 3, 2011 at 7:22 AM, Alex Waite wrote:
> >
> > Check out http://backuppc.sourceforge.net/, it's perl-based backup tool,
> > using rsync and doing exactly what you ask for.
> >
> I have looked at BackupPC before (and it is a nice piece of
> software), and it does hardlink across all
Direct I/O (assuming you mean O_DIRECT on open) can be a bit fiddly, but I
doubt it's out of reach. The main difficulty is allocating a buffer with
appropriate alignment.
I put together a library to facilitate O_DIRECT I/O a while back:
http://stromberg.dnsalias.org/~dstromberg/odirect/
odirect
What are people's favorite rsync --link-dest frontends? Why?
I personally want something based on GTK+ and/or HTML, with scheduling, but
I don't necessarily want to limit the discussion to frontends with these
attributes.
Here's a list of some of them, but I doubt it's very complete:
http://www.
It'd be pretty cool if rsync supported use of O_DIRECT on platforms that
support it, with or without my odirect package:
http://dcs.nac.uci.edu/~strombrg/odirect/
I say this because rsync is sometimes used to move a mountain of data,
just once. So there's little point in rsync toasting one's bu
On Wed, 03 Aug 2005 15:53:06 -0700, Wayne Davison wrote:
> On Wed, Aug 03, 2005 at 03:27:13PM -0700, Dan Stromberg wrote:
>> skipping directory /.
>
> You didn't specify -r, so it's skipping all directories. See also -a.
Doh!
I know better than that. Dumb mista
On Wed, 03 Aug 2005 16:30:28 -0400, Keith Warno wrote:
> * <[EMAIL PROTECTED]> [03/08/2005 1531EDT]:
>> On Wed, 03 Aug 2005 14:15:36 -0400, Keith Warno wrote:
>>
>> > * <[EMAIL PROTECTED]> [02/08/2005 2008EDT]:
>> >>
>> >> What's the easiest way, with rsync, to back up all local filesystems to
>
On Wed, 03 Aug 2005 14:15:36 -0400, Keith Warno wrote:
> * <[EMAIL PROTECTED]> [02/08/2005 2008EDT]:
>>
>> What's the easiest way, with rsync, to back up all local filesystems to
>> another host, ignoring anything on an NFS volume.
>>
>> For example, is there a way of giving a list of mount poin
What's the easiest way, with rsync, to back up all local filesystems to
another host, ignoring anything on an NFS volume.
For example, is there a way of giving a list of mount points but still
have --one-file-system work?
Thanks!
--
To unsubscribe or change options: https://lists.samba.org/ma
I've been trying to extract some of my clients' data from a flakey lustre
filesystem, and as a result, took some time to write up a page about
pulling data out of a semi-crashy filesystem. The page includes a variety
of ways of doing so, those ways including but not being limited to
rsync and a p
On Thu, 17 Mar 2005 13:04:03 -0800, Dan Stromberg wrote:
>
> I am running:
>
> + cd /oas
> + rsync -a --rsh=/usr/bin/ssh --rsync-path=/dcs/packages/gnu/bin/rsync
> --progress [EMAIL PROTECTED]:/oas .
>
> ...and I can see that it sometimes transfers some files, how
I am running:
+ cd /oas
+ rsync -a --rsh=/usr/bin/ssh --rsync-path=/dcs/packages/gnu/bin/rsync
--progress [EMAIL PROTECTED]:/oas .
...and I can see that it sometimes transfers some files, however, it
appears to be consistently missing this directory:
/oas/projects/pers/oct2004
What might cause
On Wed, 2004-12-01 at 01:04 -0600, John Van Essen wrote:
> On Tue, 30 Nov 2004, Dan Stromberg <[EMAIL PROTECTED]> wrote:
> >
> > We were doing a roughly 1 terabyte transfer, and upon running a python
> > script to verify the integrity of the transfer, we discovered
On Tue, 2004-11-30 at 12:45 -0800, Wayne Davison wrote:
> On Tue, Nov 30, 2004 at 12:10:05PM -0800, Dan Stromberg wrote:
> > Is there any precedent for rsync creating 0 length files that should've
> > had content in them?
>
> Are you using 2.6.3? Older rsync versions d
Is there any precedent for rsync creating 0 length files that should've
had content in them? IE, has anyone ever seen this before?
We were doing a roughly 1 terabyte transfer, and upon running a python
script to verify the integrity of the transfer, we discovered a small
number of files that wer
On Tue, 2004-11-23 at 15:00 -0800, Dan Stromberg wrote:
> What sorts of options might I give to rsync, to optimize transferring
> data from one networked filesystem to another?
>
> Transferring directly between the fileservers is not an option in this
> case, for technical rea
What sorts of options might I give to rsync, to optimize transferring
data from one networked filesystem to another?
Transferring directly between the fileservers is not an option in this
case, for technical reasons.
What sorts of options (or system modifications) might I use to help
rsync get t
Are there any characters that can occur in filenames that will choke
rsync?
We're transferring lots of data, and some of our users' filenames appear
to have their high bit set.
I don't expect it to cope with filenames having a \0 or / in them
(sometimes created over appletalk shares - strange bu
On Tue, 2004-10-26 at 16:15, Wayne Davison wrote:
> On Sun, Oct 24, 2004 at 10:46:01AM -0700, Dan Stromberg wrote:
> > We're copying from 3 NFS mounts of 3 GFS volumes, into 1 NFS mount of
> > 1 Lustre volume.
>
> Since you're merging 3 sources into one destinati
errors,
but on the recopying of what should be immutable files)? Or could it
mean that our source or destination filesystem(s) have data integrity
problems?
Thanks!
--
Dan Stromberg DCS/NACS/UCI <[EMAIL PROTECTED]>
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/
an estimate of how much work will need to be done
> (perhaps based on file-size differences as a crude estimate). Such a
> pre-pass would slow down rsync, so it's not something that would be
> universally useful (probably only for smallish sets of files over a slow
> net connec
I
see over double the throughput on an AIX system (1M vs 64K).
Thanks.
--
Dan Stromberg DCS/NACS/UCI <[EMAIL PROTECTED]>
signature.asc
Description: This is a digitally signed message part
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before pos
nc worked like a champ for months, but suddenly every time I try to
sync, it's getting stuck.
truss'ing appears to show things are hung up in select()'s.
Any ideas?
--
Dan Stromberg DCS/NACS/UCI <[EMAIL PROTECTED]>
signature.asc
Description: This is a digitally signe
s kind of option, because the
password can show up in ps or shell histories.
--
Dan Stromberg DCS/NACS/UCI <[EMAIL PROTECTED]>
signature.asc
Description: This is a digitally signed message part
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before post
27;s not hard to modify strace to usleep a while
on various syscalls. I did it once for read/write/send/recv, but it
never made it into the mainline. :(
--
Dan Stromberg DCS/NACS/UCI <[EMAIL PROTECTED]>
signature.asc
Description: This is a digitally signed message part
--
To unsubscribe o
On Tue, Jul 16, 2002 at 10:50:03AM +1000, Martin Pool wrote:
> On 15 Jul 2002, Dan Stromberg <[EMAIL PROTECTED]> wrote:
>
> > The issue was that demand
> > paging would glitch from .nfs* for no good reason.
>
> That is an extremely unconvincing argument for
On Sat, Jul 13, 2002 at 10:22:29AM +1000, Martin Pool wrote:
> On 12 Jul 2002, Dan Stromberg <[EMAIL PROTECTED]> wrote:
>
> > Because when we update, for example, bash, everbody's bash is going to
> > die on them if we don't keep around backups (segfault as you
On Fri, Jul 12, 2002 at 02:59:11PM +1000, Martin Pool wrote:
> On 11 Jul 2002, Dan Stromberg <[EMAIL PROTECTED]> wrote:
>
> > > I don't get what you are doing. Where did these insecure
> > > suid root files come from in the first place?
> >
> >
ven realize they need to do parsing to get reasonable behavior
from a security perspective.
In other words, if you insist, so be it.
--
Dan Stromberg UCI/NACS/DCS
msg04555/pgp0.pgp
Description: PGP signature
On Tue, Jul 09, 2002 at 02:36:22PM -0700, jw schultz wrote:
> On Tue, Jul 09, 2002 at 11:03:25AM -0700, Dan Stromberg wrote:
> > On Mon, Jul 08, 2002 at 02:04:57PM -0700, jw schultz wrote:
> > > The default behavior should not modify files. The general
> > > purpose i
as the trip will take,
> but before we work on that, we want to add the dumptruck, palmpilot, and
> grapefruit spoon features."
I want to remove the misfeature that throws broken glass in front of
your own wheels. Is that so bad?
--
Dan Stromberg
n't say this. We simply must have set[ug]id, but
only for secure binaries. Having an insecure setuid root ~ file is a
vastly larger problem than not being able to run some ~ file without
minimal sysadmin intervention first.
--
Dan Stromberg UCI/NACS/DCS
msg04519/pgp0.pgp
Description: PGP signature
log file. Does this sound like a reasonable solution?
Perl should be avoided. Perl is proof that sysadmins don't grok
language design.
--
Dan Stromberg UCI/NACS/DCS
msg04518/pgp0.pgp
Description: PGP signature
x27;h'},
X+#ifdef HAVE_CHMOD
X+ {"priv-backups",'s', POPT_ARG_NONE, &priv_backups},
X+#endif
X {"backup", 'b', POPT_ARG_NONE, &make_backups},
X {"dry-run", 'n', POPT_ARG_NONE, &dry_run},
X {"sparse", 'S', POPT_ARG_NONE, &sparse_files},
SHAR_EOF
$TOUCH -am 06210941102 options-priv-backups &&
chmod 0644 options-priv-backups ||
echo "restore of options-priv-backups failed"
set `wc -c options-priv-backups`;Wc_c=$1
if test "$Wc_c" != "1339"; then
echo original size 1339, current size $Wc_c
fi
exit 0
--
Dan Stromberg UCI/NACS/DCS
msg04403/pgp0.pgp
Description: PGP signature
--force appears to take care of this.
On Mon, Apr 22, 2002 at 06:23:08PM -0700, Dan Stromberg wrote:
> I'm trying to switch to using rsync for updating a huge software library
> containing binaries, text files, symlinks, and so on. We've been using
> something homegrown which
rmdir(packages/ctime-5.0) : File exists
rsync: symlink "packages/ctime-5.0" -> "ctime-5.0b3": File exists
rsync error: partial transfer (code 23) at main.c(883)
Is there a reasonably simple way around this?
TIA.
--
Dan Stromberg
73 matches
Mail list logo
