If you are continually copying large numbers of files with rsync, maybe
you would be better off just using cp. Do you need to keep older
versions with the same file name, meaning an incremental backup?
I generally start by just doing a cp of the src directory to the backup
location. When the number of files is large, I just do,
cp -Rfp /src_loc_path/ /backup_loc_path/ &> copylog
This gives me a record of anything that might not have been copied due
to permission issues, etc. It's best to do this from a windows admin
account.
Then I run rsync on subsequent days so that rsync is just dealing with
changes made on any one given day and not trying to copy very large
directories. I always use a log file so I can check to make sure it has
finished and got everything it was supposed to.
#!/bin/bash
function backup {
# write some stuff to the log file
echo " " >> $1
echo " " >> $1
echo "***** new log entry *****" >> $1
date >> $logfile_path
# run the backup
rsync -v -a -r -p -b --suffix=.back -E -t --log-file=$1 $2 $3
# convert the log file format to windows EOL
unix2dos $1
}
# give a location for the log file, src, and backup loc
logfile_path="/logfile_path/log.txt"
backup_src_path="/backup_src_path/"
backup_destination_path="/backup_destination_path/"
backup $logfile_path $backup_src_path $backup_destination_path
I just have this script attached to a windows shortcut. That way I can
run it with a double click, or add the shortcut to the task manager if I
want to schedule backups. I guess I should be using cron to get the full
quantity of linux points, but I haven't got around to it.
If you wanted to, you could make a script that would check if the backup
destination directory exists, and if it doesn't, just do cp. If it does
exist, then run rsync.
I don't know how much volume it rsync designed to manage, especially if
it needs to keep a connection over a net, but if you are running out of
resources you may want to look at a different tool.
There are other options, like rdiff and unison. I have used unison, but
it is more a mirroring tool and I found the incremental backup options
gave me errors. I you are just trying to mirror directories or machines,
then it is a good tool.
LMH
Richard Ivarson wrote:
Bruno, is this rsync problem maybe related to my rsync problem which I
mentioned recently in the thread "Rsync stops inmid of synchronisation" ?
(My latest post there was: http://article.gmane.org/gmane.os.cygwin/132270
... but I don't know how to link to the entire thread.)
-Richard
Am 19.03.2012 11:51, schrieb Bruno Galindro da Costa:
Hi!
I'm try to copy some files from windows to Linux using rsync but,
after some short of time, an error was showed. Here is the log:
ERROR: out of memory in flist_expand [sender]
rsync error: error allocating core memory buffers (code 22) at
/home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/util.c(117)
[sender=3.0.9]
rsync: connection unexpectedly closed (35795723020 bytes received so
far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at
io.c(601) [receiver=3.0.7]
rsync: connection unexpectedly closed (102 bytes received so far)
[generator]
rsync error: error allocating core memory buffers (code 22) at
io.c(601) [generator=3.0.7]
Bellow is the command that I used:
/usr/bin/rsync
--exclude-from=/opt/backup_ti/tmp/FILESERVER_exclude_from.tmp
-avzR --delete-excluded --rsh="ssh -o UserKnownHostsFile=/dev/null -o
StrictHostKeyChecking=no -p 31060"
root@127.0.0.1:"/cygdrive/c/fileserver"
/backup/fileserver/bkp_files_20120318225522
Bellow is the /opt/backup_ti/tmp/FILESERVER_exclude_from.tmp content:
- TEMP
- RECYCLE.BIN
- System
Is there any workaround that I can use to "solve" this problem?
Maybe using an option or change a configuration file on cygwin...
Search in google, I've found this solution
(http://blog.rudnyi.ru/2011/04/problem-with-rsync.html ), but, before
apply that, I like to see If there is another option to use.
--
Att.
Bruno Galindro da Costa
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
