-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Using --no-inc-recursive would increase memory usage not reduce it. That option forces rsync to keep the entire tree in memory.
On 03/19/2015 03:10 PM, devz...@web.de wrote: > Hi Aron, > > i hope it`s ok for you if i bring this back on-list. Your issue and > the way or possible fix to resolve it may be interesting for others > too (that may include future searches etc) > > so with 3.1.1 we are a step further.... > > i don`t really have a clue what`s happening here but my next step > would be taking a closer look on how the memory usage of rsync on > the client and server grows. > > you could log it like this: while true;do ps -eo > vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile > > does it grow continuously? does the oom situation reproducibly > happen at a certain size ? what`s the client and server platform? > how many files? (-> https://rsync.samba.org/FAQ.html#5 ! ) > > regards roland > > *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:* "Aron > Rotteveel" <rotteveel.a...@gmail.com> *An:* devz...@web.de > *Betreff:* Re: rsync 3.0.9 segmentation fault In addition to my > last message: > > * Client (sender) has 16GB's or RAM, of which only 6.5GB is used > during peak. * I tried using --no-inc-recursive, but it does not > solve the issue. > > What currrently is puzzling me is the question of why I am > receiving these errors when my server seems to have plenty of > memory to spare. > > -- Best regards / Met vriendelijke groet, > > Aron Rotteveel > > 2015-03-19 11:52 GMT+01:00 Aron Rotteveel > <rotteveel.a...@gmail.com>: > > Hi Roland, > > I just upgrade both the client and host to 3.1.1 and seem to > memory related issues now: > > ERROR: out of memory in make_file [sender] rsync error: error > allocating core memory buffers (code 22) at util2.c(102) > [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c, > line=102): about to call exit(22) [Receiver] _exit_cleanup(code=22, > file=io.c, line=1633): about to call exit(22) > ---------------------------------------------------------------------------- > > rsnapshot encountered an error! The program was invoked with these > options: /usr/bin/rsnapshot -c \ > /home/remotebackup/hosts/<redacted>/rsnapshot.conf sync > ---------------------------------------------------------------------------- > > -- Best regards / Met vriendelijke groet, > > Aron Rotteveel > > 2015-03-18 23:43 GMT+01:00 <devz...@web.de>: > > Hi, > > rsync 3.0.9 is quite ancient, more than 3 years old. A lot of bugs > have been fixed since then. > > Is there a chance to update to the latest rsync version and retry > with that ? > > regards Roland > > *Gesendet:* Dienstag, 17. März 2015 um 11:51 Uhr *Von:* "Aron > Rotteveel" <rotteveel.a...@gmail.com> *An:* rsync@lists.samba.org > *Betreff:* rsync 3.0.9 segmentation fault Hi, > > I am experiencing segfaults when transferring files via rsync > though sudo. Setup: > > - Backupserver initiates the rsync command with --delete -vvv > --no-inc-recursive --numeric-ids --delete-excluded --relative > --rsync-path=/home/backupuser/rsync-wrapper.sh - rsync-wrapper.sh > (on the client) contains /usr/bin/sudo /usr/bin/rsync "$@"; - user > "backupuser" has sudo access to the rsync command - Both host and > client are running 3.0.9 > > The transfer starts and some files are actually transferred. Once a > certain file is reached (plain PHP file, no special characters or > any other peculiarities) it segfaults. > > rsync host output: > > [sender] > make_file(<redacted>/libraries/phputf8/mbstring/strlen.php,*,2) > rsync: connection unexpectedly closed (51261222 bytes received so > far) [Receiver] rsync error: unexplained error (code 139) at > io.c(605) [Receiver=3.0.9] [Receiver] _exit_cleanup(code=12, > file=io.c, line=605): about to call exit(139) > ---------------------------------------------------------------------------- > > rsnapshot encountered an error! The program was invoked with > these options: /usr/bin/rsnapshot -c \ > /home/remotebackup/hosts/<redacted>/rsnapshot.conf sync > ---------------------------------------------------------------------------- > > ERROR: /usr/bin/rsync returned 139 while processing > backupuser@<redacted>:/backup/ > > Client output when using gdb to debug the coredump: > > warning: no loadable sections found in added symbol-file > system-supplied DSO at 0x7fff015fd000 Core was generated by > `/usr/bin/rsync --server --sender -vvvlogDtprRe.Lsf --numeric-ids . > /backup'. Program terminated with signal 11, Segmentation fault. #0 > 0x00000035cda7b441 in memcpy () from /lib64/libc.so.6 > > Any help would be greatly appreciated. Please let me know if > additional info is required to properly debug this issue. -- Best > regards / Met vriendelijke groet, > > Aron Rotteveel -- Please use reply-all for most replies to avoid > omitting the mailing list. To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync Before posting, > read: http://www.catb.org/~esr/faqs/smart-questions.html > > > - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Florida k...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlULH/cACgkQVKC1jlbQAQcs2wCfWcwjC5Ljed22UAwjkrcf1p8s vyIAoJ8AkX6OqxdNJFZjkNMjccMFc65Y =PP+Q -----END PGP SIGNATURE----- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
