Yes, I removed "--no-inc-recursive", without success. -- Best regards / Met vriendelijke groet,
Aron Rotteveel 2015-03-27 12:24 GMT+01:00 Kevin Korb <k...@sanitarium.net>: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Have you tried removing --no-inc-recursive yet? > > On 03/27/2015 07:19 AM, Aron Rotteveel wrote: > > Hi Roland, > > > > Thanks for the reply. Memory usage on both machines seem fine. The > > server has 4GB's of RAM, of which about 3GB is used during the file > > list build and about 1.5GB is used during the actual transfer. The > > client has 16GB of RAM with a peak usage of 8.5GB. > > > > I just tried three transfers in a row and it consistently breaks at > > a certain point, after which I get the "ERROR: out of memory in > > flist_expand [sender]" error. There is not much special to mention > > regarding the file on which it breaks: it's a 22KB JPEG file with > > no special attributes. > > > > The backup server is running Debian 7.8, the client runs on CentOS > > 5.11. > > > > A `find . | wc -l` in the backup directory results in 7434013 > > files. > > > > -- Best regards / Met vriendelijke groet, > > > > Aron Rotteveel > > > > 2015-03-19 20:10 GMT+01:00 <devz...@web.de > > <mailto:devz...@web.de>>: > > > > Hi Aron, > > > > i hope it`s ok for you if i bring this back on-list. Your issue > > and the way or possible fix to resolve it may be interesting for > > others too (that may include future searches etc) > > > > so with 3.1.1 we are a step further.... > > > > i don`t really have a clue what`s happening here but my next step > > would be taking a closer look on how the memory usage of rsync on > > the client and server grows. > > > > you could log it like this: while true;do ps -eo > > vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile > > > > does it grow continuously? does the oom situation reproducibly > > happen at a certain size ? what`s the client and server platform? > > how many files? (-> https://rsync.samba.org/FAQ.html#5 ! ) > > > > regards roland > > > > *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:* "Aron > > Rotteveel" <rotteveel.a...@gmail.com > > <mailto:rotteveel.a...@gmail.com>> *An:* devz...@web.de > > <mailto:devz...@web.de> *Betreff:* Re: rsync 3.0.9 segmentation > > fault In addition to my last message: > > > > * Client (sender) has 16GB's or RAM, of which only 6.5GB is used > > during peak. * I tried using --no-inc-recursive, but it does not > > solve the issue. > > > > What currrently is puzzling me is the question of why I am > > receiving these errors when my server seems to have plenty of > > memory to spare. > > > > -- Best regards / Met vriendelijke groet, > > > > Aron Rotteveel > > > > 2015-03-19 11:52 GMT+01:00 Aron Rotteveel > > <rotteveel.a...@gmail.com>: > > > > Hi Roland, > > > > I just upgrade both the client and host to 3.1.1 and seem to memory > > related issues now: > > > > ERROR: out of memory in make_file [sender] rsync error: error > > allocating core memory buffers (code 22) at util2.c(102) > > [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c, > > line=102): about to call exit(22) [Receiver] _exit_cleanup(code=22, > > file=io.c, line=1633): about to call exit(22) > > > ---------------------------------------------------------------------------- > > > > > rsnapshot encountered an error! The program was invoked with > > these options: /usr/bin/rsnapshot -c \ > > /home/remotebackup/hosts/<redacted>/rsnapshot.conf sync > > > ---------------------------------------------------------------------------- > > > > -- Best regards / Met vriendelijke groet, > > > > Aron Rotteveel > > > > 2015-03-18 23:43 GMT+01:00 <devz...@web.de>: > > > > Hi, > > > > rsync 3.0.9 is quite ancient, more than 3 years old. A lot of bugs > > have been fixed since then. > > > > Is there a chance to update to the latest rsync version and retry > > with that ? > > > > regards Roland > > > > *Gesendet:* Dienstag, 17. März 2015 um 11:51 Uhr *Von:* "Aron > > Rotteveel" <rotteveel.a...@gmail.com> *An:* rsync@lists.samba.org > > *Betreff:* rsync 3.0.9 segmentation fault Hi, > > > > I am experiencing segfaults when transferring files via rsync > > though sudo. Setup: > > > > - Backupserver initiates the rsync command with --delete -vvv > > --no-inc-recursive --numeric-ids --delete-excluded --relative > > --rsync-path=/home/backupuser/rsync-wrapper.sh - rsync-wrapper.sh > > (on the client) contains /usr/bin/sudo /usr/bin/rsync "$@"; - user > > "backupuser" has sudo access to the rsync command - Both host and > > client are running 3.0.9 > > > > The transfer starts and some files are actually transferred. Once a > > certain file is reached (plain PHP file, no special characters or > > any other peculiarities) it segfaults. > > > > rsync host output: > > > > [sender] > > make_file(<redacted>/libraries/phputf8/mbstring/strlen.php,*,2) > > rsync: connection unexpectedly closed (51261222 bytes received so > > far) [Receiver] rsync error: unexplained error (code 139) at > > io.c(605) [Receiver=3.0.9] [Receiver] _exit_cleanup(code=12, > > file=io.c, line=605): about to call exit(139) > > > ---------------------------------------------------------------------------- > > > > > rsnapshot encountered an error! The program was invoked with > > these options: /usr/bin/rsnapshot -c \ > > /home/remotebackup/hosts/<redacted>/rsnapshot.conf sync > > > ---------------------------------------------------------------------------- > > > > > ERROR: /usr/bin/rsync returned 139 while processing > > backupuser@<redacted>:/backup/ > > > > Client output when using gdb to debug the coredump: > > > > warning: no loadable sections found in added symbol-file > > system-supplied DSO at 0x7fff015fd000 Core was generated by > > `/usr/bin/rsync --server --sender -vvvlogDtprRe.Lsf --numeric-ids . > > /backup'. Program terminated with signal 11, Segmentation fault. #0 > > 0x00000035cda7b441 in memcpy () from /lib64/libc.so.6 > > > > Any help would be greatly appreciated. Please let me know if > > additional info is required to properly debug this issue. -- Best > > regards / Met vriendelijke groet, > > > > Aron Rotteveel -- Please use reply-all for most replies to avoid > > omitting the mailing list. To unsubscribe or change options: > > https://lists.samba.org/mailman/listinfo/rsync Before posting, > > read: http://www.catb.org/~esr/faqs/smart-questions.html > > > > > > > > > > - -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. ke...@futurequest.net (work) > Orlando, Florida k...@sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iEYEARECAAYFAlUVPe8ACgkQVKC1jlbQAQdJfQCdFpMSsACi2df9P8eJZQAi4qeB > xA8Amwbpj9PGpzeQfrd4VnAytU9dN1uc > =1Ayf > -----END PGP SIGNATURE----- >
-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
