I am now running with --delete --numeric-ids --relative but the problem still persists.
-- Best regards / Met vriendelijke groet, Aron Rotteveel 2015-03-27 14:22 GMT+01:00 Kevin Korb <k...@sanitarium.net>: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Try also removing --delete-excluded. Without those two options there > should be no reason for rsync to require gigs of RAM. Well, unless > the other system has rsync 2.x. > > On 03/27/2015 07:29 AM, Aron Rotteveel wrote: > > Yes, I removed "--no-inc-recursive", without success. > > > > -- Best regards / Met vriendelijke groet, > > > > Aron Rotteveel > > > > 2015-03-27 12:24 GMT+01:00 Kevin Korb <k...@sanitarium.net > > <mailto:k...@sanitarium.net>>: > > > > Have you tried removing --no-inc-recursive yet? > > > > On 03/27/2015 07:19 AM, Aron Rotteveel wrote: > >> Hi Roland, > > > >> Thanks for the reply. Memory usage on both machines seem fine. > >> The server has 4GB's of RAM, of which about 3GB is used during > >> the file list build and about 1.5GB is used during the actual > >> transfer. The client has 16GB of RAM with a peak usage of 8.5GB. > > > >> I just tried three transfers in a row and it consistently breaks > >> at a certain point, after which I get the "ERROR: out of memory > >> in flist_expand [sender]" error. There is not much special to > >> mention regarding the file on which it breaks: it's a 22KB JPEG > >> file with no special attributes. > > > >> The backup server is running Debian 7.8, the client runs on > >> CentOS 5.11. > > > >> A `find . | wc -l` in the backup directory results in 7434013 > >> files. > > > >> -- Best regards / Met vriendelijke groet, > > > >> Aron Rotteveel > > > >> 2015-03-19 20:10 GMT+01:00 <devz...@web.de > >> <mailto:devz...@web.de> <mailto:devz...@web.de > >> <mailto:devz...@web.de>>>: > > > >> Hi Aron, > > > >> i hope it`s ok for you if i bring this back on-list. Your issue > >> and the way or possible fix to resolve it may be interesting for > >> others too (that may include future searches etc) > > > >> so with 3.1.1 we are a step further.... > > > >> i don`t really have a clue what`s happening here but my next > >> step would be taking a closer look on how the memory usage of > >> rsync on the client and server grows. > > > >> you could log it like this: while true;do ps -eo > >> vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile > > > >> does it grow continuously? does the oom situation reproducibly > >> happen at a certain size ? what`s the client and server > >> platform? how many files? (-> https://rsync.samba.org/FAQ.html#5 > >> ! ) > > > >> regards roland > > > >> *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:* "Aron > >> Rotteveel" <rotteveel.a...@gmail.com > >> <mailto:rotteveel.a...@gmail.com> > >> <mailto:rotteveel.a...@gmail.com > > <mailto:rotteveel.a...@gmail.com>>> *An:* devz...@web.de > > <mailto:devz...@web.de> > >> <mailto:devz...@web.de <mailto:devz...@web.de>> *Betreff:* Re: > > rsync 3.0.9 segmentation > >> fault In addition to my last message: > > > >> * Client (sender) has 16GB's or RAM, of which only 6.5GB is used > >> during peak. * I tried using --no-inc-recursive, but it does not > >> solve the issue. > > > >> What currrently is puzzling me is the question of why I am > >> receiving these errors when my server seems to have plenty of > >> memory to spare. > > > >> -- Best regards / Met vriendelijke groet, > > > >> Aron Rotteveel > > > >> 2015-03-19 11:52 GMT+01:00 Aron Rotteveel > >> <rotteveel.a...@gmail.com <mailto:rotteveel.a...@gmail.com>>: > > > >> Hi Roland, > > > >> I just upgrade both the client and host to 3.1.1 and seem to > >> memory related issues now: > > > >> ERROR: out of memory in make_file [sender] rsync error: error > >> allocating core memory buffers (code 22) at util2.c(102) > >> [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c, > >> line=102): about to call exit(22) [Receiver] > >> _exit_cleanup(code=22, file=io.c, line=1633): about to call > >> exit(22) > > > > > > > > - -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. ke...@futurequest.net (work) > Orlando, Florida k...@sanitarium.net (personal) > Web page: http://www.sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2 > > iEYEARECAAYFAlUVWZIACgkQVKC1jlbQAQfQ4wCeOjzVgtBt0t9LQ4Mf9X3kOhjF > tEcAoJAh158PF51O3Vnn8alkd7q0iSHQ > =pQeg > -----END PGP SIGNATURE----- > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >
-- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
