On Friday 09 September 2005 09:55, you wrote: > Hi, > > Arunav Mandal wrote: > > The director is running on xeon 2.8Ghz having 4GB of RAM on Hardware > > Raid0 and the storage demaon is on dual P4 3Ghz having 1GB ram and 1TB > > disk spool(xfs) also on Hardware Raid0. Most of clients are on P4 3gig > > machine with 1Gb ram running linux and windows.At peak there were > > atleast 40Jobs running at the same time. I will give a example of > > slowness below. As you can see the Rate is only at 568.2 KB/s before > > spooling it was around 8000KB/S. At the time of this client backup there > > were around 10 jobs running. As you can see this backup took around > > 7hrs. So my question is having latest hardware and enough memory why > > this is slow and what could be done to improve it. > > You've got to consider other things as well. > First, the overall throughput - how long does it take all jour jobs to > complete? Assuming that client speed is a limit, concurrent jobs should > improve this. > Second, network bandwith - If one client alone could saturate your > backup server's connection, multiple concurrent jobs won't be faster. > Your backup time window - sometimes it might be better to have clients > spool data overnight and only wait for job completion (despooling) > during work hours than having some clients waiting through the night and > loaded by spooling data during daytime. > > Arno
Average full backups of clients take around 4 to 5 hrs and for servers it last upto 10 to 12hrs. For servers we are talking about 52gig. Munin shows peak network peak at 3Mbps so that is not much. Since I set concurrent jobs to 150 there is no clients waiting but to spool data takes much time and the problem starts when people bring laptops to office and take back in evening. I am not worried about despooling as we got SDLT600 2U autoloader from Tandberg and that is very fast on Ultra wide Scsi 320. The only thing I want is how can I make clients spool faster. Arunav. > > > 08-Sep 15:47 No prior Full backup Job record found. > > 08-Sep 15:47 No prior or suitable Full backup found. Doing FULL backup. > > 08-Sep 15:47 : Start Backup JobId 1012, > > Job=backup-abc.2005-09-08_15.47.00 07-Sep 15:48 Spooling data ... > > 07-Sep 21:41 Committing spooled data to Volume. Despooling > > 13,235,326,697 bytes ... > > 07-Sep 21:56 Sending spooled attrs to the Director. Despooling > > 122,253,435 bytes ... > > 08-Sep 22:12 Bacula 1.36.3 (22Apr05): 08-Sep-2005 22:12:50 > > JobId: 1012 > > Job: backup-abc.2005-09-08_15.47.00 > > Backup Level: Full (upgraded from Incremental) > > Client: abc-fd > > FileSet: "linux-default" 2005-09-08 00:20:01 > > Pool: "daily" > > Storage: "SDLT6K-1" > > Start time: 08-Sep-2005 15:47:02 > > End time: 08-Sep-2005 22:12:50 > > FD Files Written: 396,170 > > SD Files Written: 396,170 > > FD Bytes Written: 13,151,694,548 > > SD Bytes Written: 13,208,137,447 > > Rate: 568.2 KB/s > > Software Compression: None > > Volume name(s): SDLT005 > > Volume Session Id: 21 > > Volume Session Time: 1126100490 > > Last Volume Bytes: 154,989,306,412 > > Non-fatal FD errors: 0 > > SD Errors: 0 > > FD termination status: OK > > SD termination status: OK > > Termination: Backup OK > > > > > > > > > > ----- Original Message ----- From: "Arno Lehmann" <[EMAIL PROTECTED]> > > To: "Arunav Mandal" <[EMAIL PROTECTED]> > > Cc: <bacula-users@lists.sourceforge.net> > > Sent: Thursday, September 08, 2005 9:09 PM > > Subject: Re: [Bacula-users] Slow backups > > > >> Hi, > >> > >> Arunav Mandal wrote: > >>> I am running 1.36.3 on both director and storage deamon and diff. > >>> versions on 150 clients. Recently I started spooling to disk first > >>> then to tape with around 100 concurrent connections for director and > >>> storage deamons. Now the backups seems to take time Rate seems to be > >>> around 700KB/s before spooling and concurrent connections it was > >>> around 8000KB/s. I know too many clients are backing at the same time > >>> is that the reason? > >> > >> I can't follow you here, but yes, having too many clients working > >> concurrently can reduce the overall performance. You've got to find > >> your personal optimal number of concurrent jobs depending on overall > >> network throughput, server power, storage and spool speed - I don't > >> think that it's possible to tell you what the perfect setup is for you. > >> > >>> Also when all the clients start backing up at the same time I can't > >>> log onto the console since there is no free connections left is there > >>> any way I can atleast reserve one connection for the console? > >> > >> Two possible reasons: > >> Not enough connections allowed - change the numer in the config. > >> Server load too high - wait, or, better, have less clients run > >> simultaneously. Remember that each job means a fork and a process, > >> takes memory, cpu time, and loads the catalog database. That's all > >> stuff you need to work on the console, too. > >> > >>> Thx, > >>> > >>> Arunav. > >> > >> Arno > >> > >> > >> -- > >> IT-Service Lehmann [EMAIL PROTECTED] > >> Arno Lehmann http://www.its-lehmann.de -- Arunav Mandal - amandal at trolltech.com Trolltech AS - Waldemar Thranes gt. 98, NO-0175 Oslo, Norway Phone: +47 21 60 27 22, Fax: +47 21 60 48 02 ------------------------------------------------------- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
