Re: [Bacula-users] Bacula backup speed

2015-12-17 Thread Robert Oschwald
We got approx 80 mio entries in the file table and I confirm Alan. Before, we had MySQL 5.5 with 24GB buffer ram, but attribute insertions took ages. Then we migrated the DB to PostgreSQL and the issue resolved. 1 year ago, we migrated the physical backup Server to a vCenter VM and lowered the

Re: [Bacula-users] Bacula backup speed

2015-12-17 Thread Uwe Schuerkamp
On Tue, Dec 15, 2015 at 05:16:42PM +, Alan Brown wrote: > > MySQL works ok for small sites but doesn't scale well. PostgreSQL is a > heavy load on small installations but will keep running long after MySQL > has decided to use all your system ram and swap too. The breakeven point > is about

Re: [Bacula-users] Bacula backup speed

2015-12-16 Thread Phil Stracchino
On 12/15/15 20:22, Bryn Hughes wrote: > MySQL is clever enough that it won't assign memory until it actually > needs it. This is true to a point. It is NOT true of the InnoDB buffer pool, which is pre-allocated at startup time and cannot be dynamically resized prior to MySQL 5.7. It is generall

Re: [Bacula-users] Bacula backup speed

2015-12-16 Thread Uwe Schuerkamp
I fully agree with Bryn here, 8GB would be overkill for a 300MB database. Just make sure mysql has enough memory to keep your largest DB in RAM, so increasing the buffer pool if necessary always seems like a good option. > > We have 659,172 entries in the File table. That is quite a small catalo

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Gary R. Schmidt
Hi Dave, On 2015-12-16 12:39, Lewis, Dave wrote: >> -Original Message- >> From: Alan Brown [mailto:a...@mssl.ucl.ac.uk] >> Sent: Tuesday, December 15, 2015 8:08 AM >> To: Lewis, Dave; bacula-users@lists.sourceforge.net >> Subject: Re: [Bacula-users] Bacula b

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Lewis, Dave
> -Original Message- > From: Heitor Faria [mailto:hei...@bacula.com.br] > Sent: Tuesday, December 15, 2015 8:21 AM > To: Alan Brown > Cc: Lewis, Dave; bacula-users@lists.sourceforge.net > Subject: Re: [Bacula-users] Bacula backup speed > ... > > Suggestion: http:

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Lewis, Dave
> -Original Message- > From: Alan Brown [mailto:a...@mssl.ucl.ac.uk] > Sent: Tuesday, December 15, 2015 8:08 AM > To: Lewis, Dave; bacula-users@lists.sourceforge.net > Subject: Re: [Bacula-users] Bacula backup speed ... > What size is your database? ("select cou

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Bryn Hughes
On 2015-12-15 04:50 PM, Lewis, Dave wrote: >> -Original Message- >> From: Uwe Schuerkamp [mailto:uwe.schuerk...@nionex.net] >> Sent: Tuesday, December 15, 2015 4:47 AM >> To: Lewis, Dave >> Cc: bacula-users@lists.sourceforge.net >> Subject: Re

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Lewis, Dave
> From: Josh Fisher [mailto:jfis...@pvct.com] > Sent: Tuesday, December 15, 2015 6:57 AM > To: bacula-users@lists.sourceforge.net > Subject: Re: [Bacula-users] Bacula backup speed > ... > > What else is writing to this same disk when the metadata is being > written?

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Lewis, Dave
> -Original Message- > From: Uwe Schuerkamp [mailto:uwe.schuerk...@nionex.net] > Sent: Tuesday, December 15, 2015 4:47 AM > To: Lewis, Dave > Cc: bacula-users@lists.sourceforge.net > Subject: Re: [Bacula-users] Bacula backup speed > ... > Hi Dave, > > how

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Alan Brown
On 15/12/15 13:20, Heitor Faria wrote: > Suggestion: http://bacula.us/tuning/ Whilst that page is a good starting point, a couple of points are flat out wrong: EG: setting maximum block size in tapes - DON'T SET THIS - EVER (unless the driver manufacturer advises it) Compression should only _

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Heitor Faria
>> We are running MySQL, and the database is on the same server as the director. >> The disk that the database is on is a 7200 RPM, 3 Gb/s SATA disk. > > What size is your database? ("select count(*) from File;") > What write speeds are you actually achieving to the database drive? > (MB/s and io

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Alan Brown
at first, THEN look at the attribute despooling issue. > Thanks, > Dave > > > -Original Message- > From: Bryn Hughes [mailto:li...@nashira.ca] > Sent: Monday, December 14, 2015 4:37 PM > To:bacula-users@lists.sourceforge.net > Subject: Re: [Bacula-users] Bacula back

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Josh Fisher
anks, > Dave > > > -Original Message- > From: Bryn Hughes [mailto:li...@nashira.ca] > Sent: Monday, December 14, 2015 4:37 PM > To: bacula-users@lists.sourceforge.net > Subject: Re: [Bacula-users] Bacula backup speed > > Sounds pretty clear that you have some performance

Re: [Bacula-users] Bacula backup speed

2015-12-15 Thread Uwe Schuerkamp
On Mon, Dec 14, 2015 at 09:12:06PM +, Lewis, Dave wrote: > Hi, > > Thanks. I ran it again with attribute spooling. That sped up the backup of > data to the disk pool - instead of 6 hours it took less than 2 - but writing > the file metadata afterwards took nearly 6 hours. > > 12-Dec 18:24 j

Re: [Bacula-users] Bacula backup speed

2015-12-14 Thread Lewis, Dave
@lists.sourceforge.net Subject: Re: [Bacula-users] Bacula backup speed Sounds pretty clear that you have some performance issues on your database. What are you running (MySQL/Postgresql/Sqlite/etc) for your DB? Is it running on the same server as your director? What kind of disk storage do you have under

Re: [Bacula-users] Bacula backup speed

2015-12-14 Thread Bryn Hughes
.@informatik.uni-bremen.de] > Sent: Thursday, December 10, 2015 6:14 AM > To: bacula-users@lists.sourceforge.net > Subject: Re: [Bacula-users] Bacula backup speed > > On 10.12.2015 01:06, Lewis, Dave wrote: >> Does anyone know what's causing the OS backups to be so slow and what &

Re: [Bacula-users] Bacula backup speed

2015-12-14 Thread Lewis, Dave
about 700 KB/s, but the total elapsed time increased. Thanks, Dave -Original Message- From: Christian Manal [mailto:moen...@informatik.uni-bremen.de] Sent: Thursday, December 10, 2015 6:14 AM To: bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Bacula backup speed On

Re: [Bacula-users] Bacula backup speed

2015-12-10 Thread Lewis, Dave
arfiles to tape. Thanks! Dave From: Uwe Schuerkamp [uwe.schuerk...@nionex.net] Sent: Thursday, December 10, 2015 6:30 AM To: Lewis, Dave Cc: bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Bacula backup speed On Thu, Dec 10, 2015 at 12:06:42

Re: [Bacula-users] Bacula backup speed

2015-12-10 Thread Christian Manal
On 10.12.2015 01:06, Lewis, Dave wrote: > Does anyone know what’s causing the OS backups to be so slow and what I > can do to speed them up? Hi, the problem might be number of files, as in, writing all the file metadata to the catalog could very well be your bottle neck. Try enabling attribute s

Re: [Bacula-users] Bacula backup speed

2015-12-10 Thread Uwe Schuerkamp
On Thu, Dec 10, 2015 at 12:06:42AM +, Lewis, Dave wrote: > Hi, > > I'm configuring Bacula backups and sometimes it is very slow to back up to > disk or tape, around 1 MB/s and sometimes slower. I'm wondering why it is > sometimes so slow and if there is something I can do differently that wi