'Ondrej PLANKA (Ignum profile)' wrote: >Thanks :) >Which type of MySQL storage engine are you using? MyISAM or InnoDB for >large Bacula system? >Can you please copy/paste your MySQL configuration? I mean my.cnf file
Please re-read this thread and you should find what you are looking for. >Thanks, Ondrej. > > >Henrik Johansen napsal(a): >> 'Ondrej PLANKA (Ignum profile)' wrote: >> >>> Hello Henrik, >>> >>> what are you using? MySQL? >>> >> >> Yes - all our catalog servers run MySQL. >> >> I forgot to mention this in my last post - we are Bacula System >> customers and they have proved to very supportive and competent. >> >> If you are thinking about doing large scale backups with Bacula I can >> only encourage you to get a support subscription - it is worth every >> penny. >> >> >> >>> Thanks, Ondrej. >>> >>> 'Mingus Dew' wrote: >>> >>>> Henrik, >>>> Have you had any problems with slow queries during backup or restore >>>> jobs? I'm thinking about http://bugs.bacula.org/view.php?id=1472 >>>> specifically, and considering that the bacula.File table already has 73 >>>> million rows in it and I haven't even successfully ran the "big" job >>>> yet. >>>> >>> Not really. >>> >>> We have several 10+ million file jobs - all run without problem (backup >>> and restore). >>> >>> I am aware of the fact that a lot of Bacula users run PG ( Bacula >>> Systems also does recommend PG for larger setups ) but nevertheless >>> MySQL has served us very well so far. >>> >>> >>>> Just curious as a fellow Solaris deployer... >>>> >>>> Thanks, >>>> Shon >>>> >>>> On Fri, Oct 8, 2010 at 3:30 PM, Henrik Johansen >>>> <hen...@scannet.dk<mailto:hen...@scannet.dk >>>> <mailto:hen...@scannet.dk%3cmailto:hen...@scannet.dk>>> wrote: >>>> 'Mingus Dew' wrote: >>>> All, >>>> I am running Bacula 5.0.1 on Solaris 10 x86. I'm currently running >>>> MySQL 4.1.22 for the database server. I do plan on upgrading to a >>>> compatible version of MySQL 5, but migrating to PostgreSQL isn't an >>>> option at this time. >>>> >>>> I am trying to backup to tape a very large number of files for a >>>> client. While the data size is manageable at around 2TB, the number of >>>> files is incredibly large. >>>> The first of the jobs had 27 million files and initially failed because >>>> the batch table became "Full". I changed the myisam_data_pointer size >>>> to a value of 6 in the config. >>>> >>>> This job was then able to run successfully and did not take too long. >>>> >>>> I have another job which has 42 million files. I'm not sure what that >>>> equates to in rows that need to be inserted, but I can say that I've >>>> not been able to successfully run the job, as it seems to hang for >>>> over 30 hours in a "Dir inserting attributes" status. This causes >>>> other jobs to backup in the queue and once canceled I have to restart >>>> Bacula. >>>> >>>> I'm looking for way to boost performance of MySQL or Bacula (or both) >>>> to get this job completed. >>>> >>>> You *really* need to upgrade to MySQL 5 and change to InnoDB - there is no >>>> way in hell that MySQL 4 + MyISAM is going to perform decent in your >>>> situation. >>>> Solaris 10 is a Tier 1 platform for MySQL so the latest versions are >>>> always available from http://www.mysql.com in the native pkg format so >>>> there really >>>> is no excuse. >>>> >>>> We run our Bacula Catalog MySQl servers on Solaris (OpenSolaris) so >>>> perhaps I can give you some pointers. >>>> >>>> Our smallest Bacula DB is currently ~70 GB (381,230,610 rows). >>>> >>>> Since you are using Solaris 10 I assume that you are going to run MySQL >>>> off ZFS - in that case you need to adjust the ZFS recordsize for the >>>> filesystem that is going to hold your InnoDB datafiles to match the >>>> InnoDB block size. >>>> >>>> If you are using ZFS you should also consider getting yourself a fast >>>> SSD as a SLOG (or to disable the ZIL entirely if you dare) - all InnoDB >>>> writes to datafiles are O_SYNC and benefit *greatly* from an SSD in >>>> terms of write / transaction speed. >>>> >>>> If you have enough CPU power to spare you should try turning on >>>> compression for the ZFS filesystem holding the datafiles - it also can >>>> accelerate DB writes / reads but YMMV. >>>> >>>> Lastly, our InnoDB related configuration from my.cnf : >>>> >>>> # InnoDB options skip-innodb_doublewrite >>>> innodb_data_home_dir = /tank/db/ >>>> innodb_log_group_home_dir = /tank/logs/ >>>> innodb_support_xa = false >>>> innodb_file_per_table = true >>>> innodb_buffer_pool_size = 20G >>>> innodb_flush_log_at_trx_commit = 2 >>>> innodb_log_buffer_size = 128M >>>> innodb_log_file_size = 512M >>>> innodb_log_files_in_group = 2 >>>> innodb_max_dirty_pages_pct = 90 >>>> >>>> >>>> >>>> Thanks, >>>> Shon >>>> >>>> ------------------------------------------------------------------------------ >>>> Beautiful is writing same markup. Internet Explorer 9 supports >>>> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. >>>> Spend less time writing and rewriting code and more time creating great >>>> experiences on the web. Be a part of the beta today. >>>> http://p.sf.net/sfu/beautyoftheweb >>>> >>>> _______________________________________________ >>>> Bacula-users mailing list >>>> Bacula-users@lists.sourceforge.net<mailto:Bacula-users@lists.sourceforge.net >>>> >>>> <mailto:bacula-us...@lists.sourceforge.net%3cmailto:Bacula-users@lists.sourceforge.net>> >>>> https://lists.sourceforge.net/lists/listinfo/bacula-users >>>> >>>> >>>> -- >>>> Med venlig hilsen / Best Regards >>>> >>>> Henrik Johansen >>>> hen...@scannet.dk<mailto:hen...@scannet.dk >>>> <mailto:hen...@scannet.dk%3cmailto:hen...@scannet.dk>> >>>> Tlf. 75 53 35 00 >>>> >>>> ScanNet Group >>>> A/S ScanNet >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Beautiful is writing same markup. Internet Explorer 9 supports >>>> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. >>>> Spend less time writing and rewriting code and more time creating great >>>> experiences on the web. Be a part of the beta today. >>>> http://p.sf.net/sfu/beautyoftheweb >>>> >>>> _______________________________________________ >>>> Bacula-users mailing list >>>> Bacula-users@lists.sourceforge.net >>>> <mailto:Bacula-users@lists.sourceforge.net> >>>> https://lists.sourceforge.net/lists/listinfo/bacula-users >>>> >>> -- >>> Med venlig hilsen / Best Regards >>> >>> Henrik Johansen >>> hen...@scannet.dk <mailto:hen...@scannet.dk> >>> Tlf. 75 53 35 00 >>> >>> ScanNet Group >>> A/S ScanNet >>> >>> >>> >>> >>> >> >> -- Med venlig hilsen / Best Regards Henrik Johansen hen...@scannet.dk Tlf. 75 53 35 00 ScanNet Group A/S ScanNet ------------------------------------------------------------------------------ Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
