Thanks :) Which type of MySQL storage engine are you using? MyISAM or InnoDB for large Bacula system? Can you please copy/paste your MySQL configuration? I mean my.cnf file
Thanks, Ondrej. Henrik Johansen napsal(a): > 'Ondrej PLANKA (Ignum profile)' wrote: > >> Hello Henrik, >> >> what are you using? MySQL? >> > > Yes - all our catalog servers run MySQL. > > I forgot to mention this in my last post - we are Bacula System > customers and they have proved to very supportive and competent. > > If you are thinking about doing large scale backups with Bacula I can > only encourage you to get a support subscription - it is worth every > penny. > > > >> Thanks, Ondrej. >> >> 'Mingus Dew' wrote: >> >>> Henrik, >>> Have you had any problems with slow queries during backup or restore >>> jobs? I'm thinking about http://bugs.bacula.org/view.php?id=1472 >>> specifically, and considering that the bacula.File table already has 73 >>> million rows in it and I haven't even successfully ran the "big" job >>> yet. >>> >> Not really. >> >> We have several 10+ million file jobs - all run without problem (backup >> and restore). >> >> I am aware of the fact that a lot of Bacula users run PG ( Bacula >> Systems also does recommend PG for larger setups ) but nevertheless >> MySQL has served us very well so far. >> >> >>> Just curious as a fellow Solaris deployer... >>> >>> Thanks, >>> Shon >>> >>> On Fri, Oct 8, 2010 at 3:30 PM, Henrik Johansen >>> <hen...@scannet.dk<mailto:hen...@scannet.dk >>> <mailto:hen...@scannet.dk%3cmailto:hen...@scannet.dk>>> wrote: >>> 'Mingus Dew' wrote: >>> All, >>> I am running Bacula 5.0.1 on Solaris 10 x86. I'm currently running >>> MySQL 4.1.22 for the database server. I do plan on upgrading to a >>> compatible version of MySQL 5, but migrating to PostgreSQL isn't an >>> option at this time. >>> >>> I am trying to backup to tape a very large number of files for a >>> client. While the data size is manageable at around 2TB, the number of >>> files is incredibly large. >>> The first of the jobs had 27 million files and initially failed because >>> the batch table became "Full". I changed the myisam_data_pointer size >>> to a value of 6 in the config. >>> >>> This job was then able to run successfully and did not take too long. >>> >>> I have another job which has 42 million files. I'm not sure what that >>> equates to in rows that need to be inserted, but I can say that I've >>> not been able to successfully run the job, as it seems to hang for >>> over 30 hours in a "Dir inserting attributes" status. This causes >>> other jobs to backup in the queue and once canceled I have to restart >>> Bacula. >>> >>> I'm looking for way to boost performance of MySQL or Bacula (or both) >>> to get this job completed. >>> >>> You *really* need to upgrade to MySQL 5 and change to InnoDB - there is no >>> way in hell that MySQL 4 + MyISAM is going to perform decent in your >>> situation. >>> Solaris 10 is a Tier 1 platform for MySQL so the latest versions are >>> always available from http://www.mysql.com in the native pkg format so >>> there really >>> is no excuse. >>> >>> We run our Bacula Catalog MySQl servers on Solaris (OpenSolaris) so >>> perhaps I can give you some pointers. >>> >>> Our smallest Bacula DB is currently ~70 GB (381,230,610 rows). >>> >>> Since you are using Solaris 10 I assume that you are going to run MySQL >>> off ZFS - in that case you need to adjust the ZFS recordsize for the >>> filesystem that is going to hold your InnoDB datafiles to match the >>> InnoDB block size. >>> >>> If you are using ZFS you should also consider getting yourself a fast >>> SSD as a SLOG (or to disable the ZIL entirely if you dare) - all InnoDB >>> writes to datafiles are O_SYNC and benefit *greatly* from an SSD in >>> terms of write / transaction speed. >>> >>> If you have enough CPU power to spare you should try turning on >>> compression for the ZFS filesystem holding the datafiles - it also can >>> accelerate DB writes / reads but YMMV. >>> >>> Lastly, our InnoDB related configuration from my.cnf : >>> >>> # InnoDB options skip-innodb_doublewrite >>> innodb_data_home_dir = /tank/db/ >>> innodb_log_group_home_dir = /tank/logs/ >>> innodb_support_xa = false >>> innodb_file_per_table = true >>> innodb_buffer_pool_size = 20G >>> innodb_flush_log_at_trx_commit = 2 >>> innodb_log_buffer_size = 128M >>> innodb_log_file_size = 512M >>> innodb_log_files_in_group = 2 >>> innodb_max_dirty_pages_pct = 90 >>> >>> >>> >>> Thanks, >>> Shon >>> >>> ------------------------------------------------------------------------------ >>> Beautiful is writing same markup. Internet Explorer 9 supports >>> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. >>> Spend less time writing and rewriting code and more time creating great >>> experiences on the web. Be a part of the beta today. >>> http://p.sf.net/sfu/beautyoftheweb >>> >>> _______________________________________________ >>> Bacula-users mailing list >>> Bacula-users@lists.sourceforge.net<mailto:Bacula-users@lists.sourceforge.net >>> >>> <mailto:bacula-us...@lists.sourceforge.net%3cmailto:Bacula-users@lists.sourceforge.net>> >>> https://lists.sourceforge.net/lists/listinfo/bacula-users >>> >>> >>> -- >>> Med venlig hilsen / Best Regards >>> >>> Henrik Johansen >>> hen...@scannet.dk<mailto:hen...@scannet.dk >>> <mailto:hen...@scannet.dk%3cmailto:hen...@scannet.dk>> >>> Tlf. 75 53 35 00 >>> >>> ScanNet Group >>> A/S ScanNet >>> >>> >>> ------------------------------------------------------------------------------ >>> Beautiful is writing same markup. Internet Explorer 9 supports >>> standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. >>> Spend less time writing and rewriting code and more time creating great >>> experiences on the web. Be a part of the beta today. >>> http://p.sf.net/sfu/beautyoftheweb >>> >>> _______________________________________________ >>> Bacula-users mailing list >>> Bacula-users@lists.sourceforge.net >>> <mailto:Bacula-users@lists.sourceforge.net> >>> https://lists.sourceforge.net/lists/listinfo/bacula-users >>> >> -- >> Med venlig hilsen / Best Regards >> >> Henrik Johansen >> hen...@scannet.dk <mailto:hen...@scannet.dk> >> Tlf. 75 53 35 00 >> >> ScanNet Group >> A/S ScanNet >> >> >> >> >> > > ------------------------------------------------------------------------------ Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
