On Monday 04 April 2005 09:55, Michael 'buk' Scherer wrote: > Good day. > > Willing, yes. > Being able to, maybe. > > Details please. :)
Well, there are two obvious ways of speeding up inserting of attributes. 1. Cache the attributes, and combine them with the MD5/SHA1 signature that follows each attribute, and then do a single insert rather than an insert of the attribute followed by an update for the MD5/SHA1 signature. This would benefit all DB versions (SQLite, MySQL and PostgreSQL). Note, the user can turn off MD5 signatures so they may not always exist. Also, they are not generated for directories, and other special files. 2. For PostgreSQL, implement code that pre-allocates FileIds. Then use them from the pre-allocated pool. This will save one database access per File record inserted. The code for this probably exists in a previous submission by a user (Jamie ffolliott). However, I rejected that submission because the code also removed all locking, which would cause database corruption. Best regards, Kern > > M. > > Kern Sibbald wrote: > > Hello, > > > > Are you willing to do some C programming to get a performance > > improvement? > > > > Best regards, Kern > > > > On Monday 04 April 2005 05:42, Matt White wrote: > >>Hi - I've been evaluating Bacula for a while now, and I'm getting set to > >>transition from Amanda to Bacula. I'm backing up 17 hosts (Linux and > >>Tru64 Unix), currently to a 26 tape TL-891 (DLT-IV) library. I've got > >>my new tape drive in (30 tape MSL6030 with an Ultrium-2), and for the > >>last week or so, I've been running backups of several hosts with Bacula > >>to test everything. I'm running full dumps every night, since I wanted > >>to test spanning tapes and the autochanger. > >> > >>One thing that I'm concerned with is that it seems that the 'sending > >>spool attrs to the director' step, which I understand is inserting/ > >>updating attributes in the database, takes a VERY long time. I'm using > >>a PostgreSQL database, and the director and storage daemon are on the > >>same host (asclbackup). > >> > >>Below is a backup report - first of all, am I reading this right? It > >>looks to me like the actual backup is finished by around 22:42 (data > >>actually transferred to asclbackup by 21:15, and then another 1.5 hours > >>or so to dump it to tape), and then the next 8 hours are spent dealing > >>with the attributes. I'd really like to be able to speed that up a > >>bit... > >> > >>On the up side, the MSL6030 library works perfectly with bacula. > >> > >>Any ideas are appreciated...if you need config files to look at, the > >>sd and dir conf files are at: > >> > >>http://borris.usask.ca/bacula/bacula-dir.conf > >>http://borris.usask.ca/bacula/bacula-sd.conf > >> > >>with passwords taken out, of course :-) > >> > >>02-Apr 19:00 asclbackup-dir: Start Backup JobId 33, > >>Job=Arts-Backup.2005-04-02_19.00.02 > >>02-Apr 19:01 asclbackup-sd: Spooling data ... > >>02-Apr 21:15 asclbackup-sd: Committing spooled data to Volume. > >>Despooling 33,703,037,103 bytes ... > >>02-Apr 22:42 asclbackup-sd: Sending spooled attrs to the Director. > >>Despooling 317,625,650 bytes ... > >>03-Apr 06:35 asclbackup-dir: Bacula 1.36.2 (28Feb05): 03-Apr-2005 > >> 06:35:37 JobId: 33 > >> Job: Arts-Backup.2005-04-02_19.00.02 > >> Backup Level: Full > >> Client: arts > >> FileSet: "Arts FileSet" 2005-04-02 08:00:04 > >> Pool: "Monthly" > >> Storage: "MSL6000" > >> Start time: 02-Apr-2005 19:00:03 > >> End time: 03-Apr-2005 06:35:37 > >> FD Files Written: 1,079,169 > >> SD Files Written: 1,079,169 > >> FD Bytes Written: 33,484,543,201 > >> SD Bytes Written: 33,630,232,809 > >> Rate: 802.3 KB/s > >> Software Compression: None > >> Volume name(s): MTH0003 > >> Volume Session Id: 3 > >> Volume Session Time: 1112487694 > >> Last Volume Bytes: 103,959,793,626 > >> Non-fatal FD errors: 0 > >> SD Errors: 0 > >> FD termination status: OK > >> SD termination status: OK > >> Termination: Backup OK > >> > >>03-Apr 06:35 asclbackup-dir: Begin pruning Jobs. > >>03-Apr 06:35 asclbackup-dir: No Jobs found to prune. > >>03-Apr 06:35 asclbackup-dir: Begin pruning Files. > >>03-Apr 06:35 asclbackup-dir: No Files found to prune. > >>03-Apr 06:35 asclbackup-dir: End auto prune. > > > > ------------------------------------------------------- > > SF email is sponsored by - The IT Product Guide > > Read honest & candid reviews on hundreds of IT Products from real users. > > Discover which products truly live up to the hype. Start reading now. > > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click > > _______________________________________________ > > Bacula-users mailing list > > Bacula-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/bacula-users -- Best regards, Kern ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
