* Arno Lehmann <a...@its-lehmann.de>: > 27.02.2010 14:46, Stan Meier wrote: > > 1. Keeping configuration sane: With more than 120 servers, we need to > > find a way to keep the configuration files readable. Our servers all > > follow some naming scheme, for example, we got "appserver01" through > > "appserver08" or "webcache01" through "webcache04". We think we should > > split client configurations for each server group, so the file > > "clientdefs/appserver.conf" would define all appserver0X clients. > > Furthermore, most of those servers will need a default job performed > > (/etc, /root, /opt and so on). While it's easy to reuse a "JobDefs" > > stanza to actually define all those jobs, isn't there any way to > > "group" those servers? Do we really have to define more than 120 jobs, > > one for each server? > > The way to go, in my opinion, is to create the actual configuration > dynamically - you can include script output into the configuration > *anywhere*... now use a script that creates the actual configuration > dynamically, starting with a template where you insert the client name.
While you are right and creating a configuration based on scripts is quite easy (and has added benefits, for example that you dan define one file pool per server group), we still have to deal with 120 backup jobs. But, since you didn't jump on that part of my question, I presume there is no solution to that? > > 2. Backup availability: One plan would be to use a large part of the > > 24TB available as a FilePool (or several). Each job would then write > > it's data to that pool. A Copy job could copy the data to tape later > > on - with the advantage that restores of recent data would be quite > > fast since they would still be sitting on disk. Before running the > > backup the next day, we would simply recycle those file volumes. Is > > that a reasonable strategy? > > Yes. Properly set up, that's a very reasonable approach. You'll need > to understand retention times and how to select jobs for migration in > detail. I see several things here which we will have to look at. Please correct me if I'm wrong or if I forgot anything: 1. Concurrency: We will need to investigate all the different places in Bacula where job concurrency, concurrent pool/storage usage and connection limits are defined and adjust them to "fit together" as well as optimize them to the I/O operations limit of our raid storage. 2. Scheduling: Ideally, the copy job would start as soon as all the backup jobs have finished. But since the Schedule resource does not allow references to Job names, we are pretty much screwed in that department and will probably have to resort to a fixed schedule. 3. Job selection: You already pointed that one out. Would it be as easy as just selecting all uncopied jobs from a given pool? Since all volumes in the pool are recycle before backup starts (or while it is running), naturally, the only uncopied jobs would be those that were written to disk recently. So, for now, our backup plan would be something like: 1. Start actual backup 1.1 Start backup jobs in parallel 1.2 (Possibly erase all used volumes in a given pool) 1.3 Get data from clients, write to File pools 1.4 wait until all backup operations have finished 2. Copy all recent jobs to tape 3. Backup catalog I don't see how we can synchronize more than 120 backup jobs yet, to be honest. We could run backup at 9pm and the copy job at 10am to allow for a large margin of error, but that just doesn't feel like a proper solution. Stan ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
