I scoped the problem as two major projects: > > > > 1) implementation of "copy pools" -- where files written to a pool > > were automatically also written to up to 3 additional pools > using the > > same volume selection criteria as exist now (essentially > getting the > > SD to act as a FD to more than one FDs to ensure > synchronous updates, > > or creating a SD-mux function to split the FD data stream to N SDs). > > In a certain sense this is already implemented in 1.37. 1.37 > permits a job to start a second job in a way that makes the > second job duplicate the "since" > date/time. This effectively allows making identical copies > (except for files that change during the backup).
Hmm. Does the second job transfer the data from the FD again? If so, then that doesn't (IMHO) quite do what I want to do here. I really want to transfer the data only once (the only guarantee we have of getting the same data on all the copies) and create the replicas on the server side. > > 2) implementation of pool to pool migration as discussed on the list > > previously. > > Pool to Pool, or direct writing by the SD to several devices > within one Job > both require a few more changes to the SD. All the basic > data structures > exist so that the SD can have multiple I/O packets open, but > as of 1.37, the > SD only has a single I/O packet per job. In older versions > of Bacula, there > were only DEVICE stuructures, one per physical device. Now, > there are DCR > (device control records) that sit "above" the DEVICE > structure. There can be > multiple DCRs that use the same DEVICE in 1.37 -- this is no > problem. > However, the next generalization, and most of the code is in, > is to allow a > job to have multiple DCRs. The job must be taught to open > multiple devices > (and thus create multiple DCRs), and then either read from > one DCR and write > to another (copy), and/or write to multiple DCRs. (As a side issue, I'm beginning to wonder if overall we need a more generalized job manager. This is sort of sounding like we need something like JCL, and then this could all be handled in a more systematic way. That's a much bigger project, though.) OK, I can see how that could work. > The only part of this implementation that I have not worked > out in my head is > how to deal with the catalog. If there are two identical > backups of a Job on > two different Volumes, the Director is not currently aware of > it. There > needs to be some way of flagging two copies as being > identical, then using > only one of those copies for any given restore. What I was thinking with the copypool idea was to have multiple volume records for a file, and sorting by copypool order, eg pool A has copypools B and C. During backup, the file is stored on a volume in pool A, and also stored on volumes selected from pool B and C (finite number of these, to avoid impacting performance significantly). The database would reflect 1 file record with multiple volume records pointing to the volumes selected from pool A, B, and C, with an indication of a pool priority (based on the sequence of the copypools) to indicate which version to try first. When traversing the file table for a restore, you can retrieve the list of volume records containing that file, and iterate through them either in copypool priority sequence or try them at random, prioritizing for volumes currently mounted and available. If the volume you try is in the changer and available, use it, else try the next one. > Also, there > needs to be a > bit more work in implementing some better user interface for > "archiving" i.e. > taking Volumes (possible identical copies) out of the Volume > pool, and then > later re-introducing them. With the approach above, just taking the volumes in and out of the changers does the job for you. No new wheels needed. ------------------------------------------------------- This SF.Net email is sponsored by: Power Architecture Resource Center: Free content, downloads, discussions, and more. http://solutions.newsforge.com/ibmarch.tmpl _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
