On 2013-12-19 12:06 PM, Greg Woods wrote: > On Thu, 2013-12-19 at 11:02 -0500, Dan Langille wrote: >> There was a lot in the above and I couldn't parse it all >> just now. > > Sorry about that; I suppose it would help if I just explicitly stated > what I really want. I have a main backup pool where all the backup jobs > write. This pool is manually copied to the archive pool using Copy > jobs. > What I want is for volumes in the main pool to be recycled relatively > quickly (because space is limited), while the files are retained much > longer in the archive pool. I'd like restore attempts to automatically > figure out when the files are still available in the backup pool and > use > it when possible, but get them from the archive pool if necessary. > That's what I'm working toward.
Let others review this suggestion before trying it: I set my FILE and JOB retentions high. 3 years. Then I set my VOLUME retention lower. Whichever retention period expires first, that's the one which counts. <== I will refer to that as 'first one counts' later in this email. Therefore, in your archive pool, set your VOLUME retention to 3 years, for example. And in your main pool, set it to 5 weeks, for example. Does that help? >> Instead, I'll point you to something I wrote a few years ago. >> If you have questions after that, please ask again. >> >> http://dan.langille.org/2010/02/08/bacula-retention-periods/ > > OK, good. It's good that we're having this discussion. If I understand > any of this at all (which I may not), there are a couple of things in > there that seem not to match my experience so far. For instance, in > that > document it says "you can’t restore a given file after the File > retention passed". But that's not really true. You can't restore it in > the "usual" way, by marking files, presumably because the list of > available files for marking is created using those File records that > aren't there any more. The next sentence in my post is: If the Job retention has not passed, you can still restore the whole job. So yes, you have it right. > But you *can* restore those files by using a > regular expression; I've had to do this because I have lost a lot of my > File records by initially setting File Retention too low. I'm not sure > how it actually does this, but there must be some information written > to > the backup volume itself that can be used to determine what files are > there. It's much slower than a database access, but I have restored > directory trees and individual files this way. That's interesting. > > The document also says "Volumes are eligible for recycling once ALL > retention periods have passed." But that doesn't seem to be true > either; I have apparently seen Bacula recycle some volumes even though > the Job retention period has not passed. Which means, presumably, that > there are Job records in the database that point to backup files that > aren't there any more, since the Volume has been recycled. Yes, that statement seems to be wrong. As demonstrated by my 'first one counts' claim above. >>> Perhaps the Recycling Algorithm will help: >> >> http://www.bacula.org/en/dev-manual/main/main/Automatic_Volume_Recycling.html > > > That does help, and I've looked at this before, but it's not always > clear (to me) exactly how it works, even after reading that. > > I notice that the paragraph on Automatic Pruning says that you can't > restore files once the File records are gone, but I know that's not > true > because I have done it. I suspect the documentation may be a few > versions old and I can't necessarily trust everything it says. You've done it a different way. Not the 'usual way'. > It also states that when volume recycling happens, "all Jobs that are > older than the VolumeRetention period will be pruned", presumably > regardless of what I set for Job Retention, which leads to some of my > confusion about what these retention periods really mean. It's even > more confusing (to me) when you throw in Copy jobs, so that the same > files may be on multiple Volumes in different pools. Does what you've observed make sense when you consider my claim of 'first one counts'? I am sure that is mentioned in the docs. -- Dan Langille - http://langille.org/ ------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
