Justin, I don't have answers to all your questions (really, to any of your specific questions, anyhow). I do have some thoughts, though.
Please keep in mind that compared to many on this list, I'm a bacula novice. I might get something wrong. I also haven't entirely understood the problems you're facing, so maybe some of this might sound irrelevant. Please ignore if so. However, I think there might be a few things I can help with here. 1. I know that bacula can define 3 kinds of retention: File, Job, and Volume. File retention means "how long should I keep records about these files in the catalog", NOT "how long should I keep these files in the bacula backup volumes". So if you prune file retention records you will delete information about which files are in a job, but will still be able to restore that job later. Maybe you want to save catalog space for older backup jobs. The consequences of pruning job and volume records are different. A previously used volume with no jobs in the catalog is subject to recycling and reuse. 2. There are default retention periods coded into bacula that will be used if the administrator does not specify their own retention periods. I don't know what they are, but you can look them up. I think maybe 60 days? Not sure. 3. To my knowledge, bacula stores File, Job, and Volume retention time periods in the director configuration file (bacula-dir.conf). The manual can be more specific about this, but at minimum you should be able to specify these options under either the jobdef, job, or pool resource (I think pool resource maybe can't use file retention, only job or volume retention?). Maybe you can also specify retention periods at the FD level, but I haven't used that and am not sure if it's possible or how it works. I would think that the FD probably wouldn't be able to specify, say, volume retention time since I imagine that's outside the scope of the FD. ****Retention periods specified in the pool over-ride all other retention periods!**** 4. If bacula doesn't store any file, job, or volume retention times in the catalog, it's probably to save on database space. From a programming perspective, there's no reason to store retention times in the catalog. Bacula can simply look at the relevant configuration files, check the current time/date, and then check the catalog for file, job, or volumes that are older than the specified retention period for those items. This wouldn't save a lot of database space for jobs or volumes, but file retention... oh boy. Yeah, for file retention it'd save a LOT of database space. Think of all the posix compliant time/date codes, one for every copy of a file ever backed up. Those bytes add up over millions and billions of file entries. Regards, Robert Gerber 402-237-8692 r...@craeon.net On Sun, Oct 20, 2024 at 6:15 AM Justin Case <jus7inc...@gmail.com> wrote: > I have browsed the catalog database a bit more and I saw one thing that > might explain the problem I am having, but I don't have sufficient > technical background on the Bacula implementation details of different > versions. > > I am running bacula-dir 15.0.2 and bacularis 3.1.0. The Bacula manual and > Bacularis offer “Job Retention” and “File Retention” in the pool resource, > supposedly overriding the same fields in the client resource. I am not > using these fields in the client resource, and in the client table there > are the job and file retention default values. > I am only using the job and file retention fields in the pool resources in > Bacularis. > What irritates me is that in the pool DB table there are no columns for > job and for file retention! > Are these values stored elsewhere or has my DB not been initially set up > or not correctly upgraded? > > If the pool resource retention values for jobs and files are missing in my > DB this would mean that bacula-dir is using the values from the client > resources, which is consistent with what I am seeing (with 2 exceptions): > The oldest jobs are 6 months old (job retention default value). > Exceptions: (a) catalog backup jobs go back to the date when the > installation was set up (b) two other jobs have altogether 4 instances that > are older than 6 months. > > So open questions for me are: > > (1) where are job retention and file retention stored for the pool > resource? > > (2) if in the pool table, why does my pool table not have corresponding > columns for job and file retention? How may I fix this? > > (3) why are there catalog backup jobs older than the default job retention > of 180d in my catalog, they should have been expired? > > (4) why are there 4 other job instances older than 180d? > > Bonus questions: > > (5) it seems that only jobs of types Backup, Copy Job and Copy (Migration > also? I dont use those) are pruned. Is this the correct behavior? > > (6) Is there a builtin mechanism to expire and prune Admin jobs and Verify > jobs? > > > > On 20. Oct 2024, at 12:21, Justin Case <jus7inc...@gmail.com> wrote: > > Hi Bill, > > please find my response inline below: > > On 19. Oct 2024, at 22:59, Bill Arlofski <w...@protonmail.com> wrote: > > On Wednesday, October 16th, 2024 at 06:15, Justin Case < > jus7inc...@gmail.com> wrote: > > I am wondering why I am seeing in my catalog lots of jobs older than the > JobRetention defined in the pools, and also older than the default > JobRetention assumed for the clients. > The volume recycling seems to work fine adhering to the VolumeRetention in > the pools. > > > To me it is a mystery, probably be cause I overlook some dependencies I am > not aware of. > Can someone please help me understanding this. > > > Hello Justin, > > From the rest of this thread, it appears that your prunining is working as > configured/expected. > > > I doubt that, at least no as intended. > > Where are you seeing the old jobs? bconsole? BWeb? Bacularis? > > > Bacularis. > > They could be coming from the `jobhisto` table. > > And you can prove this by querying the `job` and `jobhisto` tables and > compare the results. > > > job has all the jobs > jobhisto is empty > > Still sounds to me like job expiry not work as intended - maybe config is > wrong, but I don’t understand where the mistake is. You find the config > excerpt in my initial post. > > Can you help? > > All the best, > J/C > > > _______________________________________________ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users >
_______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
