Thank you so much Martin,
it all makes sense now! regarding (4), the old jobs are on volumes in state
Error, those do not have an expiry date.
What do you suggest how to proceeds with these? purge the old volumes in Error
state and then run a pruning, will that clear out the old jobs?
> On 21. Oct 2024, at 17:28, Martin Simmons <mar...@lispworks.com> wrote:
>
>>>>>> On Sun, 20 Oct 2024 13:15:16 +0200, Justin Case said:
>>
>> I have browsed the catalog database a bit more and I saw one thing that
>> might explain the problem I am having, but I don't have sufficient technical
>> background on the Bacula implementation details of different versions.
>>
>> I am running bacula-dir 15.0.2 and bacularis 3.1.0. The Bacula manual and
>> Bacularis offer “Job Retention” and “File Retention” in the pool resource,
>> supposedly overriding the same fields in the client resource. I am not using
>> these fields in the client resource, and in the client table there are the
>> job and file retention default values.
>> I am only using the job and file retention fields in the pool resources in
>> Bacularis.
>> What irritates me is that in the pool DB table there are no columns for job
>> and for file retention!
>> Are these values stored elsewhere or has my DB not been initially set up or
>> not correctly upgraded?
>>
>> If the pool resource retention values for jobs and files are missing in my
>> DB this would mean that bacula-dir is using the values from the client
>> resources, which is consistent with what I am seeing (with 2 exceptions):
>> The oldest jobs are 6 months old (job retention default value). Exceptions:
>> (a) catalog backup jobs go back to the date when the installation was set up
>> (b) two other jobs have altogether 4 instances that are older than 6 months.
>>
>> So open questions for me are:
>>
>> (1) where are job retention and file retention stored for the pool resource?
>
> They are read from the config file. You can confirm the values by using the
> "show pool" command in bconsole.
>
>> (2) if in the pool table, why does my pool table not have corresponding
>> columns for job and file retention? How may I fix this?
>
> There is nothing to fix. Even for the client, the value originates from the
> config file and is stored in the DB when bacula-dir starts.
>
>> (3) why are there catalog backup jobs older than the default job retention
>> of 180d in my catalog, they should have been expired?
>>
>> (4) why are there 4 other job instances older than 180d?
>
> I don't know. Can you show the output from "llist jobid=<nnn>" where <nnn> is
> the jobid of some of these jobs?
>
>> Bonus questions:
>>
>> (5) it seems that only jobs of types Backup, Copy Job and Copy (Migration
>> also? I dont use those) are pruned. Is this the correct behavior?
>
> Yes, that's what happens.
>
>> (6) Is there a builtin mechanism to expire and prune Admin jobs and Verify
>> jobs?
>
> Probably not usefully. The problem is that these job types don't have a
> meaningful "pool" or "client" in the job definition, so Bacula doesn't prune
> using the expected retention periods.
>
> I run an admin job that does this to remove them (and also jobs with no data):
>
> delete from job where (type in ('V', 'D') or (type = 'B' and jobbytes = 0 and
> jobfiles = 0)) and starttime < now()-interval '2 months';
>
> __Martin
>
>>
>>
>>
>>> On 20. Oct 2024, at 12:21, Justin Case <jus7inc...@gmail.com> wrote:
>>>
>>> Hi Bill,
>>>
>>> please find my response inline below:
>>>
>>>> On 19. Oct 2024, at 22:59, Bill Arlofski <w...@protonmail.com
>>>> <mailto:w...@protonmail.com><mailto:w...@protonmail.com
>>>> <mailto:w...@protonmail.com>>> wrote:
>>>>
>>>> On Wednesday, October 16th, 2024 at 06:15, Justin Case
>>>> <jus7inc...@gmail.com
>>>> <mailto:jus7inc...@gmail.com><mailto:jus7inc...@gmail.com
>>>> <mailto:jus7inc...@gmail.com>>> wrote:
>>>>
>>>>> I am wondering why I am seeing in my catalog lots of jobs older than the
>>>>> JobRetention defined in the pools, and also older than the default
>>>>> JobRetention assumed for the clients.
>>>>> The volume recycling seems to work fine adhering to the VolumeRetention
>>>>> in the pools.
>>>>>
>>>>
>>>>> To me it is a mystery, probably be cause I overlook some dependencies I
>>>>> am not aware of.
>>>>> Can someone please help me understanding this.
>>>>
>>>> Hello Justin,
>>>>
>>>> From the rest of this thread, it appears that your prunining is working as
>>>> configured/expected.
>>>
>>> I doubt that, at least no as intended.
>>>
>>>> Where are you seeing the old jobs? bconsole? BWeb? Bacularis?
>>>
>>> Bacularis.
>>>
>>>> They could be coming from the `jobhisto` table.
>>>>
>>>> And you can prove this by querying the `job` and `jobhisto` tables and
>>>> compare the results.
>>>
>>> job has all the jobs
>>> jobhisto is empty
>>>
>>> Still sounds to me like job expiry not work as intended - maybe config is
>>> wrong, but I don’t understand where the mistake is. You find the config
>>> excerpt in my initial post.
>>>
>>> Can you help?
>>>
>>> All the best,
>>> J/C
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
