[Bacula-users] newbie questions

[Justin Case]

Hello fellow users,

I am new to Bacula and currently having a hard time wrapping my head around all 
the concepts that Bacula offers to implement my backup strategy.

I intend to use Bacula Community Edition for my private machines, this is not 
for professional purposes, there is no customer involved. I am the admin and 
the customer at the same time.

So far I have only set up various client backup solutions (TimeMachine, 
HyperBackup, ghettoVCB), and Bacula is my first attempt to implement a 
centralized multi-tiered backup solution for a diverse set of clients.

I will try to explain, what I intend to do and where I am currently stuck, 
hoping someone would be interested to point me in the right direction and give 
their valuable advice.

I will only use disk storage and S3 buckets, I do not own any tape drives.

There are up to 10 FDs (Linux and macOS), one of the SDs runs on the same 
machine as the Director and the SD which is going to write to the disks and to 
the S3 bucket(s). The remote S3 bucket will also be implemented by myself using 
minIO.

The Linux FD running on the same host as the Dir and SD is going to have jobs 
for:
-onsite SMB shares for Synology servers in a DMZ, so that I do not need to run 
an FD in the DMZ which would need to reach into my LAN.
-diverse non-Bacula backups form diverse systems, such as ESXi ghettoVCB, macOS 
TimeMachine, onsite minIO S3 buckets used by Synology DSM HyperBackup

Each backup job is intended to create 3 redundant “copies” (not meaning the 
Copy concept of Bacula) for safety reasons:
- Tier 1 (fastest): 1 to n  internal SATA hard disk drives (ideally no manual 
intervention when individual disks 1,.., n-1 fill up)
- Tier 2 (medium speed): 1 to m  external USB hard disk drives (ideally no 
manual intervention when individual disks 1,.., n-1 fill up)
- Tier 3 (slowest): 1 or more offsite S3 buckets implemented using minIO Docker 
container on Synology DSM

For restore, tier 1 backups are preferred. If tier 1 volumes are corrupted, the 
redundant copies in tier 2 should be used. If the tier 2 copy also is corrupted 
(e.g. onsite fire or flood event), then tier 3 should be used.

Retention schemes would be nothing fancy, something along the line of
- Full once per month
- Differential once per week
- Incremental once per day

I am currently trying to understand how to configured storage devices and pools 
to implement storage tier 1 and tier 2.
I found examples in the documentation, so far all them only use 1 disk drive, 
assuming that all jobs fit on that drive.
As I have the luxury of more than 1 drive per tier, I would like to understand, 
if it is possible to define a pool on top of more than one disk device (e.g. 
using AutoChanger with disk devices instead of tape devices?), so that first 
one device is used and if it is full, further volumes are automatically written 
to the next disk device (without the need for manual intervention).

The other thing not yet clear to me is, what the best matching Bacula concepts 
are to write efficiently to all 3 tiers.
Is it possible to make a job write each file it is backing up to more than 1 
pool, so that it creates as many redundant copies? Or is that not possible, so 
that first tier 1 Backup jobs must be run, and then Copy jobs for Tier 2 and 
then Copy jobs for tier 3? Would my intended use as described above (prefer 
restore from tier 1, since it restores the fastest. Next priority is tier 2, 
last priority is tier 3) be possible on volume copies created by Copy jobs?

I would be grateful, if someone knows of a similar setup and can give me a 
pointer to publicly available example configs for that, or is willing to share 
(excerpts) from their own similar setup configs.

Also concrete improvements for or problems you see with my strategy are welcome 
feedback!

Thanks for you consideration and your time!
 JC











___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] newbie questions

[Justin Case]

Hello Heitor,

thanks for your quick response - I have seen your videos on YT!

> On 13. Mar 2022, at 23:44, Heitor Faria  wrote:
> 
>> Hello fellow users,
> 
> Hello Justin,
> 
>> I will only use disk storage and S3 buckets, I do not own any tape drives.
>> 
>> There are up to 10 FDs (Linux and macOS), one of the SDs runs on the same
>> machine as the Director and the SD which is going to write to the disks and 
>> to
>> the S3 bucket(s). The remote S3 bucket will also be implemented by myself 
>> using
>> minIO.
> 
> Bacula can work with minIO, but you don't need. Bacula S3 Driver can write to 
> any S3 compatible cloud.

Certainly. However, I do want my data under my own authority - only.

> 
>> The Linux FD running on the same host as the Dir and SD is going to have jobs
>> for:
>> -onsite SMB shares for Synology servers in a DMZ, so that I do not need to 
>> run
>> an FD in the DMZ which would need to reach into my LAN.
>> -diverse non-Bacula backups form diverse systems, such as ESXi ghettoVCB, 
>> macOS
>> TimeMachine, onsite minIO S3 buckets used by Synology DSM HyperBackup
> 
> GhetoVCB is a very well-intentioned, but extremely poor solution to perform 
> VSphere VM backups in terms of performance and features. If you want a 100% 
> open source solution, I would migrate to Proxmox.

Sure, I know about the Proxmox backup features. Still, for the next time I will 
run on ESXi, and this need a solution for that platform.

>> Each backup job is intended to create 3 redundant “copies” (not meaning the 
>> Copy
>> concept of Bacula) for safety reasons:
>> - Tier 1 (fastest): 1 to n  internal SATA hard disk drives (ideally no manual
>> intervention when individual disks 1,.., n-1 fill up)
>> - Tier 2 (medium speed): 1 to m  external USB hard disk drives (ideally no
>> manual intervention when individual disks 1,.., n-1 fill up)
>> - Tier 3 (slowest): 1 or more offsite S3 buckets implemented using minIO 
>> Docker
>> container on Synology DSM
> 
> I would use one Cloud Storage with the CacheRetention directive for tiers 1 
> and 3. External USB (ew!) can be a Bacula Copy Job or a completely different 
> redundant Job with its own Full and partials chain (e.g. Job1-cloud; 
> Job1-usb).

Let’s see if I understand that correctly. What you are saying is that I should 
treat tier 1 as the cache for tier 3 and set the retention periods properly. 
Right?

Yes, USB is terrible, yet a cheap solution for me, as I have some seriously 
sized HDDs laying around which were retired from some arrays due to size 
reasons, not for error reasons. I have put them in external cases with USB 3.1 
Gen2 controllers and connected them using a USB 3.0 hub. As long as the jobs do 
not write to several HDDs simultaneously, performance wouldn’t be too bad.

What are the drawbacks of a Copy job as compared to a independent Backup job? 
If I understand correctly that independent Backup jobs would tie up client and 
network resources, and Copy jobs do not. What’s the catch with the Copy jpb 
results to be used as tier 2?

>> As I have the luxury of more than 1 drive per tier, I would like to 
>> understand,
>> if it is possible to define a pool on top of more than one disk device (e.g.
>> using AutoChanger with disk devices instead of tape devices?), so that first
>> one device is used and if it is full, further volumes are automatically 
>> written
>> to the next disk device (without the need for manual intervention).
> 
> As for disks, I would just use LVM or any sort of RAID to present Bacula. 
> More practical.

Brilliant, will do that! (Sometimes I get drawn into a rabbit hole of 
complexity while overlooking an easier and better solution.)

Thank you for your time and support!
 JC



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] newbie questions

[Justin Case]

Coming back to this point:

> On 13. Mar 2022, at 23:44, Heitor Faria  wrote:
> 
>> As I have the luxury of more than 1 drive per tier, I would like to 
>> understand,
>> if it is possible to define a pool on top of more than one disk device (e.g.
>> using AutoChanger with disk devices instead of tape devices?), so that first
>> one device is used and if it is full, further volumes are automatically 
>> written
>> to the next disk device (without the need for manual intervention).
> 
> As for disks, I would just use LVM or any sort of RAID to present Bacula. 
> More practical.

for tier 1, internal disks, absolutely yes.

for tier 2, external USB disks, I would have to acquire a hardware RAID multi 
disk case, which in itself is not a problem, but disk-by-disk expansion would 
be painful. For tier 2 I would rather like to work with individually adressable 
external disks. Is it possible to tweak Bacula into using an autochanger to 
fill those disks one by one while the filled disks stay attached to the USB hub?


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Baculum - how to edit JobDefs

[Justin Case]

Dear all,

I am looking for a better solution for the following problem:

The JobDefs for DefaultJob is buried in really many resources, but it cannot be 
edited in Baculum.

I had to edit JobDefs in the bacula-dir.conf to be able to get rid of the 
default resources (schedules, jobs, pools, storage) that came with the 
container I am using.

Is there a way to edit JobDefs in Baculum?

All the best,

 JC

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Baculum - how to edit JobDefs

[Justin Case]

Marcin, thank you, I actually found later on where to edit it in Baculum.

> On 16. Mar 2022, at 07:22, Marcin Haba  wrote:
> 
> Hello Justin,
> 
> Could you tell us what is the problem with JobDefs? I am asking about it 
> because I am not sure if you don't know where to edit JobDefs or you receive 
> an error when you save JobDefs. If it is an error, could you paste the output 
> here?
> 
> Thanks in advance for additional information.
> 
> Best regards,
> Marcin Haba (gani)
> 
> On Tue, 15 Mar 2022 at 21:12, Justin Case  <mailto:jus7inc...@gmail.com>> wrote:
> Dear all,
> 
> I am looking for a better solution for the following problem:
> 
> The JobDefs for DefaultJob is buried in really many resources, but it cannot 
> be edited in Baculum.
> 
> I had to edit JobDefs in the bacula-dir.conf to be able to get rid of the 
> default resources (schedules, jobs, pools, storage) that came with the 
> container I am using.
> 
> Is there a way to edit JobDefs in Baculum?
> 
> All the best,
> 
>  JC
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net <mailto:Bacula-users@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/bacula-users 
> <https://lists.sourceforge.net/lists/listinfo/bacula-users>
> 
> 
> -- 
> "Greater love hath no man than this, that a man lay down his life for his 
> friends." Jesus Christ
> 
> "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za 
> przyjaciół swoich." Jezus Chrystus

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: about default volume numbering in pools

[Justin Case]

Dear all,

I have a number of pools for different clients/use cases, and each pool has its 
own name prefix for disk volume labeling.
When I look into the folder where all he pool files are created I see the 
different pool name prefixes, and the volume numbers are uniquely incremented 
-across- all the pools on that disk device. Is that the normal behavior?

Naively I was assuming the autolabeling would start with 0001 for each 
different pool name prefix, however, it does not seem to work like that.

All the best,
 JC



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Newbie problem: Catalog backup job not executing RunScript?

[Justin Case]

Dear all,

I can’t get my Catalog backup job to work properly. The documentation seems to 
be written before  the Perl script for making the sql db copy was introduced 
(it refers to a shell script). 
When I run the job’s RunScript commands in a shell they actually work, so the 
file /opt/bacula/working/bacula.sql is created and deleted.
I guess I misunderstand how those runscripts work? 
I don’t see where my mistake is  (but I am sure I made one).

The log does not really show whether the RunScript directives were executed:


bacula-dir JobId 31: End auto prune.
bacula-dir JobId 31: No Files found to prune.
bacula-dir JobId 31: Begin pruning Files.
bacula-dir JobId 31: No Jobs found to prune.
bacula-dir JobId 31: Begin pruning Jobs older than 6 months .
bacula-dir JobId 31: Bacula bacula-dir 11.0.5 (03Jun21):
  Build OS:   x86_64-pc-linux-gnu debian 11.1
  JobId:  31
  Job:
CatalogBackup
.2022-03-16_21.50.59_11
  Backup Level:   Full
  Client: "
bacula-fd
" 11.0.5 (03Jun21) x86_64-pc-linux-gnu,debian,11.1
  FileSet:"
Catalog
" 2022-03-08 22:48:58
  Pool:   "
unraid-catalog-full
" (From Command input)
  Catalog:"MyCatalog" (From Pool resource)
  Storage:"
unraid-tier1-storage
" (From Pool resource)
  Scheduled time: 16-Mar-2022 21:50:59
  Start time: 16-Mar-2022 21:51:01
  End time:   16-Mar-2022 21:51:02
  Elapsed time:   1 sec
  Priority:   10
  FD Files Written:   0
  SD Files Written:   0
  FD Bytes Written:   0 (0 B)
  SD Bytes Written:   0 (0 B)
  Rate:   0.0 KB/s
  Software Compression:   None
  Comm Line Compression:  None
  Snapshot/VSS:   no
  Encryption: no
  Accurate:   no
  Volume name(s): 
  Volume Session Id:  1
  Volume Session Time:1647463790
  Last Volume Bytes:  705 (705 B)
  Non-fatal FD errors:1
  SD Errors:  0
  FD termination status:  OK
  SD termination status:  OK
  Termination:Backup OK -- with warnings
bacula-sd JobId 31: Sending spooled attrs to the Director. Despooling 0 bytes 
...
bacula-sd JobId 31: Elapsed time=00:00:01, Transfer rate=0  Bytes/second

Here is the problem:

bacula-fd JobId 31:  Could not stat "/opt/bacula/working/bacula.sql": 
ERR=No such file or directory


bacula-dir JobId 31: Max Volume jobs=1 exceeded. Marking Volume 
"unraid-catalog-full-vol-0042" as Used.
bacula-sd JobId 31: Wrote label to prelabeled Volume 
"unraid-catalog-full-vol-0042" on File device "Tier1-FileChgr1-Dev1" 
(/mnt/tier1-01)
bacula-sd JobId 31: Labeled new Volume "unraid-catalog-full-vol-0042" on File 
device "Tier1-FileChgr1-Dev1" (/mnt/tier1-01).
bacula-dir JobId 31: Using Device "Tier1-FileChgr1-Dev1" to write.
bacula-dir JobId 31: Created new Volume="unraid-catalog-full-vol-0042", 
Pool="unraid-catalog-full", MediaType="tier1-int-user0-folder1-file" in catalog.
bacula-dir JobId 31: Start Backup JobId 31, 
Job=CatalogBackup.2022-03-16_21.50.59_11



Job {
  Name = "CatalogBackup"
  Description = "Backup MyCatalog"
  Type = "Backup"
  Level = "Full"
  Messages = "Standard"
  Storage = "unraid-tier1-storage"
  Pool = "unraid-catalog-full"
  Fileset = "Catalog"
  Schedule = "EveryNight"
  JobDefs = "DefaultJob"
  Runscript {
RunsWhen = "Before"
RunsOnSuccess = no
RunsOnClient = no
Command = "/opt/bacula/scripts/make_catalog_backup.pl MyCatalog"
  }
  Runscript {
RunsWhen = "After"
FailJobOnError = no
RunsOnSuccess = no
RunsOnClient = no
Command = "/opt/bacula/scripts/delete_catalog_backup"
  }
  Runscript {
RunsWhen = "After"
FailJobOnError = no
RunsOnSuccess = no
RunsOnClient = no
Console = "purge volume action=all allpools storage=unraid-tier1-storage"
  }
  Priority = 11
  AllowIncompleteJobs = no
  AllowDuplicateJobs = no
}

Schedule {
  Name = "EveryNight"
  Run = Level="Full" at 20:00
}

Fileset {
  Name = "Catalog"
  Include {
File = "/opt/bacula/working/bacula.sql"
Options {
  Signature = "Md5"
}
  }
}

Thanks for any help.
 JC





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Does this Pruning job work for all volumes/pools that I have?

[Justin Case]

Dear all

as I do not yet understand the whole picture fully, let me show you what I 
currently have as a pruning job.
Will the job as defined below actually prune all volumes for all pools that I 
have? If possibly not, what would I need to do? Would I need a prune job for 
each pool?

Thank you for your time,
 JC

Job {
  Name = "Pruning"
  Description = "Prune all expired volumes"
  Type = "Admin"
  Schedule = "EveryNight"
  JobDefs = "DefaultJob"
  PruneJobs = yes
  PruneFiles = yes
  PruneVolumes = yes
  Runscript {
RunsWhen = "Before"
RunsOnClient = no
Console = "prune expired volume yes"
  }
  Priority = 12
  AllowDuplicateJobs = no
}
JobDefs {
  Name = "DefaultJob"
  Type = "Backup"
  Level = "Incremental"
  Messages = "Standard"
  Storage = "unraid-tier1-storage"
  Pool = "Default"
  Client = "bacula-fd"
  Fileset = "EmptyFileset"
  WriteBootstrap = "/opt/bacula/working/%c.bsr"
  MaxFullInterval = 2678400
  SpoolAttributes = yes
  Priority = 10
  AllowIncompleteJobs = no
  AllowDuplicateJobs = no
}
Fileset {
  Name = "EmptyFileset"
  EnableVss = no
}
bacula-dir JobId 32: End auto prune.
bacula-dir JobId 32: No Files found to prune.
bacula-dir JobId 32: Begin pruning Files.
bacula-dir JobId 32: No Jobs found to prune.
bacula-dir JobId 32: Begin pruning Jobs older than 6 months .
bacula-dir JobId 32: Bacula 11.0.5 (03Jun21): 16-Mar-2022 21:58:33
  JobId:  32
  Job:Pruning 
.2022-03-16_21.58.31_58
  Scheduled time: 16-Mar-2022 21:58:31
  Start time: 16-Mar-2022 21:58:33
  End time:   16-Mar-2022 21:58:33
  Termination:Admin OK
bacula-dir JobId 32: Start Admin JobId 32, Job=Pruning.2022-03-16_21.58.31_58
bacula-dir JobId 32: console command: run BeforeJob "prune expired volume yes"

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Newbie problem: Catalog backup job not executing RunScript?

[Justin Case]

Thank you Bill, your help was on spot - it works.

> On 17. Mar 2022, at 06:18, Bill Arlofski via Bacula-users 
>  wrote:
> 
> On 3/16/22 14:55, Justin Case wrote:
> 
> Hello Justin,
> 
> I suspected it could be possible that the "RunsOnSuccess = no" - which makes 
> no sense in a "Before" RunScript - might be
> causing this issue:
> 
>>   Runscript {
>> RunsWhen = "Before"
>> RunsOnSuccess = no <--- Here
>> RunsOnClient = no
>> Command = "/opt/bacula/scripts/make_catalog_backup.pl MyCatalog"
>>   }
> 
> I just tested in my 11.0.5 environment, and this is indeed your problem!
> 
> So, I would consider this to be a bug, albeit one that is quite the corner 
> case. :)
> 
> Just remove that line, reload your config and you should be all set.
> 
> 
> Best regards,
> Bill
> 
> --
> Bill Arlofski
> w...@protonmail.com
> 
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] How to remove all volumes?

[Justin Case]

Dear all,

again another newbie question:

I need to use a different disk drive for Bacula and I would like to start over 
in the sense of deleting all volume files in a clean way, i.e. not corrupting 
the catalog. The goal is to free up the space occupied by all Bacula volumes.

I know that I could migrate all volumes, but that takes more time than starting 
over. Losing all backups of the past few days after setting up Bacula is OK for 
me.

I checked the manual what else I could do and it seems that one way would be to 
set each volume manually to purged, but that would not delete the volume but it 
would just be recycled/re-used. That would basically not help me to free the 
space on the disk currently used by Bacula. Also it would be a manual effort as 
I already have >150 volumes.

All the best,
 JC





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] How to remove all volumes?

[Justin Case]

Thank you Heitor,

if I understand those directives correctly, it will only truncate expired 
volumes. My volumes are by far not yet expired, still I want to truncate them. 
Is that possible?

Best,
 JC

> On 19. Mar 2022, at 16:30, Heitor Faria  wrote:
> 
>> Dear all,
> 
> Hello Justin,
> 
>> I need to use a different disk drive for Bacula and I would like to start 
>> over
>> in the sense of deleting all volume files in a clean way, i.e. not corrupting
>> the catalog. The goal is to free up the space occupied by all Bacula volumes.
> 
> Normally one user does not really need to free the volume space until the 
> VolumeRetention is expired and the Volume is demanded by a new backup, but I 
> understand some users might want to have better control of the current real 
> necessary backup disk occupation to fulfill configured retentions.
> Please refer to my small post: 
> https://www.bacula.lat/truncate-bacula-volumes-to-free-disk-space/?lang=en
> 
>> All the best,
> 
> Regards,
> 
>> JC
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
> -- 
> MSc Heitor Faria (Miami/USA) 
> Bacula LATAM CIO 
> 
> mobile1: + 1 909 655-8971 
> mobile2: + 55 61 98268-4220 
> [ https://www.linkedin.com/in/msc-heitor-faria-5ba51b3 ] 
>   [ http://www.bacula.com.br/ ] 
> 
> América Latina 
> [ http://bacula.lat/ | bacula.lat ] | [ http://www.bacula.com.br/ | 
> bacula.com.br ]
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] How to remove all volumes?

[Justin Case]

Or asked differently: Is it possible to declare all volumes as expired without 
having to execute a command for each existing volume individually? (something 
like “set all volumes to expired”)?

> On 19. Mar 2022, at 20:30, Justin Case  wrote:
> 
> Thank you Heitor,
> 
> if I understand those directives correctly, it will only truncate expired 
> volumes. My volumes are by far not yet expired, still I want to truncate 
> them. Is that possible?
> 
> Best,
> JC
> 
>> On 19. Mar 2022, at 16:30, Heitor Faria  wrote:
>> 
>>> Dear all,
>> 
>> Hello Justin,
>> 
>>> I need to use a different disk drive for Bacula and I would like to start 
>>> over
>>> in the sense of deleting all volume files in a clean way, i.e. not 
>>> corrupting
>>> the catalog. The goal is to free up the space occupied by all Bacula 
>>> volumes.
>> 
>> Normally one user does not really need to free the volume space until the 
>> VolumeRetention is expired and the Volume is demanded by a new backup, but I 
>> understand some users might want to have better control of the current real 
>> necessary backup disk occupation to fulfill configured retentions.
>> Please refer to my small post: 
>> https://www.bacula.lat/truncate-bacula-volumes-to-free-disk-space/?lang=en
>> 
>>> All the best,
>> 
>> Regards,
>> 
>>> JC
>>> ___
>>> Bacula-users mailing list
>>> Bacula-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>> 
>> -- 
>> MSc Heitor Faria (Miami/USA) 
>> Bacula LATAM CIO 
>> 
>> mobile1: + 1 909 655-8971 
>> mobile2: + 55 61 98268-4220 
>> [ https://www.linkedin.com/in/msc-heitor-faria-5ba51b3 ] 
>>  [ http://www.bacula.com.br/ ] 
>> 
>> América Latina 
>> [ http://bacula.lat/ | bacula.lat ] | [ http://www.bacula.com.br/ | 
>> bacula.com.br ]
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] IRC channel on freenode

[Justin Case]

Hello all,

the #bacula channel on freenode feels rather deserted. 

Is there a reason why the channel has not been migrated to Libera? Maybe there 
would be more fellows joining?

Just my 2 cents,
 JC

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] IRC channel on freenode

[Justin Case]

Awesome, to what IRC network is the English Telegram group bridged and what is 
the IRC channel name for it?

> On 19. Mar 2022, at 21:27, Heitor Faria  wrote:
> 
> We have a 60 ppl English group in Tg: https://t.me/baculabackup
> 30 in Spanish: https://t.me/baculaes
> 1300 in Portuguese: https://t.me/baculabr
> 
> Rgds 
> --
> MSc Heitor Faria (Miami/USA)
> CIO Bacula LatAm
> mobile1: + 1 909 655-8971
> mobile2: + 55 61 98268-4220
> [ http://bacula.lat/]
> 

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] IRC channel on freenode

[Justin Case]

What a pity, I don’t do Telegram by personal decision.
Thank you for the info.

> On 19. Mar 2022, at 21:42, Heitor Faria  wrote:
> 
> Awesome, to what IRC network is the English Telegram group bridged and what 
> is the IRC channel name for it?
> We used to have a bridge in the past to the main group. I belive Dan Langille 
> was the most active admin. But the IRC group was kind of dead like you also 
> observed, and I think we just gave up maitaining the bridge.
> 
> Regards,
> -- 
> 
> MSc Heitor Faria (Miami/USA)
> Bacula LATAM CIO
> mobile1: + 1 909 655-8971
> mobile2: + 55 61 98268-4220
>     
>  
> América Latina
> bacula.lat  | bacula.com.br 
> 

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] How to remove all volumes?

[Justin Case]

I think I am not good enough on bconsole yet, I tried different things but I am 
getting nowhere.

Things I tried in bconsole:

(a) "purge action=truncate", then selected "2" for jobs
  it tells me it purged a number of jobs, but no volume got truncated

(b) "purge volume action=truncate allpools storage=unraid-tier1-storage”
  I get: No Volumes found to perform the command.

(c) truncate, then selected the storage and a pool,
 I get: No Volumes found to perform the command.

(d) truncate pool=unraid-minio-full storage=unraid-tier1-storage
 I get: No Volumes found to perform the command.

Something that would not work for me is if I had to manually enumerate each 
volume to be purged and truncated (as there are just too many).

Could someone please help me out giving more detailed directions which 
command(s) to run in bconsole with what parameters?

> On 19. Mar 2022, at 21:30, Heitor Faria  wrote:
> 
> Hello Justin,
> 
>> Or asked differently: Is it possible to declare all volumes as expired 
>> without
>> having to execute a command for each existing volume individually? (something
>> like “set all volumes to expired”)?
> 
> You can use the purge command to recycle the desired volumes prematurely. 
> 
> Rgds.
> -- 
> MSc Heitor Faria (Miami/USA) 
> Bacula LATAM CIO 
> 
> mobile1: + 1 909 655-8971 
> mobile2: + 55 61 98268-4220 
> [ https://www.linkedin.com/in/msc-heitor-faria-5ba51b3 ] 
>   [ http://www.bacula.com.br/ ] 
> 
> América Latina 
> [ http://bacula.lat/ | bacula.lat ] | [ http://www.bacula.com.br/ | 
> bacula.com.br ]
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] How to remove all volumes?

[Justin Case]

I read up the console manual on the purge command and also looked at help purge.
The tried:
(e) purge action=truncate allpools storage=unraid-tier1-storage
  when asked for a pool I selected a small one that is the on disk and I 
get a list of 2 volumes. When entering a media id I get this:
  Enter a Volume name or *MediaId: 46 
  sql_get.c:1337 Media record for Volume name "46" not found.

Sounds kinda wrong to me, as it was listing this volume right beforehand.

What am I doing wrong?

> On 19. Mar 2022, at 22:20, Justin Case  wrote:
> 
> I think I am not good enough on bconsole yet, I tried different things but I 
> am getting nowhere.
> 
> Things I tried in bconsole:
> 
> (a) "purge action=truncate", then selected "2" for jobs
>  it tells me it purged a number of jobs, but no volume got truncated
> 
> (b) "purge volume action=truncate allpools storage=unraid-tier1-storage”
>  I get: No Volumes found to perform the command.
> 
> (c) truncate, then selected the storage and a pool,
> I get: No Volumes found to perform the command.
> 
> (d) truncate pool=unraid-minio-full storage=unraid-tier1-storage
> I get: No Volumes found to perform the command.
> 
> Something that would not work for me is if I had to manually enumerate each 
> volume to be purged and truncated (as there are just too many).
> 
> Could someone please help me out giving more detailed directions which 
> command(s) to run in bconsole with what parameters?
> 
>> On 19. Mar 2022, at 21:30, Heitor Faria  wrote:
>> 
>> Hello Justin,
>> 
>>> Or asked differently: Is it possible to declare all volumes as expired 
>>> without
>>> having to execute a command for each existing volume individually? 
>>> (something
>>> like “set all volumes to expired”)?
>> 
>> You can use the purge command to recycle the desired volumes prematurely. 
>> 
>> Rgds.
>> -- 
>> MSc Heitor Faria (Miami/USA) 
>> Bacula LATAM CIO 
>> 
>> mobile1: + 1 909 655-8971 
>> mobile2: + 55 61 98268-4220 
>> [ https://www.linkedin.com/in/msc-heitor-faria-5ba51b3 ] 
>>  [ http://www.bacula.com.br/ ] 
>> 
>> América Latina 
>> [ http://bacula.lat/ | bacula.lat ] | [ http://www.bacula.com.br/ | 
>> bacula.com.br ]
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] How to remove all volumes?

[Justin Case]

in bconsole: list volume (or list media) display that the volumes are still 
marked as Used, not as Purged. So I guess none of the commands I tried (see 
below) did achieve that the volumes were marked as Purged.

What is the right way to do that (without manually iterating through all volume 
names or IDs)?

> On 19. Mar 2022, at 22:20, Justin Case  wrote:
> 
> I think I am not good enough on bconsole yet, I tried different things but I 
> am getting nowhere.
> 
> Things I tried in bconsole:
> 
> (a) "purge action=truncate", then selected "2" for jobs
>  it tells me it purged a number of jobs, but no volume got truncated
> 
> (b) "purge volume action=truncate allpools storage=unraid-tier1-storage”
>  I get: No Volumes found to perform the command.
> 
> (c) truncate, then selected the storage and a pool,
> I get: No Volumes found to perform the command.
> 
> (d) truncate pool=unraid-minio-full storage=unraid-tier1-storage
> I get: No Volumes found to perform the command.
> 
> Something that would not work for me is if I had to manually enumerate each 
> volume to be purged and truncated (as there are just too many).
> 
> Could someone please help me out giving more detailed directions which 
> command(s) to run in bconsole with what parameters?
> 
>> On 19. Mar 2022, at 21:30, Heitor Faria  wrote:
>> 
>> Hello Justin,
>> 
>>> Or asked differently: Is it possible to declare all volumes as expired 
>>> without
>>> having to execute a command for each existing volume individually? 
>>> (something
>>> like “set all volumes to expired”)?
>> 
>> You can use the purge command to recycle the desired volumes prematurely. 
>> 
>> Rgds.
>> -- 
>> MSc Heitor Faria (Miami/USA) 
>> Bacula LATAM CIO 
>> 
>> mobile1: + 1 909 655-8971 
>> mobile2: + 55 61 98268-4220 
>> [ https://www.linkedin.com/in/msc-heitor-faria-5ba51b3 ] 
>>  [ http://www.bacula.com.br/ ] 
>> 
>> América Latina 
>> [ http://bacula.lat/ | bacula.lat ] | [ http://www.bacula.com.br/ | 
>> bacula.com.br ]
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] IRC channel on freenode

[Justin Case]

As expressed, it is a personal decision.
I want to limit the amount of chat apps I install and use.
The evaluation and decision was made long before the recent conflict in Ukraine 
and is mainly based on security and benefits considerations.
Technical security is not about my social security number (in fact no system 
can protect your information from being ready by any 3rd party, if you put it 
into any publicly accessible group, no matter how technically secure it might 
be), but that`s a different can of worms and I guess off topic for this mailing 
list.


> On 20. Mar 2022, at 02:25, Heitor Faria  wrote:
> 
> What a pity, I don’t do Telegram by personal decision.
> If war-wise: UFC didn't ban Volkov that fought today in London under the 
> Russian flag. ATP, Miami Open, Indian-Wells didn't ban Medvedev. It is not 
> like you are financing Sukhois or whatever. I'm sure Ukraine could survive 
> with or without you. 
> If security wise: it is a technical group. It is not like you are going to 
> enter your social security number (if American) or any sensitive information.
> 
> Regards,
> -- 
> 
> MSc Heitor Faria (Miami/USA)
> Bacula LATAM CIO
> mobile1: + 1 909 655-8971
> mobile2: + 55 61 98268-4220
>     
>  
> América Latina
> bacula.lat  | bacula.com.br 
> 

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] IRC channel on freenode

[Justin Case]

Well there are many other parameters not mentioned yet and based on my 
individual risk assessment I ranked the solutions and made a cut-off.
Something that is security-wise absolutely terrible is Discord, yet it carries 
so many OSS support groups that the benefit was worth the security risks.

If you want suggestions or opinions: I would suggest Matrix.org for the Bacula 
support groups (best client so far: Element.io ). Security 
is very configurable, and one may participate without any personal information 
(e.g. just a fake email address) and one may use an own matrix home server or 
go via a VPN/Proxy/Tor to connect. Since there are no central servers needed it 
is way harder to control by governments (as you know: some countries like 
Russia do require commercial chat services to provide government access 
interfaces). The UX is similar to other modern chats like Discord, Mattermost, 
Slack. Many OSS support groups are already on Matrix.

Just my 2 cents

> On 20. Mar 2022, at 11:14, Heitor Faria  wrote:
> 
> Hello Justin,
> As expressed, it is a personal decision.
> I want to limit the amount of chat apps I install and use.
> Technical security is not about my social security number (in fact no system 
> can protect your information from being ready by any 3rd party, if you put it 
> into any publicly accessible group, no matter how technically secure it might 
> be),
> I respect your opinion, but our mobile phones already have enough bloatware 
> for the FANGs to snoop information and to play Pavlov's Dogs experiments with 
> us. I can't idulge myself in this way because I gotta be where my consumer 
> is. 
> Nevertheless: "there are cases where Signal can be less private and less safe 
> [than Telegram] depending on your definition – for example Signal requires 
> use a phone number for connection with other users, which can have 
> significant physical safety risks for many people who are targeted by 
> stalkers. Telegram for example allows the use of a username instead which can 
> be significantly safer for many people."
> but that`s a different can of worms and I guess off topic for this mailing 
> list.
> I don't think this is off-topic at all, since we are discussing other ways of 
> user's community communication. And frankly speaking: email is extremely 
> dated and prone to unreliability.
> 
> Regards,
> -- 
> 
> MSc Heitor Faria (Miami/USA)
> Bacula LATAM CIO
> mobile1: + 1 909 655-8971
> mobile2: + 55 61 98268-4220
>     
>  
> América Latina
> bacula.lat  | bacula.com.br 
> 

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] How to remove all volumes?

[Justin Case]

Thank you Chris and Martin!

What I then did was use the Baculum Volumes view, filter for all volumes that 
are not purged and set them to Purged. It is easier in Baculum than I initially 
thought.
Then I used a job to run: truncate volume allpools storage=mystorage 

That was easy enough for me, but it is great to have Chris’es scripts now!

All the best
JC


> On 21. Mar 2022, at 16:48, Chris Wilkinson  wrote:
> 
> I wrote a couple of scripts to help with this problem. Start by deleting all 
> the jobs that you don’t want anymore. This should mark all the associated 
> volumes as ‘purged’. That can be done in bconsole or Baculum.
> 
> This one deletes any ‘purged’ volume records from the catalog.
> 
> #!/bin/bash
> #Usage: sudo ./delete-purged-volumes.sh any-char
> #Delete purged Bacula volumes. Do not delete if arg is empty
> if [[ -z $1 ]]; then
>   echo "Not deleting"
> fi
> for vol in $(echo "list volume" | bconsole | grep Purged | awk '{print $4}')
> do
>   if [[ ! -z $1 ]]; then
> echo "delete yes volume=$vol" | bconsole > /dev/null
> echo "Volume $vol deleted"
>   else
> echo "Volume $vol not deleted"
>   fi
> done
> 
> This one deletes any volumes from the filesystem that are not now in the 
> catalog. These are files in the filesystem that have no entry in the catalog.
> 
> #!/bin/bash
> #Usage: sudo -u bacula ./delete-orphened-volumes.sh target-dir any-char
> #Delete orphaned Bacula volumes. Do not delete if second arg is empty
> if [[ -z $1 ]]; then
>   echo "Please supply target directory"
>   exit
> fi
> if [[ ! -d $1 ]]; then
>   echo "Target does not exist"
>   exit
> fi
> if [[ -z $2 ]]; then
>   echo "Not deleting"
> fi
> cd $1  # change to target directory
> for vol in $(find . -maxdepth 1 -type f -printf '%f\n')
> do
>   echo "list volume=$vol" | bconsole | if grep --quiet "No results to list"; 
> then
> if [[ ! -z $2 ]]; then
>   rm $1$vol
>   echo "Orphaned file $1$vol deleted"
> else
>   echo "Orphaned file $1$vol not deleted"
> fi
>   fi
> done
> 
> Best
> -Chris-
> 
> 
> 
> 
>> On 21 Mar 2022, at 12:42, Martin Simmons > <mailto:mar...@lispworks.com>> wrote:
>> 
>> You can use the "delete volume" command in bconsole to remove a volume from
>> the catalog.  After that, you can safely rm it from the filesystem.
>> 
>> __Martin
>> 
>> 
>>>>>>> On Sat, 19 Mar 2022 13:08:41 +0100, Justin Case said:
>>> 
>>> Dear all,
>>> 
>>> again another newbie question:
>>> 
>>> I need to use a different disk drive for Bacula and I would like to start 
>>> over in the sense of deleting all volume files in a clean way, i.e. not 
>>> corrupting the catalog. The goal is to free up the space occupied by all 
>>> Bacula volumes.
>>> 
>>> I know that I could migrate all volumes, but that takes more time than 
>>> starting over. Losing all backups of the past few days after setting up 
>>> Bacula is OK for me.
>>> 
>>> I checked the manual what else I could do and it seems that one way would 
>>> be to set each volume manually to purged, but that would not delete the 
>>> volume but it would just be recycled/re-used. That would basically not help 
>>> me to free the space on the disk currently used by Bacula. Also it would be 
>>> a manual effort as I already have >150 volumes.
>>> 
>>> All the best,
>>> JC
>>> 
>>> 
>>> 
>>> 
>>> 
>>> ___
>>> Bacula-users mailing list
>>> Bacula-users@lists.sourceforge.net 
>>> <mailto:Bacula-users@lists.sourceforge.net>
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>> 
>> 
>> 
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net 
>> <mailto:Bacula-users@lists.sourceforge.net>
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: Correct FD setup for macOS 10.15

[Justin Case]

Dear all,

I just made a test installation of bacula-fd on macOS 10.15 using homebrew (so 
I did not need to compile anything).
I configured the bacula-fd.conf and set up a fileset for “/Users” to backup all 
user data. I enabled the option HfsPlusSupport, but the machine actually does 
use APFS, is that a problem?
I also set up corresponding pools and scheduled jobs for the storage tiers.

Now running the job for storage tier 1 I can see that the FD (running as root) 
on the macOS machine cannot access a lot of directories.

Some examples:

22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
"/Users/moses/Pictures/Photos Library.photoslibrary": ERR=Operation not 
permitted
22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
"/Users/moses/Desktop": ERR=Operation not permitted
22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
"/Users/moses/Library/Application Support/CallHistoryTransactions": 
ERR=Operation not permitted
22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
"/Users/moses/Library/Application Support/com.apple.sharedfilelist": 
ERR=Operation not permitted
22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
"/Users/moses/Library/Application Support/com.apple.TCC": ERR=Operation not 
permitted
22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
"/Users/moses/Library/Application Support/FileProvider": ERR=Operation not 
permitted
22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
"/Users/moses/Library/Application Support/AddressBook": ERR=Operation not 
permitted
22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
"/Users/moses/Library/Application Support/CallHistoryDB": ERR=Operation not 
permitted
22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
"/Users/moses/Library/Autosave Information": ERR=Operation not permitted
22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
"/Users/moses/Library/IdentityServices": ERR=Operation not permitted
etc.
Does someone have a proper setup for the macOS FD and the filesets that runs 
fine and also restore will work as expected?

This is my current bacula-fd.conf on macOS:

Director {
  Name = bacula-dir
  Password = “snip"
}

Director {
  Name = bacula-mon
  Password = “snip"
  Monitor = yes
}

FileDaemon {  # this is me
  Name = macmachine-fd
  FDport = 9102  # where we listen for the director
  WorkingDirectory = /usr/local/var/lib/bacula
  Pid Directory = /usr/local/var/run
  Maximum Concurrent Jobs = 20
  Plugin Directory = /usr/local/Cellar/bacula-fd/11.0.6/lib
}

Messages {
  Name = Standard
  director = bacula-dir = all, !skipped, !restored, !saved



Here are the relevant directives from bacula-dir.conf:

Client {
  Name = "macmachine-fd"
  Address = "mac.lan.net"
  FdPort = 9102
 Password = “snip"
  Catalog = "MyCatalog"
  FileRetention = 5184000
  JobRetention = 15552000
  AutoPrune = yes
}

Job {
  Name = "mac-users-tier1"
  Type = "Backup"
  Level = "Full"
  Messages = "Standard"
  Storage = "unraid-tier1-storage"
  Pool = "mac-users-full"
  FullBackupPool = "mac-users-full"
  IncrementalBackupPool = "mac-users-incr"
 Client = “macmachine-fd"
 Fileset = “m-uacsers"
  Schedule = "First-Fri-Full"
  JobDefs = "DefaultJob"
  Enabled = yes
  AllowIncompleteJobs = no
  AllowDuplicateJobs = no
}

Similarly for Tier 2

Schedule {
  Name = "First-Fri-Full"
  Description = "First Friday of the month - Full"
  Run = Level="Full" 1st fri at 22:00
  Run = Level="Incremental" sun,mon,tue,wed,thu,sat at 22:00
  Run = Level="Incremental" Accurate=yes 2nd,3rd,4th,5th fri at 22:00
}

Fileset {
  Name = "mac-users"
  Description = "All user home directories"
  EnableVss = no
  Include {
File = "/Users"
Options {
  HfsPlusSupport = "Yes"
}
  }
}

Pool {
  Name = "mac-users-full"
  PoolType = "Backup"
  LabelFormat = "mac-users-full-vol-"
  ActionOnPurge = "Truncate"
  MaximumVolumes = 100
  MaximumVolumeJobs = 1
  MaximumVolumeBytes = 500
  VolumeRetention = 6048000
  AutoPrune = no
  Catalog = "MyCatalog"
}
Pool {
  Name = "mac-users-incr"
  PoolType = "Backup"
  LabelFormat = "mac-users-incr-vol-"
  ActionOnPurge = "Truncate"
  MaximumVolumes = 20
  MaximumVolumeJobs = 6
  MaximumVolumeBytes = 500
  VolumeRetention = 6048000
  AutoPrune = no
  Catalog = "MyCatalog"
}

All the best,
 JC


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: Bacula-FD on QubesOS

[Justin Case]

Dear all,

is someone already using Bacula-FD on QubesOS?
Does that work at all?
What is the UX? Does one need a separate FD per VM?

All the best
 JC
 


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Bacula-FD on QubesOS

[Justin Case]

Yes Jose, so far for the theory.
I was rather interested if someone already has experiences with QubesOS and 
Bacula-FD, hence the questions I had.

> On 23. Mar 2022, at 15:26, Jose Alberto  wrote:
> 
> You should be able to install the FD without problems.
> 
> What I have read is that QubesOS is x86_64 and RPM.
> 
> 
> 
> On Tue, Mar 22, 2022 at 12:47 PM Justin Case  <mailto:jus7inc...@gmail.com>> wrote:
> Dear all,
> 
> is someone already using Bacula-FD on QubesOS?
> Does that work at all?
> What is the UX? Does one need a separate FD per VM?
> 
> All the best
>  JC
> 
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net <mailto:Bacula-users@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/bacula-users 
> <https://lists.sourceforge.net/lists/listinfo/bacula-users>
> 
> 
> -- 
> #
> #   Sistema Operativo: Debian  #
> #Caracas, Venezuela  #
> #

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Correct FD setup for macOS 10.15

[Justin Case]

Hi Serge,

interesting. However, the URL you gave does not work. Also no bacula(-fd) in 
the official macOS appstore.

I found in the mailing list archive that it is possible to grant full disk 
access rights to bacula-fd, and since I did that I got far less access errors 
(the test backup has not yet completed), in fact so far only 3:

Could not stat 
"/Users/user1/Library/Containers/com.apple.mail/Data/DataVaults": ERR=Operation 
not permitted

Could not stat "/Users/user2/Library/Application 
Support/Firefox/Profiles/cb63fvn5.default/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite-wal":
 ERR=No such file or directory

Could not stat 
"/Users/user2/Library/Containers/com.apple.mail/Data/DataVaults": ERR=Operation 
not permitted

Not sure why that is, as bacula-fd should have full access rights now.

Does someone know what’s going on there and how to fix it?

All the best,
 JC

> On 23. Mar 2022, at 13:37, UGE Serge (FR2010)  wrote:
> 
> Hello Justin,
> 
> I installed facula-fd on 3 macs, and I got at each time the same error than 
> you:
> "Could not open directory "/Users/moses/Pictures/Photos 
> Library.photoslibrary": ERR=Operation not permitted"
> 
> It should not work like this. After many tests, the only way I found to make 
> it run with no errors is:
> 
> - Download form https://macappstore.org/bacula-fd/ 
> <https://macappstore.org/bacula-fd/>
> - Install bacula-fd
> - Run it to see if you have the error.
> - If yes, remove completly bacula-fd
> - Re-install it.
> 
>  and then it works. I do not know why but it does the job (backup + 
> restore).
> 
> I hope this can help you.
> 
> Serge UGE
> 
> De: "Justin Case" 
> À: "bacula-users" 
> Envoyé: Mardi 22 Mars 2022 13:52:03
> Objet: [Bacula-users] Q: Correct FD setup for macOS 10.15
> 
> Dear all,
> 
> I just made a test installation of bacula-fd on macOS 10.15 using homebrew 
> (so I did not need to compile anything).
> I configured the bacula-fd.conf and set up a fileset for “/Users” to backup 
> all user data. I enabled the option HfsPlusSupport, but the machine actually 
> does use APFS, is that a problem?
> I also set up corresponding pools and scheduled jobs for the storage tiers.
> 
> Now running the job for storage tier 1 I can see that the FD (running as 
> root) on the macOS machine cannot access a lot of directories.
> 
> Some examples:
> 
> 22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
> "/Users/moses/Pictures/Photos Library.photoslibrary": ERR=Operation not 
> permitted
> 22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
> "/Users/moses/Desktop": ERR=Operation not permitted
> 22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
> "/Users/moses/Library/Application Support/CallHistoryTransactions": 
> ERR=Operation not permitted
> 22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
> "/Users/moses/Library/Application Support/com.apple.sharedfilelist": 
> ERR=Operation not permitted
> 22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
> "/Users/moses/Library/Application Support/com.apple.TCC": ERR=Operation not 
> permitted
> 22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
> "/Users/moses/Library/Application Support/FileProvider": ERR=Operation not 
> permitted
> 22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
> "/Users/moses/Library/Application Support/AddressBook": ERR=Operation not 
> permitted
> 22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
> "/Users/moses/Library/Application Support/CallHistoryDB": ERR=Operation not 
> permitted
> 22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
> "/Users/moses/Library/Autosave Information": ERR=Operation not permitted
> 22-Mar 13:28 macmachine-fd JobId 116:  Could not open directory 
> "/Users/moses/Library/IdentityServices": ERR=Operation not permitted
> etc.
> Does someone have a proper setup for the macOS FD and the filesets that runs 
> fine and also restore will work as expected?
> 
> This is my current bacula-fd.conf on macOS:
> 
> Director {
>   Name = bacula-dir
>   Password = “snip"
> }
> 
> Director {
>   Name = bacula-mon
>   Password = “snip"
>   Monitor = yes
> }
> 
> FileDaemon {  # this is me
>   Name = macmachine-fd
>   FDport = 9102  # where we listen for the director
>   WorkingDirectory = /usr/local/var/lib/bacula
>   Pid Directory = /usr/local/var/run
>   Maximum Concurren

Re: [Bacula-users] Q: Correct FD setup for macOS 10.15

[Justin Case]

Thanks for the explanation,

I think that’s correct. Later backups only showed the problem that the mail 
container cannot be opened, no other access errors.

To get rid of the errors I will exclude these pathes.

All the best,
 JC

> On 23. Mar 2022, at 21:13, Rowdy  wrote:
> 
> Hi,
> 
> The permission errors probably result from macOS System Integrity Protection 
> (SIP).  This post sort of explains it, but does not provide a solution (there 
> may not be a solution, other than disabling SIP):
> 
> https://apple.stackexchange.com/questions/337951/access-to-library-containers-com-apple-mail-data-denied
>  
> 
> 
> The "missing file" error most likely relates to a Firefox temporary file that 
> was present when Bacula gathered a list of files to backup, but missing when 
> it actually tried to archive it.
> 
> Rowdy

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] An special day for honoring someone who has done a extremely nice job in the open source world

[Justin Case]

Dear Kern

the kudos are well deserved and yes, it takes some time to understand Bacula’s 
concepts for people -like me- who are not used to tape libraries - and the time 
it takes is well deserved. I am happy with Bacula and happy that I took the 
time to get some understanding how to use it.

I can only say thank you for the time and effort you dedicated to develop 
Bacula, it is helping me a lot and I can imagine that hundreds, if not 
thousands or more people are relying on Bacula and are grateful for what you 
achieved.

Puerto Rico and Barcelona, two so beautiful places I have visited, indeed I 
wish I would be living in one of these.

All the best to you Kern and a long and healthy life and happy times,
JC

> On 24. Mar 2022, at 17:55, Kern Sibbald  wrote:
> 
> Hello Egoitz,
> 
> Thanks for your very kind email with birthday wishes.  I am really honored 
> that you call me a mentor, thanks.  I have always done my best to produce a 
> high end backup product (though it is a bit complicated) that is robust and 
> reliable with all the features needed by the community as well as small 
> enterprises.  Though it is sometimes hard, I feel that we succeeded in 
> creating a friendly and helpful email list.
> 
> Thanks for using Bacula and for your email.
> 
> Concerning your plugin:  in general we accept all contributions that are 
> useful, follow our current design conditions (in the Developers document) and 
> for which we have a signed CAA (copyright assignment agreement).  This 
> agreement allows Bacula Systems to use the code, but even more important for 
> me, it protect all Bacula users from someone introducing code then claiming 
> we are using his proprietary code (this is what Bareos attempted to do to 
> Bacula). I can remember only on case where we we not able to use the code, 
> and that was code developed by UKFast (ISP) and released by the developer 
> without permission.  We we never able to get permission to use the code from 
> UKFast.  This code implemented quotas.  Instead of integrating unauthorized 
> code, we wrote our own simpler and more efficient quota code.  Note, Bareos 
> integrated the unauthorized code, so perhaps one day their users may be 
> exposed to license problems.
> 
> If you finish your code, and you are willing to submit a CAA, then I 
> recommend that you submit it to Bacula (Now that I am retired, Eric decides 
> exactly what is integrated and what is not).   In my opinion, it would be a 
> very nice addition.
> 
> I am very pleased that Bacula Systems has agreed to look after maintaining 
> the community version and continue adding the Bacula Systems new features.  
> This means that Bacula will be getting better and better and continue to 
> adapt to the ever changing IT backup/restore needs.
> 
> Thank you very much for recognizing my contributions, and above all thank you 
> for using Bacula and developing code for it.  I wish you all the best in the 
> future.
> 
> Kind regards,
> Kern
> (Currently in Puerto Rico until May then back to Barcelona).
> 
> 
> 
> On 3/21/2022 4:40 AM, ego...@ramattack.net wrote:
>> 
>> Good morning people :)
>> 
>> 
>> Today is a special day. Is the birthday of someone now deservedly retired. 
>> This person is Kern Sibbald. I had the opportunity of knowing about his 
>> birthday, through one nice person in this mailing lists. Kern is much more 
>> than a nice codder, who has write a extremely important tool which is the 
>> base of most of our backups. At least for me is one of my mentors, one of 
>> the persons from which I would like to learn because it has done a nice job 
>> with Bacula. For all these reasons, I wanted to say "thank you so much Kern 
>> and have an extremely happy birthday!!".
>> 
>> I wanted too, for making the most of this lines, to ask a little gift to 
>> Kern :) :). I'm working as some of you know, in building a Bacula pluggin 
>> for creating an open source delta encoding plugin for the fd. It would be 
>> hugely nice :) :) (as someone told me too in this list) if official Delta 
>> plugin of Bacula, could be distributed with the Community source of Bacula. 
>> I'll go on writting my own plugin anyway, because I wanted to learn how it 
>> works and for being able to customize some of the backups I do here, but it 
>> would be really nice to have the Delta plugin as part of the Community 
>> edition of this nice piece of software. I had to try :) :) .
>> 
>> Anyway and independently of what Kern decides about the gift I have asked :) 
>> :) (I had to try it... mainly after someone encouraged asking it :) :) ), I 
>> wanted to emphasize my recognition about Kern's person, due to all his 
>> contributions to the open source world. These ideas, anyway, would never 
>> change in my mind about you.
>> 
>> So for ending this email, I think there are not more appropiate words for 
>> being remarked as the following ones : "Kern, Thank you so much".
>> 
>> 
>> Cheers :)
>> 
> 
> 
> 
> 

[Bacula-users] Autochanger does not create volumes (on mounted external USB drive)

[Justin Case]

Hi there,

in my setup I call internal drives connected by SATA “Tier 1”, and external 
drives connected by USB “Tier 2”.
I have separate pools for jobs that write to tier 1 and for jobs that write to 
tier 2.
Each given pool contains only volumes stored on 1 tier, so each pool is either 
a tier pool or a tier 2 pool, but never both.

For Tier 1 I have a working autochanger that created volumes in the pools that 
I use for tier 1 (only).

For Tier 2 I have another autochanger that is supposed to create volumes in the 
pooks that i use for tier 2 (only), but it doesn’t do that. It gives me:
The job needs media. Please mount volume unraid-libvirt-full-tier2-vol-0543 or 
label a new one for storage: "Tier2-FileChgr1-Dev1" (/mnt/tier2-01), pool: 
unraid-libvirt-full-tier2, media type: tier2-ext-usb1-folder1-file.

Any chance to get this to work?

Here are the definitions from the bacula-sd.config:

Storage {
  Name = "bacula-sd"
  WorkingDirectory = "/opt/bacula/working"
  PidDirectory = "/opt/bacula/working"
  PluginDirectory = "/opt/bacula/plugins"
  MaximumConcurrentJobs = 20
}
Device {
  Name = "Restore-FileChgr1-Dev99"
  Description = "Use this folder as default location for restore jobs"
  MediaType = "restore-int-user0-folder1"
  DeviceType = "File"
  ArchiveDevice = "/mnt/restore"
  RemovableMedia = no
  RandomAccess = yes
  AutomaticMount = yes
  LabelMedia = yes
  Autochanger = yes
  AutoSelect = no
  DriveIndex = 99
}
Device {
  Name = "Tier1-FileChgr1-Dev1"
  Description = "unRAID local file system folder 1"
  MediaType = "tier1-int-user0-folder1-file"
  DeviceType = "File"
  ArchiveDevice = "/mnt/tier1-01"
  RemovableMedia = no
  RandomAccess = yes
  AutomaticMount = yes
  LabelMedia = yes
  AlwaysOpen = no
  Autochanger = yes
  MaximumConcurrentJobs = 5
}
Device {
  Name = "Tier1-FileChgr1-Dev2"
  Description = "unRAID local file system folder 2"
  MediaType = "tier1-int-user0-folder2-file"
  DeviceType = "File"
  ArchiveDevice = "/mnt/tier1-02"
  RemovableMedia = no
  RandomAccess = yes
  AutomaticMount = yes
  LabelMedia = yes
  AlwaysOpen = no
  Autochanger = yes
  MaximumConcurrentJobs = 5
  DriveIndex = 1
}
Device {
  Name = "Tier2-FileChgr1-Dev1"
  Description = "unRAID external USB disk 1 folder 1"
  MediaType = "tier2-ext-usb1-folder1-file"
  DeviceType = "File"
  ArchiveDevice = "/mnt/tier2-01"
  RemovableMedia = yes
  RandomAccess = yes
  AutomaticMount = yes
  LabelMedia = yes
  AlwaysOpen = no
  Autochanger = yes
  MaximumConcurrentJobs = 5
}
Device {
  Name = "Tier2-FileChgr1-Dev2"
  Description = "unRAID external USB disk 2 folder 1"
  MediaType = "tier2-ext-usb2-folder1-file"
  DeviceType = "File"
  ArchiveDevice = "/mnt/tier2-02"
  RemovableMedia = yes
  RandomAccess = yes
  AutomaticMount = yes
  LabelMedia = yes
  AlwaysOpen = no
  Autochanger = yes
  MaximumConcurrentJobs = 5
}
Messages {
  Name = "Standard"
  Director = bacula-dir = All
}
Autochanger {
  Name = "tier1-int-user0-folder-changer"
  Description = "This changer makes all tier 1 folders on internal user0 file 
system accessible"
  Device = "Restore-FileChgr1-Dev99"
  Device = "Tier1-FileChgr1-Dev1"
  Device = "Tier1-FileChgr1-Dev2"
  ChangerDevice = "/dev/null"
  ChangerCommand = "/dev/null"
}
Autochanger {
  Name = "tier2-ext-usb-changer"
  Description = "This changer makes all tier 2 folders on external USB mass 
storage accessible"
  Device = "Tier2-FileChgr1-Dev1"
  Device = "Tier2-FileChgr1-Dev2"
  ChangerDevice = "/dev/null"
  ChangerCommand = "/dev/null"
}

Here are some definitions of storage, pools and jobs from bacula-dir.config:

Job {
  Name = "unraid-libvirt-tier1"
  Description = "Job to backup unRAID Libvirt to Tier 1"
  Type = "Backup"
  Level = "Full"
  Messages = "Standard"
  Storage = "unraid-tier1-storage"
  Pool = "unraid-libvirt-full-tier1"
  FullBackupPool = "unraid-libvirt-full-tier1"
  IncrementalBackupPool = "unraid-libvirt-incr-tier1"
  Client = "bacula-fd"
  Fileset = "unraid-libvirt"
  Schedule = "First-Mon-Full"
  JobDefs = "DefaultJob"
  Enabled = yes
  AllowIncompleteJobs = no
  AllowDuplicateJobs = no
}
Job {
  Name = "unraid-libvirt-tier2"
  Description = "Job to backup unRAID Libvirt to Tier 2"
  Type = "Backup"
  Level = "Full"
  Messages = "Standard"
  Storage = "unraid-tier2-storage"
  Pool = "unraid-libvirt-full-tier2"
  FullBackupPool = "unraid-libvirt-full-tier2"
  IncrementalBackupPool = "unraid-libvirt-incr-tier2"
  Client = "bacula-fd"
  Fileset = "unraid-libvirt"
  Schedule = "Third-Mon-Full"
  JobDefs = "DefaultJob"
  Enabled = yes
  AllowIncompleteJobs = no
  AllowDuplicateJobs = no
}

Storage {
  Name = "unraid-tier1-storage"
  Description = "Tier 1 storage on unraid"
  SdPort = 9103
 Address = “redacted"
 Password = “redacted"
  Device = "tier1-int-user0-folder-changer"
  MediaType = "tier1-int-user0-folder1-file"
  Autochanger = "unraid-tier1-storage"
  MaximumConcurrentJobs = 10
}
Storage {
  Name = "unraid-tier2-storage"
  Description = "Tier 2 storage on unraid"
 

[Bacula-users] SOLVED: Autochanger does not create volumes (on mounted external USB drive)

[Justin Case]

> On 24. May 2022, at 16:40, Justin Case  wrote:
> 
> Hi there,
> 
> in my setup I call internal drives connected by SATA “Tier 1”, and external 
> drives connected by USB “Tier 2”.
> I have separate pools for jobs that write to tier 1 and for jobs that write 
> to tier 2.
> Each given pool contains only volumes stored on 1 tier, so each pool is 
> either a tier pool or a tier 2 pool, but never both.
> 
> For Tier 1 I have a working autochanger that created volumes in the pools 
> that I use for tier 1 (only).
> 
> For Tier 2 I have another autochanger that is supposed to create volumes in 
> the pooks that i use for tier 2 (only), but it doesn’t do that. It gives me:
> The job needs media. Please mount volume unraid-libvirt-full-tier2-vol-0543 
> or label a new one for storage: "Tier2-FileChgr1-Dev1" (/mnt/tier2-01), pool: 
> unraid-libvirt-full-tier2, media type: tier2-ext-usb1-folder1-file.
> 
> Any chance to get this to work?

Actually the definitions and setup in Bacula works fine. 

The problem was that the external USB drives were not writeable due to unknown 
reasons. After reformatting and setting up the folder structure all worked fine.

I am sorry if someone spent a lot of time reading my setup and scratched their 
heads where the problem may be.

All the best
 J/C



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] How to reclaim volume space occupied by incomplete jobs?

[Justin Case]

Hi there,

I am using Bacula since some weeks now, but operationally I am a complete noob. 
Haven’t run through many use cases yet.

Since I have included my laptop as a job the incrementals run relatively quick, 
but the fulls take long. Sometime I put the laptop to sleep without thinking 
about the backup!
If that happens during a job for the laptop, then the job stops if the connect 
does not work for some time. This leads to an errored job that has a lot of 
data in volumes, but the job is incomplete. I run that job later. OK, from a 
safety point of vie no problem at all.

But what about the space occupied by the incomplete job?

What is an efficient way to reclaim this space using the Baculum UI?

Deleting the incomplete job does not do the trick I suppose? What are 
additional steps that are required?

Thanks for your experienced feedback!
J/C

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] How to reclaim volume space occupied by incomplete jobs?

[Justin Case]

Thank you so much, Bill, for your excellent explanation. It was 100% clear to 
me what needs to be done.

I am happy to tell you that nearly all your proposals are already implemented 
in my setup.
They only bit left to do is to put your scripts in as jobs.

I currently have only this maintenance job:

Job {
  Name = "Pruning"
  Description = "Prune all expired volumes"
  Type = "Admin"
  Schedule = "EveryNight"
  JobDefs = "DefaultJob"
  PruneJobs = yes
  PruneFiles = yes
  PruneVolumes = yes
  Enabled = yes
  Runscript {
RunsWhen = "Before"
RunsOnClient = no
Console = "prune expired volume yes"
  }
  Priority = 12
  AllowDuplicateJobs = no
}

Thank you again,
 J/C

> On 11. Jun 2022, at 22:09, Bill Arlofski via Bacula-users 
>  wrote:
> 
> On 6/10/22 17:08, Justin Case wrote:
>> Hi there,
>> I am using Bacula since some weeks now, but operationally I am a complete 
>> noob. Haven’t run through many use cases yet.
>> Since I have included my laptop as a job the incrementals run relatively 
>> quick, but the fulls take long. Sometime I put the laptop to sleep without 
>> thinking about the backup!
>> If that happens during a job for the laptop, then the job stops if the 
>> connect does not work for some time. This leads to an errored job that has a 
>> lot of data in volumes, but the job is incomplete. I run that job later. OK, 
>> from a safety point of vie no problem at all.
>> But what about the space occupied by the incomplete job?
>> What is an efficient way to reclaim this space using the Baculum UI?
>> Deleting the incomplete job does not do the trick I suppose? What are 
>> additional steps that are required?
>> Thanks for your experienced feedback!
>> J/C
> 
> 
> Hello Justin,
> 
> 
> Short answer:
> 
> You can not delete data inside Bacula volumes.
> 
> 
> Long answer:
> 
> To begin the process of seeing if you can reclaim these volume(s), deleting 
> the failed laptop jobs is the correct first step. This removes any mention of 
> those jobs and files from the catalog, which means when you look to see what 
> jobs are stored on the volume(s) that those failed jobs used (the next step 
> to do), you will see either no jobs associated with the volume(s), or any 
> other jobs that may have also used them.
> 
> When you check the "Jobs on volume" in Baculum, if there are no jobs 
> associated with a volume, Bacula will just prune/purge/recycle and re-use 
> them as necessary once a pruning operation takes place.
> 
> If there are other jobs that used these volumes, you can just let things be 
> and the volumes will be pruned/purged/recycled and re-used once the volume 
> reaches its retention period - as above.
> 
> If you want to reclaim the volumes sooner, you can delete the other jobs that 
> are on the volume(s) - of course making the right decisions about important 
> jobs and not deleting those.
> 
> But if you keep any jobs that wrote to the volume(s) and don't delete them, 
> they are still in the catalog and you will need to wait for the volume 
> retention period to pass before Bacula will re-use the volume(s).
> 
> 
> Future hints:
> 
> You may want to create a special pool for the laptop and maybe even set in 
> this pool "MaximumVolumeJObs = 1". Then, you can be guaranteed that only 
> laptop jobs will ever be on volumes in this pool, and that only one laptop 
> job will ever be on any given volume in this pool.
> 
> Then, when you run into this situation again and you want to clean up, it 
> will be as easy as just deleting the failed job(s), and then the volume(s) 
> that the deleted job(s) used will be ready for re-use as detailed in the 
> first paragraph. :)
> 
> Additionally, if you really want/need to reclaim the actual disk space these 
> volumes use, there are some additional steps to take:
> 
> First, I would globally disable pruning in the Director's 'Director{}' block 
> by adding "Autoprune = no". If you are running an older version of Bacula, 
> this option may not exist.
> 
> Then, set "ActionOnPurge = truncate" in the new pool.
> 
> Next create an Admin type job that runs nightly before the backup jobs run 
> which will run a small script which calls some bconsole commands to do some 
> catalog maintenance, including pruning and then truncating the pruned/purged 
> volumes.
> 
> 
> The script that my NightlyMaint Admin job runs looks like the following:
> 8<
> #!/bin/bash
> 
> bcbin="/opt/comm-bacula/sbin/bconsole"
> bcconfig="/opt/comm-bacula/include/bconsole.conf"
> storage=&q

Re: [Bacula-users] How to reclaim volume space occupied by incomplete jobs?

[Justin Case]

Excellent, good to know. 
I was actually wondering why you were scripting this instead of simply having 
several Command items in the Job.
It is clear now and makes 100% sense to me.

> On 12. Jun 2022, at 00:14, Bill Arlofski via Bacula-users 
>  wrote:
> 
> On 6/11/22 15:33, Justin Case wrote:
>> Thank you so much, Bill, for your excellent explanation. It was 100% clear 
>> to me what needs to be done.
>> I am happy to tell you that nearly all your proposals are already 
>> implemented in my setup.
>> They only bit left to do is to put your scripts in as jobs.
>> I currently have only this maintenance job:
>> Job {
>> Name = "Pruning"
>> Description = "Prune all expired volumes"
>> Type = "Admin"
>> Schedule = "EveryNight"
>> JobDefs = "DefaultJob"
>> PruneJobs = yes
>> PruneFiles = yes
>> PruneVolumes = yes
>> Enabled = yes
>> Runscript {
>> RunsWhen = "Before"
>> RunsOnClient = no
>> Console = "prune expired volume yes"
>> }
>> Priority = 12
>> AllowDuplicateJobs = no
>> }
>> Thank you again,
>>  J/C
> 
> Hey Justin, you are welcome!
> 
> One additional comment and some hopefully useful information...
> 
> I always use the 'Command =' vs the 'Console =' option in RunScripts. There 
> are a few reasons:
> 
> - All output from the script, like echo commands in bash script, print() 
> commands in Python, or stdout/stderr from commands run in the script are then 
> included in the Job's job so you can see what happened, and when just by 
> looking at the job's log. Nice and neat, all in one place.
> 
> - In a script, I have full control as to what order things run in, and I can 
> also trap errors, and exit accordingly, and fail the job if necessary.
> 
> - If you have multiple 'Console =' or 'Command =' lines in a RunScript, you 
> cannot be guaranteed as to the order in which they are called/run. This 
> information is direct from the developers, and as seen/demonstrated/proven by 
> one of my colleagues on the Support team. (Thank you Uli!) :)
> 
> - The 'Console =' command associates any output to "JobId = 0" and these 
> outputs don't always appear immediately in the joblog (ie: bacula.log in a 
> Messages{} 'Append = ' log file) ast the time they occur. Seems that they can 
> be buffered and just appear randomly roo when you type 'messages' in a 
> bconsole session.
> 
> - Additionally, these log outputs for "jobId = 0" do not get inserted into 
> the catalog database, so a bconsole "ll joblog jobid=0" will return nothing, 
> and it would not matter if it did, you'd still have no idea to which job 
> these 'console =' outputs actually occurred during.
> 
> Hope this helps!
> 
> 
> Best regards,
> Bill
> 
> -- 
> Bill Arlofski
> w...@protonmail.com <mailto:w...@protonmail.com>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net <mailto:Bacula-users@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/bacula-users 
> <https://lists.sourceforge.net/lists/listinfo/bacula-users>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: Bacula for TrueNAS Core?

[Justin Case]

Hello all,

I was looking for a Bacula Plugin for a recent TrueNAS Core (bacula-fd to 
backup its content, bacula-dir, bacula-sd to backup other machines’ content). I 
can see that there has been a Bacula Plugin maintained by iX-Systems for 
FreeNAS in the past, but it has been abandoned by iX-Systems.

Since TrueNAS Core is the closest we get for free to something like an 
enterprise storage appliance, I was assuming it makes a lot of sense and there 
is a lot of demand to have Bacula for TrueNAS Core. I haven’t found it in the 
Community Plugins either.

I saw that someone on this list was compiling Bacula for TrueNAS Core, so it 
must be feasible.

Are just too few people here using TrueNAS Core? Or has everyone gone through 
compiling it for themselves? Wouldn’t it be more  “user-friendly” to have a 
plugin? Maybe no-one has sufficient motivation to make and maintain one (yeah, 
I know someone is going to respond “why don’t you do it?”). 
Or is there another explanation?

Or is there actually a Bacula Plugin for TrueNAS Core and I just did not look 
right and did not find it?

All the best, I am looking forward to your thoughts on this.
 J/C

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: regarding bconsole.conf on Bacula client machines

[Justin Case]

Hi all,

when installing the bacula-client packageon Debian I end up i /etc/bacula with
bacula-fd.conf
and
bconsole.conf

I have 2 questions, that are still open although I searched through the main 
documentation and read the chapters containing bconsole.conf:

(1) In bacula-fd.conf I understand how to set up the FileDaemon resource. I 
also understand how to set up the Director resource for bacula-dir, so that the 
actual backup works. There always is a second Director resource for bacula-mon 
and I do not understand what it used for and where I find the secret string in 
Baculum UI on the Bacula Director machine to put in Password for the Director 
resource on the Bacula FD client machine.

(2) What is the use for the bconsole.conf file on the Bacula FD client machine? 
Is it a means to allow the admin of the client machine to take a look at the 
jobs on the Bacula Director machine concerning this client machine? If I wanted 
this to work, where in Baculum UI on the Bacula Director machine do I find the 
secret string to put in Password for the Director resource in bcsonsole.conf on 
the client machine?

Thanks for considering my questions and all the best,
 J/C






___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: regarding bconsole.conf on Bacula client machines

[Justin Case]

Thanks, I understand.
Still the partial questions are open where I find the corresponding secret 
strings in Baculum. I am not complaining, just want to make explicit that not 
all parts of the questions are answered yet.

Where ein Baculum do I find the bconsole secret for the bcsonsole config file? 
I understand that it must be somehow possible to restrict the bconsole access 
using ACLs. I am just trying to understand where in the most basic config I 
would find the secret (which menu path in Baculum).

Also: on what operating systems there is a tray monitor available that then 
uses the bacula-mon interface? Is this a Windows-specific thing? (So this 
interface is for localhost use only I I should find the corresponding secret 
string in the config for the tray app?). I am using mostly macOS and Debian, 
some FreeBSD - is this relevant on these platforms? (I don’t have Windows at 
all. Lucky me.)

Thanks again,
 J/C


> On 15. Jun 2022, at 16:39, Josh Fisher  wrote:
> 
> 
> On 6/15/22 08:12, Justin Case wrote:
>> Hi all,
>> 
>> when installing the bacula-client packageon Debian I end up i /etc/bacula 
>> with
>> bacula-fd.conf
>> and
>> bconsole.conf
>> 
>> I have 2 questions, that are still open although I searched through the main 
>> documentation and read the chapters containing bconsole.conf:
>> 
>> (1) In bacula-fd.conf I understand how to set up the FileDaemon resource. I 
>> also understand how to set up the Director resource for bacula-dir, so that 
>> the actual backup works. There always is a second Director resource for 
>> bacula-mon and I do not understand what it used for and where I find the 
>> secret string in Baculum UI on the Bacula Director machine to put in 
>> Password for the Director resource on the Bacula FD client machine.
> 
> 
> The second bacula-mon Director interface is for the tray monitor app running 
> on the client machine. The tray monitor acts as a simplified director only 
> for retrieving info from the bacula-fd daemon.
> 
>> (2) What is the use for the bconsole.conf file on the Bacula FD client 
>> machine? Is it a means to allow the admin of the client machine to take a 
>> look at the jobs on the Bacula Director machine concerning this client 
>> machine? If I wanted this to work, where in Baculum UI on the Bacula 
>> Director machine do I find the secret string to put in Password for the 
>> Director resource in bcsonsole.conf on the client machine?
> 
> 
> That is the configuration file for the bconsole command line utility used to 
> interface with Director and Storage daemons.
> 
> 
>> 
>> Thanks for considering my questions and all the best,
>>  J/C
>> 
>> 
>> 
>> 
>> 
>> 
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net 
>> <mailto:Bacula-users@lists.sourceforge.net>
>> https://lists.sourceforge.net/lists/listinfo/bacula-users 
>> <https://lists.sourceforge.net/lists/listinfo/bacula-users>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: one job for a group of "identical" machines using the same fileset?

[Justin Case]

Hi there,

in the age of virtualization and cloud technology such as Kubernetes I would 
assume that people might have a lot of VMs that use the same setup where they 
may store container user data under the same path for each machine.

While I see that it is possible to define just one fileset to use for all these 
VMs, I seem to miss a way to define a job that is run for a group of clients 
(e.g. sequentially for each of them to not overload the VM host) for the same 
fileset. 

Is there way in Bacula to achieve this or is this a hard limitation that a need 
an individual job per each bacula-fd machine?

Just looking for clarification, before I proceed setting up a larger number of 
jobs (which is a pain, even if you can use the templating in Baculum).

Thanks and best,
 J/C



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: one job for a group of "identical" machines using the same fileset?

[Justin Case]

Thanks, found it under Director > Configure Director > JobDefs

> On 15. Jun 2022, at 18:28, Bill Arlofski via Bacula-users 
>  wrote:
> 
> On 6/15/22 10:21, Justin Case wrote:
>> Hi there,
>> 
>> in the age of virtualization and cloud technology such as Kubernetes I would 
>> assume that people might have a lot of VMs that use the same setup where 
>> they may store container user data under the same path for each machine.
>> 
>> While I see that it is possible to define just one fileset to use for all 
>> these VMs, I seem to miss a way to define a job that is run for a group of 
>> clients (e.g. sequentially for each of them to not overload the VM host) for 
>> the same fileset.
>> 
>> Is there way in Bacula to achieve this or is this a hard limitation that a 
>> need an individual job per each bacula-fd machine?
>> 
>> Just looking for clarification, before I proceed setting up a larger number 
>> of jobs (which is a pain, even if you can use the templating in Baculum).
>> 
>> Thanks and best,
>>  J/C
> 
> With Bacula, it is "One Client Per Job"
> 
> So in your example, you would create a Job template, that uses a JobDefs{} 
> resource containing all t
> he things common to the
> jobs (Fileset, Priority, Messages, Storage, Pool, etc), then for each Job, 
> just use that JobDefs and only override the things
> that will be different, like the Client, for example.
> 
> 
> Best regards,
> Bill
> 
> 
> --
> Bill Arlofski
> w...@protonmail.com
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: fileset "magic"

[Justin Case]

Hi all,

I am somewhat struggling with the fileset algorithm, noob birth pains I guess.

I have a bunch of VMs that have mounted(!!) docker container appdata in 
/mnt/cdat-.

So I wish to backup /mnt/cdat-* on each of these VMs, meaning I wish that the 
content of each subdirectory in /mnt where the name starts with "cdat-“ gets 
backed up.

So I looked into the main manual for fileset directive syntax. And I found this 
example:

FileSet { 
 Name = "Full Set” 
 Include { 
Options { 
 wilddir = "/home/a*” 
 wilddir = "/home/b*" 
}
Options {
RegexDir = ".*”
exclude = yes 
  } 
File = /home 
}
} 

So what I did is this (and it does not work, just returns 1 file, and that is 
wrong):

Fileset {
  Name = “cadat"
  EnableVss = no
  EnableSnapshot = no
  Include {
Options {
  OneFS = no
  RegexDir = "/mnt/cdat-.*"
}
Options {
  OneFS = no
  Exclude = yes
  RegexDir = ".*"
}
File = "/mnt"
  }
}

I know it must seem kinda obvious where the problem is for those who have been 
around with bacula for a while. For me it is kinda “magic”.

Where is my mistake?

Thanks for helping out!
 J/C




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: fileset "magic" - still weird

[Justin Case]

I re-read the chapter about filesets and fileset options.
In order to better understand what is happening I simplified the fileset as 
follows:

> Fileset {
>  Name = “cadat"
>  EnableVss = no
>  EnableSnapshot = no
>  Include {
>Options {
>  OneFS = no
>  #RegexDir = "/mnt/cdat-.*"
>}
>  #  Options {
>  #OneFS = no
>  #   Exclude = yes
>  #RegexDir = ".*”
>  #  }
>File = "/mnt"
>  }
> }


I was hoping it would then backup everything in /mnt (yes they are all 
different filesystems, but OneFS is set to no).

Again, nothing was backed up.

I do not understand this result. I thought I had understood what is in the 
manual about filesets, but obviously I did not.



> On 15. Jun 2022, at 19:55, Justin Case  wrote:
> 
> Hi all,
> 
> I am somewhat struggling with the fileset algorithm, noob birth pains I guess.
> 
> I have a bunch of VMs that have mounted(!!) docker container appdata in 
> /mnt/cdat-.
> 
> So I wish to backup /mnt/cdat-* on each of these VMs, meaning I wish that the 
> content of each subdirectory in /mnt where the name starts with "cdat-“ gets 
> backed up.
> 
> So I looked into the main manual for fileset directive syntax. And I found 
> this example:
> 
> FileSet { 
>Name = "Full Set” 
>Include { 
>   Options { 
>wilddir = "/home/a*” 
>wilddir = "/home/b*" 
>   }
>   Options {
>   RegexDir = ".*”
>   exclude = yes 
> } 
>   File = /home 
>   }
> } 
> 
> So what I did is this (and it does not work, just returns 1 file, and that is 
> wrong):
> 
> Fileset {
>  Name = “cadat"
>  EnableVss = no
>  EnableSnapshot = no
>  Include {
>Options {
>  OneFS = no
>  RegexDir = "/mnt/cdat-.*"
>}
>Options {
>  OneFS = no
>  Exclude = yes
>  RegexDir = ".*"
>}
>File = "/mnt"
>  }
> }
> 
> I know it must seem kinda obvious where the problem is for those who have 
> been around with bacula for a while. For me it is kinda “magic”.
> 
> Where is my mistake?
> 
> Thanks for helping out!
> J/C
> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: fileset "magic" - still weird

[Justin Case]

I understand what the example does, it is very much standard.
Alas, it does not touch any of the points where my use case is special:
- it enumerates all inclusions and exclusions - that is explicitely what I 
cannot and do not want to do as each FD machine has different subfolders, but 
all start with the same prefix
- it does not employ wildcard characters for folder names

Thank you for the example. To be honest, I did not learn anything new that 
would allow me to solve my problem.

Thank you again , though, for considering my questions.
J/C

> On 15. Jun 2022, at 23:23, sru...@gemneye.org wrote:
> 
> On 2022-06-15 13:47, Justin Case wrote:
>> I re-read the chapter about filesets and fileset options.
>> In order to better understand what is happening I simplified the
>> fileset as follows:
>>> Fileset {
>>> Name = “cadat"
>>> EnableVss = no
>>> EnableSnapshot = no
>>> Include {
>>>   Options {
>>> OneFS = no
>>> #RegexDir = "/mnt/cdat-.*"
>>>   }
>>> #  Options {
>>> #OneFS = no
>>> #   Exclude = yes
>>> #RegexDir = ".*”
>>> #  }
>>>   File = "/mnt"
>>> }
>>> }
>> I was hoping it would then backup everything in /mnt (yes they are all
>> different filesystems, but OneFS is set to no).
>> Again, nothing was backed up.
>> I do not understand this result. I thought I had understood what is in
>> the manual about filesets, but obviously I did not.
>>> On 15. Jun 2022, at 19:55, Justin Case  wrote:
>>> Hi all,
>>> I am somewhat struggling with the fileset algorithm, noob birth pains I 
>>> guess.
>>> I have a bunch of VMs that have mounted(!!) docker container appdata in 
>>> /mnt/cdat-.
>>> So I wish to backup /mnt/cdat-* on each of these VMs, meaning I wish that 
>>> the content of each subdirectory in /mnt where the name starts with "cdat-“ 
>>> gets backed up.
>>> So I looked into the main manual for fileset directive syntax. And I found 
>>> this example:
>>> FileSet {
>>>  Name = "Full Set”
>>>  Include {
>>> Options {
>>>  wilddir = "/home/a*”
>>>  wilddir = "/home/b*"
>>> }
>>> Options {
>>> RegexDir = ".*”
>>> exclude = yes
>>>   }
>>> File = /home
>>> }
>>> }
>>> So what I did is this (and it does not work, just returns 1 file, and that 
>>> is wrong):
>>> Fileset {
>>> Name = “cadat"
>>> EnableVss = no
>>> EnableSnapshot = no
>>> Include {
>>>   Options {
>>> OneFS = no
>>> RegexDir = "/mnt/cdat-.*"
>>>   }
>>>   Options {
>>> OneFS = no
>>> Exclude = yes
>>> RegexDir = ".*"
>>>   }
>>>   File = "/mnt"
>>> }
>>> }
>>> I know it must seem kinda obvious where the problem is for those who have 
>>> been around with bacula for a while. For me it is kinda “magic”.
>>> Where is my mistake?
>>> Thanks for helping out!
>>> J/C
> Below is an example Fileset I use which has includes and excludes.
> 
> Fileset {
>  Name = "Firewall Full"
>  Include {
>File = "/"
>File = "/boot"
>File = "/home"
>File = "/var"
>Options {
>  Compression = "Gzip"
>  Signature = "Md5"
>  Exclude = "Yes"
>  WildDir = "/ISO"
>  WildFile = "/.journal"
>  WildFile = "/.fsck"
>}
>  }
> }
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: fileset "magic" - still weird

[Justin Case]

Hello John,

I found this approach quite compelling:

> On 16. Jun 2022, at 00:24, John Lockard  wrote:
> 
> |sh -c ’echo /mnt/cdat-*’
No matter what characters i use for the ‘ I get "Error sending include list.” 
(already put in the actual pathes for sh and echo). Log: bacula-dir JobId 0: 
Fatal error: Error running program: /usr/bin/sh -c `/usr/bin/echo 
/mnt/containerdata-*`. ERR=No such file or directory

Running the same command in the shell on the host works.

So I tried this: RegexDir = ""^(?!cdat-).*

I turns out the the “?!” unfortunately leads to terminally crapping up the 
Baculum in my Bacula-Server container. I actually need to remove all the 
containers images and reainstall it. That doesn’t sound right, but its not a 
solution for me.

I am trying with regexdir for inclusion now.

Thanks anyways,
 J/C


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: ConnectToDirector not working as documented for FD Director directive

[Justin Case]

Hi there,

I have a client behind a firewall and the director is unable to connect to this 
client, but the client may connect to the director and to the SD.

In the main manual is documented that one should to this:

on the Director side in the Client directive: 
  AllowFDConnections = yes

on the FD side in the Director directive:
Director {
  Name = bacula-dir
  Password = “redacted”
#  DirPort = 9101
  Address = bacula-dir.lan.net
  ConnectToDirector = yes
  ReconnectionTime = 40 min
}

I have done this, but the bacula-fd complains:

bacula-fd: ERROR TERMINATION at parse_conf.c:1157
Config error: Keyword "ConnectToDirector" not permitted in this resource.
Perhaps you left the trailing brace off of the previous resource.
: line 22, col 20 of file /etc/bacula/bacula-fd.conf
  ConnectToDirector = yes

Actually it would also complain about DirPort in the same way if I remove the 
comment character.

It looks like the documentation does not fit the behaviour of the Debian 
bacula-client package.

What can I do now?

Best
 J/C



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: regarding bconsole.conf on Bacula client machines

[Justin Case]

Thank you Marcin,

I wasn’t aware the Baculum could also be installed on a FD host. Interesting to 
know.
Actually my question was not clearly phrased as I did mean it. I did not mean 
to look the password up on the FD host side in Baculum.

What I did mean was, where in Baculum on the director host would I find the 
secret strings to use in the bconsole file director directive on the FD host 
side. The same for the secret string for the monitor director directive in the 
bacula-fd.conf on the FD machine. The initial setup suggests that both secret 
strings are not identical to each other.

Sorry for the unclear phrasing.
Best
 J/C

> On 16. Jun 2022, at 06:58, Marcin Haba  wrote:
> 
> Hello Justin,
> 
> On Wed, 15 Jun 2022 at 16:55, Justin Case  <mailto:jus7inc...@gmail.com>> wrote:
> Thanks, I understand.
> Still the partial questions are open where I find the corresponding secret 
> strings in Baculum. I am not complaining, just want to make explicit that not 
> all parts of the questions are answered yet.
> 
> Where ein Baculum do I find the bconsole secret for the bcsonsole config 
> file? I understand that it must be somehow possible to restrict the bconsole 
> access using ACLs. I am just trying to understand where in the most basic 
> config I would find the secret (which menu path in Baculum).
> 
> This password is stored in files in two places: in your file daemon config 
> and in tray monitor config. If you want to see the password on the Baculum 
> interface, you need to install Baculum API on the file daemon host with 
> configuration feature enabled (bfdjson configured). After you need to connect 
> this host to your Baculum Web. At the end you need to go to:
> 
> Main menu -> Clients -> [ select the client ] -> Tab: Configure file daemon 
> -> Section: Director
> 
> If you have everything (Bacula and Baculum API/Web) already configured on one 
> host with the configuration feature enabled in API, you don't need to add any 
> additional API host.
> 
> In the same way you can monitor storage daemons and directors. In this case 
> the director configuration for the monitor connection will be in:
> 
> Main menu -> Director -> Tab: Configure director -> Console
> 
> and storage daemon configuration for the monitor will be in:
> 
> Main menu -> Storage -> [ select the storage ] -> Tab: Configure storage 
> daemon -> Section: Director
> 
> Best regards,
> Marcin Haba (gani)
>  
> Also: on what operating systems there is a tray monitor available that then 
> uses the bacula-mon interface? Is this a Windows-specific thing? (So this 
> interface is for localhost use only I I should find the corresponding secret 
> string in the config for the tray app?). I am using mostly macOS and Debian, 
> some FreeBSD - is this relevant on these platforms? (I don’t have Windows at 
> all. Lucky me.)
> 
> Thanks again,
>  J/C
> 
> 
>> On 15. Jun 2022, at 16:39, Josh Fisher > <mailto:jfis...@jaybus.com>> wrote:
>> 
>> 
>> On 6/15/22 08:12, Justin Case wrote:
>>> Hi all,
>>> 
>>> when installing the bacula-client packageon Debian I end up i /etc/bacula 
>>> with
>>> bacula-fd.conf
>>> and
>>> bconsole.conf
>>> 
>>> I have 2 questions, that are still open although I searched through the 
>>> main documentation and read the chapters containing bconsole.conf:
>>> 
>>> (1) In bacula-fd.conf I understand how to set up the FileDaemon resource. I 
>>> also understand how to set up the Director resource for bacula-dir, so that 
>>> the actual backup works. There always is a second Director resource for 
>>> bacula-mon and I do not understand what it used for and where I find the 
>>> secret string in Baculum UI on the Bacula Director machine to put in 
>>> Password for the Director resource on the Bacula FD client machine.
>> 
>> 
>> The second bacula-mon Director interface is for the tray monitor app running 
>> on the client machine. The tray monitor acts as a simplified director only 
>> for retrieving info from the bacula-fd daemon.
>> 
>>> (2) What is the use for the bconsole.conf file on the Bacula FD client 
>>> machine? Is it a means to allow the admin of the client machine to take a 
>>> look at the jobs on the Bacula Director machine concerning this client 
>>> machine? If I wanted this to work, where in Baculum UI on the Bacula 
>>> Director machine do I find the secret string to put in Password for the 
>>> Director resource in bcsonsole.conf on the client machine?
>> 
>> 
>> That is the configuration file 

Re: [Bacula-users] Q: ConnectToDirector not working as documented for FD Director directive

[Justin Case]

Thanks for picking this up. This is the complete bacula-fd.conf:

# File daemon Name to
#
#
# Copyright (C) 2000-2020 Kern Sibbald
# License: BSD 2-Clause; see file LICENSE-FOSS
#

#
# List Directors who are permitted to contact this File daemon
#
Director {
  Name = bacula-dir
  Password = “redacted"
#  DirPort = 9101
  Address = bacula-dir.lan.net
  ConnectToDirector = yes
  ReconnectionTime = 40 min
}
#
# Restricted Director, used by tray-monitor to get the
#   status of the file daemon
#
#Director {
#  Name = bacula-mon
#  Password = “redacted"
#  Monitor = yes
#}

#
# "Global" File daemon configuration specifications
#
FileDaemon {  # this is me
  Name = redacted-fd
#  FDport = 9102  # where we listen for the director
  WorkingDirectory = /var/lib/bacula
  Pid Directory = /run/bacula
  Maximum Concurrent Jobs = 20
  Plugin Directory = /usr/lib/bacula
#  FDAddress = 127.0.0.1
}

# Send all messages except skipped files back to Director
Messages {
  Name = Standard
  director = bacula-dir = all, !skipped, !restored
}


> On 17. Jun 2022, at 14:16, Josh Fisher  wrote:
> 
> Did you follow the error message's suggestion? "Perhaps you left the trailing 
> brace off of the previous resource" (or A previous resource in the 
> bacula-fd.conf file) We would need to see the entire bacula-fd.conf file in 
> order to help.
> 
> On 6/15/22 22:36, Justin Case wrote:
>> Hi there,
>> 
>> I have a client behind a firewall and the director is unable to connect to 
>> this client, but the client may connect to the director and to the SD.
>> 
>> In the main manual is documented that one should to this:
>> 
>> on the Director side in the Client directive:
>>   AllowFDConnections = yes
>> 
>> on the FD side in the Director directive:
>> Director {
>>   Name = bacula-dir
>>   Password = “redacted”
>> #  DirPort = 9101
>>   Address = bacula-dir.lan.net
>>   ConnectToDirector = yes
>>   ReconnectionTime = 40 min
>> }
>> 
>> I have done this, but the bacula-fd complains:
>> 
>> bacula-fd: ERROR TERMINATION at parse_conf.c:1157
>> Config error: Keyword "ConnectToDirector" not permitted in this resource.
>> Perhaps you left the trailing brace off of the previous resource.
>> : line 22, col 20 of file /etc/bacula/bacula-fd.conf
>>   ConnectToDirector = yes
>> 
>> Actually it would also complain about DirPort in the same way if I remove 
>> the comment character.
>> 
>> It looks like the documentation does not fit the behaviour of the Debian 
>> bacula-client package.
>> 
>> What can I do now?
>> 
>> Best
>>  J/C
>> 
>> 
>> 
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: ConnectToDirector not working as documented for FD Director directive

[Justin Case]

Yes, I tried to make sense of this error message, but there is no other 
directive above director in this config file (see my other email, I have 
attached the config file there).

> On 17. Jun 2022, at 14:16, Josh Fisher  wrote:
> 
> Did you follow the error message's suggestion? "Perhaps you left the trailing 
> brace off of the previous resource" (or A previous resource in the 
> bacula-fd.conf file) We would need to see the entire bacula-fd.conf file in 
> order to help.



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: regarding bconsole.conf on Bacula client machines

[Justin Case]

Dear Marcin, thank you so much for you time, I tried it, here are my 
experiences:

> On 17. Jun 2022, at 05:17, Marcin Haba  wrote:
> 
> Main menu -> Director -> Tab: Configure director -> Director
> 

Putting this pw in bconsole.config on the FD machine allows me successfully to 
access the director by using bconsole command on the FD machine. 

> If you want to use in bconsole.conf file a connection with using a Console, 
> then you need to look for the password to:
> 
> Main menu -> Director -> Tab: Configure director -> Console

Putting this pw in bconsole.config on the FD machine does NOT allow me to 
access the director by using bconsole command on the FD machine: 

Connecting to Director bacula-dir.lan.net:9101
Director authorization problem.
Most likely the passwords do not agree.


> For the password for the Tray Monitor configuration you need to look:

I do not really nkow what the Tray monitor is as I use Baculum on the 
bacula-dir = bacula-sd Linux host through firefox on my macos laptop. I have no 
idea where a tray monitor would be usefully located in my setup - because I 
still don’t have much experience and the full picture about bacula - yet.

So thank you for this information, I will archive this for when I may need it:

> 
> for FD password:
> 
> Main menu -> Clients -> [ select the client ] -> Tab: Configure file daemon 
> -> Section: Director
> 
> for SD password:
> 
> Main menu -> Storage -> [ select the storage ] -> Tab: Configure storage 
> daemon -> Section: Director
> 
> for DIR password:
> 
> Main menu -> Director -> Tab: Configure director -> Console
> 
> Best regards,
> Marcin Haba (gani)
> 
> On Thu, 16 Jun 2022 at 13:11, Justin Case  wrote:
> Thank you Marcin,
> 
> I wasn’t aware the Baculum could also be installed on a FD host. Interesting 
> to know.
> Actually my question was not clearly phrased as I did mean it. I did not mean 
> to look the password up on the FD host side in Baculum.
> 
> What I did mean was, where in Baculum on the director host would I find the 
> secret strings to use in the bconsole file director directive on the FD host 
> side. The same for the secret string for the monitor director directive in 
> the bacula-fd.conf on the FD machine. The initial setup suggests that both 
> secret strings are not identical to each other.
> 
> Sorry for the unclear phrasing.
> Best
>  J/C
> 
>> On 16. Jun 2022, at 06:58, Marcin Haba  wrote:
>> 
>> Hello Justin,
>> 
>> On Wed, 15 Jun 2022 at 16:55, Justin Case  wrote:
>> Thanks, I understand.
>> Still the partial questions are open where I find the corresponding secret 
>> strings in Baculum. I am not complaining, just want to make explicit that 
>> not all parts of the questions are answered yet.
>> 
>> Where ein Baculum do I find the bconsole secret for the bcsonsole config 
>> file? I understand that it must be somehow possible to restrict the bconsole 
>> access using ACLs. I am just trying to understand where in the most basic 
>> config I would find the secret (which menu path in Baculum).
>> 
>> This password is stored in files in two places: in your file daemon config 
>> and in tray monitor config. If you want to see the password on the Baculum 
>> interface, you need to install Baculum API on the file daemon host with 
>> configuration feature enabled (bfdjson configured). After you need to 
>> connect this host to your Baculum Web. At the end you need to go to:
>> 
>> Main menu -> Clients -> [ select the client ] -> Tab: Configure file daemon 
>> -> Section: Director
>> 
>> If you have everything (Bacula and Baculum API/Web) already configured on 
>> one host with the configuration feature enabled in API, you don't need to 
>> add any additional API host.
>> 
>> In the same way you can monitor storage daemons and directors. In this case 
>> the director configuration for the monitor connection will be in:
>> 
>> Main menu -> Director -> Tab: Configure director -> Console
>> 
>> and storage daemon configuration for the monitor will be in:
>> 
>> Main menu -> Storage -> [ select the storage ] -> Tab: Configure storage 
>> daemon -> Section: Director
>> 
>> Best regards,
>> Marcin Haba (gani)
>>  
>> Also: on what operating systems there is a tray monitor available that then 
>> uses the bacula-mon interface? Is this a Windows-specific thing? (So this 
>> interface is for localhost use only I I should find the corresponding secret 
>> string in the config for the tray app?). I am using mostly macOS and Debian, 
>> 

Re: [Bacula-users] Q: ConnectToDirector not working as documented for FD Director directive

[Justin Case]

That is the normal debian package “bacula-client”. Which other package would I 
use instead?

> On 17. Jun 2022, at 21:08, Josh Fisher  wrote:
> 
> You must have an old version of bacula-fd on the client. The 
> ConnectToDirector was implemented in version 11.0.
> 
> 
> On 6/17/22 10:00, Justin Case wrote:
>> Yes, I tried to make sense of this error message, but there is no other 
>> directive above director in this config file (see my other email, I have 
>> attached the config file there).
>> 
>>> On 17. Jun 2022, at 14:16, Josh Fisher  wrote:
>>> 
>>> Did you follow the error message's suggestion? "Perhaps you left the 
>>> trailing brace off of the previous resource" (or A previous resource in 
>>> the bacula-fd.conf file) We would need to see the entire bacula-fd.conf 
>>> file in order to help.
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: ConnectToDirector not working as documented for FD Director directive

[Justin Case]

The Debian FDs are 9.x, the DIR and SD are 11.x.
It is just 1 Debian FD that needs ConnectToDirector, so I guess I just need to 
upgrade the FD on this one host.

> On 19. Jun 2022, at 13:49, Josh Fisher  wrote:
> 
> Debs are available from the www.bacula.org website. You can build from 
> source. I'm not sure what version is in the normal Debian repositories, but I 
> think it must be older than 11.0. You can see the version by running 
> bacula-fd -? from the command line. But to use the ConnectToDirector feature, 
> you will have to upgrade to version 11.x on all clients and Director and 
> Storage daemons.
> 
> 
> On 6/17/22 17:35, Justin Case wrote:
>> That is the normal debian package “bacula-client”. Which other package would 
>> I use instead?
>> 
>>> On 17. Jun 2022, at 21:08, Josh Fisher  wrote:
>>> 
>>> You must have an old version of bacula-fd on the client. The 
>>> ConnectToDirector was implemented in version 11.0.
>>> 
>>> 
>>> On 6/17/22 10:00, Justin Case wrote:
>>>> Yes, I tried to make sense of this error message, but there is no other 
>>>> directive above director in this config file (see my other email, I have 
>>>> attached the config file there).
>>>> 
>>>>> On 17. Jun 2022, at 14:16, Josh Fisher  wrote:
>>>>> 
>>>>> Did you follow the error message's suggestion? "Perhaps you left the 
>>>>> trailing brace off of the previous resource" (or A previous resource 
>>>>> in the bacula-fd.conf file) We would need to see the entire 
>>>>> bacula-fd.conf file in order to help.
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: one job for a group of "identical" machines using the same fileset?

[Justin Case]

Good to know, I guess for commercial use.

For my private homelab commercial plugins are out of reach. Or is there such a 
plugin for free?

Best
 J/C

> On 19. Jun 2022, at 21:27, Radosław Korzeniewski  
> wrote:
> 
> Hello,
> 
> śr., 15 cze 2022 o 18:22 Justin Case  <mailto:jus7inc...@gmail.com>> napisał(a):
> Hi there,
> 
> in the age of virtualization and cloud technology such as Kubernetes I would 
> assume that people might have a lot of VMs that use the same setup where they 
> may store container user data under the same path for each machine.
> 
> In Bacula all such technology has a dedicated plugin which handles all your 
> concerns very well. 
> For example you can backup all your VM in a single Host or Cluster with just 
> one job configuration.
> You can even go very Kubernetes way by selecting what containers to backup 
> using Labels attached to containers when using the Kubernetes plugin with a 
> single job configuration.
> 
> best regards
> -- 
> Radosław Korzeniewski
> rados...@korzeniewski.net <mailto:rados...@korzeniewski.net>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Bacula for TrueNAS Core?

[Justin Case]

To be 100% clear: the term “plugin” here is meant in the context of TrueNAS 
Core, so a plugin FOR TrueNAS Core (not a plugin for Bacula).

> On 19. Jun 2022, at 21:42, Radosław Korzeniewski  
> wrote:
> 
> Hello,
> 
> śr., 15 cze 2022 o 12:50 Justin Case  <mailto:jus7inc...@gmail.com>> napisał(a):
> Or is there actually a Bacula Plugin for TrueNAS Core and I just did not look 
> right and did not find it?
> 
> We need to commit to terminology as every person can understand different 
> questions here.
> Bacula has a feature called Plugins which allow Bacula to backup and restore 
> different components using these plugins.
> 
> So, in my understanding the term: "a Bacula Plugin for TrueNAS Core" means a 
> feature for Bacula to be able to backup and restore data (files, 
> configuration, etc.) available on your TrueNAS server. This is a case when 
> your TrueNAS box flames in fire then you will be able to recover all your 
> data to the new TrueNAS box. 
> 
> But reading on the internet what iX-Systems Plugins are I think it is not 
> what I understand with the above term.
> 
> best regards
> -- 
> Radosław Korzeniewski
> rados...@korzeniewski.net <mailto:rados...@korzeniewski.net>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: one job for a group of "identical" machines using the same fileset?

[Justin Case]

Interesting, ist there an overview of available Community plugins, or where 
would I best start to look around?

> On 20. Jun 2022, at 10:37, Radosław Korzeniewski  
> wrote:
> 
> Hello,
> 
> niedz., 19 cze 2022 o 23:29 Justin Case  <mailto:jus7inc...@gmail.com>> napisał(a):
> Good to know, I guess for commercial use.
> 
> For my private homelab commercial plugins are out of reach. Or is there such 
> a plugin for free?
> 
> Some plugins are available with the Community version. Others are non-free, 
> proprietary, you need to purchase.
> 
> best regards
> -- 
> Radosław Korzeniewski
> rados...@korzeniewski.net <mailto:rados...@korzeniewski.net>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Bacula for TrueNAS Core?

[Justin Case]

Thats right. Such a plugin is something like a wrapper/installer that creates a 
jail environment for an existing application and hopefully provides a nice 
interface. So in the case of Bacula it would be a jail where all components are 
installed, so that one could use the director there to pull data in from 
clients and store them on ZFS, or, use it as a bacula-fd to backup zfs datasets 
to a third bacula-sd.

> On 20. Jun 2022, at 10:33, Radosław Korzeniewski  
> wrote:
> 
> Hello,
> 
> niedz., 19 cze 2022 o 23:45 dmitri maziuk  > napisał(a):
> On 2022-06-19 2:42 PM, Radosław Korzeniewski wrote:
> 
> > But reading on the internet what iX-Systems Plugins are I think it is 
> > not what I understand with the above term.
> 
> TrueNAS is a specialized version of OpenBSD that comes with web-based UI 
> to ZFS storage subsystem.
> 
> I know what TrueNAS is. :)
>  
> A plugin is a binary packaged for install via 
> said UI.
> 
> So it is a TrueNAS Plugin, because it extends TrueNAS functionality, right?
>  
> best regards
> -- 
> Radosław Korzeniewski
> rados...@korzeniewski.net 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: one job for a group of "identical" machines using the same fileset?

[Justin Case]

Thanks Phil,

I have implemented exactly this in the meanwhile.

Best
 J/C

> On 20. Jun 2022, at 17:08, Phil Stracchino  wrote:
> 
> On 6/20/22 06:33, Justin Case wrote:
>> Interesting, ist there an overview of available Community plugins, or where 
>> would I best start to look around?
>>> On 20. Jun 2022, at 10:37, Radosław Korzeniewski >> <mailto:rados...@korzeniewski.net>> wrote:
>>> 
>>> Hello,
>>> 
>>> niedz., 19 cze 2022 o 23:29 Justin Case >> <mailto:jus7inc...@gmail.com>> napisał(a):
>>> 
>>>Good to know, I guess for commercial use.
>>> 
>>>For my private homelab commercial plugins are out of reach. Or is
>>>there such a plugin for free?
> 
> 
> FYI, what I do for this application is I have a JobDef for all of the 
> machines that use that fileset, then a Job for each one that basically just 
> specifies that set of defaults, the job name, and the client hostname.
> 
> 
> 
> -- 
>  Phil Stracchino
>  Babylon Communications
>  ph...@caerllewys.net
>  p...@co.ordinate.org
>  Landline: +1.603.293.8485
>  Mobile:   +1.603.998.6958
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Debian Debs only for old Debians, Debian 11 is current, maybe also build Debs for them? was: Q: ConnectToDirector not working as documented for FD Director directive

[Justin Case]

Oh wow, the 11.0.2 Debs are for Debian 10 or older. Debian 10 already is quite 
old and has gone home to die soon.

Debian 11 is the train we are riding.

Have there been announcements when the Debs will be made available for Debian 
11, too?

Cheers
 J/C

> On 19. Jun 2022, at 13:49, Josh Fisher  wrote:
> 
> Debs are available from the www.bacula.org website. You can build from 
> source. I'm not sure what version is in the normal Debian repositories, but I 
> think it must be older than 11.0. You can see the version by running 
> bacula-fd -? from the command line. But to use the ConnectToDirector feature, 
> you will have to upgrade to version 11.x on all clients and Director and 
> Storage daemons.
> 
> 
> On 6/17/22 17:35, Justin Case wrote:
>> That is the normal debian package “bacula-client”. Which other package would 
>> I use instead?
>> 
>>> On 17. Jun 2022, at 21:08, Josh Fisher  wrote:
>>> 
>>> You must have an old version of bacula-fd on the client. The 
>>> ConnectToDirector was implemented in version 11.0.
>>> 
>>> 
>>> On 6/17/22 10:00, Justin Case wrote:
>>>> Yes, I tried to make sense of this error message, but there is no other 
>>>> directive above director in this config file (see my other email, I have 
>>>> attached the config file there).
>>>> 
>>>>> On 17. Jun 2022, at 14:16, Josh Fisher  wrote:
>>>>> 
>>>>> Did you follow the error message's suggestion? "Perhaps you left the 
>>>>> trailing brace off of the previous resource" (or A previous resource 
>>>>> in the bacula-fd.conf file) We would need to see the entire 
>>>>> bacula-fd.conf file in order to help.
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: How to define a working storage for file volumes across more than 1 device?

[Justin Case]

Hello dear all,

Abstractly spoken I need a concept for my Tier 2 to be able to write file 
volumes to more than just 1 (HDD) Device (or MediaType), filling the HDDs up 
one after the other. By this I mean that the individual HDDs are -NOT- 
abstracted away by a logical layer device such as some RAID level (in such a 
case we would just have 1 Device (or MediaType)).

(How) can this be done with Bacula?

To give you a different perspective on the problem that I am trying to solve:

If I use physical tape drives and autochangers I can always buy more tapes to 
expand my storage. If the changer would only work with a limited set of tapes 
(lets say tapes in a magazine that the autochanger can mechanically change 
without human/manual help) it is more like the situation with volumes in files 
instead of tapes. The autochanger can write to volumes until the HDD is full, 
then the auto changing is limited to the set of files held by this HDD. New 
backups can only be written if volumes get recycled.

And then comes the day when the HDD is full and the retention does not yet 
allow for recycling.
This is where I am today.

What I tried so far: (- and it does not work for me-)   (at the bottom of 
this email please find attached all my directives for storage for my Director 
und my SD (both on the same machine, same container).

I have structured my storage into tiers, where Tier 1 is held on internal SATA 
HDDs and Tier 2 is held on external HDDs connected via USB3. 

I would like to focus on Tier 2 storage for this posting (Tier 1 is not a 
problem as it is held on a HDD array that can be expanded by replacing smaller 
HDDs with larger HDDs without impacting existing file volumes.)

Tier 2 storage is declared as follows in the director:

Storage {
  Name = "tier2-storage"
  SdPort = 9103
  Address = "redacted"
  Password = "redacted"
  Device = "tier2-ext-usb-changer"
  MediaType = "tier2-ext-usb1-folder1-file"
  Autochanger = yes
  MaximumConcurrentJobs = 10
}
 
The device tier2-ext-usb1-folder1-file is an autochanger that has access to two 
file volume changer devices, where /mnt/tier2-01 is on USB HDD 1 and 
/mnt/tier2-02 is on USB HDD 2. These HDDs 1 and 2 are held in the same USB 
case, but are addressed individually, they are explicitly not combined in a 
RAID and (I would like to keep it this way).

Autochanger {
Name = "tier2-ext-usb-changer"
Device = "Tier2-FileChgr1-Dev1"
Device = "Tier2-FileChgr1-Dev2"
ChangerDevice = "/dev/null"
ChangerCommand = "/dev/null”
}

Device {
Name = "Tier2-FileChgr1-Dev1"
MediaType = "tier2-ext-usb1-folder1-file"
DeviceType = "File"
ArchiveDevice = "/mnt/tier2-01"
RemovableMedia = no
RandomAccess = yes
AutomaticMount = yes
LabelMedia = yes
AlwaysOpen = no
Autochanger = yes
MaximumConcurrentJobs = 5
}

Device {
Name = "Tier2-FileChgr1-Dev2"
MediaType = "tier2-ext-usb2-folder1-file"
DeviceType = "File"
ArchiveDevice = "/mnt/tier2-02"
RemovableMedia = no
RandomAccess = yes
AutomaticMount = yes
LabelMedia = yes
AlwaysOpen = no
Autochanger = yes
MaximumConcurrentJobs = 5
}

As in the Director Storage directive declared due to MediaType = 
"tier2-ext-usb1-folder1-file” merely HDD 1 is used (and the MediaType directive 
is mandatory for Storage in the Director).

Is there a way to allow the autochanger tier2-ext-usb-changer to also create 
und use volumes on HDD2? This autochanger actually comprises of 2 “File” 
devices, but now can only use one of them due to the different media types (and 
I understand that the changing is meant regarding the volumes within each 
“File" device, meaning for file volumes, that it can create/access the file 
volumes within a given folder. What I imagine is that it could use file volumes 
from any of the 2 Devices Tier2-FileChgr1-Dev1 and Tier2-FileChgr1-Dev2).

If there is no way to get this to work as I imagine it?

If not so, what is the correct way to handle my current situation: HDD 1 is 
full now, no further volumes can be written to HDD1, only existing volumes on 
HDD 1 could be recycled, but the retention time does not yet allow this. 

How can I expand my USB-based Tier 2 storage to hold further volumes? 
Note: I already have HDD 2 there in the same USB case waiting to be filled. 
Adding further HDDs in the same USB case is not a problem, but each HDD must be 
addressed individually (the USB case intentionally is not using a RAID 
controller).

Coming back to the tape magazine analogy, I now have an autochanger where 1 
magazine is full and it has a second magazine with empty tapes, but both 
magazines have a different MediaType. Is there a chance that the second 
magazine gets used?

All the best,
 J/C


For completeness sake all relevant directives for the described situation:


bacula-sd.conf


Storage {
Name = "bacula-sd"
WorkingDirectory = "/opt/bacula/working"
PidDirectory = "/opt/bacula/working"
PluginDirectory = "/opt/bacula/plugins"
MaximumConcurre

Re: [Bacula-users] Q: How to define a working storage for file volumes across more than 1 device?

[Justin Case]

Thanks, great pointer, I will look into that.

> On 27. Jun 2022, at 17:14, Bill Arlofski via Bacula-users 
>  wrote:
> 
> On 6/27/22 04:41, Justin Case wrote:
>> Hello dear all,
>> 
>> Abstractly spoken I need a concept for my Tier 2 to be able to write file 
>> volumes to more than just 1 (HDD) Device (or MediaType), filling the HDDs up 
>> one after the other. By this I mean that the individual HDDs are -NOT- 
>> abstracted away by a logical layer device such as some RAID level (in such a 
>> case we would just have 1 Device (or MediaType)).
>> 
>> (How) can this be done with Bacula?
> 
> Hello Justin,
> 
> You are looking for Josh's 'vchanger'
> 
> 
> Hope this helps!
> Bill
> 
> --
> Bill Arlofski
> w...@protonmail.com
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: How to define a working storage for file volumes across more than 1 device?

[Justin Case]

These HDDs were taken out of an array and replaced with bigger ones. They may 
have different sizes. I want to have only one big USB case where I can flexibly 
add/remove HDDs for different use cases, there are not only the Bacula Tier 2 
HDDs. I want to keep it flexible.

> On 28. Jun 2022, at 12:12, Radosław Korzeniewski  
> wrote:
> 
> Hello,
> 
> pon., 27 cze 2022 o 13:22 Justin Case  <mailto:jus7inc...@gmail.com>> napisał(a):
> By this I mean that the individual HDDs are -NOT- abstracted away by a 
> logical layer device such as some RAID level
> 
> Just curious why?
> 
> --
> Radosław Korzeniewski
> rados...@korzeniewski.net <mailto:rados...@korzeniewski.net>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: How to define a working storage for file volumes across more than 1 device?

[Justin Case]

Keep in mind: this is a homelab environment for private use, not a professional 
one with paying customers. Different mindset.

> On 28. Jun 2022, at 12:12, Radosław Korzeniewski  
> wrote:
> 
> Hello,
> 
> pon., 27 cze 2022 o 13:22 Justin Case  <mailto:jus7inc...@gmail.com>> napisał(a):
> By this I mean that the individual HDDs are -NOT- abstracted away by a 
> logical layer device such as some RAID level
> 
> Just curious why?
> 
> --
> Radosław Korzeniewski
> rados...@korzeniewski.net <mailto:rados...@korzeniewski.net>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: How to define a working storage for file volumes across more than 1 device?

[Justin Case]

Thats not what I said. It shouldn’t matter anyways.

> On 28. Jun 2022, at 12:28, Radosław Korzeniewski  
> wrote:
> 
> Hello,
> 
> wt., 28 cze 2022 o 12:21 Justin Case  <mailto:jus7inc...@gmail.com>> napisał(a):
> Keep in mind: this is a homelab environment for private use, not a 
> professional one with paying customers. Different mindset.
> 
> I do not understand. I never pay for RAID in my homelab for private use. Why 
> do you have to pay for it?
> Just curious.
> -- 
> Radosław Korzeniewski
> rados...@korzeniewski.net <mailto:rados...@korzeniewski.net>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: ConnectToDirector not working as documented for FD Director directive

[Justin Case]

I eventually removed the 9.X FD (from the Debian standard repos) from this 
Debian machine and installed the 13.X FD (from the Community Edition Debs). 
Running it manually works fine, it connects to the Director (11.x) and the 
backup job runs successfully.

There is one thing still unclear to me: 
From the Debian standard repos there is still the /etc/init.d/bacula-fd script 
to start and stop the FD. However, the pathes in the script fit where the 
standard repos install the FD (/usr/sbin/bacula-fd) and its configuration 
(/etc/bacula/bacula-fd). 
Surely I can edit /etc/init.d/bacula-fd to put in where the Community Edition 
Debs put the FD and config.

My questions is this: does a proper bacula-fd service script for /etc/init.d 
usually come with the Community Edition Debian Debs? (in my case it did not 
seem so)

All the best,
 J/C

> On 19. Jun 2022, at 14:36, Justin Case  wrote:
> 
> The Debian FDs are 9.x, the DIR and SD are 11.x.
> It is just 1 Debian FD that needs ConnectToDirector, so I guess I just need 
> to upgrade the FD on this one host.
> 
>> On 19. Jun 2022, at 13:49, Josh Fisher  wrote:
>> 
>> Debs are available from the www.bacula.org website. You can build from 
>> source. I'm not sure what version is in the normal Debian repositories, but 
>> I think it must be older than 11.0. You can see the version by running 
>> bacula-fd -? from the command line. But to use the ConnectToDirector 
>> feature, you will have to upgrade to version 11.x on all clients and 
>> Director and Storage daemons.
>> 
>> 
>> On 6/17/22 17:35, Justin Case wrote:
>>> That is the normal debian package “bacula-client”. Which other package 
>>> would I use instead?
>>> 
>>>> On 17. Jun 2022, at 21:08, Josh Fisher  wrote:
>>>> 
>>>> You must have an old version of bacula-fd on the client. The 
>>>> ConnectToDirector was implemented in version 11.0.
>>>> 
>>>> 
>>>> On 6/17/22 10:00, Justin Case wrote:
>>>>> Yes, I tried to make sense of this error message, but there is no other 
>>>>> directive above director in this config file (see my other email, I have 
>>>>> attached the config file there).
>>>>> 
>>>>>> On 17. Jun 2022, at 14:16, Josh Fisher  wrote:
>>>>>> 
>>>>>> Did you follow the error message's suggestion? "Perhaps you left the 
>>>>>> trailing brace off of the previous resource" (or A previous resource 
>>>>>> in the bacula-fd.conf file) We would need to see the entire 
>>>>>> bacula-fd.conf file in order to help.
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: Connection errors on daily backup with ConnectToDirector

[Justin Case]

Hi all

I have a client machine (FD 13) in a firewalled admin network that cannot be 
connected to by the Director (11), so it is using ConnectToDirector = yes, i.e 
the FD connects through the firewall to the Director.

The client machine is backed up to two different tiers, so there are two daily 
jobs backing up the machine. 
The backup to both tiers works fine, - however:

the first backup generates a larger number of error messages before it starts:

Error: getmsg.c:217 Malformed message: [bsockcore.c:265 Unable to connect to 
Director daemon on bacula-dir.lan.net:9101. ERR=Connection refused

Then the backup runs through fine, it says it terminates with warnings 
(although there are NO warnings in the log, but the aforementioned errors)

  Termination:Backup OK -- with warnings

The second backup later runs though without those errors.

It looks to me like the FD was initially not connected to the director, but 
then it it does successfully connect. I am not sure if this is the correct 
interpretation.

Generally its not nice to have errors in the log as I need to check for them 
every day..

Is there a way to better configure the FD or DIrector so that the errors do not 
occur?


The FD config:

Director {
  Name = bacula-dir
  Password = “redacted"
#  DirPort = 9101
  Address = bacula-dir-redacted
  ConnectToDirector = yes
  ReconnectionTime = 40 min
}

FileDaemon {  # this is me
  Name = redacted
#  FDport = 9102  # where we listen for the director
  WorkingDirectory = /var/lib/bacula
  Pid Directory = /run/bacula
  Maximum Concurrent Jobs = 20
  Plugin Directory = /usr/lib/bacula
#  FDAddress = 127.0.0.1
}




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Connection errors on daily backup with ConnectToDirector

[Justin Case]

Thanks for your interest, Bill, I will respond to your requests inline:

> On 20. Jul 2022, at 01:11, Bill Arlofski via Bacula-users 
>  wrote:
> 
> On 7/19/22 16:55, Justin Case wrote:
> >
>> Error: getmsg.c:217 Malformed message: [bsockcore.c:265 Unable to connect to 
>> Director daemon on bacula-dir.lan.net:9101. ERR=Connection refused
> 
> Can you show the whole log line where this is displayed?
bacula-dir JobId 1595: Error: getmsg.c:217 Malformed message: [bsockcore.c:265 
Unable to connect to Director daemon on bacula-dir.lan.net:9101. ERR=Connection 
refused
> Curious to know if it is coming from the FD called `redacted` (looks to be), 
> and if it is a "JobId 0" message or if it is
> actually associated to an actual backup jobid.

It is coming from the director and is associated to the actual job ID.

> Maybe just increasing the `ReconnectionTime` to something very big would 
> prevent this?

You mean the job just starts at the unlucky moment when the reconnect happens? 
These messages happened every night since I enabled this FD to be backed up 
(just 2 nights so far). Isn’t it very low probability that the reconnect always 
happens just in the moment when the job starts?

> No idea, but now that I am curious, I think I will do a little testing here 
> later. :)
> 
> Best regards,
> Bill
> 
> --
> Bill Arlofski
> w...@protonmail.com
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Connection errors on daily backup with ConnectToDirector

[Justin Case]

inline response continued:

> On 20. Jul 2022, at 01:38, Bill Arlofski via Bacula-users 
>  wrote:
> 
> On 7/19/22 16:55, Justin Case wrote:
>> Error: getmsg.c:217 Malformed message: [bsockcore.c:265 Unable to connect to 
>> Director daemon on bacula-dir.lan.net:9101. ERR=Connection refused
> 
> 
> Hello Justin,
> 
> My guess is that the FD is configured to call the Director, and it tries, but 
> fails either due to a firewall blocking it, or,

The firewall is _not_ blocking the FD connecting to the director. However, the 
firewall is blocking the Director from connecting to the FD.

> more likely, the Director has not been configured to allow this Client to 
> connect to the Director.
> 
> Then, when you run the job, the Director simply calls the client as normal.
> 
> In the Client{} resource on the Director for this Client, you need:
> 8<
> AllowFDConnections = yes
> 8<

Excuse me, I forgot to show the config of the client in the director (and yes, 
I allow FD connections):

Client {
 Name = "myfd"
 Address = “myfd-fqdn"
  FdPort = 9102
 Password = “redacted
  Catalog = "MyCatalog"
  FileRetention = 5184000
  JobRetention = 15552000
  AutoPrune = yes
  SDCallsClient = no
  AllowFDConnections = yes
  MaximumConcurrentJobs = 2
}

Job {
 Name = “job1"
  Client = "myfd"
  JobDefs = "tier1"
  Enabled = yes
  AllowIncompleteJobs = no
  AllowDuplicateJobs = no
}
Job {
 Name = “job2"
 Client = “mfdy"
  JobDefs = "tier2"
  Enabled = yes
  AllowIncompleteJobs = no
  AllowDuplicateJobs = no
}

JobDefs {
  Name = "tier1"
  Type = "Backup"
  Level = "Full"
  Messages = "Standard"
  Storage = "tier1-storage"
  Pool = "full-tier1"
  FullBackupPool = "full-tier1"
  IncrementalBackupPool = "incr-tier1"
  Fileset = "various"
 Schedule = “sched1"
}

JobDefs {
  Name = "tier2"
  Type = "Backup"
  Level = "Full"
  Messages = "Standard"
  Storage = "tier2-storage"
  Pool = "full-tier2"
  FullBackupPool = "full-tier2"
  IncrementalBackupPool = "incr-tier2"
  Fileset = "various"
  Schedule = "sched2"
}

> See, if makes no sense to me that if the Client is not connected, or cannot 
> connect that the Director can (or would) call out
> to it to start a job with "AllowFDConnections = yes" set for this client.

> The connection should be already established By the Client -> to the Director 
> and there would be no connection refused
> messages generated by the client - w
> hich is why I asked about the JobId 0 possibility for these log entries.

Sure. The FD can always connect.

> BTW, I just tested this on a client that is local to the Director (ie: no 
> firewalls between the two) and it works as
> expected. No errors.
> 
> If I run the Director in foreground and debug mode, I can see after a few 
> seconds of it starting up, an incoming connection
> from the Client I configured, and then the authentication and finally the 
> inbound connection allowed:
> 8<
> [...snip a lot of Director startup stuff...]
> bacula-dir: authenticate.c:267-0 authenticate user agent: Hello speedy-fd 
> fdcallsdir 1 tlspsk=100
> bacula-dir: authenticate.c:286-0 *** No Dir compression to UA
> bacula-dir: authenticatebase.cc:335-0 TLSPSK Local need 100
> bacula-dir: authenticatebase.cc:365-0 TLSPSK Remote need 100
> bacula-dir: authenticatebase.cc:563-0 TLSPSK Start PSK
> bacula-dir: authenticatebase.cc:365-0 TLSPSK Remote need 100
> bacula-dir: authenticatebase.cc:563-0 TLSPSK Start PSK
> bacula-dir: bnet.c:96-0 TLS serve
> r negotiation established.
> bacula-dir: bnet.c:155-0 TLS client negotiation established.
> bacula-dir: cram-md5.c:68-0 send: auth cram-md5 challenge 
> <695684628.1658273108@bacula-dir> ssl=0
> bacula-dir: cram-md5.c:132-0 cram-get received: auth cram-md5 
> <1878454514.1658273108@speedy-fd> ssl=0
> bacula-dir: cram-md5.c:132-0 cram-get received: auth cram-md5 
> <40069706.1658273108@speedy-fd> ssl=0
> bacula-dir: cram-md5.c:156-0 sending resp to challenge: hG/bnB+/sA+oS8RQ+S+nEA
> bacula-dir: cram-md5.c:156-0 sending resp to challenge: 9+BJT+cVo5tAb6/dZ/+9nB
> bacula-dir: cram-md5.c:75-0 send: auth cram-md5 challenge 
> <472182750.1658273108@bacula-dir> ssl=0
> bacula-dir: authenticate.c:368-0 FDCallsDir OK for speedy-fd
> 8<
> 
> The first line shows the client connecting inbound to the Director and that 
> 'fdcallsdir = 1' for this client called
> 'speedy-fd' which allows the inbound attempt and authentication processes, 
> then the last line shows the connection has been
> allowed and is OK.
> 
> 
> Hope
> this helps,
> Bill
> 
> --
> Bill Arlofski
> w...@protonmail.com
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Connection errors on daily backup with ConnectToDirector

[Justin Case]

> On 20. Jul 2022, at 12:06, Martin Simmons  wrote:
> 
> There are two levels of error here: firstly, the "Connection refused" occurs
> when the client tries to connect to the Director; then that eventually
> succeeds and the error message is sent back to the Director when it doesn't
> expect it which leads to the "Malformed message" error.
> 
> Does the client machine have permanent access to the Director machine (via the
> firewall)?  It looks to me like it might only have intermittent access, so
> gets "Connection refused" for some time before it connects.

It has permanent access. However, the machine running the Dir and SD is busy 
with local FD jobs (as in “Dir, SD and FD on same host”) when also the jobs for 
remote FDs are initiated. The jobs I had so far for remote hosts just waited 
with a cogwheel showing in Baculum. The job for the FD with ConnectToDirector 
also shows the cogwheel, but it seems at the point in time when its turn has 
come, the connection messages are occuring.

> Alternatively, does "bacula-dir.lan.net" resolve to more than one address on
> the client machine?  If so, maybe some of the addresses fail to connect.

Nope. This is a CNAME and it resolves to exactly 1 FQDN and exactly 1 IP 
address.

> Also, does "(FD 13)" and "Director (11)" mean that the client machine is
> running Bacula version 13 and the Director is running Bacula version 11?  

This.

> That
> is not a supported configuration (it must be Director = Storage >= Client).

Oh well I did not know this. Then I would have to go down to 11 for this FD. 
Will do so later today.

> 
> __Martin
> 
> 
>>>>>> On Wed, 20 Jul 2022 00:55:56 +0200, Justin Case said:
>> 
>> Hi all
>> 
>> I have a client machine (FD 13) in a firewalled admin network that cannot be 
>> connected to by the Director (11), so it is using ConnectToDirector = yes, 
>> i.e the FD connects through the firewall to the Director.
>> 
>> The client machine is backed up to two different tiers, so there are two 
>> daily jobs backing up the machine. 
>> The backup to both tiers works fine, - however:
>> 
>> the first backup generates a larger number of error messages before it 
>> starts:
>> 
>> Error: getmsg.c:217 Malformed message: [bsockcore.c:265 Unable to connect to 
>> Director daemon on bacula-dir.lan.net:9101. ERR=Connection refused
>> 
>> Then the backup runs through fine, it says it terminates with warnings 
>> (although there are NO warnings in the log, but the aforementioned errors)
>> 
>>  Termination:Backup OK -- with warnings
>> 
>> The second backup later runs though without those errors.
>> 
>> It looks to me like the FD was initially not connected to the director, but 
>> then it it does successfully connect. I am not sure if this is the correct 
>> interpretation.
>> 
>> Generally its not nice to have errors in the log as I need to check for them 
>> every day..
>> 
>> Is there a way to better configure the FD or DIrector so that the errors do 
>> not occur?
>> 
>> 
>> The FD config:
>> 
>> Director {
>>  Name = bacula-dir
>>  Password = “redacted"
>> #  DirPort = 9101
>>  Address = bacula-dir-redacted
>>  ConnectToDirector = yes
>>  ReconnectionTime = 40 min
>> }
>> 
>> FileDaemon {  # this is me
>>  Name = redacted
>> #  FDport = 9102  # where we listen for the director
>>  WorkingDirectory = /var/lib/bacula
>>  Pid Directory = /run/bacula
>>  Maximum Concurrent Jobs = 20
>>  Plugin Directory = /usr/lib/bacula
>> #  FDAddress = 127.0.0.1
>> }
>> 
>> 
>> 
>> 
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Connection errors on daily backup with ConnectToDirector

[Justin Case]

Hi Martin please find my response inline:

> On 20. Jul 2022, at 20:39, Martin Simmons  wrote:
> 
>>>>>> On Wed, 20 Jul 2022 16:25:15 +0200, Justin Case said:
>> 
>>> On 20. Jul 2022, at 12:06, Martin Simmons  wrote:
>>> 
>>> There are two levels of error here: firstly, the "Connection refused" occurs
>>> when the client tries to connect to the Director; then that eventually
>>> succeeds and the error message is sent back to the Director when it doesn't
>>> expect it which leads to the "Malformed message" error.
>>> 
>>> Does the client machine have permanent access to the Director machine (via 
>>> the
>>> firewall)?  It looks to me like it might only have intermittent access, so
>>> gets "Connection refused" for some time before it connects.
>> 
>> It has permanent access. However, the machine running the Dir and SD is busy
>> with local FD jobs (as in “Dir, SD and FD on same host”) when also the jobs
>> for remote FDs are initiated. The jobs I had so far for remote hosts just
>> waited with a cogwheel showing in Baculum. The job for the FD with
>> ConnectToDirector also shows the cogwheel, but it seems at the point in time
>> when its turn has come, the connection messages are occuring.
> 
> It is not clear when the "Connection refused" messages actually occurred,
> because they are queued until a connection exists and then sent to that
> connection.  They probably occurred sometime before the connection existed.

I see.

> "Connection refused" comes from the OS errno ECONNREFUSED, which suggests a
> genuine networking problem at the point in time.  It usually means that the
> server is not listening on the target port, or some firewall is blocking it.

AFAIK the server is 
(a) running during the time when the jobs are started (obviously),
(b) the firewall block the Director from connecting to the FD. The FD may 
connect to the Director any time.
I have no idea whether the Director is listening permanently, though.

> Since you don't have a Schedule clause in the bacula-fd.conf, the bacula-fd
> will try to connect to the Director when the bacula-fd starts.  It will try to
> connect every 5 seconds until it succeeds.  

Why every 5 seconds?
The FD config has ReconnectionTime = 40 min
Let me remove this clause and restart the FD. Maybe it helps.

> Maybe it can't connect at that point for some reason?

I doubt it. AFAIK there is nothing there that would prevent this.

What I can imagine is that at some point in time the TCP connection from the FD 
to the Dir stops working (maybe some timeout in a firewall table?) and the FD 
only tried to reconnect in a 40 min interval. This could explain the problem.

I have now removed the ReconnectionTime = 40 min clause. If that does not help 
I will put in a schedule on the FD for the connection.

If none of this helps I will downgrade the FD from 13 to 11 (as the Dir is on 
11).

The curious thing is this: If I trigger the job manually at any point in time, 
I just works without any of the connection error messages.

> The problem is repeatable for me if I start the bacula-fd before starting the
> bacula-dir.
> 
> __Martin

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Connection errors on daily backup with ConnectToDirector

[Justin Case]

Bill, thanks for clarification. So the problem remains a mystery, so far.

> On 20. Jul 2022, at 21:33, Bill Arlofski via Bacula-users 
>  wrote:
> 
> 
>>>  It will try to connect every 5 seconds until it succeeds.
>> Why every 5 seconds?
>> The FD config has ReconnectionTime = 40 min
>> Let me remove this clause and restart the FD. Maybe it helps.
> 
> 
> Hello Justin,
> 
> Don't confuse the two different things.
> 
> On startup, the FD will immediately attempt to connect to the Director, and 
> if it fails, it will keep trying, every 5 seconds.
> 
> 
> The ReconnectionTime is defined as: ...the connection initiated by the 
> FileDeamon to the Director will be reinitialized at a regular interval 
> specified by the ReconnectionTime directive.
> 
> 
> So, after 40 minutes (the default) the FD will (apparently) drop the 
> connection and reconnect - at which point it would try every 5 seconds until 
> it succeeds.
> 
> I of course assume that if a job is running this ReconnectionTime 
> will/must/should/might be ignored until the next 40 minutes passes, etc. :)
> 
> 
> Best regards,
> Bill
> 
> -- 
> Bill Arlofski
> w...@protonmail.com
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Connection errors on daily backup with ConnectToDirector

[Justin Case]

Hey Bill, thanks for spending time on this!

> On 20. Jul 2022, at 21:46, Bill Arlofski via Bacula-users 
>  wrote:
> 
> 
> Justin,
> 
> I know what you told us, but I think we have a situation that I (and Martin) 
> described:

I understand your experiment, but it is not like that here.

> - FD cannot connect to Director due to firewall

It can.

> - Director CAN connect to FD (I know, I know... :)

It cannot.

> - Job starts, Director connects to FD and receives all the queued "Cannot 
> connect" messages
> - Job runs fine
> 
> 
> Here is how I tested:
> 
> - In my FD config I set in the the Director{} block:
> 
>  - ConnectToDirector = yes
>  - A BOGUS IP address for the `Address =` setting
> 
> 
> I killed and restarted the FD in foreground and debug mode, and I see that it 
> goes on to try to connect to an IP address that
> is not taken on my network
> 8<
> speedy-fd: events.c:48-0 Events: code=FD0001 daemon=speedy-fd ref=0x238e 
> type=daemon source=*Daemon* text=Filed startup
> 13.0.0 (04Jul22)
> speedy-fd: filed.c:296-0 filed: listening on port 9102
> speedy-fd: bnet_server.c:90-0 Addresses 0.0.0.0:9102
> speedy-fd: bsockcore.c:354-0 Current 10.1.1.99:9101 All 10.1.1.99:9101
> speedy-fd: bsockcor
> e.c:443-0 Could not connect to server Director daemon 10.1.1.99:9101. ERR=No 
> route to host
> speedy-fd: bsockcore.c:253-0 Unable to connect to Director daemon on 
> 10.1.1.99:9101. ERR=No route to host
> speedy-fd: bsockcore.c:354-0 Current 10.1.1.99:9101 All 10.1.1.99:9101
> speedy-fd: bsockcore.c:443-0 Could not connect to server Director daemon 
> 10.1.1.99:9101. ERR=No route to host
> speedy-fd: bsockcore.c:253-0 Unable to connect to Director daemon on 
> 10.1.1.99:9101. ERR=No route to host
> 8<
> 
> Meanwhile, from the Director, I do a `status client=` and BAMM.. Director 
> connects to Client and I get the FD's status -
> so a Job would also work in this manner.
> 
> 
> From your Director, can you try:

good thinking. This was the first thing I checked when I saw the errors, 
though. I usually try everything i can think of before I turn to the mailing 
list, but of course you cannot know what I tried, as I did not mention it.

> # telnet  9102

Connection refused

> And from the Client:
> 
> # telnet  9101

no telnet there, using netcat instead, the connection gets established. I can 
write stuff, after some “invalid keywords” the connection is closed by the 
director.

to be sure I tried again with other port numbers that have no daemons running. 
netcat returns immediately (due to port being closed).

> And show us the results?

see above.

In the mean while the schedule ran again.

today: no connection error messages.

OK OK, but why. What was different? I did some experiments earlier, so the job 
did run twice before.

Also I ran another longer running job on the other tier, but actually the 
problematic job did not queue up but ran through immediately (so both jobs ran 
simultaneously) and no errors were thrown.

When a few minutes ago the schedule started the longer running job also was 
started as “incremental” and had no files to be backed up (because it ran a few 
hours before and no changes had been made in the fileset.

Finally, I had commented out the Reconnect clause.

Hard to say what was the reason. 

I will observe whether tomorrow it will or will not throw connection  errors 
and will report back in both cases. And I will not make any experiments on 
Bacula before the schedule runs tomorrow.

> 
> Thank you!
> Bill
> 
> --
> Bill Arlofski
> w...@protonmail.com
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Connection errors on daily backup with ConnectToDirector

[Justin Case]

The last two scheduled runs of the job again had the connection errors (and 
again the the backup was still taken fine). yesterday I even ran the longer 
running job a few hours ahead to see if this was the reason why the connection 
error disappeared the other night - but that was not it. Also commenting out 
the reconnect clause didn’t make a difference.

the only two things I am aware I can do now to check it out further:

(1) use a connection schedule for the FD
(2) downgrade the FD from 13 to 11 (can this really be the cause?)

> On 20. Jul 2022, at 22:35, Justin Case  wrote:
> 
> 
> Hey Bill, thanks for spending time on this!
> 
>> On 20. Jul 2022, at 21:46, Bill Arlofski via Bacula-users 
>> > <mailto:bacula-users@lists.sourceforge.net>> wrote:
>> 
>> 
>> Justin,
>> 
>> I know what you told us, but I think we have a situation that I (and Martin) 
>> described:
> 
> I understand your experiment, but it is not like that here.
> 
>> - FD cannot connect to Director due to firewall
> 
> It can.
> 
>> - Director CAN connect to FD (I know, I know... :)
> 
> It cannot.
> 
>> - Job starts, Director connects to FD and receives all the queued "Cannot 
>> connect" messages
>> - Job runs fine
>> 
>> 
>> Here is how I tested:
>> 
>> - In my FD config I set in the the Director{} block:
>> 
>>  - ConnectToDirector = yes
>>  - A BOGUS IP address for the `Address =` setting
>> 
>> 
>> I killed and restarted the FD in foreground and debug mode, and I see that 
>> it goes on to try to connect to an IP address that
>> is not taken on my network
>> 8<
>> speedy-fd: events.c:48-0 Events: code=FD0001 daemon=speedy-fd ref=0x238e 
>> type=daemon source=*Daemon* text=Filed startup
>> 13.0.0 (04Jul22)
>> speedy-fd: filed.c:296-0 filed: listening on port 9102
>> speedy-fd: bnet_server.c:90-0 Addresses 0.0.0.0:9102
>> speedy-fd: bsockcore.c:354-0 Current 10.1.1.99:9101 All 10.1.1.99:9101
>> speedy-fd: bsockcor
>> e.c:443-0 Could not connect to server Director daemon 10.1.1.99:9101. ERR=No 
>> route to host
>> speedy-fd: bsockcore.c:253-0 Unable to connect to Director daemon on 
>> 10.1.1.99:9101. ERR=No route to host
>> speedy-fd: bsockcore.c:354-0 Current 10.1.1.99:9101 All 10.1.1.99:9101
>> speedy-fd: bsockcore.c:443-0 Could not connect to server Director daemon 
>> 10.1.1.99:9101. ERR=No route to host
>> speedy-fd: bsockcore.c:253-0 Unable to connect to Director daemon on 
>> 10.1.1.99:9101. ERR=No route to host
>> 8<
>> 
>> Meanwhile, from the Director, I do a `status client=` and BAMM.. 
>> Director connects to Client and I get the FD's status -
>> so a Job would also work in this manner.
>> 
>> 
>> From your Director, can you try:
> 
> good thinking. This was the first thing I checked when I saw the errors, 
> though. I usually try everything i can think of before I turn to the mailing 
> list, but of course you cannot know what I tried, as I did not mention it.
> 
>> # telnet  9102
> 
> Connection refused
> 
>> And from the Client:
>> 
>> # telnet  9101
> 
> no telnet there, using netcat instead, the connection gets established. I can 
> write stuff, after some “invalid keywords” the connection is closed by the 
> director.
> 
> to be sure I tried again with other port numbers that have no daemons 
> running. netcat returns immediately (due to port being closed).
> 
>> And show us the results?
> 
> see above.
> 
> In the mean while the schedule ran again.
> 
> today: no connection error messages.
> 
> OK OK, but why. What was different? I did some experiments earlier, so the 
> job did run twice before.
> 
> Also I ran another longer running job on the other tier, but actually the 
> problematic job did not queue up but ran through immediately (so both jobs 
> ran simultaneously) and no errors were thrown.
> 
> When a few minutes ago the schedule started the longer running job also was 
> started as “incremental” and had no files to be backed up (because it ran a 
> few hours before and no changes had been made in the fileset.
> 
> Finally, I had commented out the Reconnect clause.
> 
> Hard to say what was the reason. 
> 
> I will observe whether tomorrow it will or will not throw connection  errors 
> and will report back in both cases. And I will not make any experiments on 
> Bacula before the schedule runs tomorrow.
> 
>> 
>> Thank you!
>> Bill
>> 
>> --
>> Bill Arlofski
>> w...@protonmail.com <mailto:w...@protonmail.com>
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net 
>> <mailto:Bacula-users@lists.sourceforge.net>
>> https://lists.sourceforge.net/lists/listinfo/bacula-users 
>> <https://lists.sourceforge.net/lists/listinfo/bacula-users>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Connection errors on daily backup with ConnectToDirector

[Justin Case]

I think it is great support here from  you people!

Today I think I might have understood what is happening, and Bill’s 
explanations about what might be going on were probably correct in the core, 
but not in the details.

Let me try to lay out what I think is going on and where I had my problem 
understanding it in the first place:

After an update of syslog-ng the syslogging of the FD client host started to 
work (it was configured a long time ago but somehow it never worked before the 
update for the syslog-ng server that came in the last days) I began to see 
where and WHEN(!) the error messages originated.

It is - as you guys are saying - the FD generating these errors, which are 
logged without delay in my central syslog-ng server:

2022-07-25 12:57:30 
bsockcore.c:265 Unable to connect to Director daemon on 
bacula-dir.lan.net:9101. ERR=Connection refused

The eye-opener were the timestamps, which explained what is happening (more on 
that later).
My problem so far was that the error messages shown in Baculum had the 
timestamp of the Director when the Director sees the error messages, not when 
they happened!

25-Jul 22:00 bacula-dir JobId 1725: Error: getmsg.c:217 Malformed message: 
[bsockcore.c:265 Unable to connect to Director daemon on 
bacula-dir.lan.net:9101. ERR=Connection refused

Note the different timestamp. In the first message it is the timestamp of the 
FD client host when the error occurs there. In the second message you see the 
timestamp of the Director host when the first error message gets delivered from 
the FD to the Director.

So what you guys said is correct: the Director accepts the error messaged from 
the FD only when a job runs for the FD. Even if the FD connects to the Director 
many times during the day, the error messages are held back by the FD until a 
job actually runs and then they are ingested for the first job that runs on the 
current day. This also explains why there are no errors when a similar job runs 
shortly after to backup to the other tier storage

Because so far I was only seeing the Director timestamp I was misled that the 
error actually happens at the time when the job runs. I now understand that 
this is not correct, and I think you guys also mentioned it, but I didn’t pick 
it up consciously enough to understand what this means.

Now that I can see the timestamp from the FD when the errors actually happen on 
the FD host I can now confirm:

(1) the Director is definitely reachable for the FD at the time when the job 
runs (as I alway also stated), this is why the error messages show the 
timestamp of when the job runs, as it always is able to run due to availability 
of the Director.

(2) the Director is NOT reachable at some scheduled times each day when the 
contained is shut down for third party backup reasons (the firewall has nothing 
to do with this). And this is the time frame when the errors actually occur and 
can now be seen in syslog-ng.

I suppose if I now schedule the FD only connect to the Director when the job 
runs, the errors should go away. I will try this and report back.

One last thing is still unclear to me. Today I saw 455 connection errors in the 
Baculum Messages window, but only 38 connection errors in syslog-ng. This is 
weird, as I am (1) using syslog over TCP, and (2) I think I should see a higher 
or the same number of connection errors in syslog-ng as compared to in Baculum 
Messages window. However it is the over way around and considerably more errors 
on the Director side than on the FD side (syslog).
Can this be explained?

All the best,
 J/C
 

> On 25. Jul 2022, at 18:04, Martin Simmons  wrote:
> 
>> On Mon, 25 Jul 2022 15:50:15 +, Bill Arlofski said:
>> 
>> On Monday, July 25th, 2022 at 08:54, Martin Simmons =
>> wrote:
>>> 
>>> You could try running bacula-fd with debugging output. Unfortunately,
>>> it doesn't include timestamps, but you can do it like this:
>> 
>> Hey Martin, Not sure if this is recent or not, but:
>> 8<
>> $ /opt/comm-bacula/sbin/bacula-fd -?
>> Copyright (C) 2000-2022 Kern Sibbald.
>> 
>> Version: 13.0.0 (04 July 2022)
>> 
>> Usage: bacula-fd [-f -s] [-c config_file] [-d debug_level]
>> -c use  as configuration file
>> -d [,]  set debug level to , debug tags to 
>> 
>> -dt  print a timestamp in debug output<--=
>> -- TimeStamps
>> 
>> -f   run in foreground (for debugging)
>> -g   groupid
>> -k   keep readall capabilities
>> -m   print kaboom output (for debugging)
>> -P   do not create pid file
>> -s   no signals (for debugging)
>> -t   test configuration file and exit
>> -T   set trace on
>> -u   userid
>> -v   verbose user messages
>> -?   print this message.
>> 8<
> 
> Thanks, I didn't know that.
> 
> So this will be simpler:
> 
> bacul

[Bacula-users] Q: Baculum throws error that bconsole cannot connect

[Justin Case]

Hello all,

I am having a problem with Baculum currently. To explain the context I need to 
start somewhere else:

Baculum is running in a Docker container 
(pwa666/bacula-server:11-postgresql-latest) on my unRAID NAS, this container 
contains Baculum, Director, the SD and an FD for backing up local data. 
(https://hub.docker.com/pwa666/bacula-server)

As macOS TimeMachine stopped working properly after unRAID upgrade to 6.10.3 I 
disabled TimeMachine in unRAID and installed the TimeMachine container 
(https://github.com/mbentley/docker-timemachine). This container runs with a 
br0 network and the TimeMachine service is using a different IP than the unRAID 
server, but in the same subnet.
(Yes, macOS backup up twice, by Bacula and by TimeMachine)

Since I have this TimeMachine container Baculum threw an error on the UI that 
bconsole had problems connecting to the director at localhost:9101.

Consequently I replaced in bconsole.conf localhost with the FQDN of the 
director, which is a CNAME for the FQDN of the unRAID server:

bconsole.conf:

Director {
  Name = "bacula-dir"
  DIRport = 9101
  address = bacdir.foo.net
  Password = "redacted"
}

After this, Baculum first works a while, but then it complains that bconsole 
fails to connect to the Director due to authentication problems:

"Error code: 4

Message: Problem with connection to bconsole. Output=>Connecting to Director 
bacula-dir.lan.net:9101 Director authorization problem. Most likely the 
passwords do not agree. If you are using TLS, there may have been a certificate 
validation error during the TLS handshake. For help, please see 
http://www.bacula.org/rel-manual/en/problems/Bacula_Frequently_Asked_Que.html, 
Exitcode=>1" (btw, the mentioned URL is dead, I didn't find an FAQ)
  
bacula-dir.conf:

Director {
  Name = "bacula-dir"
  Messages = "Daemon"
  QueryFile = "/opt/bacula/scripts/query.sql"
  WorkingDirectory = "/opt/bacula/working"
  PidDirectory = "/opt/bacula/working"
  MaximumConcurrentJobs = 20
  Password = "redacted"
}

The redacted passwords for both Director resources are identical, so a password 
mismatch is not the problem here.

As can be seen there are no TLS directives, so even if TLS is used per default 
it would us the Password directives for the TLS PSK (if I understood it 
correctly). There are not TLS certificates provides by me for bacula. However, 
the unRAID server UI uses a certificate, and bacdir is not one of the names in 
the certificate's SAN. But AFAIK Bacula looks only at the CN, not at the names 
in SAN, correct?

This is the first time I am seeing such an error in Bacula, and I have a small 
number of remote FDs that might possibly also connect using TLS (although in no 
config file there are TLS directives).

If this error was about a missing TLS certificate, then why am I not seeing 
such an error for any of the remote FDs?

Any has an idea where the problem might be? 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Baculum throws error that bconsole cannot connect

[Justin Case]

Actually connecting to bacdir:9101 using telnet fails, and I do see a lot of 
related connection errors in syslog from a FD that is configured to connect to 
the Director on a schedule. Actually it started trying to connect when the 
schedule started, BUT - after the end of the schedule it still is trying to 
connect to the Director. At first glance this sounds wrong to me.

The question is rather why the director isn’t running. When re-starting the 
container it runs, and after a minute or so it stops. I will also try to solve 
this with the container guy...

> On 29. Jul 2022, at 16:48, Justin Case  wrote:
> 
> 
> Hello all,
> 
> I am having a problem with Baculum currently. To explain the context I need 
> to start somewhere else:
> 
> Baculum is running in a Docker container 
> (pwa666/bacula-server:11-postgresql-latest) on my unRAID NAS, this container 
> contains Baculum, Director, the SD and an FD for backing up local data. 
> (https://hub.docker.com/pwa666/bacula-server)
> 
> As macOS TimeMachine stopped working properly after unRAID upgrade to 6.10.3 
> I disabled TimeMachine in unRAID and installed the TimeMachine container 
> (https://github.com/mbentley/docker-timemachine). This container runs with a 
> br0 network and the TimeMachine service is using a different IP than the 
> unRAID server, but in the same subnet.
> (Yes, macOS backup up twice, by Bacula and by TimeMachine)
> 
> Since I have this TimeMachine container Baculum threw an error on the UI that 
> bconsole had problems connecting to the director at localhost:9101.
> 
> Consequently I replaced in bconsole.conf localhost with the FQDN of the 
> director, which is a CNAME for the FQDN of the unRAID server:
> 
> bconsole.conf:
> 
> Director {
>  Name = "bacula-dir"
>  DIRport = 9101
>  address = bacdir.foo.net
>  Password = "redacted"
> }
> 
> After this, Baculum first works a while, but then it complains that bconsole 
> fails to connect to the Director due to authentication problems:
> 
> "Error code: 4
> 
> Message: Problem with connection to bconsole. Output=>Connecting to Director 
> bacula-dir.lan.net:9101 Director authorization problem. Most likely the 
> passwords do not agree. If you are using TLS, there may have been a 
> certificate validation error during the TLS handshake. For help, please see 
> http://www.bacula.org/rel-manual/en/problems/Bacula_Frequently_Asked_Que.html,
>  Exitcode=>1" (btw, the mentioned URL is dead, I didn't find an FAQ)
> 
> bacula-dir.conf:
> 
> Director {
>  Name = "bacula-dir"
>  Messages = "Daemon"
>  QueryFile = "/opt/bacula/scripts/query.sql"
>  WorkingDirectory = "/opt/bacula/working"
>  PidDirectory = "/opt/bacula/working"
>  MaximumConcurrentJobs = 20
>  Password = "redacted"
> }
> 
> The redacted passwords for both Director resources are identical, so a 
> password mismatch is not the problem here.
> 
> As can be seen there are no TLS directives, so even if TLS is used per 
> default it would us the Password directives for the TLS PSK (if I understood 
> it correctly). There are not TLS certificates provides by me for bacula. 
> However, the unRAID server UI uses a certificate, and bacdir is not one of 
> the names in the certificate's SAN. But AFAIK Bacula looks only at the CN, 
> not at the names in SAN, correct?
> 
> This is the first time I am seeing such an error in Bacula, and I have a 
> small number of remote FDs that might possibly also connect using TLS 
> (although in no config file there are TLS directives).
> 
> If this error was about a missing TLS certificate, then why am I not seeing 
> such an error for any of the remote FDs?
> 
> Any has an idea where the problem might be? 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] bsmtp from within a container

[Justin Case]

One problem I ignored so far is that the Bacula Docker container I use 
(pwa666/bacula-server:11-postgresql-latest) cannot send emails to me.

bsmtp only seems to use the output of the “hostname” command as the hostname 
during the negotiation phase. this hostname is some arbitrary ID, not a FQDN. 

3422f1072002 --> HELO 3422f1072002

Setting the hostname fails in the container (but thats something to talk about 
to the container maintainer).

my mailserver complains when the hostname used during negotiation is not a FQDN.

bsmtp: bsmtp.c:124-0 Fatal malformed reply from mailserver.dummy.net: 504 5.5.2 
<3422f1072002>: Helo command rejected: need fully-qualified hostname

I did not find a bsmtp option to override the hostname for smtp client. such an 
option might be useful.

But my mailserver also requires the sending mail user to authenticate and from 
what I have seen in the documentation bsmtp cannot do that, correct?

So I would have to use some other smtp client like “mail”? (which is not in the 
container, again a problem to clarify elsewhere)

Best,
 J/C

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bsmtp from within a container

[Justin Case]

I run the mailserver put its basically a tightly baked postfix dovecot under 
Synology DSM UI. So I won’t manually change config files. But “Ignore 
authorization for LAN connections” sounds reasonable, I have activated that 
now. Lets see if that helps.

(BTW, bacula-dir and mail server are on different machines. its a home lab, 
yes, but its vast ;)

This does, however, not solve the problem that the hostname is not an FQDN and 
that it cannot be overridden with bsmtp. So I am still 100% away from a working 
solution :(

> On 1. Aug 2022, at 23:12, dmitri maziuk  wrote:
> 
> On 2022-08-01 3:57 PM, Justin Case wrote:
> 
>> bsmtp: bsmtp.c:124-0 Fatal malformed reply from mailserver.dummy.net: 504 
>> 5.5.2 <3422f1072002>: Helo command rejected: need fully-qualified hostname
> 
> Who runs the mailserver? Typically you's set "permit-mynetworks" before 
> "reject-XX-helo-hostname" and add the docker ip ranges to "mynetworks" -- 
> assuming it's postfix.
> 
> Since docker uses private ip ranges, these ips should never appear on "the 
> Internet" side of the mailserver, i.e. it's not opening the relay for 
> everyone: only to docker containers (which could be a problem too dep. on how 
> far they trust their customers).
> 
> $.02
> Dima
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bsmtp from within a container

[Justin Case]

Hi Bill,

the container I use is maintained by another person. I will suggest it.
If I would be using Bacula in a platform maintained by me I already had postfix 
installed...

That container not even has apt…. what can I say.

> On 2. Aug 2022, at 22:05, Bill Arlofski via Bacula-users 
>  wrote:
> 
> On 8/2/22 13:16, Justin Case wrote:
>> I run the mailserver put its basically a tightly baked postfix dovecot under 
>> Synology DSM UI. So I won’t manually change config files. But “Ignore 
>> authorization for LAN connections” sounds reasonable, I have activated that 
>> now. Lets see if that helps.
>> (BTW, bacula-dir and mail server are on different machines. its a home lab, 
>> yes, but its vast ;)
>> This does, however, not solve the problem that the hostname is not an FQDN 
>> and that it cannot be overridden with bsmtp. So I am still 100% away from a 
>> working solution :(
> 
> I always install a postfix MTA on my local Bacula Director, listening on 
> 127.0.0.1, and then from there, I can set all the outbound SMTP relay(s) and 
> any auth that is required.
> 
> Typically, the local postfix is just there on the local Director because 
> bsmtp is a simple, one-shot deal. If it cannot connect to the host to deliver 
> the message for any reason, that email is gone.
> 
> With a local postfix MTA, bsmtp sends the messages to thge local postfix, the 
> messages are locally queued and always delivered. Your case takes this a 
> couple steps further (configuring auth etc) than I usually ever need to go 
> with postfix and Bacula, but it is still what I would recommend trying. :)
> 
> 
> Best regards,
> Bill
> 
> -- 
> Bill Arlofski
> w...@protonmail.com
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Connection errors on daily backup with ConnectToDirector

[Justin Case]

This chapter was not yet closed and I have interesting further results 
(actually a bug report).
I opened another thread here where bacula-dir was crashing. Today I found the 
cause.

A macOS FD was earlier using normal connection from bacula-dir, but later I 
adapted it to connect to bacula-dir on its own. So far so good? No. Hang on, I 
am coming to it.

Under macOS I am using homebrew bacula-fd and in the meanwhile there was an 
upgrade from 11 to 13. And at that point I made a subtle mistake: I copied 
bacula-fd.conf from the 11 install to the 13 install. The problem with this is 
that I overlooked this directive:

  Plugin Directory = /usr/local/Cellar/bacula-fd/11.0.6/lib

I guess you immediately see where the problem is, the 13.0.0 bacula-fd binary 
is using bpipe plugin 11.0.6 binary.

The result of this seems to be that bacula-fd connects a lot more aggressively 
to bacula-dir (don’t ask me why, I don’t know it).

In an ideal world this shouldn’t pose a problem, but in the container for 
bacula-dir 11 that I am using it caused bacula-dir to crash with a segmentation 
violation.

This is why I am reporting this.

Since I corrected the library path I don’t see the crashes any more and no 
aggressive quick and numerous connections from the fd to the dir.

But! It is still the same director binary and there must be some bug if a 
misbehaving FD can actually make the director crash. This is a security 
problem, as someone could incapacitate the backups when they run and stop doing 
that during the phase when backups are not running, so it would be hard to 
troubleshoot. One could argue it is my own fault if I use v13 FDs with v11 
diriectors. From a security perspective it is a problem with the 
implementation. As long as a v11 director accepts v13 clients, it must not 
crash. 

I don’t know how to better report this, but some developer would probably take 
a look where the problem is with bacula-dir crashing if it gets a lot of FD 
connections quickly over a longer time. I have only tried it with v13 FDs, but 
I suspect it would also happen with v11 FDs.

All the best
 J/C

> On 25. Jul 2022, at 22:33, Justin Case  wrote:
> 
> I think it is great support here from  you people!
> 
> Today I think I might have understood what is happening, and Bill’s 
> explanations about what might be going on were probably correct in the core, 
> but not in the details.
> 
> Let me try to lay out what I think is going on and where I had my problem 
> understanding it in the first place:
> 
> After an update of syslog-ng the syslogging of the FD client host started to 
> work (it was configured a long time ago but somehow it never worked before 
> the update for the syslog-ng server that came in the last days) I began to 
> see where and WHEN(!) the error messages originated.
> 
> It is - as you guys are saying - the FD generating these errors, which are 
> logged without delay in my central syslog-ng server:
> 
> 2022-07-25 12:57:30   
> bsockcore.c:265 Unable to connect to Director daemon on 
> bacula-dir.lan.net:9101 <http://bacula-dir.lan.net:9101/>. ERR=Connection 
> refused
> 
> The eye-opener were the timestamps, which explained what is happening (more 
> on that later).
> My problem so far was that the error messages shown in Baculum had the 
> timestamp of the Director when the Director sees the error messages, not when 
> they happened!
> 
> 25-Jul 22:00 bacula-dir JobId 1725: Error: getmsg.c:217 Malformed message: 
> [bsockcore.c:265 Unable to connect to Director daemon on 
> bacula-dir.lan.net:9101 <http://bacula-dir.lan.net:9101/>. ERR=Connection 
> refused
> 
> Note the different timestamp. In the first message it is the timestamp of the 
> FD client host when the error occurs there. In the second message you see the 
> timestamp of the Director host when the first error message gets delivered 
> from the FD to the Director.
> 
> So what you guys said is correct: the Director accepts the error messaged 
> from the FD only when a job runs for the FD. Even if the FD connects to the 
> Director many times during the day, the error messages are held back by the 
> FD until a job actually runs and then they are ingested for the first job 
> that runs on the current day. This also explains why there are no errors when 
> a similar job runs shortly after to backup to the other tier storage
> 
> Because so far I was only seeing the Director timestamp I was misled that the 
> error actually happens at the time when the job runs. I now understand that 
> this is not correct, and I think you guys also mentioned it, but I didn’t 
> pick it up consciously enough to understand what this means.
> 
> Now that I can see the timestamp from the FD when the errors actually happen 
> on the FD host I can now confirm:
> 
> (1) the Director is defini

Re: [Bacula-users] bsmtp from within a container

[Justin Case]

The container uses the container ID as hostname. nothing I can do about it with 
DNS.
I will retire the Synology mail server at somepoint but that is months in the 
future.

I disabled authentication for local networks, but still:
504 5.5.2 <3422f1072002>: Helo command rejected: need fully-qualified hostname

> On 2. Aug 2022, at 22:29, dmitri maziuk  wrote:
> 
> On 2022-08-02 2:16 PM, Justin Case wrote:
>> I run the mailserver put its basically a tightly baked postfix dovecot under 
>> Synology DSM UI. So I won’t manually change config files. But “Ignore 
>> authorization for LAN connections” sounds reasonable, I have activated that 
>> now. Lets see if that helps.
> 
> It has to know 172.x is a "LAN" connection... if they don't have a way to set 
> $mynetworks, I think you might want to add a raspi to your home lab to run a 
> proper postfix instance. ;)
> 
>> This does, however, not solve the problem that the hostname is not an FQDN 
>> and that it cannot be overridden with bsmtp. So I am still 100% away from a 
>> working solution :(
> 
> It's common enough, half of them get "localhost" from the resolver anyway and 
> happily stick it in the mail header. I tend to specify From: addresses like 
> "win-acme-on-server-X@mydomain" to know where it came from -- and if anyone 
> decides to reply, they can keep the bounce.
> 
> As far as mail delivery goes, FQDN is not needed for anything. It's only 
> there for that UCE check which should be disabled for "LAN connections".
> 
> PS. if bsmtp gets its hostname from the resolver, you might be able to fool 
> it by setting up a nameserver for docker ips. Or maybe get names from docker 
> network -- but I never looked into that.
> 
> Dima
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bugs/Problems with ConnectToDirector

[Justin Case]

Dear Bacula developers,

This chapter goes further. It seems there is not merely a problem with 
bacula-dir crashing if an FD is aggressively connecting, also the FD connection 
scheduling and connection aggressivity seems to need some more thought.

Today the bacula-dir crashed again. This time the other FD (this is Debian) 
went rampant, no apparent reason. It just connected to the dir like crazy.

Something that also seems to need more adjustment is the connection scheduling.

When an FD is running before the schedule starts, it will start connecting as 
scheduled, but it will never stop. I think it should stop when the schedule 
ends.

When an FD is started during the schedule window, it will NOT start connecting 
to the director. This is clearly now what one would want. An started after the 
start time point and during the scheduled duration should connect to the 
director. Only if started before the scheduled time and after the duration of 
the schedule the FD shold not connect to the director.

The director needs to be fixed so that it does not crash if an FD is 
reconnecting very aggressively. And FDs should be more limited in the 
connection frequency (this does not mean the parameter ReconnectionTime, but 
the time wait before initiating another connection after unsuccessful 
connection trial).

All the best,
 J/C



> On 2. Aug 2022, at 22:42, Justin Case  wrote:
> 
> This chapter was not yet closed and I have interesting further results 
> (actually a bug report).
> I opened another thread here where bacula-dir was crashing. Today I found the 
> cause.
> 
> A macOS FD was earlier using normal connection from bacula-dir, but later I 
> adapted it to connect to bacula-dir on its own. So far so good? No. Hang on, 
> I am coming to it.
> 
> Under macOS I am using homebrew bacula-fd and in the meanwhile there was an 
> upgrade from 11 to 13. And at that point I made a subtle mistake: I copied 
> bacula-fd.conf from the 11 install to the 13 install. The problem with this 
> is that I overlooked this directive:
> 
>   Plugin Directory = /usr/local/Cellar/bacula-fd/11.0.6/lib
> 
> I guess you immediately see where the problem is, the 13.0.0 bacula-fd binary 
> is using bpipe plugin 11.0.6 binary.
> 
> The result of this seems to be that bacula-fd connects a lot more 
> aggressively to bacula-dir (don’t ask me why, I don’t know it).
> 
> In an ideal world this shouldn’t pose a problem, but in the container for 
> bacula-dir 11 that I am using it caused bacula-dir to crash with a 
> segmentation violation.
> 
> This is why I am reporting this.
> 
> Since I corrected the library path I don’t see the crashes any more and no 
> aggressive quick and numerous connections from the fd to the dir.
> 
> But! It is still the same director binary and there must be some bug if a 
> misbehaving FD can actually make the director crash. This is a security 
> problem, as someone could incapacitate the backups when they run and stop 
> doing that during the phase when backups are not running, so it would be hard 
> to troubleshoot. One could argue it is my own fault if I use v13 FDs with v11 
> diriectors. From a security perspective it is a problem with the 
> implementation. As long as a v11 director accepts v13 clients, it must not 
> crash. 
> 
> I don’t know how to better report this, but some developer would probably 
> take a look where the problem is with bacula-dir crashing if it gets a lot of 
> FD connections quickly over a longer time. I have only tried it with v13 FDs, 
> but I suspect it would also happen with v11 FDs.
> 
> All the best
>  J/C
> 
>> On 25. Jul 2022, at 22:33, Justin Case > <mailto:jus7inc...@gmail.com>> wrote:
>> 
>> I think it is great support here from  you people!
>> 
>> Today I think I might have understood what is happening, and Bill’s 
>> explanations about what might be going on were probably correct in the core, 
>> but not in the details.
>> 
>> Let me try to lay out what I think is going on and where I had my problem 
>> understanding it in the first place:
>> 
>> After an update of syslog-ng the syslogging of the FD client host started to 
>> work (it was configured a long time ago but somehow it never worked before 
>> the update for the syslog-ng server that came in the last days) I began to 
>> see where and WHEN(!) the error messages originated.
>> 
>> It is - as you guys are saying - the FD generating these errors, which are 
>> logged without delay in my central syslog-ng server:
>> 
>> 2022-07-25 12:57:30  
>> bsockcore.c:265 Unable to connect to Director daemon on 
>> bacula-dir.lan.net:9101 <http://bacula-dir.lan.net:9101/>. ERR=Connection 
>> refused
>> 
>

Re: [Bacula-users] bsmtp from within a container

[Justin Case]

Thank you Josh, I got it now.
I had to relax 2 SPAM settings, but didn’t need to ignore authentication for 
local machines.
If someone later on needs details, let me know.
Thanks again!
 J/C

> On 4. Aug 2022, at 16:24, Josh Fisher  wrote:
> 
> 
> On 8/2/22 16:46, Justin Case wrote:
>> The container uses the container ID as hostname. nothing I can do about it 
>> with DNS.
>> I will retire the Synology mail server at somepoint but that is months in 
>> the future.
>> 
>> I disabled authentication for local networks, but still:
>> 504 5.5.2 <3422f1072002>: Helo command rejected: need fully-qualified 
>> hostname
> 
> 
> Fix the Synology mail server instead of the container. Look at advanced 
> security rules (Mail Delivery > Security > Advanced) for the 'Reject HELO 
> hostnames without fully qualified domain name (FQDN)' and 'Reject unknown 
> HELO hostnames' rule settings.
> 
> 
>> 
>>> On 2. Aug 2022, at 22:29, dmitri maziuk  wrote:
>>> 
>>> On 2022-08-02 2:16 PM, Justin Case wrote:
>>>> I run the mailserver put its basically a tightly baked postfix dovecot 
>>>> under Synology DSM UI. So I won’t manually change config files. But 
>>>> “Ignore authorization for LAN connections” sounds reasonable, I have 
>>>> activated that now. Lets see if that helps.
>>> It has to know 172.x is a "LAN" connection... if they don't have a way to 
>>> set $mynetworks, I think you might want to add a raspi to your home lab to 
>>> run a proper postfix instance. ;)
>>> 
>>>> This does, however, not solve the problem that the hostname is not an FQDN 
>>>> and that it cannot be overridden with bsmtp. So I am still 100% away from 
>>>> a working solution :(
>>> It's common enough, half of them get "localhost" from the resolver anyway 
>>> and happily stick it in the mail header. I tend to specify From: addresses 
>>> like "win-acme-on-server-X@mydomain" to know where it came from -- and if 
>>> anyone decides to reply, they can keep the bounce.
>>> 
>>> As far as mail delivery goes, FQDN is not needed for anything. It's only 
>>> there for that UCE check which should be disabled for "LAN connections".
>>> 
>>> PS. if bsmtp gets its hostname from the resolver, you might be able to fool 
>>> it by setting up a nameserver for docker ips. Or maybe get names from 
>>> docker network -- but I never looked into that.
>>> 
>>> Dima
>>> 
>>> 
>>> ___
>>> Bacula-users mailing list
>>> Bacula-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>> 
>> 
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net 
>> <mailto:Bacula-users@lists.sourceforge.net>
>> https://lists.sourceforge.net/lists/listinfo/bacula-users 
>> <https://lists.sourceforge.net/lists/listinfo/bacula-users>
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net <mailto:Bacula-users@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/bacula-users 
> <https://lists.sourceforge.net/lists/listinfo/bacula-users>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Need help with fileset declaration - what's my mistake?

[Justin Case]

greetings fellas,

i am using the following fileset on a number of VMs running docker containers 
to backup the container config and user data - and it works for me. It does 
backup all /mnt/containerdata..* folders with content, nothing else:

Fileset {
  Name = "various-appdata"
  EnableVss = no
  EnableSnapshot = no
  Include {
Options {
  OneFs = no
  RegexDir = "^/mnt/containerdata-.*/"
}
Options {
  OneFs = no
  Exclude = yes
  RegexDir = "^/mnt/.*/"
}
File = "/mnt"
  }
 

Today I extended the fileset to do the exact same, but not for 
/mnt/containerdata.*, but instead for /home/dockerman/containerdata-.*

> Fileset {
>   Name = "various-appdata"
>   Description = "Container application data from various machines set up like 
> tpl050-203"
>   EnableVss = no
>   EnableSnapshot = no
>   Include {
> Options {
>   OneFs = no
>   RegexDir = "^/mnt/containerdata-.*/"
> }
> Options {
>   OneFs = no
>   Exclude = yes
>   RegexDir = "^/mnt/.*/"
> }
> File = "/mnt"
>   }
>   Include {
> Options {
>   OneFs = no
>   RegexDir = "^/home/dockerman/containerdata-.*/"
> }
> Options {
>   OneFs = no
>   Exclude = yes
>   RegexDir = "^/home/dockerman/.*/"
> }
> File = "/home/dockerman"
>   }
> }

While the first Include still works as intended and as in the past, the 
additional Include does not. The second Include backups everything in 
/home/dockerman - except for the containerdata-.* folders, so actually the 
exact opposite of what I intended it to do.

I am really puzzled by this outcome, as the Include directives are identical 
except for the path prefix (/mnt vs. /home/dockerman).

I assume a fileset composed of 2 include directives is the sum of the fileset 
defined by each include directive alone. Is that correct?

Where is my mistake?

Thanks for your time!



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] I am missing Debs for Buster 32 bit (for Raspberry Pi machines running RaspiOS)

[Justin Case]

Hello, this goes to the people who manage the debian packages for Bacula 
Community

I know that Buster is old and that there are packages for the 64 bit kernel. 
That is great as long as you are not running any Raspberry Pis with RaspiOS 
(https://www.raspberrypi.com/software/operating-systems/ 
). The last RaspiOS 
release is still based on Debian Buster 32 (!) bit.

The last version available for this is Bacula 9 and my problem is that I need 
the Raspi FD to connect to the Director (some small physical fallback machines 
in a management network behind a firewall). As this works reliably with 13.0.1 
(and not the earlier versions starting with 11) I actually need 13.0.1 debs für 
Buster 32 bit.

Is there any chance that we can get this?

All the best,
 J/C



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] I am missing Debs for Buster 32 bit (for Raspberry Pi machines running RaspiOS)

[Justin Case]

You mean the script will also work on RaspiOS 32 bit? (note: Raspbian != RasiOS)
There are a number of VMs and physical raspis running the RaspiOS 32 bit.
So I actually need a solution to be able to get or make Bacula debs of 13.0.1 
for RaspiOS 3 2bit.
I cannot just reinstall all these machines with Raspbian 64.

> On 26. Aug 2022, at 14:14, Chris Wilkinson  wrote:
> 
> I compiled v11/postgres for 64 bit Raspbian. Its working OK backing up to B2 
> cloud. I have a script for the compile if you need it. There shouldn't be any 
> issue with v13.
> 
> Regards
> Chris Wilkinson
> 
> On Fri, 26 Aug 2022, 12:06 pm Justin Case,  <mailto:jus7inc...@gmail.com>> wrote:
> Hello, this goes to the people who manage the debian packages for Bacula 
> Community
> 
> I know that Buster is old and that there are packages for the 64 bit kernel. 
> That is great as long as you are not running any Raspberry Pis with RaspiOS 
> (https://www.raspberrypi.com/software/operating-systems/ 
> <https://www.raspberrypi.com/software/operating-systems/>). The last RaspiOS 
> release is still based on Debian Buster 32 (!) bit.
> 
> The last version available for this is Bacula 9 and my problem is that I need 
> the Raspi FD to connect to the Director (some small physical fallback 
> machines in a management network behind a firewall). As this works reliably 
> with 13.0.1 (and not the earlier versions starting with 11) I actually need 
> 13.0.1 debs für Buster 32 bit.
> 
> Is there any chance that we can get this?
> 
> All the best,
>  J/C
> 
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net <mailto:Bacula-users@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/bacula-users 
> <https://lists.sourceforge.net/lists/listinfo/bacula-users>

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] I am missing Debs for Buster 32 bit (for Raspberry Pi machines running RaspiOS)

[Justin Case]

Hi Sebastian,

so far no problem. Then read on what I wrote:

"The last version available for this is Bacula 9 and my problem is that I need 
the Raspi FD to connect to the Director (some small physical fallback machines 
in a management network behind a firewall). As this works reliably with 13.0.1 
(and not the earlier versions starting with 11) I actually need 13.0.1 debs für 
Buster 32 bit."

TBH, V9.x is more then 2y old. I think for a current Debian bullseye this is 
way too old.

But no worries, I compiled 13.0.1 on Buster with the help of Chris Wilkinson.

> On 26. Aug 2022, at 18:17, Sebastian Suchanek  wrote:
> 
> Am 26.08.2022 um 14:19 schrieb Justin Case:
>> You mean the script will also work on RaspiOS 32 bit? (note: Raspbian != 
>> RasiOS)
>> [...]
> 
> To be honest, I've never heard of a system called "RaspiOS" exactly, but 
> "Raspberry Pi OS" is definetly just a rename of what was formerly called 
> "Raspbian" - as it also stated on the Raspberry homepage:
> 
>   https://www.raspberrypi.com/software/
> 
> Concerning your original question: Unfortunately, I don't run a Raspberry Pi 
> on Buster, but I do run three of them under Raspberry Pi OS (a.k.a. Raspbian) 
> Stretch and one under Bullseye. For both versions, Bacula came in the 
> original Raspbian repository. (Bacula 7.4.4 on Stretch and Bacula 9.6.7 on 
> Bullseye.) I've set up Bacula FDs on all of my RasPis from the respective 
> Raspbian repository and they all work well.
> 
> Assuming that "RaspiOS" indeed is something different then "Raspberry Pi 
> OS"/"Raspbian" - have you already checked the corresponding repository for 
> Bacula?
> 
> 
> HTH,
> 
> Sebastian
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Need help with fileset declaration - what's my mistake?

[Justin Case]

> On 26. Aug 2022, at 19:53, Martin Simmons  wrote:
> 
> show fileset="various-appdata”

FileSet: name=various-appdata IgnoreFileSetChanges=0
  O f
  RD ^/mnt/containerdata-.*/
  N
  O fe
  RD ^/mnt/.*/
  N
  I /mnt
  N
  O f
  RD ^/home/dockerman/containerdata-.*/
  N
  O fe
  RD ^/home/dockerman/.*/
  N
  I /home/dockerman
  N

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Need help with fileset declaration - what's my mistake?

[Justin Case]

> On 26. Aug 2022, at 19:53, Martin Simmons  wrote:
> 
> 
> Also, does it work if you only have the /home/dockerman Include?
> 


That does work, with the following caveat:

So actually, the fileset did back up the /home/../containerdata.. stuff, but 
also other files outside of /home/,,,/containderdata, which I did not intend to 
happen.

in the /home/dockerman there are only dot-files sich as .Xauthority and 
.ssh/config 
Why would those have been backed up by this fileset?

And then I realized that this was also the case for the compound fileset! The 
fileset does actually back up the containerdata-* folder contents both in 
/home/dockerman and in /mnt. The amount of data in /home is just much smaller 
and in baculum the order of the files shown suggested that files other then 
/home/../containerdata-… were backed up, and the files on containerdata 
couldn’t be seen in the file list.

Still the same caveat as above holds for the compound fileset, too. I don’t 
understand that.



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Need help with fileset declaration - what's my mistake?

[Justin Case]

Martin, thanks, I admit that for me the fileset definition is one of the more 
confusing things in Bacula. Thanks for the explanation, however, honestly, I 
don’t get it. 

I adapted the fileset according to your proposal, and this is what baculum 
makes from it:

Fileset {
Name = "test-appdata"
EnableVss = no
EnableSnapshot = no
Include {
File = "/home/dockerman"
Options {
OneFs = "No"
RegexDir = "^/home/dockerman/containerdata-"
}
Options {
OneFs = "No"
Exclude = "Yes"
RegexDir = "^/home/dockerman/[^/]+$"
}
}

Running this results in actually nearly the same result. The delta of the 
outcome are actually 3 files (gleaned form the numbers of the jobs, I don’t 
know which files that are). 

Does the order of the declarations (as seen above) have anything to do with 
this result?

> On 26. Aug 2022, at 21:24, Martin Simmons  wrote:
> 
>>>>>> On Fri, 26 Aug 2022 20:41:22 +0200, Justin Case said:
>> 
>> So actually, the fileset did back up the /home/../containerdata.. stuff, but
>> also other files outside of /home/,,,/containderdata, which I did not intend
>> to happen.
>> 
>> in the /home/dockerman there are only dot-files sich as .Xauthority and
>> .ssh/config Why would those have been backed up by this fileset?
>> 
>> And then I realized that this was also the case for the compound fileset!
>> The fileset does actually back up the containerdata-* folder contents both
>> in /home/dockerman and in /mnt. The amount of data in /home is just much
>> smaller and in baculum the order of the files shown suggested that files
>> other then /home/../containerdata-… were backed up, and the files on
>> containerdata couldn’t be seen in the file list.
>> 
>> Still the same caveat as above holds for the compound fileset, too. I don’t
>> understand that.
> 
> It does that because your fileset definition tells it to :-)
> 
> The problem is that "^/home/dockerman/.*/" only matches directories two levels
> below /home/dockerman, such as /home/dockerman/dir1/dir2.  Everything above
> that is included, for example directories like /home/dockerman/dir1 and files
> like /home/dockerman/file1 and /home/dockerman/dir1/file1.
> 
> I think this does what you want:
> 
>  Include {
>Options {
>  OneFs = no
>  # override the Exclude for this prefix
>  RegexDir = "^/home/dockerman/containerdata-"
>}
>Options {
>  OneFs = no
>  Exclude = yes
>  # disallow all top-level things not explicitly included above
>  Regex = "^/home/dockerman/[^/]+$"
>}
>File = "/home/dockerman"
>  }
> 
> Note that the paths inside the /home/dockerman/containerdata-... directories
> will be included implicitly because they don't match the excluded Regex.  The
> Options (except Exclude) from the last clause will be used for those paths.
> 
> __Martin



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q. storage groups for a given pool using several external USB HDDs?

[Justin Case]

Greetings,

I am interested in the new V13 storage groups feature.
If I wanted a given pool to use file folumes on several externally connected 
USB HDDs:

-the devices would be connected all the time, would the director fill the 
devices one ofter the other, or would it scatter the jobs randomly across the 
devices?

-may I extend the list of storages in a given pool to extend the available 
storage space by adding USB HDD devices to the storage? (I mean distinct HDD 
devices, not enlarging the storage of a RAID!)

-This would mean that I don’t need to use vchanger vor this?

Thanks for shedding a bit more light on this new feature.

Best
 J/C



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: what do JobtoVerify, WriteVerifyList ?

[Justin Case]

Hi all,

I am using Baculum to manage my Bacula V11 installation.
Today I wanted to define a verify job for the first time.

In Baculum I find 3 fields in the Verify section:

VerifyJob: AFAU this is the name of the job which should be verified (I am 
using Level “Data).

JobToVerify: no idea what this is, sounds like the same thing as VerifyJob. I 
found no documentation on this.

WriteVerifyList: no idea what this is. I found no documentation on this.


Does somebody know what JobToVerify and WriteVerifyList are doing and would 
care to explain it a bit?

Thanks for you time,
 J/C





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Need help with fileset declaration - what's my mistake?

[Justin Case]

Martin, you are right, my bad, I overlooked that you were not using RegexDir, 
but Regex for the exclude.

I am very glad that your solution works, thank you so much. I have to admit, on 
that evening I had been mucking around a lot more and also a friend, and none 
of our attempts worked as intended.

So still, I find the definition of filesets very confusing, I hope I will get a 
better understanding through your example.

Thank you again!
 J/C


> On 30. Aug 2022, at 15:44, Martin Simmons  wrote:
> 
> The order of the File declarations relative to the Options declarations
> doesn't matter.  However, the order of the Options declarations relative to
> each other does matter (Bacula uses the first Options that matches based on
> RegexDir etc).
> 
> The problem below is that you are using RegexDir instead of Regex in the
> Exclude Options clause.
> 
> __Martin
> 
> 
>>>>>> On Fri, 26 Aug 2022 22:37:08 +0200, Justin Case said:
>> 
>> Martin, thanks, I admit that for me the fileset definition is one of the 
>> more confusing things in Bacula. Thanks for the explanation, however, 
>> honestly, I don’t get it. 
>> 
>> I adapted the fileset according to your proposal, and this is what baculum 
>> makes from it:
>> 
>> Fileset {
>> Name = "test-appdata"
>> EnableVss = no
>> EnableSnapshot = no
>> Include {
>> File = "/home/dockerman"
>> Options {
>> OneFs = "No"
>> RegexDir = "^/home/dockerman/containerdata-"
>> }
>> Options {
>> OneFs = "No"
>> Exclude = "Yes"
>> RegexDir = "^/home/dockerman/[^/]+$"
>> }
>> }
>> 
>> Running this results in actually nearly the same result. The delta of the 
>> outcome are actually 3 files (gleaned form the numbers of the jobs, I don’t 
>> know which files that are). 
>> 
>> Does the order of the declarations (as seen above) have anything to do with 
>> this result?
>> 
>>> On 26. Aug 2022, at 21:24, Martin Simmons  wrote:
>>> 
>>>>>>>> On Fri, 26 Aug 2022 20:41:22 +0200, Justin Case said:
>>>> 
>>>> So actually, the fileset did back up the /home/../containerdata.. stuff, 
>>>> but
>>>> also other files outside of /home/,,,/containderdata, which I did not 
>>>> intend
>>>> to happen.
>>>> 
>>>> in the /home/dockerman there are only dot-files sich as .Xauthority and
>>>> .ssh/config Why would those have been backed up by this fileset?
>>>> 
>>>> And then I realized that this was also the case for the compound fileset!
>>>> The fileset does actually back up the containerdata-* folder contents both
>>>> in /home/dockerman and in /mnt. The amount of data in /home is just much
>>>> smaller and in baculum the order of the files shown suggested that files
>>>> other then /home/../containerdata-… were backed up, and the files on
>>>> containerdata couldn’t be seen in the file list.
>>>> 
>>>> Still the same caveat as above holds for the compound fileset, too. I don’t
>>>> understand that.
>>> 
>>> It does that because your fileset definition tells it to :-)
>>> 
>>> The problem is that "^/home/dockerman/.*/" only matches directories two 
>>> levels
>>> below /home/dockerman, such as /home/dockerman/dir1/dir2.  Everything above
>>> that is included, for example directories like /home/dockerman/dir1 and 
>>> files
>>> like /home/dockerman/file1 and /home/dockerman/dir1/file1.
>>> 
>>> I think this does what you want:
>>> 
>>> Include {
>>>   Options {
>>> OneFs = no
>>> # override the Exclude for this prefix
>>> RegexDir = "^/home/dockerman/containerdata-"
>>>   }
>>>   Options {
>>> OneFs = no
>>> Exclude = yes
>>> # disallow all top-level things not explicitly included above
>>> Regex = "^/home/dockerman/[^/]+$"
>>>   }
>>>   File = "/home/dockerman"
>>> }
>>> 
>>> Note that the paths inside the /home/dockerman/containerdata-... directories
>>> will be included implicitly because they don't match the excluded Regex.  
>>> The
>>> Options (except Exclude) from the last clause will be used for those paths.
>>> 
>>> __Martin
>> 
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q. storage groups for a given pool using several external USB HDDs?

[Justin Case]

Hi Ana, thank you for chiming in, see my response inline:

> On 1. Sep 2022, at 13:38, Ana Emília M. Arruda  wrote:
> 
> On Tue, Aug 30, 2022 at 3:43 PM Justin Case  <mailto:jus7inc...@gmail.com>> wrote:
> Greetings,
> 
> I am interested in the new V13 storage groups feature.
> If I wanted a given pool to use file folumes on several externally connected 
> USB HDDs:
> 
> -the devices would be connected all the time, would the director fill the 
> devices one ofter the other, or would it scatter the jobs randomly across the 
> devices?
> 
> This will depend on the storage group policy you use. The default 
> "ListedOrder" will always use the first storage in the storage group list 
> unless there is a connection failure. The LeastUsed policy will load balance 
> the storage usage between the storages defined in the list.
> 
> It will not spread jobs randomly across the devices or it will not skip a 
> storage if it gets full.
> 
> -may I extend the list of storages in a given pool to extend the available 
> storage space by adding USB HDD devices to the storage? (I mean distinct HDD 
> devices, not enlarging the storage of a RAID!)
> Yes.
>  
> 
> -This would mean that I don’t need to use vchanger vor this?
> vchanger and storage groups have different purposes, but none will skip a 
> device if it gets full.
> 

So to me it seems Storage Groups could actually be a good way to dynamically  
extend USB storage (although requiring manual interaction). when a device is 
full, and using ListedOrder, I could move the full device  to the end of the 
list and could I still do restore jobs from all devices in the list?

I was thinking vchanger would let me use a bunch of individual disks and fill 
them up one after the other… without me needing to manually interact with it 
after initial correct setup… weird.


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Schedules verify job fails

[Justin Case]

Hi

I am confused why I am getting this error:


03-Sep 22:00 bacula-dir JobId 2481: Fatal error: verify.c:73 Unimplemented 
Verify level 73(I)
03-Sep 22:00 bacula-dir JobId 2481: Error: Bacula bacula-dir 11.0.6 (10Mar22):

—snip—

 Verify Level:   Incremental
 Verify JobId:   0
 Files Examined: 0
 Non-fatal FD errors:1
 FD termination status:  
 Termination:*** Verify Error ***

The Verify Level shown above is definitely wrong, here is the definition (and 
it sets Data as Level!)(I removed the schedule resource name):

Job {
  Name = “job1-verify"
  Level = "Data"
  Pool = “pool1"
  FullBackupPool = “pool1"
  IncrementalBackupPool = “pool2"
  Fileset = “users"
  VerifyJob = “job1"
  JobDefs = “jdef1-verify"
  Enabled = yes
  AllowIncompleteJobs = no
  AllowDuplicateJobs = no
}
JobDefs {
  Name = “jdef1-verify"
  Type = "Verify"
  Level = "Data"
  Messages = "Standard"
  Storage = “storage1"
  Pool = "Default"
  Client = “client-fd"
  Fileset = "EmptyFileset"
  WriteBootstrap = "/bootstrap/%c_%n.bsr"
  Priority = 40
}

Any ideas?

Thanks for your time,
 J/C




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: Baculum, Pools, Next Pool vs Copy Pool

[Justin Case]

Hi there,

while in the process of trying out copy jobs I realized that Baculum has two 
directives for pools:

Next Pool: according to the documentation this is used for migration jobs and 
for copy jobs

Copy Pool: not found in the documentation. I can only guess that this suggests 
that Copy Pool should be used for copy jobs and Next Pool for migration jobs, 
but I haven’t tried it.

Does anyone know more about this “Copy Pool” directive? Does it actually do 
anything?

Best,
 J/C




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: Copy jobs dummy client problem

[Justin Case]

Hi there,

I took this snipped from the main documentation about copy jobs:

# # Fake client for copy jobs # 
Client { 
Name = None 
 Address = localhost 
 Password = "NoNe” 
 Catalog = MyCatalog 
} 

# # Default template for a CopyDiskToTape Job # JobDefs { 

Name = CopyDiskToTape 
Type = Copy 
Messages = StandardCopy 
Client = None 
FileSet = None 
Selection Type = PoolUncopiedJobs 
Maximum Concurrent Jobs = 10 
SpoolData = No 
Allow Duplicate Jobs = Yes 
Cancel Queued Duplicates = No 
Cancel Running Duplicates = No 
Priority = 13 
} 

It says: "The Copy Job runs without using the File daemon by copying the data 
from the old backup Volume to a different Volume in a different Pool.” So there 
is no need for a working Client.

This does not seem to be entirely factual. I configured it as suggested above, 
but then the Director gives me:

04-Sep 14:19 bacula-dir JobId 0: Fatal error: authenticatebase.cc:435 Director 
unable to authenticate with File Daemon at "localhost:9102". Possible causes: 
Passwords or names not the same or

The copy job still works, but I think it is not suitable to provoke such an 
error.

Why would the Director actually connect to the client? May I suppress this?

Thanks for considering my question,
 J/C







___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: How to verify copy jobs?

[Justin Case]

Hi there,

today I switched to using copy jobs to create the tier 2 backup (instead of 
backup jobs from the original source). I realized that the copied job on tier 2 
storage has the exact same name as the job on tier from which it was copied.

I am now unclear how I can schedule verify jobs for the tier copies. Previously 
I used the name of the tier 2 back up to identify the job that should be 
verified. Now the tier 2 copy job has the same name as the tier 1 backup job. 
How do I discern them when scheduling (I could not use the job IDs as there are 
not known when the verify job is scheduled).

Any ideas how to achieve this?

Thanks for considering my question,
 J/C




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Schedules verify job fails

[Justin Case]

Martin, excellent advice. it says at the job info:

level=Data

then under the schedule info:

Run Level=Full
..
Run Level=Incremental
..
Run Level=Incremental

according to the schedule it is Incremental

The precedence of which level is used cannot be derived from the "show job" 
alone.
I was assuming that the job level declared within the job or jobdef would be 
overriding levels declared elsewhere. The manual albeit explains that the level 
declared in the schedule takes precedence. For me this is a bit unfortunate, as 
I then need an own set of schedules for backup jobs with runs for Full and 
Incremental, and another set for verify jobs with a run for level=Data and 
possibly additional runs for other verify levels. considering the number of 
schedules I have defined for use this would be a lot.

Do you see a more elegant solution?

I guess it wouldn’t work if I added a run for level=Data to all of my 
schedules, as Bacula wouldn’t know how to match the runs with the correct level 
to the fitting job type? Or would the Director know that runs with level=Data 
only apply to jobs with type=Verify, and levels Full and Incremental apply to 
backup, migrate and copy jobs?

Cheers,
 J/C


> On 5. Sep 2022, at 17:47, Martin Simmons  wrote:
> 
> Maybe the output of
> 
> show job=job1-verify
> 
> will show where the level is being set to Incremental?  It might be the
> Schedule resource for example.
> 
> __Martin
> 
> 
>>>>>> On Sat, 3 Sep 2022 22:13:16 +0200, Justin Case said:
>> 
>> Hi
>> 
>> I am confused why I am getting this error:
>> 
>> 
>> 03-Sep 22:00 bacula-dir JobId 2481: Fatal error: verify.c:73 Unimplemented 
>> Verify level 73(I)
>> 03-Sep 22:00 bacula-dir JobId 2481: Error: Bacula bacula-dir 11.0.6 
>> (10Mar22):
>> 
>> —snip—
>> 
>> Verify Level:   Incremental
>> Verify JobId:   0
>> Files Examined: 0
>> Non-fatal FD errors:1
>> FD termination status:  
>> Termination:*** Verify Error ***
>> 
>> The Verify Level shown above is definitely wrong, here is the definition 
>> (and it sets Data as Level!)(I removed the schedule resource name):
>> 
>> Job {
>>  Name = “job1-verify"
>>  Level = "Data"
>>  Pool = “pool1"
>>  FullBackupPool = “pool1"
>>  IncrementalBackupPool = “pool2"
>>  Fileset = “users"
>>  VerifyJob = “job1"
>>  JobDefs = “jdef1-verify"
>>  Enabled = yes
>>  AllowIncompleteJobs = no
>>  AllowDuplicateJobs = no
>> }
>> JobDefs {
>>  Name = “jdef1-verify"
>>  Type = "Verify"
>>  Level = "Data"
>>  Messages = "Standard"
>>  Storage = “storage1"
>>  Pool = "Default"
>>  Client = “client-fd"
>>  Fileset = "EmptyFileset"
>>  WriteBootstrap = "/bootstrap/%c_%n.bsr"
>>  Priority = 40
>> }
>> 
>> Any ideas?
>> 
>> Thanks for your time,
>> J/C
>> 
>> 
>> 
>> 
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Copy jobs dummy client problem

[Justin Case]

Yes, it does match. I then replaced the dummy password with the actual FD 
password, and the error did not occur any more when starting copy jobs. This 
behaves differently than documented, that’s why I am asking. 

> On 5. Sep 2022, at 17:56, Martin Simmons  wrote:
> 
> That looks strange to me.  The "JobId 0" maybe means that it was caused by
> something else.  Does the time "04-Sep 14:19" match the sequence of times in
> the other messages about the copy job?
> 
> __Martin
> 
> 
>>>>>> On Sun, 4 Sep 2022 14:33:03 +0200, Justin Case said:
>> 
>> Hi there,
>> 
>> I took this snipped from the main documentation about copy jobs:
>> 
>> # # Fake client for copy jobs # 
>> Client { 
>> Name = None 
>> Address = localhost 
>> Password = "NoNe” 
>> Catalog = MyCatalog 
>> } 
>> 
>> # # Default template for a CopyDiskToTape Job # JobDefs { 
>> 
>> Name = CopyDiskToTape 
>> Type = Copy 
>> Messages = StandardCopy 
>> Client = None 
>> FileSet = None 
>> Selection Type = PoolUncopiedJobs 
>> Maximum Concurrent Jobs = 10 
>> SpoolData = No 
>> Allow Duplicate Jobs = Yes 
>> Cancel Queued Duplicates = No 
>> Cancel Running Duplicates = No 
>> Priority = 13 
>> } 
>> 
>> It says: "The Copy Job runs without using the File daemon by copying the 
>> data from the old backup Volume to a different Volume in a different Pool.” 
>> So there is no need for a working Client.
>> 
>> This does not seem to be entirely factual. I configured it as suggested 
>> above, but then the Director gives me:
>> 
>> 04-Sep 14:19 bacula-dir JobId 0: Fatal error: authenticatebase.cc:435 
>> Director unable to authenticate with File Daemon at "localhost:9102". 
>> Possible causes: Passwords or names not the same or
>> 
>> The copy job still works, but I think it is not suitable to provoke such an 
>> error.
>> 
>> Why would the Director actually connect to the client? May I suppress this?
>> 
>> Thanks for considering my question,
>> J/C
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Copy jobs dummy client problem

[Justin Case]

It works for me, too, but I get this error.

> On 5. Sep 2022, at 20:14, Martin Simmons  wrote:
> 
> A copy job works for me in Bacula 13.0.0 without this problem (even if the
> bacula-fd isn't running).
> 
> __Martin
> 
> 
>>>>>> On Mon, 5 Sep 2022 19:09:18 +0200, Justin Case said:
>> 
>> Yes, it does match. I then replaced the dummy password with the actual FD 
>> password, and the error did not occur any more when starting copy jobs. This 
>> behaves differently than documented, that’s why I am asking. 
>> 
>>> On 5. Sep 2022, at 17:56, Martin Simmons  wrote:
>>> 
>>> That looks strange to me.  The "JobId 0" maybe means that it was caused by
>>> something else.  Does the time "04-Sep 14:19" match the sequence of times in
>>> the other messages about the copy job?
>>> 
>>> __Martin
>>> 
>>> 
>>>>>>>> On Sun, 4 Sep 2022 14:33:03 +0200, Justin Case said:
>>>> 
>>>> Hi there,
>>>> 
>>>> I took this snipped from the main documentation about copy jobs:
>>>> 
>>>> # # Fake client for copy jobs # 
>>>> Client { 
>>>> Name = None 
>>>> Address = localhost 
>>>> Password = "NoNe” 
>>>> Catalog = MyCatalog 
>>>> } 
>>>> 
>>>> # # Default template for a CopyDiskToTape Job # JobDefs { 
>>>> 
>>>> Name = CopyDiskToTape 
>>>> Type = Copy 
>>>> Messages = StandardCopy 
>>>> Client = None 
>>>> FileSet = None 
>>>> Selection Type = PoolUncopiedJobs 
>>>> Maximum Concurrent Jobs = 10 
>>>> SpoolData = No 
>>>> Allow Duplicate Jobs = Yes 
>>>> Cancel Queued Duplicates = No 
>>>> Cancel Running Duplicates = No 
>>>> Priority = 13 
>>>> } 
>>>> 
>>>> It says: "The Copy Job runs without using the File daemon by copying the 
>>>> data from the old backup Volume to a different Volume in a different 
>>>> Pool.” So there is no need for a working Client.
>>>> 
>>>> This does not seem to be entirely factual. I configured it as suggested 
>>>> above, but then the Director gives me:
>>>> 
>>>> 04-Sep 14:19 bacula-dir JobId 0: Fatal error: authenticatebase.cc:435 
>>>> Director unable to authenticate with File Daemon at "localhost:9102". 
>>>> Possible causes: Passwords or names not the same or
>>>> 
>>>> The copy job still works, but I think it is not suitable to provoke such 
>>>> an error.
>>>> 
>>>> Why would the Director actually connect to the client? May I suppress this?
>>>> 
>>>> Thanks for considering my question,
>>>> J/C
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> ___
>>>> Bacula-users mailing list
>>>> Bacula-users@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>> 
>>> 
>> 
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: Copy jobs dummy client problem

[Justin Case]

I have that on 2 backup jobs that are copied, but not on the copy job itself. 
doe copy jobs run the before/after of all the jobs being copied?

> On 6. Sep 2022, at 13:02, Martin Simmons  wrote:
> 
> Maybe you have a run before/after directive that is causing it to run a
> bconsole command?
> 
> __Martin
> 
> 
>>>>>> On Mon, 5 Sep 2022 21:02:52 +0200, Justin Case said:
>> 
>> It works for me, too, but I get this error.
>> 
>>> On 5. Sep 2022, at 20:14, Martin Simmons  wrote:
>>> 
>>> A copy job works for me in Bacula 13.0.0 without this problem (even if the
>>> bacula-fd isn't running).
>>> 
>>> __Martin
>>> 
>>> 
>>>>>>>> On Mon, 5 Sep 2022 19:09:18 +0200, Justin Case said:
>>>> 
>>>> Yes, it does match. I then replaced the dummy password with the actual FD 
>>>> password, and the error did not occur any more when starting copy jobs. 
>>>> This behaves differently than documented, that’s why I am asking. 
>>>> 
>>>>> On 5. Sep 2022, at 17:56, Martin Simmons  wrote:
>>>>> 
>>>>> That looks strange to me.  The "JobId 0" maybe means that it was caused by
>>>>> something else.  Does the time "04-Sep 14:19" match the sequence of times 
>>>>> in
>>>>> the other messages about the copy job?
>>>>> 
>>>>> __Martin
>>>>> 
>>>>> 
>>>>>>>>>> On Sun, 4 Sep 2022 14:33:03 +0200, Justin Case said:
>>>>>> 
>>>>>> Hi there,
>>>>>> 
>>>>>> I took this snipped from the main documentation about copy jobs:
>>>>>> 
>>>>>> # # Fake client for copy jobs # 
>>>>>> Client { 
>>>>>> Name = None 
>>>>>> Address = localhost 
>>>>>> Password = "NoNe” 
>>>>>> Catalog = MyCatalog 
>>>>>> } 
>>>>>> 
>>>>>> # # Default template for a CopyDiskToTape Job # JobDefs { 
>>>>>> 
>>>>>> Name = CopyDiskToTape 
>>>>>> Type = Copy 
>>>>>> Messages = StandardCopy 
>>>>>> Client = None 
>>>>>> FileSet = None 
>>>>>> Selection Type = PoolUncopiedJobs 
>>>>>> Maximum Concurrent Jobs = 10 
>>>>>> SpoolData = No 
>>>>>> Allow Duplicate Jobs = Yes 
>>>>>> Cancel Queued Duplicates = No 
>>>>>> Cancel Running Duplicates = No 
>>>>>> Priority = 13 
>>>>>> } 
>>>>>> 
>>>>>> It says: "The Copy Job runs without using the File daemon by copying the 
>>>>>> data from the old backup Volume to a different Volume in a different 
>>>>>> Pool.” So there is no need for a working Client.
>>>>>> 
>>>>>> This does not seem to be entirely factual. I configured it as suggested 
>>>>>> above, but then the Director gives me:
>>>>>> 
>>>>>> 04-Sep 14:19 bacula-dir JobId 0: Fatal error: authenticatebase.cc:435 
>>>>>> Director unable to authenticate with File Daemon at "localhost:9102". 
>>>>>> Possible causes: Passwords or names not the same or
>>>>>> 
>>>>>> The copy job still works, but I think it is not suitable to provoke such 
>>>>>> an error.
>>>>>> 
>>>>>> Why would the Director actually connect to the client? May I suppress 
>>>>>> this?
>>>>>> 
>>>>>> Thanks for considering my question,
>>>>>> J/C
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> ___
>>>>>> Bacula-users mailing list
>>>>>> Bacula-users@lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>> 
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: How to verify copy jobs?

[Justin Case]

Greetings,

so far I don’t yet have a nice solution for the problem described earlied (see 
attached email).

Since verify jobs can only verify the latest job run with a specific name, and 
since copy jobs have the same name as the original backup job, the only way to 
verify both, the backup job AND the copy job seems to be:

(1) run backup job “examplejob"
(2) run verify job for “examplejob”
(3) run copy job for “examplejob”
(4) run verify job for “examplejob”

It is must happen in this order I guess. If I would switch (2) and (3) both 
verify jobs would verify the volumes of the copy job, correct?

So I need to provide priorities accordingly to the 4 jobs to run in this order? 
(As I don’t want to create a set of schedules for each of these jobs).

Happy to hear about better solutions you are using,
 J/C

> On 4. Sep 2022, at 18:27, Justin Case  wrote:
> 
> Hi there,
> 
> today I switched to using copy jobs to create the tier 2 backup (instead of 
> backup jobs from the original source). I realized that the copied job on tier 
> 2 storage has the exact same name as the job on tier from which it was copied.
> 
> I am now unclear how I can schedule verify jobs for the tier copies. 
> Previously I used the name of the tier 2 back up to identify the job that 
> should be verified. Now the tier 2 copy job has the same name as the tier 1 
> backup job. How do I discern them when scheduling (I could not use the job 
> IDs as there are not known when the verify job is scheduled).
> 
> Any ideas how to achieve this?
> 
> Thanks for considering my question,
> J/C
> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Q: How to verify copy jobs?

[Justin Case]

My experience with trying the steps planned in my previous email (as attached):

(2) verifies the backup job (1) as expected.

(4) does NOT verify the copy (3) on the pool and storage as provided for the 
verify job, instead it verifies the backup job (1) by switching to the pool, 
storage and volume used in (1).

Using the copy job name as VerifyJob fails with "Fatal error: Unable to find 
JobId of previous Job for this client.”

So now actually it seems to be impossible to verify copy jobs in a scheduled 
way, as we only know the names of the original backup job (1), and the name of 
the copy job (3), but we cannot know the job ID of the copy job (3) before it 
is executed.

Is it actually to verify copy jobs in a scheduled manner? If so, how?

> On 6. Sep 2022, at 15:38, Justin Case  wrote:
> 
> Greetings,
> 
> so far I don’t yet have a nice solution for the problem described earlied 
> (see attached email).
> 
> Since verify jobs can only verify the latest job run with a specific name, 
> and since copy jobs have the same name as the original backup job, the only 
> way to verify both, the backup job AND the copy job seems to be:
> 
> (1) run backup job “examplejob"
> (2) run verify job for “examplejob”
> (3) run copy job for “examplejob”
> (4) run verify job for “examplejob”
> 
> It is must happen in this order I guess. If I would switch (2) and (3) both 
> verify jobs would verify the volumes of the copy job, correct?
> 
> So I need to provide priorities accordingly to the 4 jobs to run in this 
> order? (As I don’t want to create a set of schedules for each of these jobs).
> 
> Happy to hear about better solutions you are using,
> J/C
> 
>> On 4. Sep 2022, at 18:27, Justin Case  wrote:
>> 
>> Hi there,
>> 
>> today I switched to using copy jobs to create the tier 2 backup (instead of 
>> backup jobs from the original source). I realized that the copied job on 
>> tier 2 storage has the exact same name as the job on tier from which it was 
>> copied.
>> 
>> I am now unclear how I can schedule verify jobs for the tier copies. 
>> Previously I used the name of the tier 2 back up to identify the job that 
>> should be verified. Now the tier 2 copy job has the same name as the tier 1 
>> backup job. How do I discern them when scheduling (I could not use the job 
>> IDs as there are not known when the verify job is scheduled).
>> 
>> Any ideas how to achieve this?
>> 
>> Thanks for considering my question,
>> J/C
>> 
>> 
> 



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Q: is Level a selection criterion for Migration/Copy jobs?

[Justin Case]

Hi all,

I am wondering whether Level is also a selection criterion for migration/copy 
jobs, i.e. if there are for a given Job Name a full backup from yesterday and 
an incremental backup from today, will I be able to select the full backup if I 
use the job name and Level=Full?

Thanks for considering my question,
 J/C




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users