>>>>> On Fri, 29 Dec 2023 12:35:59 -0500, Dan Langille said:
>
> On Fri, Dec 29, 2023, at 12:10 PM, Martin Simmons wrote:
> > 9.6.6 certainly displayed them for me, so I suspect a config issue.
> >
> > The messages would be omitted if !notsaved is in the Messages resource (but
> > they would still be counted as "Non-fatal FD errors" which makes it add
> > "with
> > warnings" to the status).
> >
> > Maybe that changed in the client's bacula-fd.conf when you upgraded it?
>
> That's a good idea.
>
> [17:29 r730-01 dvl ~] % sudo ls -l /usr/local/etc/bacula/bacula-fd.conf
> -rw-r----- 1 root bacula 1497 Feb 25 2023
> /usr/local/etc/bacula/bacula-fd.conf
>
> [17:31 r730-01 dvl ~] % sudo md5 /usr/local/etc/bacula/bacula-fd.conf
> MD5 (/usr/local/etc/bacula/bacula-fd.conf) = e41a7d835766f563253c0a93418a1c61
>
>
> No change since February.
>
> Let's look at snapshots taken before Dec 25, the date of the job in question.
>
> [17:32 r730-01 dvl /.zfs/snapshot] % cd autosnap_2023-12-20_00:00:09_daily
> [17:32 r730-01 dvl /.zfs/snapshot/autosnap_2023-12-20_00:00:09_daily] % sudo
> md5 usr/local/etc/bacula/bacula-fd.conf
> MD5 (usr/local/etc/bacula/bacula-fd.conf) = e41a7d835766f563253c0a93418a1c61
>
>
> I'm confident this file has not changed.
Hmm, looking at src/lib/message.h, I suspect this change broke version
compatibility in the message filtering infrastructure:
commit fd926fc4671b054234fd3d5957bc05d303d87763
Author: Eric Bollengier <e...@baculasystems.com>
Date: Fri Nov 6 21:27:05 2020 +0100
Fix unexpected connection event sent by the FD when the Message resource is
not configured
The problem is that the message types have been renumbered by moving M_EVENTS
higher up, but messages sent to the Director from other daemons use the
numeric value of the type so this is an incompatible change in the wire
protocol.
Dispite the date of this change, it looks like it first appeared in Bacula 13,
so will cause problems if a Client < 13 sends a message to a Director >= 13 as
in your case.
>
> Here is it, with some redactions:
>
> [17:33 r730-01 dvl /.zfs/snapshot/autosnap_2023-12-20_00:00:09_daily] % sudo
> cat usr/local/etc/bacula/bacula-fd.conf
> Director {
> Name = bacula-dir
> Password = "[redacted]"
>
> TLS Enable = yes
> TLS Require = yes
> TLS Verify Peer = yes
>
> # Allow only the Director to connect
> TLS Allowed CN = "bacula.example.org"
> TLS CA Certificate File = /usr/local/etc/ssl/MyBigCA.crt
>
> TLS Certificate = /usr/local/etc/ssl/r730-01.int.unixathome.org.crt
> TLS Key =
> /usr/local/etc/ssl/r730-01.int.unixathome.org.nopassword.key
> }
>
> Director {
> Name = nagios-mon
> Password = "[redacted]"
> Monitor = yes
> }
>
> FileDaemon {
> Name = r730-01-fd
> FDAddress = 10.55.0.141
> FDport = 9102
> WorkingDirectory = /var/db/bacula
> Pid Directory = /var/run
> Maximum Concurrent Jobs = 20
>
> TLS Enable = yes
> TLS Require = yes
>
> TLS CA Certificate File = /usr/local/etc/ssl/MyBigCA.crt
>
> TLS Certificate = /usr/local/etc/ssl/r730-01.int.unixathome.org.crt
> TLS Key =
> /usr/local/etc/ssl/r730-01.int.unixathome.org.nopassword.key
> }
>
> # Send all messages except skipped files back to Director
> Messages {
> Name = Standard
> director = bacula-dir = all, !skipped, !restored
> }
>
>
> >
> > __Martin
> >
> >
> >>>>>> On Mon, 25 Dec 2023 07:06:58 -0500, Dan Langille said:
> >>
> >> Hello,
> >>
> >> This is more for advising others than looking for a fix.
> >>
> >> bacula9 client mentions warnings and does not list them.
> >> bacula13 client mentions the warnings.
> >>
> >> It turns out, the missing warnings are rather important to know. The ZFS
> >> datasets in question are jailed, and need a different path to the mount
> >> point. Checking the log file, the warnings do not appear there either.
> >>
> >> Here are examples of the problem.
> >>
> >> The following email is from bacula9-client: 9.6.7_3 on FreeBSD 14. The
> >> subject of this message from Bacula mentions warnings.
> >>
> >> No warnings are supplied.
> >>
> >> From: (Bacula) d...@langille.org
> >> Subject: Bacula: Backup OK -- with warnings of r730-01-fd Incremental
> >> Sender: bac...@bacula.int.unixathome.org
> >> To: [redacted]
> >> Date: Mon, 25 Dec 2023 03:09:16 +0000 (UTC)
> >>
> >> 25-Dec 03:09 bacula-dir JobId 362083: Start Backup JobId 362083,
> >> Job=r730-01_snapshots.2023-12-25_03.05.00_24
> >> 25-Dec 03:09 bacula-dir JobId 362083: Connected to Storage
> >> "bacula-sd-04-IncrFile" at bacula-sd-04.int.unixathome.org:9103 with TLS
> >> 25-Dec 03:09 bacula-dir JobId 362083: There are no more Jobs associated
> >> with Volume "IncrAuto-04-14761". Marking it purged.
> >> 25-Dec 03:09 bacula-dir JobId 362083: All records pruned from Volume
> >> "IncrAuto-04-14761"; marking it "Purged"
> >> 25-Dec 03:09 bacula-dir JobId 362083: Recycled volume "IncrAuto-04-14761"
> >> 25-Dec 03:09 bacula-dir JobId 362083: Using Device "vDrive-IncrFile-5" to
> >> write.
> >> 25-Dec 03:09 bacula-dir JobId 362083: Connected to Client "r730-01-fd" at
> >> r730-01.int.unixathome.org:9102 with TLS
> >> 25-Dec 03:09 r730-01-fd JobId 362083: shell command: run ClientBeforeJob
> >> "/usr/local/sbin/snapshots-for-backup.sh create"
> >> 25-Dec 03:09 bacula-sd-04 JobId 362083: Recycled volume
> >> "IncrAuto-04-14761" on File device "vDrive-IncrFile-5"
> >> (/usr/local/bacula/volumes/IncrFile), all previous data lost.
> >> 25-Dec 03:09 bacula-dir JobId 362083: Max Volume jobs=1 exceeded. Marking
> >> Volume "IncrAuto-04-14761" as Used.
> >> 25-Dec 03:09 r730-01-fd JobId 362083: shell command: run ClientAfterJob
> >> "/usr/local/sbin/snapshots-for-backup.sh destroy"
> >> 25-Dec 03:09 bacula-sd-04 JobId 362083: Elapsed time=00:00:14, Transfer
> >> rate=32.35 K Bytes/second
> >> 25-Dec 03:09 bacula-sd-04 JobId 362083: Sending spooled attrs to the
> >> Director. Despooling 10,154 bytes ...
> >> 25-Dec 03:09 bacula-dir JobId 362083: Bacula bacula-dir 13.0.3 (02May23):
> >> Build OS: amd64-portbld-freebsd14.0 freebsd 14.0-RELEASE-p1
> >> JobId: 362083
> >> Job: r730-01_snapshots.2023-12-25_03.05.00_24
> >> Backup Level: Incremental, since=2023-12-25 01:51:02
> >> Client: "r730-01-fd" 9.6.7 (10Dec20)
> >> amd64-portbld-freebsd14.0,freebsd,14.0-RELEASE-p1
> >> FileSet: "r730-01 snapshots" 2023-12-25 01:30:49
> >> Pool: "IncrFile-04" (From Job IncPool override)
> >> Catalog: "MyCatalog" (From Client resource)
> >> Storage: "bacula-sd-04-IncrFile" (From Pool resource)
> >> Scheduled time: 25-Dec-2023 03:05:00
> >> Start time: 25-Dec-2023 03:09:01
> >> End time: 25-Dec-2023 03:09:16
> >> Elapsed time: 15 secs
> >> Priority: 10
> >> FD Files Written: 34
> >> SD Files Written: 34
> >> FD Bytes Written: 446,284 (446.2 KB)
> >> SD Bytes Written: 453,016 (453.0 KB)
> >> Rate: 29.8 KB/s
> >> Software Compression: None
> >> Comm Line Compression: 56.7% 2.3:1
> >> Snapshot/VSS: no
> >> Encryption: no
> >> Accurate: no
> >> Volume name(s): IncrAuto-04-14761
> >> Volume Session Id: 52
> >> Volume Session Time: 1703358455
> >> Last Volume Bytes: 455,013 (455.0 KB)
> >> Non-fatal FD errors: 3
> >> SD Errors: 0
> >> FD termination status: OK
> >> SD termination status: OK
> >> Termination: Backup OK -- with warnings
> >>
> >> 25-Dec 03:09 bacula-dir JobId 362083: Begin pruning Jobs older than 3
> >> years .
> >> 25-Dec 03:09 bacula-dir JobId 362083: No Jobs found to prune.
> >> 25-Dec 03:09 bacula-dir JobId 362083: Begin pruning Files.
> >> 25-Dec 03:09 bacula-dir JobId 362083: No Files found to prune.
> >> 25-Dec 03:09 bacula-dir JobId 362083: End auto prune.
> >>
> >>
> >> I replaced that bacula9-client: 9.6.7_3 with bacula13-client: 13.0.3 and
> >> reran the job.
> >>
> >> The warnings are now listed.
> >>
> >> 25-Dec 11:53 bacula-dir JobId 362099: Start Backup JobId 362099,
> >> Job=r730-01_snapshots.2023-12-25_11.52.58_26
> >> 25-Dec 11:53 bacula-dir JobId 362099: Connected to Storage
> >> "bacula-sd-04-IncrFile" at bacula-sd-04.int.unixathome.org:9103 with TLS
> >> 25-Dec 11:53 bacula-dir JobId 362099: There are no more Jobs associated
> >> with Volume "IncrAuto-04-14785". Marking it purged.
> >> 25-Dec 11:53 bacula-dir JobId 362099: All records pruned from Volume
> >> "IncrAuto-04-14785"; marking it "Purged"
> >> 25-Dec 11:53 bacula-dir JobId 362099: Recycled volume "IncrAuto-04-14785"
> >> 25-Dec 11:53 bacula-dir JobId 362099: Using Device "vDrive-IncrFile-5" to
> >> write.
> >> 25-Dec 11:53 bacula-dir JobId 362099: Connected to Client "r730-01-fd" at
> >> r730-01.int.unixathome.org:9102 with TLS
> >> 25-Dec 11:53 r730-01-fd JobId 362099: Connected to Storage at
> >> bacula-sd-04.int.unixathome.org:9103 with TLS
> >> 25-Dec 11:53 r730-01-fd JobId 362099: shell command: run ClientBeforeJob
> >> "/usr/local/sbin/snapshots-for-backup.sh create"
> >> 25-Dec 11:53 bacula-sd-04 JobId 362099: Recycled volume
> >> "IncrAuto-04-14785" on File device "vDrive-IncrFile-5"
> >> (/usr/local/bacula/volumes/IncrFile), all previous data lost.
> >> 25-Dec 11:53 bacula-dir JobId 362099: Max Volume jobs=1 exceeded. Marking
> >> Volume "IncrAuto-04-14785" as Used.
> >> 25-Dec 11:53 r730-01-fd JobId 362099: Could not stat
> >> "/var/db/ingress/latest_commits/.zfs/snapshot/snapshot-for-backup": ERR=No
> >> such file or directory
> >> 25-Dec 11:53 r730-01-fd JobId 362099: Could not stat
> >> "/var/db/ingress/latest_commits/.zfs/snapshot/snapshot-for-backup": ERR=No
> >> such file or directory
> >> 25-Dec 11:53 r730-01-fd JobId 362099: Could not stat
> >> "/var/db/ingress/latest_commits/.zfs/snapshot/snapshot-for-backup": ERR=No
> >> such file or directory
> >> 25-Dec 11:53 r730-01-fd JobId 362099: shell command: run ClientAfterJob
> >> "/usr/local/sbin/snapshots-for-backup.sh destroy"
> >> 25-Dec 11:53 bacula-sd-04 JobId 362099: Elapsed time=00:00:11, Transfer
> >> rate=446.2 K Bytes/second
> >> 25-Dec 11:53 bacula-sd-04 JobId 362099: Sending spooled attrs to the
> >> Director. Despooling 77,691 bytes ...
> >> 25-Dec 11:53 bacula-dir JobId 362099: Bacula bacula-dir 13.0.3 (02May23):
> >> Build OS: amd64-portbld-freebsd14.0 freebsd 14.0-RELEASE-p1
> >> JobId: 362099
> >> Job: r730-01_snapshots.2023-12-25_11.52.58_26
> >> Backup Level: Incremental, since=2023-12-25 03:09:01
> >> Client: "r730-01-fd" 13.0.3 (02May23)
> >> amd64-portbld-freebsd14.0,freebsd,14.0-RELEASE-p1
> >> FileSet: "r730-01 snapshots" 2023-12-25 01:30:49
> >> Pool: "IncrFile-04" (From Job IncPool override)
> >> Catalog: "MyCatalog" (From Client resource)
> >> Storage: "bacula-sd-04-IncrFile" (From Pool resource)
> >> Scheduled time: 25-Dec-2023 11:52:57
> >> Start time: 25-Dec-2023 11:53:01
> >> End time: 25-Dec-2023 11:53:15
> >> Elapsed time: 14 secs
> >> Priority: 10
> >> FD Files Written: 194
> >> SD Files Written: 194
> >> FD Bytes Written: 4,854,123 (4.854 MB)
> >> SD Bytes Written: 4,909,040 (4.909 MB)
> >> Rate: 346.7 KB/s
> >> Software Compression: None
> >> Comm Line Compression: 73.1% 3.7:1
> >> Snapshot/VSS: no
> >> Encryption: no
> >> Accurate: no
> >> Volume name(s): IncrAuto-04-14785
> >> Volume Session Id: 55
> >> Volume Session Time: 1703358455
> >> Last Volume Bytes: 4,919,905 (4.919 MB)
> >> Non-fatal FD errors: 3
> >> SD Errors: 0
> >> FD termination status: OK
> >> SD termination status: OK
> >> Termination: Backup OK -- with warnings
> >>
> >> 25-Dec 11:53 bacula-dir JobId 362099: Begin pruning Jobs older than 3
> >> years .
> >> 25-Dec 11:53 bacula-dir JobId 362099: No Jobs found to prune.
> >> 25-Dec 11:53 bacula-dir JobId 362099: Begin pruning Files.
> >> 25-Dec 11:53 bacula-dir JobId 362099: No Files found to prune.
> >> 25-Dec 11:53 bacula-dir JobId 362099: End auto prune.
> >>
> >> FYI, the Messages directive (with email addresses modified) appears below.
> >>
> >> Messages {
> >> Name = Standard
> >>
> >> mailcommand = "/usr/local/sbin/bsmtp -h cliff.example.org -f
> >> \"\(Bacula\) %r\" -s \"Bacula: %t %e of %c %l\" %r"
> >> operatorcommand = "/usr/local/sbin/bsmtp -h cliff.example.org -f
> >> \"\(Bacula\) %r\" -s \"Bacula: Intervention needed for %j\" %r"
> >> operator = d...@example.org = mount
> >> mail = d...@example.org = all
> >> console = all
> >>
> >> append = "/var/log/bacula/bacula.log" = all, !skipped, !restored
> >>
> >> catalog = all, !skipped, !saved
> >> }
> >>
> >> --
> >> Dan Langille
> >> d...@langille.org
> >>
> >>
> >> _______________________________________________
> >> Bacula-users mailing list
> >> Bacula-users@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >>
>
> --
> Dan Langille
> d...@langille.org
>
__Martin
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
