Morning Folks,
I was keen to use the new cloud storage feature so I've
configured things up to use it as an offsite repository of some
backup jobs.
I use the 'Next Pool' Pool parameter and then a Copy type job
on PoolUncopiedJobs
I start a job and it seems to run OK but I'm seeing dozens of
errors similar to the following and the data from 'list volumes'
is very different to what is actually present on the S3 volume.
---
10-May 07:06 ghost-sd JobId 12751: Warning: cloud_dev.c:1748
For Volume "Offsite-0183":
The Parts do not match! Metadata Volume=16915 Catalog=16889.
Correcting Catalog
---
Sometimes a single job will get multiple
---
07-May 11:04 ghost-sd JobId 12608: Warning: cloud_dev.c:1748
For Volume "Offsite-0182":
The Parts do not match! Metadata Volume=6985 Catalog=6711.
The Cloud Parts do not match! Metadata Volume=1817
Catalog=1574.
Correcting Catalog
07-May 11:47 ghost-sd JobId 12608: Warning: cloud_dev.c:1748
For Volume "Offsite-0182":
The Parts do not match! Metadata Volume=10080 Catalog=9986.
The Cloud Parts do not match! Metadata Volume=1953
Catalog=1574.
Correcting Catalog
07-May 11:54 ghost-sd JobId 12608: Warning: cloud_dev.c:1748
For Volume "Offsite-0182":
The Parts do not match! Metadata Volume=10521 Catalog=10480.
The Cloud Parts do not match! Metadata Volume=1976
Catalog=1574.
Correcting Catalog
----
Director: 9.4.2 on Solaris
Storage: 9.4.2 on Solaris (disk based), 9.4.2 on Linux
(cloud based)
Clients: 9.4.2 on Solaris, Windows and Linux
Linux sd (ghost) is using Bacula community RPMs
bacula-libs-9.4.2-1.el7.x86_64
bacula-cloud-storage-9.4.2-1.el7.x86_64
bacula-mysql-9.4.2-1.el7.x86_64
I expect it's a bug in the new plugin but thought I'd see if
anyone had suggestions on how to proceed: it could be my
misconfiguration.
Note I do have Truncate Cache = AfterUpload since I already
have a local copy of the job and restoring from cloud would be
last resort.
Thanks for the help
* list volumes
<SNIP>
Pool: Offsite
+---------+--------------+-----------+---------+---------------+----------+--------------+---------+------+-----------+-----------+---------+----------+---------------------+------------+
| MediaId | VolumeName | VolStatus | Enabled |
VolBytes | VolFiles | VolRetention | Recycle | Slot |
InChanger | MediaType | VolType | VolParts | LastWritten
| ExpiresIn |
+---------+--------------+-----------+---------+---------------+----------+--------------+---------+------+-----------+-----------+---------+----------+---------------------+------------+
| 181 | Offsite-1 | Full | 1 |
262,087,761 | 0 | 15,552,000 | 1 | 0 |
0 | CloudType | 14 | 2,866 | 2019-05-05 11:43:59 |
15,011,184 |
| 182 | Offsite-0182 | Full | 1 |
1,073,696,800 | 0 | 15,552,000 | 1 | 0
| 0 | CloudType | 14 | 13,922 | 2019-05-07
12:44:02 | 15,187,587 |
| 183 | Offsite-0183 | Append | 1 |
630,979,806 | 0 | 15,552,000 | 1 | 0 |
0 | CloudType | 14 | 17,827 | 2019-05-11 16:59:43 |
15,548,528 |
| 184 | Offsite-0184 | Append | 1 |
0 | 0 | 15,552,000 | 1 | 0 | 0 |
CloudType | 14 | 0 | NULL |
NULL |
| 185 | Offsite-0185 | Append | 1 |
0 | 0 | 15,552,000 | 1 | 0 | 0 |
CloudType | 14 | 0 | NULL |
NULL |
| 186 | Offsite-0186 | Append | 1 |
0 | 0 | 15,552,000 | 1 | 0 | 0 |
CloudType | 14 | 0 | NULL |
NULL |
| 187 | Offsite-0187 | Full | 1 |
1,073,729,432 | 0 | 15,552,000 | 1 | 0
| 0 | CloudType | 14 | 14,859 | 2019-05-06
18:51:26 | 15,123,231 |
+---------+--------------+-----------+---------+---------------+----------+--------------+---------+------+-----------+-----------+---------+----------+---------------------+------------+
Firstly the VolBytes is very wrong, on the S3 side it's
439.633 GBytes (472052843913 Bytes) and what I'd expect based on
the Copy jobs.
Secondly there are different counts of VolParts:
13916 Offsite-0182
17722 Offsite-0183
14855 Offsite-0187
724 Offsite-1
* cloud list
<SNIP>
+--------------------+-----------+----------------------+----------------------+---------------+
| Volume Name | Status | Media Type
| Pool | VolBytes |
+--------------------+-----------+----------------------+----------------------+---------------+
| Offsite-0182 | Full | CloudType
| Offsite | 1.073 GB |
| Offsite-0183 | Append | CloudType
| Offsite | 630.9 MB |
| Offsite-0187 | Full | CloudType
| Offsite | 1.073 GB |
| Offsite-1 | Full | CloudType
| Offsite | 262.0 MB |
+--------------------+-----------+----------------------+----------------------+---------------+
---sd --
Device {
Name = CloudStorage
Device Type = Cloud
Cloud = RealS3
Archive Device = /opt/bacula/cloud_backups
Maximum Part Size = 10 MB
Media Type = CloudType
LabelMedia = yes
Random Access = Yes;
AutomaticMount = yes
RemovableMedia = no
AlwaysOpen = no
}
Cloud {
Name = RealS3
Driver = "S3"
HostName = <SNIP>
BucketName = <SNIP>
AccessKey = <SNIP>
SecretKey = <SNIP>
Protocol = HTTPS
UriStyle = Path
Truncate Cache = AfterUpload
Upload = EachPart
Region = "eu-frankfurt-1"
MaximumUploadBandwidth = 35MB/s
}
--- dir ---
Job {
Name = job.copyjob.full
Type = Copy
Pool = Full-Pool
Selection Type = PoolUncopiedJobs
Messages = Standard
Client = client.fake
FileSet="none"
Maximum Concurrent Jobs = 2
}
Pool {
Name = Offsite
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Storage = ghost-changer
Maximum Volume Jobs = 1
AutoPrune = yes
Volume Retention = 6 months
Maximum Volumes = 80
Label Format = Offsite-
}
Pool {
Name = Full-Pool
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Retention = 3 months
Maximum Volume Jobs = 1
Label Format = Full-
Maximum Volumes = 60
Next Pool = "Offsite"
Storage = File1
}
Autochanger {
Name = ghost-changer
Address = ghost.<SNIP>
SDPort = 9103
Password = <SNIP>
Device = CloudStorage
Media Type = CloudType
Autochanger = ghost-changer
Maximum Concurrent Jobs = 10
TLS Enable = yes
TLS Require = no
TLS CA Certificate File = /etc/bacula/certs/cacert.pem
TLS Certificate = "<SNIP>"
TLS Key = "<SNIP>"
}
Autochanger {
Name = File1
Address = otherhost.<SNIP>
SDPort = 9103
Password = "<SNIP>"
Device = FileChgr1
Media Type = File1
Maximum Concurrent Jobs = 10
Autochanger = File1
TLS Enable = yes
TLS Require = yes
TLS CA Certificate File = /etc/bacula/certs/cacert.pem
TLS Certificate = "<SNIP>"
TLS Key = "<SNIP>"
}
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
