Jonathan,
In principle, yes, it is possible for bacula to back up to S3 compatible
cloud storage providers.
However, there are some caveats.
The short answer is:
Yes, you can run bacula cloud backup and copy jobs with S3 compatible
storage as a target, IF you are running the bacula Director (DIR) and
Storage Daemon (SD) on some *nix or BSD OS. You will NOT want to target
bacula backups to an S3 bucket 'mounted' as a drive.
You mention bacula on windows. My (cloud) setup has windows FDs, but the
DIR and SD are running in a linux virtual machine (in my case, Rocky linux
9, which is basically RHEL 9). To enable cloud functionality I have
installed the bacula-cloud plugin. I don't think that plugin is available
for directors or SDs on windows. (Can you run a DIR or SD on windows?) So
my point is, I'm pretty sure you'd have to have your bacula director and SD
running on a *nix or BSD OS instance - OR - run a docker container with
bacula already inside it. Bacularis offers some docker containers with
bacula included, so that could be an option if you are running the DIR and
SD on windows (however, I'm not sure if those containers include the cloud
plugin).
You mention an app that lets you mount cloud storage like it was a local
drive. This is a useful concept in general, but not good for bacula to use
as a storage target. The bacula cloud implementation doesn't behave exactly
like the bacula local storage implementation. It splits volumes into
'parts' to ease uploading, manages background uploads of those volume part
files, and optionally retains copies of those .part files as a local cache
to enable local restores. With a local S3 'mount' like you described,
bacula will have no way to know if the S3 connection is functional, or
whether or not the volumes have been uploaded successfully. I believe it is
essential for bacula to 'know' about the upload status of cloud volume
parts.
Bacula uses its cloud 'driver' to interface with S3 compatible buckets.
There are two drivers - the 'S3' driver, and the 'Amazon' driver. The
Amazon driver is newer (introduced in the latest bacula, 15.x). The Amazon
driver should be able to interface with many S3 compatible hosts, and is
NOT limited to the AWS service hosted by the Amazon company. I am using
Backblaze's B2 S3 compatible service, and I am rather happy with it and the
Amazon driver on bacula community edition 15.0.2
If you are running another version of bacula than 15.x, you might think
"surely, how bad can the S3 driver be?" I thought that. I was mistaken.
While in a pinch, the S3 driver can work (and does work for many people!),
it simply is less reliable and will give you strange problems that the
Amazon driver will avoid. So I recommend using bacula 15.x, and the Amazon
driver.
Assuming your DIR and SD are hosted in *nix or BSD, then here are the basic
ideas
This list necessarily omits some details, and glosses over important facts.
(I can flesh these out later if you're interested):
0. Preferably, be running bacula 15.x from the bacula community
repositories.
1. You will install the bacula-cloud plugin in your OS package manager
2. Configuration of the cloud connection is initially like a normal file
writer. in bacula-sd-conf, you specify a Cloud{} resource that contains
your bucket access and secret keys. Reference this 'cloud' device in
otherwise relatively normal SD 'devices' and 'autochangers'. This lets
bacula know it isn't dealing with a normal file volume 'device'.
3. The one thing about the Amazon driver is that you MUST specify the
BlobEndpoint option if not using an actual Amazon AWS S3 bucket. This fact
isn't very well documented.
4. Most other setup is similar to normal bacula. You'll specify your cloud
autochanger in the bacula-dir.conf
5. You can run regular backup or copy jobs with the cloud devices as the
target. In the background, volume parts will be uploaded, and the cache
either retained or truncated, depending on how you configured it in the SD
resources.
6. One thing that is different from 'regular' local bacula is that you'll
want to run an admin job that periodically looks for cloud volume .part
files that haven't been uploaded, and uploads them to the cloud device. We
often refer to this as a "cloud volume part sweeper" script. There are
simple versions of this that don't raise a red flag if certain cloud parts
routinely or repeatedly fail to upload. The simpler scripts are definitely
effective and bulletproof, but again, that warning is important IMO. I have
written a script that will do the uploading, THEN analyzes the upload
process logs and exits with an error condition to alert an admin to the
issue. I can share this script if you find any of this interesting.
7. It is important to monitor a setup like this. Things can go wrong. One
of the scarier things that can go wrong is that if the cloud volume parts
aren't uploaded to the S3 compatible bucket, then you don't actually have a
cloud backup. You just have a local backup. Of course, it is also possible
for the data stored in a bucket to 'go away' if one fails to pay the cloud
bill for long enough, so that is also an existential threat.
8. For monitoring, I strongly recommend Bill A's BaculaBackupReport script,
available on github.
I (and others) can help with any of the things mentioned above. Let me know
if any of that is interesting to you, or if you have other questions.
Regards,
Robert Gerber
402-237-8692
r...@craeon.net
On Wed, Mar 26, 2025 at 11:53 PM Jonathan Aquilina via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:
> Good Afternoon,
>
>
>
> I am contemplating thanks to another thread mentioning bacula on windows
> to use this to backup my PC and NAS to an S3 bucket.
>
>
>
> Is this something that can be done with Bacula. Note that the NAS drives
> are mapped to the pc using the ISCSI initiator to map the drives like they
> are in the machine.
>
>
>
> Is it possible to get bacula to connect and talk to an S3 bucket? I also
> have that with a special app mapped like a drive in terms of my S3 bucket.
>
>
>
>
>
> Regards,
>
> Jonathan Aquilina
>
>
>
>
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
