I have been using Bacula for over a year now and it has been providing
'passable' service though I think since day one I have been streching it to
it's limits or need a paradigm shift in how I am configuring it.
Basically, I have a single server which has direct atached disk (~128TB / 112
drives) and Tape drives (LTO4). It's main function is a centralized file server
& archival server. It has several mount points (~20) (ZFS) to break down some
structures based on file size and intended use basically spawning a new
mountpoint for anything > a couple TB or 100,000 files. Some file systems are
up to 30TB in size others are only a handful of GB. With ~4,000,000 files
anywhere from 4KiB up to 32GiB in size.
Data change is about 1-2TiB/month which is not that big of an issue. The
problem is when I need to do full backups and restores (restores mainly ever
1-2 years when I have to do forklift replacement of drives). Bottlenecks that I
see are:
- File daemon is single threaded so is limiting backup performance. Is there
was a way to start more than one stream at the same time for a single machine
backup? Right now I have all the file systems for a single client in the same
file set.
- Tied in with above, accurate backups cut into performance even more when
doing all the md5/sha1 calcs. Spliting this perhaps with above to multiple
threads would really help.
- How to stream a single job to multiple tape drives. Couldn't figure this out
so that only one tape drive is being used.
- spooling to disk first then to tape is a killer. if multiple streams could
happen at once this may mitigate this or some type of continous spooling. How
do others do this?
At this point I'm starting to look at Arkeia & Netbackup both with provide
multistreaming and tape drive pooling, but would rather stick or send my $$ to
open source if I could opposed to closed systems.
I'm at a point where I can't do a 20-30day full backup. And 'virtual fulls' are
not an answer. There's no way I can tie up tape drives for the hundreds of
tapes at 2.5 hours per tape assuming zero processing overhead. I have plenty of
cpu on the system and plenty of disk subsystem speed, just can't seem to get at
it through bacula.
So what options are available or how are others backing up huge single servers?
------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
