On 21.05.2015 19:11, Devin Reade wrote: > --On Thursday, May 21, 2015 11:09:58 AM -0500 dweimer <dwei...@dweimer.net> > wrote: > >> I have three systems two of which are using disk backup, then copy to >> tape both of those are running on CentOS with 25G volume sizes [...] >> The third system is using 46G File volumes > Thanks. Good to know. > >> I wouldn't worry about Bacula's ability, but more the capabilities of the >> file system and operating system its running on. > FreeBSD, ZFS (RAID-Z2) thus large file support, large capacity > commodity drives, but otherwise enterprise-grade gear. The > bottleneck right now is network speeds; we probably need to go to > 10GbE on the LAN, straight to the workstations. > > Believe me, I've been quite happy with Bacula over the years. This is > just a case where the data pattern is unusual. My questions about issues > was aimed more at second-order effects that might not be obvious in > smaller installations. The tradeoff of volume size against number of > volumes while satisfying retention times is one such. > > When I was saying that I'm not sure yet if Bacula is the right tool, > that was not a reflection on Bacula but on the data. Certainly I will > be using Bacula for the smaller working set, but the with the largest > amount of data being write-once, that implies that I'm writing out > that entire data set every month or two months (whatever the full > schedule is) indefinitely. > > One of the options I'm considering is something like setting up > pairs of drives in a ZFS mirror in removable drive caddies, > putting sets of the write-once data on such pairs via rsync or > some-such, and then making 3 or 4 copies of those pairs of drives > as the archival copies. That way, each pair gets written *once*, > gets put into cold storage, and not touched again (other than > perhaps running a ZFS scrub on the copies every few years). > > The big advantage of that option is that the amount of data that > needs to be written on a monthly/whatever basis is no more than > the size of the working set (via Bacula) plus the size of one > archive pair, rather than the size of the entire data set. > > The big down-side of that option is on the data management side; > the lack of the equivalent of Bacula's catalog and all the > software behind it. I've either got to create my own mechanism > for keeping the catalog of those archive pairs, or find a tool > that is already made for the purpose. Maybe CERN has something ...
CERN is using 12 instances of TSM for their administrative data (enormous). This could be replaced by 2 instances of Bacula :-) For their experimental data they are using Casper,which is open source and published on their web site. However, for this data they are not worried about a few losses -- the data arrives so fast it is hard to capture it all. I would love to modify Bacula to handle their experimental data and I have told them so, but for the moment they have not given me the chance -- it may be too late as I am a bit too old to take on such a monster project. > > Devin > > > ------------------------------------------------------------------------------ > One dashboard for servers and applications across Physical-Virtual-Cloud > Widest out-of-the-box monitoring support with 50+ applications > Performance metrics, stats and reports that give you Actionable Insights > Deep dive visibility with transaction tracing using APM Insight. > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > _______________________________________________ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > ------------------------------------------------------------------------------ One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
