Hi, 27.11.2009 13:23, Ralf Gross wrote: > [crosspost to -users and -devel list] > > Hi, > > we are happily using bacula since a few years and already backing up > some dozens of TB (large video files) to tape. > > In the next 2-3 years the amount of data will be growing to 300+ TB. > We are looking for some very pricy solutions for the primary storage > at the moment (NetApp etc). But we (I) are also looking if it is > possible to go on with the way we store the data right now. Which is > just some large raid arrays and backup2tape.
Good luck... while I agree that SAN/NAS appliances tend to look expensive, they've got their advantages when your space has to grow to really big sizes. Managing only one setup, when some physical disk arrays work together is one of these advantages. Also, if you're running a number of big RAID arrays, reliable vendor support is surely beneficial. > I've no problem with the primary storage and 10 very big raid boxes > (high availability is not needed). What frightens me is to backup all > the data. Most files will be written once and maybe never been > accessed again. Yearly full backups, and then only incrementals (using accurate backup mode) should be a usable approach. Depending on how often you expect to need recovery, you may want your boss to spend some money on a bigger tape library :-) to make sure most data can be restored without manually loading tapes. > But the data need to be online and there is a > requirement for backup and the ability to restore deleted files > (retention time can be different, going from 6 months to a couple of > years). 6 months retention time and actually pruning data would be easier with full backups more often than one year, probably. I think you should start by defining how long you want to keep your data, how to do full backups when those jobs will surely run longer than your regular backup windows (either splitting the jobs into smaller parts, or making sure you can run backups over a non-production network; measuring impact of backups on other file system accesses). > > The snapshot feature of some commerical products is a very nice > feature for taking backups and it's a huge benefit that only the > deltas are stored. You can build on snapshot capability of SAN filers with Bacula. You'll still get normal file backups, but that's an advantage IMO... the most useful aspect of those snapshots is that you get a consistent stae of the file system, and don't affect production access more than necessary. > Backup2tape means that with the classic Full/Diff/Incr setup we'll > need many tapes. Even if the data on the primary storage won't change. Sure - a backup is reliable if you've got at least two copies of your files, so for 300 TB, you'll need some tapes. But tapes are probably cheaper than the required disk capacity for a NetApp filer :-) > Backup2disk has the disadvantage that a corrupt filesystem (been > there, seen that...) can ruin TB's of backed up data. And we will need > file storage that is much bigger than the primary storage (keeping x > versions of a file....). Yup. Tapes get cheaper at that volume, especially since they don't need power when stored. > > Anyone else here with the same problem? Anyone (maybe Kern or Eric) > here that can tell if one of the upcoming new bacula features (dedup?) > could help to solve the problem with the massive amount of tapes > needed and the growing time windows and bandwidth for backups? Well, I'm not Kern or Eric - you've got Kerns feedback - but deduplication using a deduping virtual tape library (dVTL I'll call it) might be one way to go. Unfortunately, these things are, as far as I know, slow compared to real tape drives. And more expensive than a big RAID array for use with Bacula. > Any thought? I'd build something using LUN snapshots to minimize (not eliminate!) performance impact on production use of your file systems, a dedicated backup network to keep the production network from getting slow during backups, and a big tape library. For fast backups and recovery, you might add a big extra RAID array for Bacula use and implement disk to disk to tape backups using Baculas copy or migration feature. Data would end up on tapes for relatively cheap and energy-efficient storage. You'll need some nice big tape library, or employ a tape operator for a few hours a day ;-) Cheers, Arno > > Thanks, Ralf > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Bacula-devel mailing list > bacula-de...@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-devel > -- Arno Lehmann IT-Service Lehmann Sandstr. 6, 49080 Osnabrück www.its-lehmann.de ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
