I've been using bacula with S3 using s3fs for about 6 months. To say it has been flakey is an understatement and I plan to make changes over the coming months, although still using s3/glacier. s3fs is a filesystem for S3, I believe the only free one (it's GPLv2).
Backups: Small backups are fine, but large backups taking > ~2 days inevitably have problems (ie connection drops) and neither s3fs or bacula are good at resuming transfers/backups. Large backups need to be stored locally and then transfered to s3 post job. It's also extra important to backup the database straight after the initial full backup as if something goes wrong you'll have to spend a long time re-upload/downloading full backups. Restores: Restoring directly using s3fs is only possible if the volume is completely cached locally by s3fs. I've never managed a backup that hasn't had the entire volume manually downloaded (either filling the cache or just cheating and remounting), so using a small volume size is essentially. I look at the volumes required when using the restore command and manually get them to cache. My Future Plan: Ditch s3fs, it's not reliable enough. Instead I plan to use a combination of s3cmd and a simple post job script to verify correct transfer. s3cmd is just a cp/mv/rsync kind of tool and so doesn't suffer of the issues s3fs does. My only concern is correctly verifying transfers have been successful. I've never used AWS storage gateway, the economics don't work in my instance, I run multiple small sites - it's $125 a head. This might be an option for some, but I still think baculas inability to resume backups would be a problem. Perhaps if bacula had a built cache, copy and verify mechanism use of such offsite backup services would be easier and safer. I.e. do backup to local disk (to avoid large jobs taking weeks/months and potentially bombing half way through), copy these volumes to another location and then verify the copy. A similar procedure in reverse would also remove some headaches for restorations. Edward On 21 August 2012 15:39, Mr IT Guru <misteritg...@gmx.com> wrote: > > Good Afternoon All, > > On 21 Aug 2012, at 13:38, eric santelices <ericsa...@gmail.com> wrote: > > With Amazon announcement of Glacier it seems like a natural fit for Bacula. > Does anyone know if/when we would see support for this storage resource? > > > > If you wanted to utilize this, wouldn't it be better to have the OS look into > mounting this service as a file system, and then just direct bacula to use > the file system? Rather than coding specifically for bacula - I'm sure bacula > is not going to be the only project that can benefit from this, so I'm > betting that support for this will be in the underlying OS first. > > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
