how about archiving the 'tiny' items into larger tar- or zip-files on a regular basis (say every 15 minutes) and then only backup the archives.
Konstantin Khomoutov schrieb: > Hi! > > I have a somewhat unusual Bacula setup as we're backing call records > produced by a call-center software -- so the files are tiny in size, > there is very large number of them produced daily, they are all unique > and we have to keep them for several years as required by the local > law. > > Due to the uniqueness + tiny size + large quantities property of these > data, we have decided to keep only the information on the last few > backup session in the database to keep its size manageable, and resort > to "painful" restoration of old files -- for which job records are > expired by the volumes are still there -- using `bextract`. > > To do restores using `bextract` we have lists of files backed during > each backup session saved in flat files, and we also have Bacula > writing bootstrap file for each backup session. So the restoration > of old files basically rolls like this: > > 1) Know the date of the files to be restored. > 2) Find what backup session backed them up. > 3) Grab the matching bootstrap file. > 4) Use `bextract -b boostrap.bsr -V ... /output/dir`. > > The problem is that as it turned out, Bacula silently wasn't generating > bootstrap files for our backup jobs despite having > > Write bootstrap = "/path/to/bootstrap/files/bsr-%j" > > directive. Okay, that's for another question. > > So when we need to restore an old file, we know (1) and (2) but fail at > (3). I learned that `bls -j` is able to dump the list of jobs recorded > on a given volume. But what I'd really like to know is how could I > recreate bootstrap files from the output of `bls -j` run on old volumes? > > Programming the solution is not the problem but I'd like to know how do > I properly form the contents for such bootstrap files given the records > I'd parse from the output of `bls -j`. > The problem I have with interpreting them is that `bls -j` gives > me out two pairs of file+block numbers identifying each backup session. > That appears to be neatly covering the range of data occupied by a > single backup job run but how do I properly put VolFile and VolBlock > ranges if a job's data crosses the boundary of a file? > > For instance, here's a fragment from the real output for one of my > volumes (elided a bit for terseness): > > Begin Job Session Record: File:blk=338:1 SessId=142 \ > SessTime=1427996423 JobId=5007 > Job=call-recs.2015-05-12_14.00.00_41 ... > End Job Session Record: File:blk=339:10868 SessId=142 \ > SessTime=1427996423 JobId=5007 > Date=12-May-2015 14:44:22 Level=F Type=B ... > > Here, the job starts at 338:1 and ends at 339:10868, how do I write out > the bootstrap file's data to cover that whole range in it? >>From [1], it appears that I could use > > VolFile=338-339 > > to cover the files but the VolBlock spec appears to pertain to a single > VolFile. I thought of using multiple VolFile + VolBlock pairs but > that would require me to use an open range for the VolBlock of the > first pair, something like > > VolFile=338 > VolBlock=1- > VolFile=339 > VolBlock=1-10868 > > (and by the way, are blocks numbered from 0 or 1?) > > Is there a solution for my case? > Or should I maybe look for some other data from the output of `bls -j`? > > FWIW, for these backups, there's always a single client backing up to > that tape at any given time, so the jobs do not have their data > intermixed on the tape. > > 1. http://www.bacula.org/5.1.x-manuals/de/main/main/Bootstrap_File.html > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users -- Mit besten Grüßen / Kind Regards Andreas Nastke IT System Management g/d/p Markt- und Sozialforschung GmbH Ein Unternehmen der Forschungsgruppe g/d/p Richardstr. 18 D-22081 Hamburg Fon: +49 (0)40 / 29876-117 Fax: +49 (0)40 / 29876-127 nas...@gdp-group.com www.gdp-group.com Sitz der Gesellschaft ist Hamburg, Handelsregister Hamburg, HRB 40482 Geschäftsführer: Christa Braaß, Volker Rohweder ----------------------------------------------------------------------- This e-mail may contain confidential and/or privileged information. If you are not the intended recipient please notify the sender and delete this e-mail from your whole system. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden. ----------------------------------------------------------------------- ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
