Hello, > Date: Mon, 25 Jun 2007 11:54:35 +1000 > From: Jon Wilson <[EMAIL PROTECTED]> > Subject: [Bacula-users] checksumming problem > To: [email protected] > Cc: Glen Davison <[EMAIL PROTECTED]> > Message-ID: <[EMAIL PROTECTED]> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> Hi, > I have an "interesting" problem, regarding checksumming a large number > of big files prior to a storage migration. > We have a fairly old install of Bacula, v1.36, running on Redhat > Enterprise Linux v3, with a MySQL 3.23 database. > We are shortly going to upgrade our storage system. I want to check for > bit-rot before commencing the migration. Bacula seems to have some > checksums stored, so lets use them! Strictly speaking they are not checksums, but rather hash codes, or file digests. > To complicate things, many files have been moved around, in order > optimise use of different storage media. We have some scripts which > ensure Bacula backs-up the correct files, and doesn't double back-up big > files. But this means that the paths in the catalog don't match the > paths on disk. > It seems like the checksumming and base64 encoding in the version of > Bacula we have is broken. We can't just run "md5sum" on the command line > and compare against what's in the Catalog. The bug report for this is > here (fixed in newer releases): The code was not broken. The hash code is identical to the RFC defining it. However, when Bacula was originally written, there was no RFC for 64 bit encoding or I wasn't aware of it, so the 64 bit printable representation was different from what md5sum produces. > http://bugs.bacula.org/bug_view_advanced_page.php?bug_id=0000565 > My first thought was to set up a "Verify" task in our current > configuration, and get Bacula to natively check the files against the > disk. It shouldn't matter that the encoding is broken, since we will be > using the same (broken) code for verification. But when I do this, it > complains that there has been no "InitCatalog" task done. I'm not sure > whether this is related to our file moves, or because we intially > misconfgiured things. I guess we should have been checking this > functionality from the start :-( > Some questions: > 1) Are the checksums stored in Bacula any use to me at all? Yes, of course, but they are not so easy to use without a bit of work. > 2) Can I set up a "Verify" task, and compare the stored checksums > against the disk copies? Or is this going to be too hard, given that the > InitCatalog jobs was never run, and we have moved files around? The closest thing to what you want is a Verify VolumeToCatalog. It will verifiy that the hash codes stored on the Volume are the same as those in the catalog. This means that Bacula must read the whole tape to find the hash codes, but they are pre-computed, so Bacula does not actually read the data on the Volume and re-compute a hash code (considering the data such as Win32 data, ... that would be a nearly impossible task anyway). > 3) Can I write a command line checksum program, which will output > checksums, and encode them in a way I can compare with the catalog? How > can I do this in Perl, or C? I've had a look at the internals of the > Bacula libraries, and my meager C skills are not sufficient to work > things out. (This is my preferred option) There is a C++ program (actually written in C) in <bacula-source>/src/lib/md5.c that you can tweak and compile into a program that will permit you to create an "md5" program that does the same thing as md5sum, but using the Bacula 64 bit encoding scheme ... If I remember right, if you have already configured and built Bacula, you can simply: cd <bacula-source>/src/lib make md5sum and it will create a binary named md5 (I think) that you can execute. If not, just tweak the Makefile. Regards, Kern > Thanks, > -- > Jon Wilson <[EMAIL PROTECTED]> > Systems Administration Manager > PO Box H58, Australia Square, Sydney NSW 1215 > Level 2, 9 Castlereagh Street, Sydney Office Tel:+61 9231 5888 > Direct Tel:+61 2 9236 9118 Fax: +61 2 9231 5988 > www.sirca.org.au > DISCLAIMER: The contents of this email, inclusive of attachments, may be > legally privileged and confidential. Any unauthorised use of the > contents is expressly prohibited. If you have received this email > message in error or are not the intended recipient, you should destroy > the message along with any attachment(s). Unintended recipients of this > email are prohibited from retaining, disclosing, distributing or using > any information herein. This email is also subject to copyright. No part > of it should be reproduced, adapted or transmitted without the written > consent of the copyright owner. ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Bacula-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-users
