> Before I cause a panic, there is a cut/paste error in my description > below. > There is nothing wrong with Sparse + Compression, it should say "Works > fine" > not "Restore broken".
Thanks for the update -- yes, if sparse + compression were broken, I would be very worried ... :-) > > -----Original Message----- > From: Kern Sibbald [mailto:[EMAIL PROTECTED] > Sent: Friday, November 03, 2006 5:05 PM > To: Robert Nelson > Cc: [EMAIL PROTECTED]; 'Landon Fuller'; > bacula-users@lists.sourceforge.net > Subject: RE: [Bacula-devel] [Bacula-users] Encryption/Compression Conflict > in CVS > > I've got to go now. Thanks for the explaination. > I'll respond as soon as I can, but I would also like to see Landon's > response. > > >> Perhaps if I explained the problem: >> >> Currently (as of 1.39.27) >> >> No filters = Works fine >> Sparse = Works fine >> Compression = Works fine >> Encryption = Works fine >> >> Sparse + Compression = Restore broken >> Sparse + Encryption = Restore broken >> Sparse + Compression + Encryption = Restore broken >> Compression + Encryption = Restore broken >> >> With my fix (> 1.39.27): >> >> All combinations work fine. >> >> But restoration of old backups (1.39.x) using encryption alone or in >> combination with other filters is broken. Since all cases of encryption >> in >> combination with anything else were already broken, that just leaves >> encryption alone. I could special case the handling of just encryption >> so >> that it would be stored differently than when used in combination with >> other >> filters. This would allow old encrypted backups to be read at the >> expense >> of having two versions of the encryption and decryption code. >> >> Since this is a brand new feature I don't think changing the archive >> format >> at this point would affect that many people. Particularly since the >> archives are unreadable if they've used it in combination with >> compression >> or sparse file handling. >> >> I think it is safer to just say that previously written archives using >> encryption aren't readable and should be recreated. >> >> -----Original Message----- >> From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] On Behalf Of Kern >> Sibbald >> Sent: Friday, November 03, 2006 4:40 PM >> To: Robert Nelson >> Cc: [EMAIL PROTECTED]; 'Landon Fuller'; >> bacula-users@lists.sourceforge.net >> Subject: Re: [Bacula-devel] [Bacula-users] Encryption/Compression >> Conflict >> in CVS >> >> >>> This code is backwards compatible for everything except encrypted data. >>> Previously compressed backups will still work fine. >> >> I'm not 100% sure what you mean, but here are my thoughts: >> >> If it breaks something that previously worked, then it is does not fit >> with the Bacula philosophy of always being able to read Volumes written >> by >> prior versions. >> >> If something was previously broken -- i.e. could not be read -- then we >> should attempt to fix it, if at all possible. >> >> Maintaining backward Volume compatibility and fixing the problem is the >> best solution. Hopefully this is what you can do ... >> >> If we *must* create an incompatibility with Volumes written by prior >> versions of Bacula, then we need to think really hard about how to >> handle >> it because to the best of my knowledge this has never happened. >> >> If certain combinations of options created data that cannot be read >> under >> any conditions, then we need to carefully document it and inform the >> users. >> >>> >>> -----Original Message----- >>> From: [EMAIL PROTECTED] >>> [mailto:[EMAIL PROTECTED] On Behalf Of Kern >>> Sibbald >>> Sent: Friday, November 03, 2006 4:15 PM >>> To: Robert Nelson >>> Cc: [EMAIL PROTECTED]; 'Landon Fuller'; >>> bacula-users@lists.sourceforge.net >>> Subject: Re: [Bacula-users] Encryption/Compression Conflict in CVS >>> >>> >>>> Landon, >>>> >>>> I've changed the code so that the encryption code prefixes the data >>>> block >>>> with a block length prior to encryption. >>>> >>>> The decryption code accumulates data until a full data block is >>>> decrypted >>>> before passing it along to the decompression code. >>>> >>>> The code now works for all four scenarios with encryption and >>>> compression: >>>> none, encryption, compression, and encryption + compression. >>>> Unfortunately >>>> the code is no longer compatible for previously encrypted backups. >>>> >>>> I could add some more code to make the encryption only case work like >>>> before. However, since this is a new feature in 1.39 and there >>>> shouldn't >>>> be >>>> a lot of existing backups, I would prefer to invalidate the previous >>>> backups >>>> and keep the code simpler. >>>> >>>> Also I think we should have a design rule that says any data filters >>>> like >>>> encryption, compression, etc must maintain the original buffer >>>> boundaries. >>>> >>>> This will allow us to define arbitrary, dynamically extensible filter >>>> stacks >>>> in the future. >>>> >>>> What do you think? >>> >>> I'm unfortuntely not in a good position to examine this problem in >>> detail, >>> but I suggest that we should do our best to keep the old data readable >>> by >>> any kludge necessary. >>> >>> One possible solution for the new code that you have implemented is to >>> put >>> the new compressed data in a new stream -- i.e. a different one from >>> the >>> old compressed data -- this could possibly allow old Volumes to be read >>> and any new data written to Volumes will be written correctly. >>> >>> One thing to be very careful about is to make sure the length that you >>> store is bigendian-littlendian independent. Probably you have already >>> done >>> this, but if not you need to use the serialization code that is also >>> used >>> for sparse file length. >>> >>> >>>> >>>> -----Original Message----- >>>> From: [EMAIL PROTECTED] >>>> [mailto:[EMAIL PROTECTED] On Behalf Of >>>> Landon >>>> Fuller >>>> Sent: Wednesday, November 01, 2006 7:08 PM >>>> To: Michael Brennen >>>> Cc: bacula-users@lists.sourceforge.net >>>> Subject: Re: [Bacula-users] Encryption/Compression Conflict in CVS >>>> >>>> >>>> On Nov 1, 2006, at 2:20 PM, Michael Brennen wrote: >>>> >>>>> On Wednesday 01 November 2006 15:33, Arno Lehmann wrote: >>>>> >>>>>>>> This sounds like compression should be automatically disabled when >>>>>>>> encrypton is enabled. Should be useless anyway as encrypted data >>>>>>>> should >>>>>>>> no longer be compressible. >>>>>>> >>>>>>> Not if compression happens prior to encryption. :) >>>>>> >>>>>> Theoretically - yes, but I'm quite sure that encryption usually also >>>>>> compresses data. This is completely unverified and refers to >>>>>> encryption >>>>>> programs that are rather outdated by now, though... >>>>>> >>>>>> But I suppose you could inform us if encryption in Bacula also >>>>>> compresses :-) >>>>> >>>>> Landon, what is your take on this? Since you wrote the code you >>>>> seem to be >>>>> the best source on whether the openssl functions you are using >>>>> compress data. >>>> >>>> Howdy, >>>> >>>> The encryption does not include compression -- It made more sense to >>>> piggyback on the existing compression code. >>>> Also, thanks for catching this! I'm embarrassed that I forgot to test >>>> backup+restore with both compression and encryption enabled. >>>> >>>> -landonf >>>> >>>> >>>> >>>> > ------------------------------------------------------------------------- >>>> Using Tomcat but need to do more? Need to support web services, >>>> security? >>>> Get stuff done quickly with pre-integrated technology to make your job >>>> easier >>>> Download IBM WebSphere Application Server v.1.0.1 based on Apache >>>> Geronimo >>>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >>>> _______________________________________________ >>>> Bacula-users mailing list >>>> Bacula-users@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/bacula-users >>>> >>> >>> >>> Best regards, Kern >>> >>> ------------------------------------------------------------------------- >>> Using Tomcat but need to do more? Need to support web services, >>> security? >>> Get stuff done quickly with pre-integrated technology to make your job >>> easier >>> Download IBM WebSphere Application Server v.1.0.1 based on Apache >>> Geronimo >>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >>> _______________________________________________ >>> Bacula-users mailing list >>> Bacula-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/bacula-users >>> >>> >>> >>> ------------------------------------------------------------------------- >>> Using Tomcat but need to do more? Need to support web services, >>> security? >>> Get stuff done quickly with pre-integrated technology to make your job >>> easier >>> Download IBM WebSphere Application Server v.1.0.1 based on Apache >>> Geronimo >>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >>> _______________________________________________ >>> Bacula-devel mailing list >>> [EMAIL PROTECTED] >>> https://lists.sourceforge.net/lists/listinfo/bacula-devel >>> >> >> >> Best regards, Kern >> >> ------------------------------------------------------------------------- >> Using Tomcat but need to do more? Need to support web services, >> security? >> Get stuff done quickly with pre-integrated technology to make your job >> easier >> Download IBM WebSphere Application Server v.1.0.1 based on Apache >> Geronimo >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >> _______________________________________________ >> Bacula-devel mailing list >> [EMAIL PROTECTED] >> https://lists.sourceforge.net/lists/listinfo/bacula-devel >> >> > > > Best regards, Kern > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Bacula-devel mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/bacula-devel > Best regards, Kern ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
