Hello Laurent,
Thank you for the nice patch. I took a quick look at it, and it looks
*very* interesting. Thanks also for having read some of my design
ideas and for having implemented the LZO using those ideas :-)
As Dan wrote, I would appreciate it if you send in the Fiduciary
License Agreement, as all contributors have done. Just let me know
by email once you have put it in the post.
We have frozen new features for the next community version (5.2.0) and
will probably release a beta version in the next few weeks. However, this
patch looks to be very well implemented and since you have even provided
a regression script, we will consider including it before the beta
release -- we
just need your FLA.
I am teaching a Bacula course until Friday of next week, so it will probably
be Eric who integrates it. He is on the bacula-devel list, but I am
copying him
directly so that he sees this email :-)
Thanks for the nice contribution!
Best regards,
Kern
On 06/25/2011 05:47 PM, Laurent Papier wrote:
Hi,
I have written a LZO compression patch for bacula. The compression is
implemented on the file daemon side like the current gzip compression.
I have also tried to make it easier to add other compression scheme in
bacula-fd by defining new streams. I have used the idea discussed in this thread
http://copilotco.com/mail-archives/bacula-devel.2006/msg00435.html.
This patch need more testing before using it to backup critical data. It should
be considered as beta.
This patch is for the lastest git version (lastest commit
c436ea750eb8df3f7a1d5f9213fd8bbb938b1ca8).
After applying the patch, you must first run 'make configure' before doing the
traditional 'configure; make; make install'
Usage is simple. Just replace 'Compression = GZIP' with 'Compression = LZO' in
your fileset options.
I will now detail some technical part of the patch.
1. I have not modified the current gzip compression handling in bacula. So a lzo
patched bacula should be able to ask non lzo patched fd for gzip compressed
backup. And you should still be able to restore your old gzipped backup with a
lzo patched bacula. We should have full backward compatibility.
2. A new fileset option 'Zo' is passed from the dir to the fd to indicate lzo
compression. Then the compression algorithm is stored in 'struct findFOPTS' and
'struct s_included_file'.
3. I have duplicated all 'gzip' streams to more generic compressed streams as
suggested in the devel mailing thread
http://copilotco.com/mail-archives/bacula-devel.2006/msg00435.html. A
compressed stream header is written at the beginning of every compressed stream
to indicate the compression algorithm and option used to compress the stream.
/* Compressed data stream header */
typedef struct {
uint32_t magic; /* compression algo used in this compressed data
stream */
uint16_t level; /* compression level used */
uint16_t version; /* for futur evolution */
uint32_t size; /* compressed size of the original data */
} comp_stream_header;
No data integrity are done as it seems that bacula already handle this (but
maybe I'm wrong).
4. in order to restore encrypted lzo compressed data, I store the last
compressed stream found in struct r_ctx. Without this, it is not possible to
know if the data is compressed with gzip or something else.
5. the rest is just #if HAVE_LZO sections inspired by the corresponding #if
HAVE_LIBZ ones ;-)
I have made minimal testing and I have written regression test scripts. It
seems fine on my system. If you want to try this patch please use the lzo-test,
lzo-encrypt-test and sparse-lzo-test regression tests.
I would appreciate comment, feedback, suggestion about this patch.
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense..
http://p.sf.net/sfu/splunk-d2d-c1
_______________________________________________
Bacula-devel mailing list
Bacula-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel
------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Bacula-devel mailing list
Bacula-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel
