Hi Eric!!
Thank you so much for your answer. Give me please some minutes to read
your mail relaxedly. I'm not just interested in developing plugins!!! I
would love that Bacula would triumph over Veeam... and I wanted to
contribute for that!!!
Let me please read carefully and please I emphasize on being extremely
thankful for your help really Eric :) :)
I answer later :) :)
Cheers!!
El 2022-03-08 09:56, Eric Bollengier escribió:
> ATENCION
> ATENCION
> ATENCION!!! Este correo se ha enviado desde fuera de la organizacion. No
> pinche en los enlaces ni abra los adjuntos a no ser que reconozca el
> remitente y sepa que el contenido es seguro.
>
> Hello,
>
> On 3/7/22 12:44, egoitz--- via Bacula-devel wrote:
>
>> Hi!,
>>
>> After digging in plugin building documentation and checking the provided
>> examples, I have some doubts I have not been able to clarify by my own.
>> I describe them below in case you could give me a hand :). I would be
>> very thankful if you could help me a little clarifying this doubts :)
>
> I very pleased to see someone interested to develop Bacula plugins,
> specially around data compression.
>
>> - I'm working in trying to create an open source version of the delta
>> encoding plugin by using the bacula-fd plugin api. When working on it I
>> have seen Bacula's source is aware of delta and delta file
>> sequentiation.
>
> Yes, Bacula can manage "delta" or "patches" for a given file. The first
> Full backup should take the entire file, the delta_seq will be 0.
>
> During the first Incr or the Differential, the plugin can generate a
> "patch" (can be done with "diff", or "xdelta", or rsync, or whatever).
>
> This new files is based on the original file, and the delta_seq will
> be set to 1 automatically (Accurate mode should be turned on). The plugin
> must set a variable in the save_pkt to indicates that it has saved a
> patch, and at the restore time, bacula must send back the version of
> the file included in the Full, then in Diff (if any) and all Incrementals.
>
> On the plugin side, the restore code will be called for each part of
> the file.
>
>> I have seen for instance, even a .bvfs command exists for
>> showing deltas of a file id. But, what I have not found is that Bacula
>> works on that Delta files generation (patch generation, signatures,
>> etc...). I assume that Bacula in the non-fd part, acts just as a just
>> delta file holder keeping the files and stores the patch sequentiation
>> just that. Bacula keeps records of deltas in database (and file
>> storages) but only fd works with them (with probably a library like
>> librsync in the delta plugin) in the sense of applying patches over an
>> original file or even generating deltas when backup. Am I wrong?. Was
>> just for understanding the nice work done and what's already written and
>> free in Bacula's source for this purpose.
>
> I think that you got the concept, the "delta" in Bacula means that you
> must restore all parts of a file, and not just the last copy.
>
> With a program such as VMware, CBT helps the backup software to generate
> "patches" as well for example.
>
>> - By the way, I have one question about virtual files. I have not seen
>> very clear (perhaps my problem as don't understand it) how to work with
>> them. I understand the concept, but have not seen a clear example of how
>> for instance in the backup you create a virtual file, how do you see it
>> in bvfs and finally... what you get after restoring. In page 36/146 of
>> Bacula 11 for developers pdf, you say "This will create a virtual file."
>> but really you are entering in the structure :
>>
>> sp->type = FT_REG;
>> sp->statp.st_mode = 0700 | S_IFREG;
>>
>> FT_REG and S_IFREG both are for regular files.... what exactly causes a
>> virtual file to be created?. Perhaps st_size -1?.
>
> A virtual file is generated by plugins, they don't have to exist on disk.
> The name can be anything, it can also point to an existing file.
>
> The plugin code will be executed to restore the "virtual file", the result
> can be a real file on disk, or a virtual machine on Proxmox for example.
>
> In this example, we have a regular file, but it's a virtual file that may
> or may not exist on the filesystem.
>
>> Are they relevant for what I'm trying to do?. It seems Bacula handles
>> delta sequentiation so... perhaps for this purpose I shouldn't need
>> "virtual files"?.
>
> In your case, it will be virtual files that points to regular files.
>
>> - I'm planning to implement delta encoding by checking the previous day
>> file signature done by librsync. Instead of looking at the filesystem it
>> would be nice if I could take a look at that signature in the last
>> backup done (yesterday backup). Could it be possible in some manner,
>> that if I see a file passed in EventHandleBackupFile() to check if
>> yesterdays signature exists in the backup of yesterday, and then read
>> the yesterday signature from the own backup?. I mean, instead of having
>> to leave the signature in the being backed server's filesystem.
>
> You can store information in the save_pkt structure and the plugin can check
> the last version of that information with the accurate mode.
>
> In general, you can use a couple of bytes with this technique.
>
> I don't think you have enough space to store a file signature, you will
> have to use an other way to store it (a local file, a database record, ...)
>
>> - The last one :) . For restoring, and for the code seen (for instance
>> in insert_missing_delta()) I assume Bacula detects we are restoring a
>> delta compressed file. Then I assume Bacula restores apart from the own
>> initial file, patches to arrive to the day we want to restore to. Am I
>> wrong?.
>
> This is correct, bacula will send back to the plugin the data that was
> produced. Up to the plugin to reassemble the data. If one delta piece is
> missing, the restore will stop to the last correct one.
>
>> Perhaps later in a post-restore job I could run a shell script
>> that tries to find patches pending to be applied to a parent file. I
>> suppose then I could apply and the backup would become finally restored.
>> Does some other more elegant way you could advise me?.
>
> Normally, the plugin will receive patches one after the other, you can
> re-open the file on disk and apply the patch. It is also possible to
> store everything on disk and call a script to do the work at the end,
> it depends.
>
> Good luck,
>
> Best Regards,
> Eric
_______________________________________________
Bacula-devel mailing list
Bacula-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel
