Hello, I have only two more questions:
- Is there a CLI or graphical tool that I can use to browse available files
that were recently modified and journaled by CDP, and be able to drill down to
see the various versions cdp captured and then choose to restore ? The user
documentation alludes to the recovery of past versions of files but doesn't
mention any tooling to assist with this. As the CDP and journal and spool
directory don't appear to be ever cleared, this tool could run where the Bacula
file daemon runs and not have to deal with recovery points.
- Is there some kind of tooling to be able to browse recovery points
graphically (ie. drill into a list of files backed up for a particular job and
browse the contents of a particular file) ? To sweeten the pot, I'm strongly
considering using client-side encryption, which may make it hard for such tools
to function. Acronis has this functionality, but I moved away from them for
other reasons (lack of stability mostly).
Thanks,
<Myles>
---------------------------------------
After some code reading, I think I have the answers to my previous questions,
which I hope help others in the community:
- Every Plugin = "cdp: ..." line in the FileSet/Include block is interpreted as
a command that is sent individually by the fd daemon to the cdp plugin, which
responds to each command separately by looking for the indicated directory (or
deriving the directory from a user account) to find the spool directory
.cdp-sdir and the spool journal file .bcdp-cli.journal which is typically in
the same directory as the spool directory .cdp-sdir, and then the cdp fd plugin
identifies the files to add to the backup set.
- From what I can tell, the spool directory contents and journal file continues
to grow and is neither rotated nor pruned (can you confirm this?). I suppose
the advantage in this is that recent changes can be backed out even if an
external backup has not been done.
- I converted the call to cdp-client to a systemd service call and embedded the
multiple -f arguments, each for different directories to monitor, including
user directories and systems locations such as SQL database locations
- I found (which should have been obvious) that if I placed the journal file
and spool directory inside /root then I couldn't actually monitor the /root
directory due to recursion !?!? Kind of like a room with mirrors on opposite
walls.
- Bacula-fd daemon does not launch cdp-client, I have to do that outside
bacula-dir, bacula-fd and bacula-sd daemons.
Best,
<Myles>
On 2024-01-17 10:58 p.m., Myles Dear wrote:
> Thanks for the reply!
>
> I'm still missing a few important concepts:
>
> If I specify the 'Plugin = "cdp: user=x user=y user=x"' in my FileSet/Include
> block in bacula-dir.conf, I would assume that bacula-fd would be the one to
> launch cdp-client, otherwise why would I need to tell bacula director which
> users I'm monitoring and tell bacula fd that I have cdp plugin enabled?
>
> My bacula-fd runs under user "root" so it can select files across the entire
> server to back up, system and user files included.
>
> When I run cdp-client manually and specify multiple -f arguments, multiple
> directories are monitored but only a single spool directory is created. I do
> not see evidence that spool files are created for each user directory being
> monitored.
>
> If I do not specify the spool directory and journal file locations, they
> default to the home directory of the user that ran the cdp-client file (in
> this case, /root).
>
> # /opt/bacula/bin/cdp-client -f /home/usera/ -f /home/userb/
> : cdp-client.cpp:151-0 Set spool directory to: /root/.cdp-sdir
> : cdp-client.cpp:112-0 Set journal file to: /root/.bcdp-cli.journal
> : folderwatcher.cpp:126-0 Started Watching: /home/usera
> : folderwatcher.cpp:126-0 Started Watching: /home/usera/.ssh
> : folderwatcher.cpp:126-0 Started Watching: /home/usera/.cache
> : folderwatcher.cpp:126-0 Started Watching: /home/userb
> : folderwatcher.cpp:126-0 Started Watching: /home/userb/Downloads
> ...
>
> So, is the right thing to do to duplicate the configuration in my
> bacula-dir.conf and hardcode a system service to launch on startup that
> ensures that cdp-client is running under username root at all times, that
> restarts itself if it ever crashes, assuming that none of the bacula-dir,
> bacula-fd or facula-sd processes will ever launch it?
>
> The cdp-client doesn't appear to have a "user" option, so there's still some
> bacula-level integration with cdp that I don't quite understand. In other
> words, I don't understand how the one or more Plugin directives under
> FileSet/Include ultimately result in the cdp-client command being run and
> what the translation between Plugin directives and cdp-client arguments is.
>
> Thanks,
>
> <Myles>
>
> On 2024-01-17 9:18 a.m., Eric Bollengier wrote:
>
>> Hello Myles,
>>
>> On 1/16/24 18:47, MylesDearBusiness via Bacula-users wrote:
>>
>>> Hello, Bacula experts.
>>>
>>> I was pleasantly surprised to see the CDP plugin is part of the community
>>> edition. I'm hoping to take full advantage of this.
>>>
>>> I'm running it on a Ubuntu 22.04.3 LTS bare metal cloud server.
>>>
>>> After looking at the main reference manual section 47.1, I still have
>>> questions:
>>>
>>> - If I want to specify multiple users to be monitored, do I just have
>>> multiple Plugin = "cdp: userHome=/home/<username>" directives in my
>>> director FileSet / Include configuration?
>>
>> I would put only one plugin line and multiple userHome directories (or user=)
>>
>> Plugin = "cdp: user=eric user=myles"
>>
>>> - Is it legal to specify a number of Plugin directives in my FileSet
>>> Include block as well as multiple "File = /path/to/dir-to-monitor"
>>> directives all together ?
>>
>> Yes, it will work
>>
>>> - I'm seeing evidence that my /root/.cdp-sdir is the only place where the
>>> spool directory is created (I'm running my fd as root), despite me having
>>> multiple "cdp: userHome" lines, so what exactly are the semantics of this
>>> directive? The documentation says that this points to the home directory of
>>> the cdp user, which presumably would be the user running the fd, and the
>>> place where the spool directory is placed, but I want to monitor multiple
>>> places, including user home directories and key places such as
>>> /var/lib/mysql where I didn't think the Plugin = "cdp: user=<username>"
>>> directive was appropriate, so I'm using the userHome directive for these
>>> cases. Hope it works.
>>
>> The .cdp-sdir is used by the cdp daemon that is monitoring your files, you
>> should have one in each user directory.
>>
>>> - I don't see /opt/bacula/bin/cdp-client running continuously on my /bin/ps
>>> -Af command output, am I expected to launch this myself at startup?
>>
>> Yes, it should be started, I believe that we have a GUI to run it, but it
>> can be
>> started with your session as well.
>>
>>> Since I'm seeing evidence that spooling is happening, with the config I've
>>> described, in what cases is this client automatically run? For example, is
>>> this client only run when a job is launched? What is the recommended way of
>>> setting this up?
>>
>> No, it runs when the user is logged in, files that are modified are copied in
>> the spool directory, and the Bacula job will come from time to time to backup
>> all versions and do some cleanup. Doing so, the CDP will work even if you
>> are not connected to the network.
>>
>>> - Since the Linux cdp uses the inotify API, whose man page states is not
>>> recursive, am I assured that when I specify a directory that CDP will tell
>>> inotify to monitor all its subdirectories as well?
>>
>> Yes, this part is handled by the cdp daemon. I would recommend to not monitor
>> hundred of thousand directories, but really where you work.
>>
>>> - ChatGPT insists that in the file daemon configuration each directory to
>>> back up may be specified in a separate "Plugin {cdp:
>>> userHome=/home/<username>} block but I see nothing in the documentation
>>> that corroborates this, it also says that when specifying in this way, the
>>> director FileSet block may have just a single plugin line referencing no
>>> directories{ Options {...} Plugin = "cdp"}. Is my just my AI hallucinating
>>> or does this configuration work ?
>>
>> I would rather trust the users list, the code or both, but not chatgpt for
>> that :-)
>>
>> Best Regards,
>> Eric_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
