Hello,

wt., 28 sie 2018 o 23:50 George Anchev <stu...@anchev.net> napisał(a):

> On Tue, 28 Aug 2018 16:23:39 +0200 Radosław
> Korzeniewski wrote:
>
> > And this client had a name 'remote-fd', right?
>
> The actual names of 'remote*' and 'local*' are
> different but I am replacing them here for the sake of
> clarity (and privacy). But if you are asking if that
> client was renamed, the answer is no.
>
> > > I think this may be incorrect suspicion because in
> > > the past I have noticed that running an estimate
> > > for a job without IFC also resulted in Full
> > > fallback and simply adding the IFC and reloading
> > > the DIR service instantly resulted in correct
> > > incremental estimate.
> >
> > I do not understand the above statement. Is it the
> > same situation like you are experiencing now.
>
> In the past I have seen:
>
> 1. Change fileset (without IFC)
> 2. reload DIR
> 3. estimate incremental job
>
> Result: fallback to Full
>
> 4. Add IFC to fileset
> 5. repeat 2-3
>
> Result: correct incremental estimate.
> This is what I am currently seeing for 'local-fd' too.
>

No, it is not correct. Check your email from 24/08/2018, 11:53. The step 4.
do not exist for 'local-fd'.


> > This query is not correct. You are asking for
> > remote_Backup on local-fd, right?
>
> My bad:
>
> MariaDB [bacula]> SELECT StartTime, Job FROM Job WHERE JobStatus IN
> ('T','W') AND Type='B' AND Level='F' AND Name='remote_Backup' AND
> ClientId=1;
> +---------------------+--------------------------------------+
> | StartTime           | Job                                  |
> +---------------------+--------------------------------------+
> | 2018-07-15 17:39:53 | remote_Backup.2018-07-15_17.39.48_31 |
> +---------------------+--------------------------------------+
> 1 row in set (0.00 sec)
>
> > What sql query say?
>
> MariaDB [bacula]> SELECT FileSetId FROM Job WHERE JobStatus IN ('T','W')
> AND Type='B' AND Level='F' AND Name='remote_Backup' AND ClientId=1;
> +-----------+
> | FileSetId |
> +-----------+
> |        64 |
> +-----------+
> 1 row in set (0.00 sec)
>
>
OK, thank you.


> > My current experience in this area is: when you
> > change fileset, then any next incr/diff jobs will be
> > upgraded to full unless you've set ignore fileset
> > changes = yes.
>
> That doesn't contradict my experience. However I can
> also confirm that I was able to change a fileset and
> add IFC *after* having run a few incremental jobs and
> then run a next incremental job in the sequence,
> without having to upgrade it to full.
>
>
I checked the source code from the current git Bacula 9.2 and latest git
patch for 5.2. Functions get_or_create_fileset_record() in both releases
are the same, so the functionality did not changed. What we can get in this
area:

if (!jcr->fileset->ignore_fs_changes ||
       !db_get_fileset_record(jcr, jcr->db, &fsr)) {
      if (!db_create_fileset_record(jcr, jcr->db, &fsr)) {
(...)

So, when Ignore Fileset Changes parameter is set ('Yes') then a function:
db_get_fileset_record() is called which makes a following query:

"SELECT FileSetId,FileSet,MD5,CreateTime FROM FileSet "
"WHERE FileSet='%s' ORDER BY CreateTime DESC LIMIT 1"

So it gets the latest filesetid, which means in your case the Filesetid=65
and not 64 which (the fileset=64) has a perfect full job available (and the
same applies for filesetid=60 for a local-fd). This function return 0/false
when it cannot find a proper filesetid, so the function
db_create_fileset_record() creates it on request. But when this function
finds it then db_create_fileset_record() is not called, so on "Ignore
Fileset Changes = Yes" parameter set on any Fileset changes before any
estimate/job execution then a new fileset record won't be created. This is
a situation you have for your 'local-fd' where IFC parameter was set before
any estimate/job execution.

So, I think I proved above my suspicious with a little of tune, because not
the reload but any estimate/job execution will create the new fileset
record when fileset was changed.


> > To understand what is happening in your setup, you
> > need to carefully check you catalog data.
>
> I believe this is what I am doing (with your much
> appreciated help). The strange thing for me so far is
> that fallback to FileSetId=64 when it should really be
> 65. I.e. somehow Bacula does not see the latest full
> job and I suppose this may be the reason for the full
> estimate it does. IOW: it is not that the catalog is
> missing the necessary data but there is some algorithm
> which fails to detect the latest full job. And what is
> weird is that this same thing does not happen on
> 'local-fd'.
>

No it detects perfectly fine, but the IFC parameter was set tool late.
Bacula does not ignored your changes and saved it in the catalog because
there was no IFC parameter in configuration. Setting it after Bacula
notices it is useless.


>
> > You can even trace some debug messages to verify
> > what is going on Bacula side. I'm not sure if it is
> > worth the effort.
>
> I have no idea how to do that or if I would be able to
> understand those messages.
>
> > The fileset data is required for job information.
> > When you delete some jobs (manually or when
> > automatic retention time pass) then Bacula has no
> > information what fileset entries are not required
> > any more. To check it Bacula has to make a special
> > catalog query which would take time and resources,
> > which is useless when you want to save a few bytes
> > on your catalog.
>
> I see.
>
> > You could manually delete orphaned fileset data
> > using dbcheck utility: (...)
> > 10) Eliminate orphaned Filename records
> > (...)
>
> Thanks for the tip! Even after:
>
> "Deleting 61 orphaned FileSet records."
>
> Bacula still tries to use FileSetId=64 and FileSetId=65
> doesn't exist any more. Unfortunately now it is not
> possible to find out why Bacula didn't like 65. It
> might have been a "new" fileset which Bacula created
> due to the lack of IFC.
>
> Now I obviously have no choice but to run a Full
> job. :)
>

You have to plan in advance that this changes are worthless full backup and
setting IFC parameter means you are an expert, you know what you've done
and if something blows up it will be your responsibility. Keep it in mind.

best regards
-- 
Radosław Korzeniewski
rados...@korzeniewski.net
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Reply via email to