On Thu, May 05, 2011 at 12:12:10AM -0400, Dan Langille wrote:
> On May 4, 2011, at 3:26 AM, Graham Keeling wrote:
>
> > On Fri, Apr 29, 2011 at 11:11:24AM +0200, Hugo Letemplier wrote:
> > > I think that a new feature that add dependency between various job
> > > levels for the next versions of bacula could be cool.
> > > The idea is to allow pruning only for volume/jobs that aren't needed
> > > by other ones whatever are the retention time.
...
> > > What do you think about such a feature ?
> >
> > A while ago, I made a patch that does it. Nobody seemed to want it though.
> > http://www.adsm.org/lists/html/Bacula-users/2011-01/msg00308.html
>
>
> Just because you didn't find anyone that wanted it does not make it a bad
> idea.
>
> Ideas are sometimes difficult to comprehend. I didn't follow the above in a
> 30 second scanning....
>
> If you think it's a good idea. Pursue it. Give examples. Describe the
> issues, in brief, and then in general. Build a case that others can
> comprehend with minimal effort.
Thank you for the pep talk. :)
But I think that I have explained very well, in plain terms here:
http://www.adsm.org/lists/html/Bacula-users/2011-01/msg00308.html
"Bacula doesn't prevent backups that other backups depend on from being
purged."
> If you think it's a good idea, something will come of it. But nothing will
> come of it if you don't persist.
Bacula is clearly not built to think that one backup depends directly on
another one. For example, there is no database field that says 'this backup
depends on this other backup'. This would have been one of the first fields
that I would have designed into it. Instead, it heavily relies on timestamps
and sort of scoops up everything that has a timestamp that matches.
I think that this may be down to its main authors having a 'tape mentality'
that I don't fully understand, since I have never worked with tapes.
Therefore, although I think that it would be a very good idea for bacula to
not purge a backup that another backup depends upon, I do not expect that this
patch will be adopted. And I do not expect bacula to gain this ability anytime
soon, since the concept seems so alien to it.
Making bacula do my concept of the 'right thing' would require a very radical
redesign. And perhaps my 'right thing' is different to many other people's
'right thing'.
Making the patch (which I have now attached to this email) felt as if I was
subverting some basic assumptions that I did not fully understand.
Furthermore, the patch makes its own assumptions:
a) you are using one Job per Volume
b) you are not using Job / File retentions (ie, you have set them very high)
and instead rely purely on Volume retentions.
c) you are using mysql (it might work fine on postgres, but I haven't tried)
So, the patch is provided here on an 'as is' basis. If anybody likes it, or
wants to do something with it, or persist with it to get it into bacula, then
they can. :)
A note on the patch:
The main part starts in src/dird/autoprune.c with the call to
db_volume_has_dependents().
Final note:
I have actually gone very far down the path of 'very radical redesign' - see
http://burp.grke.net/ if you are interested.
Index: WORK/src/cats/protos.h
===================================================================
RCS file: /cvs/netpilot/GPL/bacula-5.0.3/WORK/src/cats/protos.h,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- WORK/src/cats/protos.h
+++ WORK/src/cats/protos.h
@@ -90,6 +90,8 @@
int db_delete_media_record(JCR *jcr, B_DB *mdb, MEDIA_DBR *mr);
/* sql_find.c */
+int db_volume_has_dependents(JCR *jcr, B_DB *mdb, int mediaid);
+void db_volume_list_dependents(JCR *jcr, B_DB *mdb, DB_LIST_HANDLER sendit, void *ctx);
bool db_find_last_job_start_time(JCR *jcr, B_DB *mdb, JOB_DBR *jr, POOLMEM **stime, int JobLevel);
bool db_find_job_start_time(JCR *jcr, B_DB *mdb, JOB_DBR *jr, POOLMEM **stime);
bool db_find_last_jobid(JCR *jcr, B_DB *mdb, const char *Name, JOB_DBR *jr);
@@ -131,6 +133,7 @@
void db_list_files_for_job(JCR *jcr, B_DB *db, uint32_t jobid, DB_LIST_HANDLER sendit, void *ctx);
void db_list_media_records(JCR *jcr, B_DB *mdb, MEDIA_DBR *mdbr, DB_LIST_HANDLER *sendit, void *ctx, e_list_type type);
void db_list_jobmedia_records(JCR *jcr, B_DB *mdb, JobId_t JobId, DB_LIST_HANDLER *sendit, void *ctx, e_list_type type);
+void db_list_jobandmedia_records(JCR *jcr, B_DB *mdb, JobId_t JobId, DB_LIST_HANDLER *sendit, void *ctx, e_list_type type);
void db_list_joblog_records(JCR *jcr, B_DB *mdb, JobId_t JobId, DB_LIST_HANDLER *sendit, void *ctx, e_list_type type);
int db_list_sql_query(JCR *jcr, B_DB *mdb, const char *query, DB_LIST_HANDLER *sendit, void *ctx, int verbose, e_list_type type);
void db_list_client_records(JCR *jcr, B_DB *mdb, DB_LIST_HANDLER *sendit, void *ctx, e_list_type type);
Index: WORK/src/cats/sql_cmds.c
===================================================================
RCS file: /cvs/netpilot/GPL/bacula-5.0.3/WORK/src/cats/sql_cmds.c,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- WORK/src/cats/sql_cmds.c
+++ WORK/src/cats/sql_cmds.c
@@ -95,10 +95,13 @@
const char *del_JobMedia = "DELETE FROM JobMedia WHERE JobId=%s";
const char *cnt_JobMedia = "SELECT count(*) FROM JobMedia WHERE MediaId=%s";
+/* Graham says: This was hacked so that it also selects jobs
+ in error, as well as those past the retention time. As of 2009-08-25, it
+ is only used in one place - ua_prune.c */
const char *sel_JobMedia =
"SELECT DISTINCT JobMedia.JobId FROM JobMedia,Job "
"WHERE MediaId=%s AND Job.JobId=JobMedia.JobId "
- "AND Job.JobTDate<%s";
+ "AND (Job.JobTDate<%s OR Job.JobStatus IN ('A','E','f'))";
/* Count Select JobIds for File deletion */
const char *count_select_job =
Index: WORK/src/cats/sql_find.c
===================================================================
RCS file: /cvs/netpilot/GPL/bacula-5.0.3/WORK/src/cats/sql_find.c,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- WORK/src/cats/sql_find.c
+++ WORK/src/cats/sql_find.c
@@ -47,6 +47,578 @@
#if HAVE_SQLITE3 || HAVE_MYSQL || HAVE_SQLITE || HAVE_POSTGRESQL || HAVE_INGRES || HAVE_DBI
+struct jmed
+{
+ long jobid;
+ long mediaid;
+ int expired;
+ int dependents; // others depend upon this one
+ int dependents_expired; // all that depend upon this one have expired
+};
+
+static int add_jmed(struct jmed ***jds, int *arrlen, const char *jobid, const char *mediaid, int expired, int dependents)
+{
+ struct jmed *j=NULL;
+ if(!(*jds=(struct jmed **)brealloc(*jds,
+ ((*arrlen)+2)*sizeof(struct jmed *)))
+ || !(j=(struct jmed *)bcalloc(1, sizeof(struct jmed))))
+ {
+ return -1;
+ }
+ j->jobid=atol(jobid?:"0");
+ j->mediaid=atol(mediaid?:"0");
+ j->expired=expired;
+ j->dependents=dependents;
+ j->dependents_expired=0;
+ (*jds)[(*arrlen)++]=j;
+ return 0;
+}
+
+/* The list comes out with the next dependent being the next in the list (if
+ the current item has the 'dependents' flag set. */
+static void get_whether_dependents_have_expired(struct jmed **jdeps, int jarrlen)
+{
+ int x=0;
+ // Go down the list backwards, figuring out whether dependents have
+ // expired.
+ for(x=jarrlen-1; x>=0; x--)
+ {
+ int y=x+1;
+
+ if(y<jarrlen)
+ {
+ if(jdeps[x]->dependents)
+ {
+ // If the next one that x depends on
+ // has expired, and all the ones that that
+ // depends on have also expired, then all those
+ // that depend on x have also expired.
+ if(jdeps[y]->dependents_expired
+ && jdeps[y]->expired)
+ jdeps[x]->dependents_expired=1;
+ else
+ // If either of the above are false, the
+ // dependents have not expired.
+ jdeps[x]->dependents_expired=0;
+ }
+ else
+ {
+ // Has no dependents, therefore dependents
+ // have expired.
+ jdeps[x]->dependents_expired=1;
+ }
+ }
+ else
+ {
+ // End of the list - has no dependents, therefore
+ // dependents have expired.
+ jdeps[x]->dependents_expired=1;
+ }
+/*
+ syslog(LOG_INFO, "get_whether: %li %li e:%d d:%d de:%d",
+ jdeps[x]->jobid,
+ jdeps[x]->mediaid,
+ jdeps[x]->expired,
+ jdeps[x]->dependents,
+ jdeps[x]->dependents_expired);
+*/
+ }
+}
+
+static void get_dependents(JCR *jcr, B_DB *mdb, struct jmed ***jdeps, int *jarrlen)
+{
+ SQL_ROW row;
+ char skip[50];
+ char curr[50]="";
+ int expired=0;
+ int has_dependents=0;
+
+ while((row=sql_fetch_row(mdb))) {
+ expired=atoi(row[9]);
+
+ // Remember, there might be more than one job in a volume.
+ // if we did not find a dependent for a job, we need to
+ // continue and test the next job in the volume.
+ // While we are using one job per volume, we will never
+ // see this.
+ if(*skip)
+ {
+ if(!strcmp(row[0], skip)) continue;
+ *skip='\0';
+ }
+
+ if(strcmp(row[0], curr))
+ {
+ if(*curr) add_jmed(jdeps, jarrlen, curr,
+ NULL, expired, has_dependents);
+ // We will be checking more
+ // jobids and need to reset has_dependents
+ has_dependents=0;
+ bsnprintf(curr, sizeof(curr), "%s", row[0]);
+ }
+
+ if( !(row[3]) // It is NULL on the last diff/incr in a chain
+ || *(row[3])==L_FULL
+ || *(row[3])==L_BASE)
+ {
+ bsnprintf(skip, sizeof(skip), "%s", row[0]);
+ // Carry on to the next job in the volume to check
+ // the dependents for.
+ continue;
+ }
+
+ // If the job in the volume is L_FULL, and the one we are
+ // checking is L_INCREMENTAL, or L_DIFFERENTIAL, the one we
+ // are checking depends upon the full.
+ if(*(row[1])==L_FULL || *(row[1])==L_BASE)
+ {
+ if((*(row[3])==L_INCREMENTAL
+ || (*(row[3])==L_DIFFERENTIAL))) {
+ // Extra check for virtuals.
+ // If a full on the left and a diff/incr on the
+ // right have the same StartTime, then the
+ // diff/incr is *not* dependent on the full.
+ // It is actually the diff/incr that the full
+ // was based on.
+ if(strcmp(row[7], row[8])) has_dependents++;
+ }
+ }
+ // Incrementals depend on previous incrementals.
+ else if(*(row[1])==L_DIFFERENTIAL || (*row[1])==L_INCREMENTAL)
+ {
+ if((*row[3])==L_INCREMENTAL) {
+ has_dependents++;
+ }
+ // Differential is since the last full backup,
+ // so this one is fine.
+ else if((*row[3])==L_DIFFERENTIAL) {
+ bsnprintf(skip, sizeof(skip), "%s", row[0]);
+ continue;
+ }
+ }
+ }
+
+ add_jmed(jdeps, jarrlen, curr, NULL, expired, has_dependents);
+
+ get_whether_dependents_have_expired(*jdeps, *jarrlen);
+}
+
+int db_volume_has_dependents(JCR *jcr, B_DB *mdb, int mediaid)
+{
+ int x=0;
+ char ed1[50];
+ char ed2[50];
+ int jarrlen=0;
+ int has_dependents=0;
+ struct jmed **jdeps=NULL;
+
+ db_lock(mdb);
+
+ /* For each Job in the Media, find all the jobs that might depend on
+ it. That is, those jobs that match the name, ClientId, FileSetId,
+ whose start time is later than our end time, and are not
+ in error conditions that might be recovered. */
+ /* 2009-08-25: Added 'JobMedia km' so as not to select jobs that have
+ no JobMedia records. We found that bacula never removes such jobs,
+ and this would cause the system to never recycle volumes that had
+ such a job as a dependency. */
+ /* The ORDER BY ... k.Level DESC on the end is in attempt to sort
+ redundant incrementals (I) that have been superceded by
+ virtualfulls (F) so that they appear first in the list and will
+ therefore get purged before the virtualfull. */
+ /* 14/04/2010: Added a wrapping select that does a LEFT JOIN, so that
+ we also get returned the JobIds that do not have dependents. */
+ /* 26/04/2010: Get the Expiry status on the outer select, so that it
+ is filled in for JobIds that do not have dependents. */
+
+ Mmsg(mdb->cmd,
+"SELECT J.JobId, J.Level, KJobId, K.Level, K.name, J.ClientId,"
+" J.FilesetId, J.StartTime, K.StartTime,"
+" if(J.JobTDate+m.Volretention<UNIX_TIMESTAMP(), 1, 0) AS Expired"
+" FROM Job J LEFT JOIN"
+" (SELECT DISTINCT j.JobId AS jJobId, k.JobId AS kJobId, k.Level,"
+" k.Name, k.FilesetId, k.StartTime"
+" FROM JobMedia jm,Job j,Job k,Media m,JobMedia km"
+" WHERE j.JobId=jm.JobId AND j.JobId!=k.JobId AND j.Name=k.Name"
+" AND j.ClientId=k.ClientId AND j.FileSetId=k.FileSetId"
+" AND m.MediaId=jm.MediaId"
+" AND k.JobId=km.JobId"
+" AND j.StartTime<=k.StartTime"
+" AND j.JobStatus NOT IN ('A','E','f')"
+" AND k.JobStatus NOT IN ('A','E','f')"
+" AND m.MediaId=%s"
+" ORDER BY j.JobId, k.StartTime ASC, k.Level DESC) as K"
+" ON J.JobId=jJobId,"
+" JobMedia jm, Media m"
+" WHERE J.JobId=jm.JobId AND m.MediaId=jm.MediaId"
+" AND m.MediaId=%s"
+" ORDER BY J.JobId, K.StartTime, K.Level DESC",
+ edit_int64(mediaid, ed1),
+ edit_int64(mediaid, ed2));
+
+ if (!QUERY_DB(jcr, mdb, mdb->cmd)) {
+ Mmsg2(&mdb->errmsg, _("Query error for Jobs in MediaId request: ERR=%s\nCMD=%s\n"), sql_strerror(mdb), mdb->cmd);
+ db_unlock(mdb);
+ return -1;
+ }
+
+ get_dependents(jcr, mdb, &jdeps, &jarrlen);
+
+ sql_free_result(mdb);
+
+ db_unlock(mdb);
+
+// Jmsg(jcr, M_INFO, 0, _("MediaId %d has %sdependents.\n"),
+// mediaid, jarrlen?"":_("no "));
+ // Free the list
+ for(x=0; x<jarrlen; x++)
+ {
+ if(jdeps[x]->dependents) has_dependents++;
+ free(jdeps[x]);
+ }
+ if(jarrlen) free(jdeps);
+
+ return has_dependents;
+}
+
+static void db_volume_get_all_dependents(JCR *jcr, B_DB *mdb, DB_LIST_HANDLER sendit, void *ctx, struct jmed ***jdeps, int *jarrlen)
+{
+ db_lock(mdb);
+
+ /* Same as in the function above, but do not specify MediaId. */
+ Mmsg(mdb->cmd,
+"SELECT J.JobId, J.Level, KJobId, K.Level, K.name, J.ClientId,"
+" J.FilesetId, J.StartTime, K.StartTime,"
+" if(J.JobTDate+m.Volretention<UNIX_TIMESTAMP(), 1, 0) AS Expired"
+" FROM Job J LEFT JOIN"
+" (SELECT DISTINCT j.JobId AS jJobId, k.JobId AS kJobId, k.Level,"
+" k.Name, k.FilesetId, k.StartTime"
+" FROM JobMedia jm,Job j,Job k,Media m,JobMedia km"
+" WHERE j.JobId=jm.JobId AND j.JobId!=k.JobId AND j.Name=k.Name"
+" AND j.ClientId=k.ClientId AND j.FileSetId=k.FileSetId"
+" AND m.MediaId=jm.MediaId"
+" AND k.JobId=km.JobId"
+" AND j.StartTime<=k.StartTime"
+" AND j.JobStatus NOT IN ('A','E','f')"
+" AND k.JobStatus NOT IN ('A','E','f')"
+" ORDER BY j.JobId, k.StartTime ASC, k.Level DESC) as K"
+" ON J.JobId=jJobId,"
+" JobMedia jm, Media m"
+" WHERE J.JobId=jm.JobId AND m.MediaId=jm.MediaId"
+" ORDER BY J.JobId, K.StartTime, K.Level DESC");
+
+ if (!QUERY_DB(jcr, mdb, mdb->cmd)) {
+ char msg[1024]="";
+ bsnprintf(msg, sizeof(msg), _("Query error for Jobs in mediadeps request: ERR=%s\nCMD=%s\n"), sql_strerror(mdb), mdb->cmd);
+ sendit(ctx, msg);
+ db_unlock(mdb);
+ return;
+ }
+
+ get_dependents(jcr, mdb, jdeps, jarrlen);
+
+ sql_free_result(mdb);
+
+ db_unlock(mdb);
+}
+
+void db_volume_list_dependents(JCR *jcr, B_DB *mdb, DB_LIST_HANDLER sendit, void *ctx)
+{
+ int x=0;
+ int jarrlen=0;
+ struct jmed **jdeps=NULL;
+ db_volume_get_all_dependents(jcr, mdb, sendit, ctx, &jdeps, &jarrlen);
+ if(ctx && jarrlen)
+ {
+ char msg[256]="";
+ //sendit(ctx, "JobIds that other JobIds depend upon:\n");
+ for(x=0; x<jarrlen; x++)
+ {
+ int y=0;
+ bsnprintf(msg, sizeof(msg),
+ _("%li:"), jdeps[x]->jobid);
+ sendit(ctx, msg);
+ if(jdeps[x]->dependents) for(y=x+1; y<jarrlen; y++)
+ {
+ bsnprintf(msg, sizeof(msg),
+ _(" %li"), jdeps[y]->jobid);
+ sendit(ctx, msg);
+ if(!jdeps[y]->dependents) break;
+ }
+ sendit(ctx, "\n");
+ }
+ }
+ // Free the list
+ for(x=0; x<jarrlen; x++) free(jdeps[x]);
+ if(jarrlen) free(jdeps);
+}
+
+static int get_mediaids_with_expiry_and_dependence_statuses(JCR *jcr, B_DB *mdb, DB_LIST_HANDLER sendit, void *ctx, struct jmed ***jds, int *arrlen)
+{
+ int x=0;
+ SQL_ROW row;
+ struct jmed **jdeps=NULL;
+ int expired=0;
+ int jarrlen=0;
+
+ db_lock(mdb);
+ Mmsg(mdb->cmd,
+ "SELECT DISTINCT m.MediaId, VolBytes>1, j.JobStatus,"
+ " JobTDate+m.VolRetention<=UNIX_TIMESTAMP(), m.VolStatus, j.JobId"
+ " FROM Media m "
+ "LEFT JOIN JobMedia jm ON jm.MediaId=m.MediaId "
+ "LEFT JOIN Job j ON j.JobId=jm.JobId ");
+
+ if (!QUERY_DB(jcr, mdb, mdb->cmd)) {
+ db_unlock(mdb);
+ return -1;
+ }
+ // Get all the jobids and mediaids.
+ while((row=sql_fetch_row(mdb))) {
+ // Ignore it if nothing was written to it.
+ if(!row[1] || *row[1]=='0') continue;
+
+ // No JobStatus means that there are no JobMedia records, and
+ // the volume can be purged.
+ if(!row[2]
+ // 'A', 'E', or 'f' also means that it failed.
+ || !strcmp(row[2], "A")
+ || !strcmp(row[2], "E")
+ || !strcmp(row[2], "f"))
+ expired=1;
+ else if(row[3] && *row[3]=='1'
+ || (row[4] &&
+ (!strcmp(row[4], "Purged") || !strcmp(row[4], "Recycle"))))
+ expired=1;
+ else
+ expired=0;
+
+ add_jmed(jds, arrlen, row[5], row[0], expired, 0);
+ }
+ if(*arrlen) (*jds)[*arrlen]=NULL;
+ sql_free_result(mdb);
+ db_unlock(mdb);
+
+ // Get a list of all the jobids and dependencies
+ db_volume_get_all_dependents(jcr, mdb, NULL, NULL, &jdeps, &jarrlen);
+
+ //for(x=0; x<jarrlen; x++)
+ // syslog(LOG_INFO, "jdeps: %li", jdeps[x]->jobid);
+
+ // Mark up our master list with the dependency information.
+ for(x=0; x<*arrlen; x++)
+ {
+ int y=0;
+ struct jmed *j=(*jds)[x];
+
+ if(!j->jobid || !j->mediaid) continue;
+
+ for(y=0; y<jarrlen; y++) if(j->jobid==jdeps[y]->jobid)
+ {
+ j->dependents=jdeps[y]->dependents;
+ j->dependents_expired=jdeps[y]->dependents_expired;
+ // Do not set 'expired', as we have already got it.
+ //j->expired=jdeps[y]->expired;
+ }
+ }
+
+ //for(x=0; x<*arrlen; x++)
+ // syslog(LOG_INFO, "%li %li e:%d d:%d de:%d",
+ // (*jds)[x]->jobid, (*jds)[x]->mediaid, (*jds)[x]->expired,
+ // (*jds)[x]->dependents, (*jds)[x]->dependents_expired);
+
+ // Free the list of mediaids that have dependents.
+ for(x=0; x<jarrlen; x++) free(jdeps[x]);
+ if(jarrlen) free(jdeps);
+
+ return 0;
+}
+
+static int create_temp_dep_table(B_DB *mdb, const char *table)
+{
+ char cmd[256]="";
+ bsnprintf(cmd, sizeof(cmd), "CREATE TEMPORARY TABLE %s (MediaId INTEGER UNSIGNED NOT NULL, Required VARCHAR(128))", table);
+ if (!db_sql_query(mdb, cmd, NULL, (void *)NULL)) {
+ return -1;
+ }
+ return 0;
+}
+
+/*
+ These are Robin's statuses:
+ 1. within retention time (Keep recent)
+ 2. out of retention, but something in retention depends on it
+ (Keep)
+ 3. out of retention time (and no dependents)
+ (Re-use now)
+ 4. out of retention, but something out of retention depends on it
+ (Re-use)
+
+ In the language of this code:
+ 1. !expired (Keep recent)
+ 2. expired, dependents, !dependents_expired (Keep)
+ 3. expired, !dependents (Re-use now)
+ 4. expired, dependents, dependents_expired (Re-use)
+*/
+
+static int insert_into_temp_dep_table(B_DB *mdb, const char *table, int mid, int expired, int dependents, int dependents_expired)
+{
+ char cmd[256]="";
+ char status[32]="";
+ if(!expired)
+ snprintf(status, sizeof(status), "%s", "Keep recent");
+ else
+ {
+ if(dependents)
+ {
+ if(!dependents_expired)
+ snprintf(status, sizeof(status), "%s", "Keep");
+ else
+ snprintf(status, sizeof(status), "%s", "Re-use");
+ }
+ else
+ snprintf(status, sizeof(status), "%s", "Re-use now");
+ }
+ bsnprintf(cmd, sizeof(cmd),
+ "INSERT INTO %s (MediaId, Required) VALUES (%u, '%s')",
+ table, mid, status);
+ if (!db_sql_query(mdb, cmd, NULL, (void *)NULL)) {
+ return -1;
+ }
+ return 0;
+}
+
+static int drop_temp_dep_table(B_DB *mdb, const char *table)
+{
+ char cmd[256]="";
+ bsnprintf(cmd, sizeof(cmd), "DROP TABLE %s", table);
+ if (!db_sql_query(mdb, cmd, NULL, (void *)NULL)) {
+ return -1;
+ }
+ return 0;
+}
+
+static int setup_temp_dep_table(B_DB *mdb, struct jmed **jds, int arrlen, DB_LIST_HANDLER *sendit, void *ctx)
+{
+ int i=0;
+ int error=0;
+
+ if(create_temp_dep_table(mdb, "KeepOrReuse")) return 1;
+
+ for(i=0; i<arrlen; i++) if(jds[i])
+ {
+ /*
+ char msg[256]="";
+ bsnprintf(msg, sizeof(msg), "'%d' '%d' '%d'\n",
+ jds[i]->mediaid,
+ jds[i]->expired,
+ jds[i]->dependents);
+
+ sendit(ctx, msg);
+ */
+ if(!error)
+ {
+ if(insert_into_temp_dep_table(mdb,
+ "KeepOrReuse", jds[i]->mediaid,
+ jds[i]->expired,
+ jds[i]->dependents,
+ jds[i]->dependents_expired)) error++;
+ }
+ free(jds[i]);
+ }
+ if(arrlen) free(jds);
+ if(error) return -1;
+ return 0;
+}
+
+/* Added by Graham: combine the job/jobmedia/media tables to have a more useful
+ view. */
+void db_list_jobandmedia_records(JCR *jcr, B_DB *mdb, uint32_t JobId,
+ DB_LIST_HANDLER *sendit, void *ctx, e_list_type type)
+{
+ char ed1[50];
+ int arrlen=0;
+ struct jmed **jd=NULL;
+
+ if(get_mediaids_with_expiry_and_dependence_statuses(jcr,
+ mdb, sendit, ctx, &jd, &arrlen))
+ return;
+
+ db_lock(mdb);
+
+ if(setup_temp_dep_table(mdb, jd, arrlen, sendit, ctx))
+ {
+ drop_temp_dep_table(mdb, "KeepOrReuse");
+ db_unlock(mdb);
+ return;
+ }
+if (JobId > 0) { // do by JobId
+ // Graham: I was previously using this...
+ // "SEC_TO_TIME(m.`VolRetention`) AS VolRetention,"
+ // ...but SEC_TO_TIME has an upper bound of 838:59:59, so it goes
+ // wrong if you have a volume retention greater than a month.
+ // So, instead, divide the seconds by 3600 to give the figure in
+ // hours. This is OK as long as we do not offer retention times
+ // broken down to intervals less than an hour.
+ Mmsg(mdb->cmd,
+ "SELECT DISTINCT j.JobId,m.MediaId,m.VolumeName,c.Name as Client,"
+ "j.Name,Level,JobStatusLong,SchedTime,StartTime,EndTime,"
+ "TIMEDIFF(`EndTime`,`StartTime`) AS Duration,"
+ "CONCAT(TRUNCATE(m.`VolRetention`/3600, 0),':00:00') AS VolumeRetention,"
+ "FROM_UNIXTIME(`JobTDate`+m.`VolRetention`) AS ExpireTime,"
+ "JobTDate+m.VolRetention AS ExpireTimeSecs,"
+ "VolBytes,FileSet,p.Name AS PoolName,"
+ "t.Name AS StorageName,"
+ "kor.Required, "
+ "j.JobId IS NULL AS isnull "
+ "FROM KeepOrReuse kor "
+ "LEFT JOIN Media m ON kor.MediaId=m.MediaId "
+ "LEFT JOIN JobMedia jm ON jm.MediaId=m.MediaId "
+ "LEFT JOIN Job j ON j.JobId=jm.JobId "
+ "LEFT JOIN Client c ON j.ClientId=c.ClientId "
+ "LEFT JOIN Pool p ON j.PoolId=p.PoolId "
+ "LEFT JOIN FileSet f ON j.FileSetid=f.FileSetid "
+ "LEFT JOIN Status s ON j.JobStatus=s.JobStatus "
+ "LEFT JOIN Storage t ON m.StorageId=t.StorageId "
+ "WHERE JobId='%s' ",
+ "ORDER BY isnull ASC, SchedTime ASC",
+ edit_int64(JobId, ed1));
+} else {
+ Mmsg(mdb->cmd,
+ "SELECT DISTINCT j.JobId,m.MediaId,m.VolumeName,c.Name as Client,"
+ "j.Name,Level,JobStatusLong,SchedTime,StartTime,EndTime,"
+ "TIMEDIFF(`EndTime`,`StartTime`) AS Duration,"
+ "CONCAT(TRUNCATE(m.`VolRetention`/3600, 0),':00:00') AS VolumeRetention,"
+ "FROM_UNIXTIME(`JobTDate`+m.`VolRetention`) AS ExpireTime,"
+ "JobTDate+m.VolRetention AS ExpireTimeSecs,"
+ "VolBytes,FileSet,p.Name AS PoolName,"
+ "t.Name AS StorageName,"
+ "kor.Required, "
+ "j.JobId IS NULL AS isnull "
+ "FROM KeepOrReuse kor "
+ "LEFT JOIN Media m ON kor.MediaId=m.MediaId "
+ "LEFT JOIN JobMedia jm ON jm.MediaId=m.MediaId "
+ "LEFT JOIN Job j ON j.JobId=jm.JobId "
+ "LEFT JOIN Client c ON j.ClientId=c.ClientId "
+ "LEFT JOIN Pool p ON j.PoolId=p.PoolId "
+ "LEFT JOIN FileSet f ON j.FileSetid=f.FileSetid "
+ "LEFT JOIN Status s ON j.JobStatus=s.JobStatus "
+ "LEFT JOIN Storage t ON m.StorageId=t.StorageId "
+ "ORDER BY isnull ASC, SchedTime ASC");
+}
+ if(!QUERY_DB(jcr, mdb, mdb->cmd)) {
+ drop_temp_dep_table(mdb, "KeepOrReuse");
+ db_unlock(mdb);
+ return;
+ }
+
+ list_result(jcr, mdb, sendit, ctx, type);
+ sql_free_result(mdb);
+ drop_temp_dep_table(mdb, "KeepOrReuse");
+
+ db_unlock(mdb);
+}
+
/* -----------------------------------------------------------------------
*
* Generic Routines (or almost generic)
Index: WORK/src/dird/autoprune.c
===================================================================
RCS file: /cvs/netpilot/GPL/bacula-5.0.3/WORK/src/dird/autoprune.c,v
retrieving revision 1.1
retrieving revision 1.3
diff -u -r1.1 -r1.3
--- WORK/src/dird/autoprune.c
+++ WORK/src/dird/autoprune.c
@@ -181,6 +181,16 @@
/* Prune only Volumes with status "Full", or "Used" */
if (strcmp(lmr.VolStatus, "Full") == 0 ||
strcmp(lmr.VolStatus, "Used") == 0) {
+ int dret=0;
+
+ /* Do not prune/purge jobs that other jobs depend upon!
+ db_volume_has_dependents() returns -1 on error, 0 if there were no
+ dependents, and 1 if there were. */
+ if((dret=db_volume_has_dependents(jcr, jcr->db, lmr.MediaId))!=0) {
+ if(dret>0) Dmsg2(100, _("Volume '%s' (%d) has dependents.\n"), lmr.VolumeName, (int)lmr.MediaId);
+ continue;
+ }
+
Dmsg2(100, "Add prune list MediaId=%d Volume %s\n", (int)lmr.MediaId, lmr.VolumeName);
count = get_prune_list_for_volume(ua, &lmr, &prune_list);
Dmsg1(100, "Num pruned = %d\n", count);
Index: WORK/src/dird/ua_output.c
===================================================================
RCS file: /cvs/netpilot/GPL/bacula-5.0.3/WORK/src/dird/ua_output.c,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- WORK/src/dird/ua_output.c
+++ WORK/src/dird/ua_output.c
@@ -385,6 +385,28 @@
/* List for all jobs (jobid=0) */
db_list_jobmedia_records(ua->jcr, ua->db, 0, prtit, ua, llist);
}
+ /* Added by Graham: Combine the job/jobmedia/media tables to get a more
+ useful view. */
+ } else if (strcasecmp(ua->argk[i], NT_("jobandmedia")) == 0) {
+ int done = FALSE;
+ for (j=i+1; j<ua->argc; j++) {
+ if (strcasecmp(ua->argk[j], NT_("ujobid")) == 0 && ua->argv[j]) {
+ bstrncpy(jr.Job, ua->argv[j], MAX_NAME_LENGTH);
+ jr.JobId = 0;
+ db_get_job_record(ua->jcr, ua->db, &jr);
+ jobid = jr.JobId;
+ } else if (strcasecmp(ua->argk[j], NT_("jobid")) == 0 && ua->argv[j]) {
+ jobid = str_to_int64(ua->argv[j]);
+ } else {
+ continue;
+ }
+ db_list_jobandmedia_records(ua->jcr, ua->db, jobid, prtit, ua, llist);
+ done = TRUE;
+ }
+ if (!done) {
+ /* List for all jobs (jobid=0) */
+ db_list_jobandmedia_records(ua->jcr, ua->db, 0, prtit, ua, llist);
+ }
/* List JOBLOG */
} else if (strcasecmp(ua->argk[i], NT_("joblog")) == 0) {
------------------------------------------------------------------------------
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today. Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
