Hello,
On 12/28/2005 6:11 PM, Beren Gamble wrote:
Hi Arno,
Sorry I didn't reply to the list... That's me on autopilot :D - Hope you had a
Merry Xmas!
No problem... and thanks, I did have some nice days.
I have it working perfectly, EXCEPT the only problem i've now got, is when
Bacula looks for volumes to use, it chooses Recycled volumes over Purged
volumes. In the hope that i'd be able to force Bacula to use the tape in my
drive, I've manually set all of my volumes to Used.. even the ones that have
never been used, but some of the volumes have never had a successful backup to
them, so bacula sees that no jobs are associated with a volume, so it marks it
Recycled, then of course, it chooses the Recycled volume to backup to. During
this process, Bacula marks the volume in my drive as Purged, as it should, but
it ignores it.
The empty volumes are the stick in the mud here.
The reason there are empty volumes is because they may have failed any time
within the last 2 weeks (we run a 2 week cycle).
Hmm. Ok, that sounds like some real problem to me :-)
IIRC, you're running something newer than 1.36.something. As far as I
know, with these versions, Bacula should always prefer any volume that
could be recycled (i.e. is purged) and that's already mounted over ones
that need extra intervention. That, at least, is one feature that's
working here since... about... forever ;-)
Now, you might check if you've got the Accept Any Volume directive set
in your config, and you could check what happens in a situatien where
- you've got a recyclable (or even recycled) volume in the tape drive,
- Bacula requests a different volume,
- and you simple issue a mount command.
Usually, in my experience, Bacula should start writing to the tape in
the drive. (I *think* this is a bug but I never investigated it - there
seems to be some potential for improving the recycling / pruning
algorithm. Basically, I suspect that Bacula, when it looks for a new
volume, first look for a pruned volume, then starts job pruning, after
that it prunes volumes, and thinks it's finished. I've seen situations
like this, and I thought a different sequence might be more useful.)
If the above does work, you should closely observe the sequence of
things Bacula does - run the DIR with debug output and see if you can
pinpoint the situation where it expects the wrong volume. I'm sure that,
during the next week, it's possible to prepare a perfecly detailed bug
reports with all sorts of necessary information ;-)
Arno
Thanks,
Beren
Arno Lehmann <[EMAIL PROTECTED]> 27/12/2005 22:17 >>>
Hello,
it took me a while to answer, partly due to the christmas days, partly
due to you writing to me personally :-) anyway...
On 12/23/2005 10:04 AM, Beren Gamble wrote:
Arno Lehmann Writes-----
Well, only to repeat myself - it's not that Bacula is designed to
overwrite valid data, instead it tries to protect that data.
I fully appreciate this, but in cases where we have single tape units, I need a setting
which overrides this and uses the tape that I supply. After all, if I put a tape in which
is outside of the retention period, a retention period which "I" have set, I
want it to be overwritten.. I don't want the data preserved. And it doesn't seem possible
to do this wich Bacula.
Hmm. I admit I still don't understand why anyone would want to overwrite
a perfect backup volume, but using the right settings for recycling,
maximum volume use duration and volume retention should allow what you want.
Keep in mind that I haven't tried this, but allowing, for example 12
hour of volume use time and setting a volume retention of 11 hours
should have the result you want: If, after, one day, the volume is still
loaded it can be recycled, and thus re-used.
Hope this helps,
and I guess I'll need to start some experiments when I've got more time
available...
Arno
Arno Lehmann <[EMAIL PROTECTED]> 23/12/2005 08:14 >>>
Hello,
On 12/23/2005 2:19 AM, Doug Sampson wrote:
FWIW.
My experience with using Bacula on tape drives is similar to Beren's. When
used on HD server, Bacula runs like a dream. But when used with a server
using tape drives, I am seeing Bacula wanting to append to volumes that
aren't full when I've specificaly requested that Bacula reuse the tape
currently in the tape drive. Bacula doesn't ask for a different tape but
appends to the tape inserted.
Well, only to repeat myself - it's not that Bacula is designed to
overwrite valid data, instead it tries to protect that data.
I have been following this thread and making adjustments to the
bacula-dir.conf file as I see fit. I've inserted the 'Volume Use Duration'
parameter and am not seeing any changes that I expect. Volumes marked
"Append" are still marked such as even after the Volume Use Duration time
parameter has passed.
I assume you didn't see the following sentence in the manual, Director
configuration, in the section dealing with Volume Use Duration.
"The use duration is checked and the Used status is set only at the end > of a job that writes to the particular volume, which means that even
though the use duration may have expired, the catalog entry will not be > updated until the next job that uses this volume is run."
Currently running 1.38.2.
Currenly not running 1.38.3 :-(
Arno
~Doug
-----Original Message-----
From: Beren Gamble [mailto:[EMAIL PROTECTED]
Sent: Wednesday, December 21, 2005 08:34 AM
To: bacula-users@lists.sourceforge.net; [EMAIL PROTECTED]
Subject: Re: [Bacula-users] Puring/Pruning doesn't seem to be
happening.
Ok, maybe i'm not picking up a concept here in the
documentation.... I've now got my Volume Use Duration at 23h,
which works beautifully, marking the volume as Used.
However, I just tried to run a backup job. All of my volumes
in the pool were set as Used. (If VolUsen didn't mark them
used, I did, just for uniformities' sake) But bacula then
found a volume which had no jobs associated with it and
decided to recycle that one instead of the volume that was in
the drive (it definitely is past the 12 day retention period).
Bacula is an absolute dream with my autochanger because tapes
are only either marked as Full or, Purged or Recycle and I
never remove a half used tape. But on the single tape
systems, I still haven't figured out how to make bacula cope
with rotating half filled volumes.. or volumes where all
backups failed and no data was written.
Kern Sibbald <[EMAIL PROTECTED]> 17/12/2005 10:56:45 >>>
I think the basic problem that you have had, and it seems
like you don't quite
understand though you have found a solution, is that
recycling for Bacula requires applying two concepts: 1. the
time Bacula can write on a Volume, 2.
the time Bacula keeps that Volume before reusing it.
Most people come with the pre-conceived idea that there is
one "Recycle" time,
when in fact Bacula is significantly more sophisticated than that.
I've reviewed the chapter you refer to below, and if one
reads the full
chapter, all the information *is* there (at least IMO). I have added
additional text that stresses the above two concept points a
bit more as well as refer the user to the "Basic Volume
Management" chapter that gives a
detailed practical example of why you want to do. The
Recycle chapter is complete in itself assuming you want to
fill all your Volumes. If not, you
must also read the Basic Volume Management chapter as well.
Concerning your comment:
"Can someone please update the documentation here
http://www.bacula.org/rel-manual/Automatic_Volume_Recycling.html
with this information, that should hopefully forestall
this question
coming up so much."
I appreciate you pointing out where you got confused.
However, this is a free
software project, and I understand recycling, so I don't
easily see why some
people don't get the concepts or I would have already changed
the doc. As a
consequence it is much more helpful if you would submit a
suggested change to
the documentation that clarifies it for you.
I suggest you look at the documentation in:
http://www.bacula.org/dev-manual/Automatic_Volume_Recycling.html
It is usually much more up to date.
On Saturday 17 December 2005 07:00, Harondel J. Sibble wrote:
On 16 Dec 2005 at 1:25, Arno Lehmann wrote:
....so....how does one force it to do it?!
Apart from the configuration directives like "prune
oldest volume" (or
something) my advice is usually "you don't".
Philisophical disagreement, If I as the admin decide that,
that is an
appropriate thing to do, I should be able to do it in some
manner. Reminds
me of the unix philosophy, build a bunch of single purpose
tools that you
can chain together to do anything you want .....
Anyhow, on the how to do it, here's what I got from an
unrelated list where
I was ranting about the same problem the OP has as I had it
for months
also.
What you need is this
Volume Use Duration =
added to your pool definition where the duration is equal
to or less than
your Volume Retention setting, in my case the relevant lines are
Volume Retention = 6d
Volume Use Duration = 6d
With this setup, Bacula FINALLY works like I expect and how
pretty much any
other backup software I've used on Windows or Unix works. It will
overwrite the tape in the drive, assuming 6 days has
passed, it marks the
tape as used and then recycles it, however it only does that at the
beginning of the next job, so the recycling happens at the
start of the
current job. Can someone please update the documentation here
http://www.bacula.org/rel-manual/Automatic_Volume_Recycling.html
with this information, that should hopefully forestall this
question coming
up so much.
From the messages on tonights job
16-Dec 21:30 fileserver-dir: Start Backup JobId 70,
Job=Company-Fileserver_Full_Back 16-Dec 21:30
fileserver-dir: Recycled
current volume "Friday3"
16-Dec 21:30 fileserver-sd: Recycled volume "Friday3" on
device "DDS-4"
(/dev/ns
This is the first week since the beginning of the year
where I've not had
to manually purge the tapes each day.
Think: if the tape goes full after 3 jobs, and I have 4
each night, my
last job will not be done.
Or, in other words, you need enough space available.
Yes, that makes perfect sense.
No, this is,in my opinion, the best way to handle backups: Avoid
overwriting valid data whenever possible.
If I setup a rotation, I should be able as the
administrator to decide what
is the appropriate retention time
Well, your assumption is not coherent with Bacula, then.
Bacula treats a
retention like "keep the data for at least this time.
Only after this
time has passed consider recycling that volume." Thus,
you should assume
that your data will not be overwriten until 12 days have
passed after the
last job on a given volume is finished.
In my experience, it still doesn't work, even 5 weeks later
when I have a 6
day retention period, I still have to manually purge the tape.
Hmm. Isn't that how Bacula works? If a volume is in the
right pool,
and it is flagged as recyclable, and automatic pruning
for the jobs
and volumes is on, AND the retention period has passed, the volume
will be considered when Bacula looks for a recyclable volume.
The key thing for the OP to remember is that bacula will
not recycle the
current volume if any other appendable volumes exist, seems
odd to me, but
that is how it is. However, from a practical pov, using the
volume use
duration statement seems to get around that just fine.
If you want to limit the time Bacula uses a volume, you
can do so by
setting the proper volume use time or number of jobs.
Yes, that needs to be added to the docs.
If I set up a pool to have backups available for a
certain time, and
I've not got enough volumes, Bacula tells me so. Either I
re-think my
backup strategy and modify the setup, or I buy more tapes.
Even with that setup, it still doesn't recycle properly.
I've seen this in
several instances where I've followed the chapter on
automatic volume
recycling in the manual, until I was shown the info at the
beginning of
this email, I was looking at dumping Bacula, which is a
GREAT piece of
software, but for this one problem.
Considering the frequency that this problem is asked about,
I'd say there
is definately a conceptual issue at hand.
--
Best regards,
Kern
(">
/\
V_V
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep
through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web.
DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
**************************************************************
*********************
Mail FROM London Borough of Harrow:
Unencrypted electronic mail is not secure and may not be
authentic, in whole or in part. You are advised to check
directly with the sender before acting upon any e-mail received.
The information contained in this message and any attachments
is confidential and is intended for receipt by the above
named addressee(s) only. If you have otherwise encountered
this message please notify its originator via +44(0)20 8863
5611 at LONDON BOROUGH OF HARROW. The unauthorised use,
disclosure, copying or alteration of this message is strictly
forbidden. The views expressed within this message are those
of the individual sender and not necessarily those of Harrow Council.
Mail TO London Borough of Harrow:
London Borough of Harrow monitors all electronic mail it
receives for Policy compliance and to protect its systems
including anti-spam and anti-virus measures.
Electronic mail does not guarantee delivery, nor notification
of non-delivery. It is suggested you contact your intended
recipient(s) by other means should confirmation of receipt be
important.
**************************************************************
*********************
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep
through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web.
DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id865&op=click
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
--
IT-Service Lehmann [EMAIL PROTECTED]
Arno Lehmann http://www.its-lehmann.de
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
