> I'm sorry about that. The version is 1.38.9.
>
> Thanks for the feedback. It sounds like Fifo is EOL'd as far as support
> in the code goes?
>
> Actually, my purpose in using Fifo's is to use bacula for archiving to
> DVD in addition to doing backups.
>
> The problem I'm solving is that I've already been burned by DVD's that,
> for whatever, reason, have an error or two on them after some time in
> storage. Rather than make 3 (or however many) DVD images of whatever I'm
> archiving, I've taken to using cpio, and piping the output of that into
> a program that performs a FEC encoding based on Vandermonde matrices.
> This, in turn, is streamed directly onto a DVD (no filesystem--just a
> stream of data.)
>
> The resulting DVD can be fed back into the FEC decoding quite well using
> dd if=/dev/dvd, and then that, of course, is piped back into cpio.
>
> Using this method, I like to make regular "permanent snapshots" of
> my /home filesystem, in additional to occasional snapshots of certain
> directories, like my digital photos.
>
> Yeah, I know it violates the principle that an "archive" should be more
> random-accessible than that (I have to play back through the whole
> stream to get any particular file) but it gives me the "best"
> combination of ease of use and administration. Using my technique (along
> with another program for splitting the streams across multiple deivces),
> I don't have to worry about figuring out which and how many files (e.g.,
> pictures) to put onto which DVD. All I need to know is the order of
> DVD's and what is in that particular stream. (kind of like a backup :-).
>
> My hope was to use bacula to make the overall operation go even more
> smoothly, basically using bacula to replace cpio + my manual
> manipulation of filenames to backup. When I saw that bacula supported a
> fifo device type, I was very excited because of the possibilities this
> opened up.
>
> Another thing that would work just as well--probably even better--is if
> bacula provided a hook for some kind of manipulation of the data stream.
> That manipulation could then be a user-supplied compression, FEC, or
> whatever. It might even make sense as an additional option in the
> FileSet definition.
>
> Anyway, that's my story. Perhaps there's a better way, with bacula as it
> exists, to achieve what I want? I'm open to suggestions, but the key
> thing is that I want to be able to use some kind of self-supplied or
> third-party FEC coding to make it so that even if there are read errors
> from the DVD volume, I can still ultimately recover all of my data.
Interesting project and ideas. Since you are using 1.38, there is a good
chance that the FIFO code does work since if I remember right it has been
tested and did work in that version. As I said, I strongly suspect that
it is broken in 1.39.x
At the moment there is no way with Bacula (that I can think of) to pass
your data to or through an external program. If you have some ideas, you
might try putting them in a Feature Request (see the Feature Request page
on www.bacula.org for details). This is something that could perhaps be
very interesting especially for users who want to encrypt the data in the
Storage daemon just before it goes on the volume.
Regards,
Kern
>
> Thanks,
> -Dan
>
>
> On Sun, 2006-10-22 at 17:40 +0200, Kern Sibbald wrote:
>> I wonder if we couldn't require a $10 donation to the project every time
>> someone forgets to include the Bacula version? I'd probably be a
>> multi-millionaire by now. :-)
>>
>> It looks like you are running some 1.39 version, and any FIFO use for
>> the
>> Device type is likely to be broken as there is no regression test for
>> it.
>> In any case, I cannot imagine any reason to use a FIFO device unless you
>> have some sort of pseudo attached tape device. You are probably wasting
>> your time trying to get it to work, unless there is no other way to do
>> what you want.
>>
>> If you are trying to write DVD images, a disk Volume would do quite
>> well.
>> If you want to write directly to DVD, the current beta seems to be in
>> pretty good shape.
>>
>> Regards,
>>
>> Kern
>>
>>
>> > I'm trying to use a fifo for backup/restore.
>> >
>> > I have the storage set up for min and max block size = 1024.
>> >
>> > My runbeforejob script uses dd with ibs=1024 obs=1024 conv=sync, to
>> pull
>> > the data from the fifo and save it to a file (saving to the file is
>> just
>> > an interim test setup; I'll do something else with the data later). I
>> > tried it with and without the conv=sync.
>> >
>> > The first few blocks seem fine. However, it always seems to write
>> fewer
>> > bytes out for the fifth block, resulting in a sixth block that starts
>> at
>> > the wrong place. This shows up in the log as something like:
>> >
>> > Enter dispatch_msg type=4 msg=monica-sd:
>> > RestoreFiles.2006-10-22_10.40.43 Error: block.c:263 Volume data error
>> at
>> > 0:6143! Wanted ID: "BB02", got "". Buffer discarded.
>> >
>> > This almost feels like a block sync error inside of bacula. If bacula
>> > was actually outputing a smaller block size, then the conv=sync would
>> > have taken care of it, so that on restore the beginning of each block
>> > would at least look right.
>> >
>> > Can anyone shed any light on this? Here is my storage definition:
>> >
>> > Device {
>> > Name = SafeDVD
>> > Archive Device = "/tmp/backup_fifo"
>> > Device Type = Fifo
>> > Media Type = "Safe DVD"
>> > Minimum block size = 1024
>> > Maximum block size = 1024
>> > AlwaysOpen = no
>> > LabelMedia = no
>> > Automatic Mount = no
>> > Random Access = no
>> > RequiresMount = no
>> > RemovableMedia = no
>> > VolumePollInterval = 10
>> > }
>> >
>> > And my scripts for backup and restore runbeforejob:
>> >
>> > Here's the one for backup:
>> >
>> > #!/bin/bash
>> > #
>> >
>> > # create a fifo
>> > #
>> > /bin/rm -f /tmp/backup_fifo
>> > /bin/mkfifo --mode=777 /tmp/backup_fifo
>> >
>> > # start the backup process reading from it
>> >
>> > ( nohup dd if=/tmp/backup_fifo of=/var/tmp/mybackup_ \
>> > ibs=1024 obs=1024 conv=sync ) &>/dev/null </dev/zero &
>> >
>> >
>> > And here's the one for restore:
>> >
>> > #!/bin/bash
>> > #
>> >
>> > # create a fifo
>> > #
>> > /bin/rm -f /tmp/backup_fifo
>> > /bin/mkfifo --mode=777 /tmp/backup_fifo
>> >
>> > # start the backup process reading from it
>> >
>> > ( nohup dd if=/var/tmp/mybackup_ of=/tmp/backup_fifo \
>> > bs=1024 ) &>/tmp/restore.err </dev/zero &
>> >
>> >
>> >
>>
> --
>
>
> .
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642_______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
Best regards, Kern
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
