Thank you Alan for the detailed response.
I was unaware of the GPL violations, this is an issue for me and it will
be removed promptly.
The block size I am aiming for was 512KB-1MB and have done extensive
research on tuning. This should also of course be within specification
of bacula-sd.
Because I am currently writing at a 64KB block, I get around 70MB/s of
throughput on a mix of compressible and non-compressible data. Even
changing it to a working 256KB improves what I am writing by about
15MB/s. Changing to any other value causes an immediate Device Or
Resource busy error.
The setup has worked well for years until my storage array doubled in
size. It is taking over a week to do a backup.
This is direct-attached storage so I don't believe the network buffer
should have an effect, right?
I'm not sure I completely understand this statement.
"
You cannot change maximum block sizes on a single volume (Tape). Doing
so is extremely likely to result in an unreadable volume past the point
where the block size has changed (reading older tapes with smaller
maximum block sizes is still ok)
"
My plan was to overwrite all tapes that may have a smaller block size.
It doesn't matter at this time if I temporarily don't have a tape backup
and I have no old archives I am concerned with.
Rewinding the tape and writing an EOF marker and rewinding again should
solve this correct?
To me it seems like a the SAS controller I am connected to but it just
seems so unlikely it isn't able to pass blocks greater than 256KB but I
have no way of verifying this.
The more I write this the more I am convincing myself it probably has
nothing to do with Bacula (rip-off version) or the configuration. I was
just hoping someone has run into something similar before.
Regards,
drewv
On 12/22/2016 11:15 AM, Alan Brown wrote:
> On 22/12/16 14:31, Drew Von Spreecken wrote:
>> Greetings,
>>
>> I have run into an issue and am looking for input. I have a SAS tape
>> autoloader with an IBM HH-LTO6 drive running the newest firmware.
>> It is currently connected to a Adaptec 78165 HBA/Raid controller via SAS.
> The maximum block size for IBM LTO6 drives (HH or full height) is 8MB
> (It's usually 16MB for HP drives)
>
>> The issue I am having is when I attempt to modify the block-size to
>> anything over 256KB that BareOS writes to tape, it fails. To simplify
>> troubleshooting I have opted to use a combination of btape and dd to test
>> block-size adjustments.
> Bareos is not Bacula
>
> (well, it _IS_ Bacula, which someone has tried to repackage and claim
> credit for. It is not supported by the Bacula community - and given the
> legal antics of the Bareos maintainer, (including egrarious GPL
> breaches) I would advise you don't bring Bareos problems to this forum -
> and further suggest that you should consider uninstalling Bareos.)
>
>
> Back on the block size topic:
>
>
> The maximum block size supported by bacula-SD is 2MB. That is sufficient
> to get at least 140MB/s write speed on a LTO6 drive for non-compressible
> data and upwards of 300MB/s for highly compressible data.
>
> device {
> ...
> Maximum block size = 2M
> ...
> }
>
>
> Attempting to increase max block size past 2MB will result in errors.
> There is no general benefit in doing so in any case (Tested and
> verified. I feel that setting larger block sizes should be allowable
> even if there's no benefit, as it causes apparent errors when the max
> block size settable in Bacula is smaller than the max allowable block
> size reported by the tape drives)
>
>
> Make sure you understand the difference between the block size (writing
> to the tape device) and the network buffer size (communications between
> -df, -dir and -sd - and needs to be set in all three config files if you
> change anything)
>
> device {
> ...
> Maximum Network Buffer Size = 262144
> ...
> }
>
> Increasing the Network buffer size from the default 64kB is advisable to
> achieve higher transfer rates on Gb/s or faster networks but there is no
> discernable benefit going past 256kB - even on 10Gb/s networks.
>
>
> You cannot change maximum block sizes on a single volume (Tape). Doing
> so is extremely likely to result in an unreadable volume past the point
> where the block size has changed (reading older tapes with smaller
> maximum block sizes is still ok)
>
>
> If changing block size in a working system, mark ALL open volumes as
> used _before_ attempting any more writes.
>
>
> Alan
>
>
>
>> Each test I perform I rewind the tape, write EOF and rewind again. I'm
>> not missing a step here, right? Should I be able to write to a tape in
>> this way with a different block size if I've used it at a different
>> (smaller) size before?
>>
>> I am querying the tape-drive in the autoloader directly, the autoloader
>> should not be part of the problem.
>>
>> Writing at anything under and at 256Kb works fine but is slow.
>>
>> The output from tapeinfo is:
>>
>> tapeinfo -f /dev/nst0
>> Product Type: Tape Drive
>> Vendor ID: 'IBM '
>> Product ID: 'ULTRIUM-HH6 '
>> Revision: 'G9P1'
>> Attached Changer API: No
>> SerialNumber: '10WT077984'
>> MinBlock: 1
>> MaxBlock: 8388608
>> SCSI ID: 0
>> SCSI LUN: 0
>> Ready: yes
>> BufferedMode: yes
>> Medium Type: 0x68
>> Density Code: 0x5a
>> BlockSize: 0
>> DataCompEnabled: no
>> DataCompCapable: yes
>> DataDeCompEnabled: yes
>> CompType:
>> DeCompType: 0xff
>> Block Position: 5
>> Partition 0 Remaining Kbytes: -1
>> Partition 0 Size in Kbytes: -1
>> ActivePartition: 0
>> EarlyWarningSize: 0
>> NumPartitions: 0
>> MaxPartitions: 3
>>
>> As you can see the block size limit for the drive itself is ~8MB...
>>
>> Here is an output from mt:
>>
>> mt -f /dev/nst0 status
>> SCSI 2 tape drive:
>> File number=0, block number=0, partition=0.
>> Tape block size 0 bytes. Density code 0x5a (no translation).
>> Soft error count since last status=0
>> General status bits on (41010000):
>> BOT ONLINE IM_REP_EN
>>
>> Here is an attempt to write at a 256K block:
>>
>> dd if=/dev/zero of=/dev/nst0 bs=256k count=1
>> 1+0 records in
>> 1+0 records out
>> 262144 bytes (262 kB) copied, 1.9402 s, 135 kB/s
>>
>> Here is the failure at 512K:
>>
>> dd if=/dev/zero of=/dev/nst0 bs=512k count=1
>> dd: error writing ‘/dev/nst0’: Device or resource busy
>> 1+0 records in
>> 0+0 records out
>> 0 bytes (0 B) copied, 1.56954 s, 0.0 kB/s
>>
>>
>> It will fail with anything over 256K, even 257K.
>>
>> There are no errors in my system logs.
>>
>> I suspect either I have a configuration error here or am missing
>> something simple OR the Adaptec 78165 raid controller is limiting the
>> block size before I write to tape. Adaptec support is unable to confirm
>> this. Is there a way I can prove this or does anyone have any guidance
>> on how to continue troubleshooting this issue?
>>
>> Thanks!
>>
>> --drewv
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Developer Access Program for Intel Xeon Phi Processors
>> Access to Intel Xeon Phi processor-based developer platforms.
>> With one year of Intel Parallel Studio XE.
>> Training and support from Colfax.
>> Order your platform today.http://sdm.link/intel
>> _______________________________________________
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>>
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
