On 30/11/15 16:57, Christoph Litauer wrote:
> Hi Alan,
>
> maybe this is an important hint …
> I thought btape doesn't use a disk at all. Instead it uses on-the-fly
> generation of test data … does it?
If it does, that could easily be your bottleneck. /dev/random isn't
normally very fast.
> If btape uses the configured spool directory I have to take a look at that
> point.
For LTO, the Spool disk MUST be at least least one SSD, preferably a
stripe of them on as fast a controller as you can afford. Standard disks
simply can't keep up with tape drives.
In addition (as I've discovered the hard way), those SSD's must be
regularly fstrimmed or mounted with discard options(*).
If behind a controller which doesn't support discard/trim operation you
will end up at the "steady state" write speed of the SSDs which is
usually 5-10% of the quoted throughput (as an example, Samsung 840Pros
drop from 250-300MB/s down to under 5MB/s write speed once they're full.
It's important to note that SSD "full" is different from Filesystem
"full" and extremely important to look at the "steady state" write
speeds if you're pumping a lot of data through the things. Only a few
reviewers benchmark this stat in terms of MB/s instead of IOPs and it's
of critical importance in this kind of operation.
(*) Just to confuse things, some OSes/distributions *Ahem*RHEDHAT*Ahem*
ignore the discard flag on performance grounds(**), so you might think
you're doing things right but find out that it's still breaking
(**) The claim is that trims during normal operation seriously degrade
performance. This is true for some old, slow crappy MLC drives that had
slow CPUs but nothing made recently suffers from this. The claims were
made in the context of "discouraging" the use of the mount flag and no
mention was made that they were silently ignoring the thing if it was
set anyway.
As a result of this discovery I've added fstrim crontabs everywhere
there are SSDs and suddenly discovered a lot more IO performance than we
were getting up to now.
>> Am 30.11.2015 um 17:00 schrieb Alan Brown <a...@mssl.ucl.ac.uk>:
>>
>> On 30/11/15 15:32, Christoph Litauer wrote:
>>> Andrew,
>>>
>>> many thanks for this hint. I installed the IBM driver and found the tape
>>> drive testing tool itdt, too.
>>> Now, using itdt and the new driver there is an improvement:
>>> LTO6: 158 MB/s without compression, 177MB/s with compression
>>> LTO4: 27 MB/s without compression, 128 MB/s with compression.
>>>
>>> For LTO6 this is in fact near to the theoretical limit (regarding random
>>> data).
>>>
>>> Using btape with the new driver I still got 153MB/s (with compression)
>>> resp. 39 MB/s (random data).
>>> Seems as if there is a bottleneck in bacula?
>>> My device configuration:
>>>
>>
>> or a bottleneck in your disks.
>>
>> What is the physical makeup of /var/spool/bacula and how many simultaneous
>> sessions have you got using that point?
>>
>>
>>
>>> Device {
>>> Name = "LTO6-1"
>>> Device Type = Tape;
>>> Media Type = LTO6
>>> Archive Device = /dev/IBMtape0n
>>> AutomaticMount = yes;
>>> AlwaysOpen = yes;
>>> RemovableMedia = yes;
>>> RandomAccess = no;
>>> Spool Directory = /var/spool/bacula
>>> Maximum Spool Size = 300G
>>> Maximum File Size = 5G
>>> Maximum Block Size = 1m
>>> }
>>>
>>>
>>>> Am 28.11.2015 um 08:56 schrieb Andrew Ryder <tire...@shaw.ca>:
>>>>
>>>> Are you using the IBM Linux tape driver? If not, I'd suggest installing it
>>>> vs the generic linux kernel's tape driver.
>>>>
>>>> http://www-933.ibm.com/support/fixcentral/swg/selectFixes?parent=Tape%2Bdrivers%2Band%2Bsoftware&product=ibm/Storage_Tape/Tape+device+drivers&release=1.0&platform=Linux&function=all
>>>>
>>>> On 11/27/2015 08:43 AM, Christoph Litauer wrote:
>>>>> Dear bacula users,
>>>>>
>>>>> I recently recognized that my tape drives do not perform as expected.
>>>>> Neither while used with bacula nor native with dd. I did many performance
>>>>> tests in the meantime but could not figure out the reason. So may I ask
>>>>> in this group what typical write speeds you have with your tape drives?
>>>>>
>>>>> I tested LTO6 (IBM Ultrium HH6) and LTO4 (Ultrium 4-SCSI). Both are part
>>>>> of two libraries connected via SAS to a LSI SAS3801E-controller. The host
>>>>> system runs SLES12.
>>>>> I used btape and dd for writing.
>>>>> I figured out the best writing performance with blocksize 512KB (LTO4)
>>>>> resp. 2MB (LTO6). Both drives with compression on.
>>>>>
>>>>> LTO4 best performance (writing zeros): 115 MB/s
>>>>> LTO4 worst performance (random data): 71.58 MB/s
>>>>>
>>>>> LTO6 best performance (zeros): 153 MB/s
>>>>> LTO6 worst performance (random): 58 MB/s
>>>>>
>>>>> Disabling compression does not change the maximum speed. In this case
>>>>> writing zeros is the same as writing random date (expected).
>>>>>
>>>>> According to Wikipedia, theoretical speed should be
>>>>> LTO4: 240 resp. 120MB/s
>>>>> LTO6: 400 resp. 160MB/s
>>>>>
>>>>> Any ideas welcome! Thanks in advance!
>>>>>
>>>>> --
>>>>> Kind regards
>>>>> Christoph
>>>>> _________________________________________
>>>>> Christoph Litauer
>>>>> Uni Koblenz, Computing Centre, Office A 022
>>>>> Postfach 201602, 56016 Koblenz
>>>>> Fon: +49 261 287-1311, Fax: -100 1311
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> _______________________________________________
>>>>> Bacula-users mailing list
>>>>> Bacula-users@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>>>
>>>
>>> --
>>> Freundliche Grüße
>>> Christoph Litauer
>>> _________________________________________
>>> Christoph Litauer
>>> Uni Koblenz, Rechenzentrum, Raum A 022
>>> Postfach 201602, 56016 Koblenz
>>> Fon: +49 261 287-1311, Fax: -100 1311
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Go from Idea to Many App Stores Faster with Intel(R) XDK
>>> Give your users amazing mobile app experiences with Intel(R) XDK.
>>> Use one codebase in this all-in-one HTML5 development environment.
>>> Design, debug & build mobile apps & 2D/3D high-impact games for multiple
>>> OSs.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
>>> _______________________________________________
>>> Bacula-users mailing list
>>> Bacula-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>
>>>
>>
>>
>>
>
> --
> Freundliche Grüße
> Christoph Litauer
> _________________________________________
> Christoph Litauer
> Uni Koblenz, Rechenzentrum, Raum A 022
> Postfach 201602, 56016 Koblenz
> Fon: +49 261 287-1311, Fax: -100 1311
>
>
>
>
>
> ------------------------------------------------------------------------------
> Go from Idea to Many App Stores Faster with Intel(R) XDK
> Give your users amazing mobile app experiences with Intel(R) XDK.
> Use one codebase in this all-in-one HTML5 development environment.
> Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
> http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
