On 08/06/15 15:58, Dimitri Maziuk wrote:
> On 2015-06-08 05:25, Denis Witt wrote:
>
>> I'm using normal (7.200 RPM) HDDs. But, as pointed out in
>> my mail, there is only one job running ("despooling Attributes") at the
>> moment. All other jobs are still waiting for execution ("waiting on
>> max storage jobs").
>
> Did you see my "Fixed: Re: concurrent jobs not working?" post from 4
> days ago? Take a look.The problem with adding more concurrent jobs (up from one) is that the increased seek load will badly affect despooling time. I benchamrked multidrive 7200rpm raid0 arrays several years ago and posted results to this list. The short version is: 0: Spooling areas _must_ be on seperate spindles to everythng else. 1: If you have spindles, you will see an exponential-"plus a bit" dropoff in overall throughput for each extra task added to the spool array from what's seen with a single job - output divided amongst jobs-in-progress, plus you've added effectively random seek latency of 8-12ms on top. 2: A raid-0 mechanical disk array is the _only_ way 7200rpm mechanical disks can keep up with LTO 1-2 tape drives for concurrent backups (simultaneous reads and writes because one spool builds whilst another is despooling to tape.) 3: You need at least 3 striped drives and no concurrent backups to keep up with LTO 3-4 - the benchmarked sequential read/write speed of 3-drive raid0 maxes out at about 360Gb/s - which is just enough if you are feeding any form of compressible data to the tape drive - the disks _must_ be able to keep well ahead of what's being fed to the tape drive or it will start shoeshining if there is the slightest hiccup. 4: When you add concurrency, you need more spindles to compensate for the seek penalties. Lots more spindles. 5: A good datacenter SSD will replace a bunch of spindles because it has virtually no seek penalties, plus decent write speeds. Datacentre SSDs continue to perform well under heavy random r/w load. Consumer ones don't - the write speeds suffer particularly badly and cheap consumer SSD used to have slower sequential write performance than spinning media, often falling to kB/s write speeds under random load (it's better now but still often falls apart quickly under heavy load) 6: No matter what you build, test, test and test again. If you think it's overspecified it probably isn't. Concurrent job spooling is not the kind of load drive designers had in mind even when considering extreme random r/w performance. 7: If you can afford it, tossing 100GB of ramdrive at the task is even better than SSD. 8: It doesn't hurt to put the database on a dedicated raid10 SSD set. My current spool area is a 6-way raid0 of Intel X25E drives. These feed 7 LTO5 drives via FC with the limiting factor being the 8Gb/s FC connections out of the bacula-sd box. Over the last 5 years that's worked pretty well at keeping up but I still get shoeshining from time to time. Concurrency plus spooling is a big win if you get it right and a major pain in the backside if you don't. Don't be tempted to attempt to run multiple concurrent jobs without spooling. It may write quickly, but the penalty on the read side is so severe as to make the tape effectively unuseable. My personal feeling is that it will shoeshine so badly the tape or drive may end up destroyed before being sucessfuly read, and if you do suceed in reading, you're looking at throughputs in the tens of kB/s range. ------------------------------------------------------------------------------ _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
