Dear Adam, In message <20190207185030.35830...@teln.shikadi.net> you wrote: > > It may also help in this case, but so would buffering instead of > spooling. I was under the impression that in the case of slow clients, > Bacula is designed to read from many clients at the same time, so that > it can get the throughput required for the tape without spooling.
You can backups from several clients in parallel, and this will create a higher total data rate, but it has also disadvantages: there is still no guarantee that there is a continuous flow of sufficient data at all times, and that data for each client are scattered all over the tape, which will significantly decrease efficientcy for a restore. It is much better to spool all the date for a client, and then write it to tape as a single chunk. Note that this is something you have to configure - there is no automatism. > This is true, but the drawback of the spool file is that you need > enough disk space to hold a full tape's worth of data for it to > perform optimally. If the spool file is not an exact multiple of the > tape size, performance will drop. This is wrong. It does not hurt if if the tape stops a few times, after writing a sufficiently big chunk of data. You just don't want to have it shoe-shining because the incoming data stream is too slow. > > This is not necessary. Or only one possible special case. > > It's not necessary, but if you do not do this, performance will suffer > and your tape will shoe shine. No, it will not. Shoe-shining happens when you cannot supply data fast enough while writing. There is absolutely no problem when the tape stops a few times between writing independent backups. > I am only referring to getting data from the spool file onto tape. > Let's say you have a 100GB spool file and you are writing to an 800GB > tape. The process will go like this: > > * Read 100GB from client, tape is idle > * Write 100GB to tape, pause tape > * Read next 100GB from client while tape is paused > * Start up tape again and write next 100GB > > Thus even if your clients can keep up with the tape 100% of the time, > you will still introduce extra shoe shining if your spool file is not > exactly one tape in size. Shoe-shining is something else. Assume youe example, and a LTO8 tape with compression, all working under optimal conditions. Then you can write 900 MB/s to the drive, i. e. writing the 100 GB spool file will take 111 seconds. That means, the tape will write continuously for nearly two minutes, then stop, then continue. This repeats seven times per tape. This is not a problem, and it is not what is called show-shining. Also, disk space is cheap - where is the problem of using a much bigger spool area? I use only LTO4 tapes so far, and I have a 1.5 TB spool area. Where is the problem? Also please note that you are kind of contradicting yourself - if you were uysing ram for buffering data, would you not still have to reserve at least a tape size of memory? Would you be willing to reserve 12 TB of RAM or this? Sorry, to me this makes no sense. > (If your spool file is larger than one tape, then you will fill up one > tape in one continuous operation which is perfect, but then the second > tape will pause once the end of the spool file has been reached which > is not ideal either.) Apparently you have a real angst for a tape stop. Don't! It is a perfectly normal thing to have. > So you can see that using a spool file is typically worse for a tape > drive, as it will almost always introduce additional stop/start cycles > (shoe shining) which would not be there otherwise, unless you have a > very slow client. single stop/start cycles != shoe shining > With my own experience writing data to tape using mbuffer and tar, a > buffer of 4GB was enough to prevent all shoe shining, and it did not > slow down the process at all. However with Bacula, my spool file must > be 800GB to achieve the same result, and even this makes the process > take much longer because the tape is idle while the spool file is > filling up the first time. Normally you have several jobs running in parallel, and while some jobs are writing their data to the spool area, other are writing the spooled data to tape. Don't consider the special case of running a single client. > I don't have 800GB available for the spool file either, which means my > choices are: OK, if this is your problem, then ask yourself how much it would cost to add sufficient storage. Come on, be real! Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de I believe you find life such a problem because you think there are the good people and the bad people. You're wrong, of course. There are, always and only, the bad people, but some of them are on oppo- site sides. - Terry Pratchett, _Guards! Guards!_ _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
