Hi Jared,
Can you post here your storage configurations?
Best regards,
Ana
On Wed, Nov 19, 2014 at 3:56 PM, Kelley, Jared <jkel...@popcap.com> wrote:
> For the sake of sharing information I am going to reply to this thread
> since I am working on the same type of setup and experiencing similar
> issues.
>
>
> I removed the setting ‘prefer mounted volumes = no' and separated my
> jobs into 3 different pools. All my jobs are running smoothly now, no
> failures. The prefer mounted volumes was the issue.
> Now I am experiencing interleaving on a single volume/tape drive with
> multiple jobs scheduled at the same time, in the same pool, as opposed to
> bacula using different volumes and different drives for jobs scheduled at
> the same time for the same pool.
>
> Here is my issue:
>
> I have 10 jobs that kick off at the same time every day. Each job
> takes anywhere from 10 minutes to 2 hours. The pool behind these jobs has
> 4 tape drives and 4 volumes(virtual tapes) assigned to it. The 10 jobs all
> kick off at the same time and they all write to the same volume,
> apparently(assuming) interleaving the data. See my paste below, Ive
> bolded start times, end times, pools and tapes. One can see these jobs all
> completed near each other and ran for over an hour. All to the same
> tape/volume, tape3 and volume *AAAACF5F6A* . I assume if I had the
> "prefer mounted volume=no" for these jobs they would use all 4 drives in
> the pool and write to separate volumes as opposed to interleaving to a
> single drive and volume. But based on the manual, reported problems and
> experienced problems with ‘prefer mounted volumes = no’ I hesitate to use
> that.
>
> My question:
> Is there another way to get multiple jobs scheduled at the same time to
> run concurrently using different drives and volumes, as assigned to the
> pool, versus all writing at the same time and interleaving the data to a
> single drive and volume even though the pool has 4 drives/volumes assigned
> to it? Or is prefer mounted volumes = no the only way to solve this?
>
> any help is greatly appreciated
>
> 19-Nov 10:06 backup02-sd JobId 925: Job write elapsed time = 01:01:23,
> Transfer rate = 1.489 M Bytes/second
>
> 19-Nov 10:06 backup02-dir JobId 925: Bacula backup02-dir 5.2.6 (21Feb12):
>
> Build OS: x86_64-pc-linux-gnu ubuntu 14.04
>
> JobId: 925
>
> Job: labsdb1.2014-11-19_09.05.00_21
>
> Backup Level: Full (upgraded from Differential)
>
> Client: "labsdb1" 2.4.4 (28Dec08)
> x86_64-pc-linux-gnu,debian,lenny/sid
>
> FileSet: "Database Backup" 2014-11-07 22:02:50
>
> P*ool: "database" (From Job resource)*
>
> Catalog: "MyCatalog" (From Client resource)
>
> *Storage: "Tape3" (From Pool resource)*
>
> *Scheduled time: 19-Nov-2014 09:05:00*
>
> * Start time: 19-Nov-2014 09:05:02*
>
> *End time: 19-Nov-2014 10:06:26*
>
> Elapsed time: 1 hour 1 min 24 secs
>
> Priority: 10
>
> FD Files Written: 2
>
> SD Files Written: 2
>
> FD Bytes Written: 5,486,802,781 (5.486 GB)
>
> SD Bytes Written: 5,486,802,988 (5.486 GB)
>
> Rate: 1489.4 KB/s
>
> Software Compression: None
>
> VSS: no
>
> Encryption: no
>
> Accurate: no
>
> *Volume name(s): AAAACF5F6A*
>
> Volume Session Id: 155
>
> Volume Session Time: 1415816791
>
> Last Volume Bytes: 1,348,311,508,992 (1.348 TB)
>
> Non-fatal FD errors: 0
>
> SD Errors: 0
>
> FD termination status: OK
>
> SD termination status: OK
>
> Termination: Backup OK
>
>
> 19-Nov 10:06 backup02-dir JobId 925: Begin pruning Jobs older than 1
> year .
>
> 19-Nov 10:06 backup02-dir JobId 925: No Jobs found to prune.
>
> 19-Nov 10:06 backup02-dir JobId 925: Begin pruning Files.
>
> 19-Nov 10:06 backup02-dir JobId 925: No Files found to prune.
>
> 19-Nov 10:06 backup02-dir JobId 925: End auto prune.
>
>
> 19-Nov 10:41 backup02-sd JobId 919: Job write elapsed time = 01:36:26,
> Transfer rate = 2.570 M Bytes/second
>
> 19-Nov 10:41 backup02-dir JobId 919: Bacula backup02-dir 5.2.6 (21Feb12):
>
> Build OS: x86_64-pc-linux-gnu ubuntu 14.04
>
> JobId: 919
>
> Job: shopd1.2014-11-19_09.05.00_15
>
> Backup Level: Full (upgraded from Differential)
>
> Client: "shopdb1" 5.0.2 (28Apr10)
> x86_64-pc-linux-gnu,debian,squeeze/sid
>
> FileSet: "varlibDatabase Backup" 2014-11-06 20:20:52
>
> *Pool: "database" (From Job resource)*
>
> Catalog: "MyCatalog" (From Client resource)
>
> *Storage: "Tape3" (From Pool resource)*
>
> *Scheduled time: 19-Nov-2014 09:05:00*
>
> * Start time: 19-Nov-2014 09:05:00*
>
> *End time: 19-Nov-2014 10:41:26*
>
> Elapsed time: 1 hour 36 mins 26 secs
>
> Priority: 10
>
> FD Files Written: 3
>
> SD Files Written: 3
>
> FD Bytes Written: 14,875,191,024 (14.87 GB)
>
> SD Bytes Written: 14,875,192,011 (14.87 GB)
>
> Rate: 2570.9 KB/s
>
> Software Compression: None
>
> VSS: no
>
> Encryption: yes
>
> Accurate: no
>
> *Volume name(s): AAAACF5F6A*
>
> Volume Session Id: 149
>
> Volume Session Time: 1415816791
>
> Last Volume Bytes: 1,375,507,574,784 (1.375 TB)
>
> Non-fatal FD errors: 0
>
> SD Errors: 0
>
> FD termination status: OK
>
> SD termination status: OK
>
> Termination: Backup OK
>
>
> 19-Nov 10:41 backup02-dir JobId 919: Begin pruning Jobs older than 7
> years .
>
> 19-Nov 10:41 backup02-dir JobId 919: No Jobs found to prune.
>
> 19-Nov 10:41 backup02-dir JobId 919: Begin pruning Files.
>
> 19-Nov 10:41 backup02-dir JobId 919: No Files found to prune.
>
> 19-Nov 10:41 backup02-dir JobId 919: End auto prune.
>
>
>
> 19-Nov 11:10 backup02-sd JobId 917: Job write elapsed time = 02:05:32,
> Transfer rate = 3.894 M Bytes/second
>
> 19-Nov 11:10 backup02-dir JobId 917: Bacula backup02-dir 5.2.6 (21Feb12):
>
> Build OS: x86_64-pc-linux-gnu ubuntu 14.04
>
> JobId: 917
>
> Job: pushdb.2014-11-19_09.05.00_13
>
> Backup Level: Full (upgraded from Differential)
>
> Client: "pushdb1" 5.0.2 (28Apr10)
> x86_64-pc-linux-gnu,debian,6.0.6
>
> FileSet: "varlibDatabase Backup" 2014-11-06 20:20:52
>
> *Pool: "database" (From Job resource)*
>
> Catalog: "MyCatalog" (From Client resource)
>
> *Storage: "Tape3" (From Pool resource)*
>
> *Scheduled time: 19-Nov-2014 09:05:00*
>
> * Start time: 19-Nov-2014 09:05:00*
>
> *End time: 19-Nov-2014 11:10:33*
>
> Elapsed time: 2 hours 5 mins 33 secs
>
> Priority: 10
>
> FD Files Written: 2
>
> SD Files Written: 2
>
> FD Bytes Written: 29,334,385,684 (29.33 GB)
>
> SD Bytes Written: 29,334,385,894 (29.33 GB)
>
> Rate: 3894.1 KB/s
>
> Software Compression: None
>
> VSS: no
>
> Encryption: no
>
> Accurate: no
>
> *Volume name(s): AAAACF5F6A*
>
> Volume Session Id: 147
>
> Volume Session Time: 1415816791
>
> Last Volume Bytes: 1,397,835,371,520 (1.397 TB)
>
> Non-fatal FD errors: 1
>
> SD Errors: 0
>
> FD termination status: OK
>
> SD termination status: OK
>
> Termination: Backup OK
>
>
> 19-Nov 11:10 backup02-dir JobId 917: Begin pruning Jobs older than 1
> year .
>
> 19-Nov 11:10 backup02-dir JobId 917: No Jobs found to prune.
>
> 19-Nov 11:10 backup02-dir JobId 917: Begin pruning Files.
>
> 19-Nov 11:10 backup02-dir JobId 917: No Files found to prune.
>
> 19-Nov 11:10 backup02-dir JobId 917: End auto prune.
>
>
> 19-Nov 11:26 backup02-sd JobId 915: Job write elapsed time = 02:21:25,
> Transfer rate = 3.655 M Bytes/second
>
> 19-Nov 11:26 backup02-dir JobId 915: Bacula backup02-dir 5.2.6 (21Feb12):
>
> Build OS: x86_64-pc-linux-gnu ubuntu 14.04
>
> JobId: 915
>
> Job: ecommdb1.2014-11-19_09.05.00_11
>
> Backup Level: Differential, since=2014-11-08 09:05:03
>
> Client: "ecommdb" 2.4.4 (28Dec08)
> x86_64-pc-linux-gnu,debian,lenny/sid
>
> FileSet: "Database Backup" 2014-11-07 22:02:50
>
> *Pool: "database" (From Job resource)*
>
> Catalog: "MyCatalog" (From Client resource)
>
> *Storage: "Tape3" (From Pool resource)*
>
> *Scheduled time: 19-Nov-2014 09:05:00*
>
> * Start time: 19-Nov-2014 09:05:00*
>
> *End time: 19-Nov-2014 11:26:27*
>
> Elapsed time: 2 hours 21 mins 27 secs
>
> Priority: 10
>
> FD Files Written: 1
>
> SD Files Written: 1
>
> FD Bytes Written: 31,016,183,168 (31.01 GB)
>
> SD Bytes Written: 31,016,183,295 (31.01 GB)
>
> Rate: 3654.6 KB/s
>
> Software Compression: None
>
> VSS: no
>
> Encryption: no
>
> Accurate: no
>
> *Volume name(s): AAAACF5F6A*
>
> Volume Session Id: 145
>
> Volume Session Time: 1415816791
>
> Last Volume Bytes: 1,407,107,681,280 (1.407 TB)
>
> Non-fatal FD errors: 0
>
> SD Errors: 0
>
> FD termination status: OK
>
> SD termination status: OK
>
> Termination: Backup OK
>
>
> From: "hei...@bacula.com.br" <hei...@bacula.com.br>
> Date: Wednesday, November 19, 2014 at 9:03 AM
> To: Josh Fisher <jfis...@pvct.com>
> Cc: "bacula-users@lists.sourceforge.net" <
> bacula-users@lists.sourceforge.net>
> Subject: Re: [Bacula-users] bacula can't make use of multi drives
>
> I think that the only problem with setting "Prefer Mounted Volumes = no" is
> that it will result in a lot of moving of a volume back and forth between
> drives. While it might not be a bug, since it works as advertised, it CAN
> be a problem. For tape, it will really slow things down because it takes a
> substantial time to move a tape out of one drive and into another. For disk
> virtual autochangers, though, the moving between "drives" is simply closing
> a disk file and then re-opening it, meaning that the delay is very
> negligible.
>
> Mr. Fisher: It's not just that. Eventually "Prefer Mounted Volumes =
> no" messes up the autochanger during a random backup and you have to
> restart storage daemon and sometimes powercycle the tape library.
>
> If there is some other issue, like a race condition that I am not aware
> of, then that would indeed be a bug in my book. But I have used
> PreferMountedVolumes=no for disk virtual autochanger without (so far)
> encountering any such problems, just the swapping between drives. And for
> disk virtual autochangers, I don't see that as a problem in and of itself.
>
> I don't know if it is already fixed or if this error happens for all
> autochangers. But last Bacula System Community Storage whitepapers goes in
> the same way. Having 2 pools is still safer and more reliable way to do
> multiple volumes writing.
>
> Regards,
> =========================================================================
> Heitor Medrado de Faria - LPIC-III | ITIL-F
> Faltam poucos dias - Treinamento Telepresencial Bacula:
> http://www.bacula.com.br/?p=2174
> 61 2021-8260 <%2B55%2061%202021-8260> | 8268-4220
> <%2B55%2061%208268-4220>
> Site: www.bacula.com.br | Facebook: heitor.faria
> <http://www.facebook.com/heitor.faria> | Gtalk: heitorfa...@gmail.com
> ==========================================================================
>
> ------------------------------
> *De: *"Josh Fisher" <jfis...@pvct.com>
> *Para: *bacula-users@lists.sourceforge.net
> *Enviadas: *Quarta-feira, 19 de novembro de 2014 14:52:42
> *Assunto: *Re: [Bacula-users] bacula can't make use of multi drives
>
> On 11/19/2014 11:10 AM, hei...@bacula.com.br wrote:
>
> Is it a bug that bacula cannot write to multiple drives for the jobs in
> the same pool? Will there be a fix. Back in March 2014 Wolfgang Denk wrote
> "it can" but how?
>
> http://adsm.org/lists/html/Bacula-users/2014-03/msg00212.html
>
> It's not a bug, but just the way it's designed.
> "Prefer Mounted Volumes=no" it's a directive that could allow to write
> multiple volumes within the same pool at the same time, however this is a
> little buggy and not recommended at this time.
>
>
> It's not a bug, but it is a little buggy??
>
> I think that the only problem with setting "Prefer Mounted Volumes = no"
> is that it will result in a lot of moving of a volume back and forth
> between drives. While it might not be a bug, since it works as advertised,
> it CAN be a problem. For tape, it will really slow things down because it
> takes a substantial time to move a tape out of one drive and into another.
> For disk virtual autochangers, though, the moving between "drives" is
> simply closing a disk file and then re-opening it, meaning that the delay
> is very negligible.
>
> If there is some other issue, like a race condition that I am not aware
> of, then that would indeed be a bug in my book. But I have used
> PreferMountedVolumes=no for disk virtual autochanger without (so far)
> encountering any such problems, just the swapping between drives. And for
> disk virtual autochangers, I don't see that as a problem in and of itself.
>
>
> Regards,
> =========================================================================
> Heitor Medrado de Faria - LPIC-III | ITIL-F
> Faltam poucos dias - Treinamento Telepresencial Bacula:
> http://www.bacula.com.br/?p=2174
> 61 2021-8260 <%2B55%2061%202021-8260> | 8268-4220
> <%2B55%2061%208268-4220>
> Site: www.bacula.com.br | Facebook: heitor.faria
> <http://www.facebook.com/heitor.faria> | Gtalk: heitorfa...@gmail.com
> ==========================================================================
>
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
