Re: [Bacula-users] Cleaning up database records
In response to "Frank Altpeter" <[EMAIL PROTECTED]>: > > My current bacula system (FreeBSD, bacula-2.0.1, mysql-4.1) has > currently some massive performance problems. One of the reasons i > think is caused by the massive amount of old and obsolete records. > For example, i had a client to backup once, which has dissappeared > some time ago. When this machine has been removed, the client has been > deleted from the bacula configuration. But this way the database > records for these clients remain in the database, thus making the db > records more and more unusable. > So, i would like to prune such records to reduce unneeded data. > > Is there any way on archiving this? dbcheck should clean this stuff up. > And, for the future, what's the best practice to avoid this? Purge the volumes prior to removing the clients. On the flip side, running dbcheck periodically is pretty much a requirement for keeping Bacula's database reasonably sized. I have it run once a month in read-only mode via cron and email us the results. When the extra stuff gets significant, I use it to go in and clean up. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Cleaning up database records
In response to Martin Simmons <[EMAIL PROTECTED]>: > >>>>> On Mon, 19 Feb 2007 08:55:07 -0500, Bill Moran said: > > > > On the flip side, running dbcheck periodically is pretty much a requirement > > for keeping Bacula's database reasonably sized. I have it run once a month > > in read-only mode via cron and email us the results. When the extra stuff > > gets significant, I use it to go in and clean up. > > I think that is overly pessimistic -- unless you are backing up lots of data > where the file names or directory names keep changing in the Filename and Path > tables. Backup up a server with a bunch of Maildirs, and you'll see this fast enough. Even without that, with 6 servers and 20 workstations being backed up, lots of files get created, renamed, deleted, on a constant basis. Every few months, the amount of unreferenced stuff is big enough to warrant cleanup. Personally, I don't think a monthly review of the state of the backup system is overly pessimistic in any way :) bacula=> select count(*) from file; count -- 19667790 (1 row) bacula=> select count(*) from filename; count 927786 (1 row) bacula=> select count(*) from path; count 655714 (1 row) > I find that those tables are dwarfed by the File table, but that is cleaned by > Bacula's own pruning to shouldn't be a problem (though there was a bug in this > in some obsolete version). I agree that the file table gets dwarfs the other two, but it doesn't change the fact that our filename and path tables get full of unreferenced stuff over time. Cleaning it out occasionally keeps database size down. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Cleaning up database records
In response to Martin Simmons <[EMAIL PROTECTED]>: > >>>>> On Mon, 19 Feb 2007 15:10:26 -0500, Bill Moran said: > > > > In response to Martin Simmons <[EMAIL PROTECTED]>: > > > > > >>>>> On Mon, 19 Feb 2007 08:55:07 -0500, Bill Moran said: > > > > > > > > On the flip side, running dbcheck periodically is pretty much a > > > > requirement > > > > for keeping Bacula's database reasonably sized. I have it run once a > > > > month > > > > in read-only mode via cron and email us the results. When the extra > > > > stuff > > > > gets significant, I use it to go in and clean up. > > > > > > I think that is overly pessimistic -- unless you are backing up lots of > > > data > > > where the file names or directory names keep changing in the Filename and > > > Path > > > tables. > > > > Backup up a server with a bunch of Maildirs, and you'll see this fast > > enough. > > Ah, good point. > > > > Even without that, with 6 servers and 20 workstations being backed up, lots > > of > > files get created, renamed, deleted, on a constant basis. Every few months, > > the amount of unreferenced stuff is big enough to warrant cleanup. > > > > Personally, I don't think a monthly review of the state of the backup system > > is overly pessimistic in any way :) > > > > bacula=> select count(*) from file; > > count > > -- > > 19667790 > > (1 row) > > > > bacula=> select count(*) from filename; > > count > > > > 927786 > > (1 row) > > > > bacula=> select count(*) from path; > > count > > > > 655714 > > (1 row) > > That's an interestingly low ratio of filename/path :-) Never really thought about it, but it does seem a little odd. It's probably an indicator of the fact that we're a devel shop. Most of our users have copies of the software project in their home directory, so lots of new paths, but copies of the same files (filenames) -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-dir won't start from a script
In response to Beren <[EMAIL PROTECTED]>: > Hi Guys, > > The bacula startup script won't start the director, however typing > "bacula-dir -c /etc/bacula/bacula-dir.conf" works fine. > > I tried putting this line into a script and running it: > > /usr/sbin/bacula-dir -c /etc/bacula/bacula-dir.conf > > It also didn't work...however, if I type it in the command line letter for > letter, it works fine. Try running the script as "sh -x script_name" and see if the debugging output helps any. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Sony LTO-2/LTO-3
In response to Fausto Barros de Sá Teles <[EMAIL PROTECTED]>: > Has anyone there ever used Bacula for backin up to Sony LTO-2/LTO-3 tapes? > I have to implement Bacula but I'm not sure if it works with this kind of > tape. We use LTO2/LTO3 tapes at work with no problems. They're Dell branded -- don't know who the underlying hardware is made by. -- Bill Moran http://www.potentialtech.com - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] 400G compressed tape only using 215G
In response to Jason King <[EMAIL PROTECTED]>: > I have an LTO2 drive with tapes that hold 200g native and 400g with > hardware compression. I have the hardware compression turned on on my > tape drive. I know because I went and manually made sure using the tape > tools that come with FreeBSD (The os the storage daemon is running on). > Last night my tape filled up with only 215G. I know it filled up because > I got a message to change tapes from the daemon. I didn't do anything > specific in Bacula to tell it to use hardware compression on the tape > drive because I didn't think I had to. I made sure hardware compression > was enabled in FreeBSD. My question is, is there a specific way to tell > bacula to take advantage of hardware compression? Perhaps a setting in > the storage daemon config file or the bacula director config file. What are you backing up? That "400G with compression" is just an average. There's no guarantee that you'll get 2x compression. Personally, I find it false advertising, but nobody's listening to me. A few years ago (pre-Bacula for me) I set up a backup system for a company that did a lot of graphics work (lots of jpegs). They were really upset that their 400G tapes only held 200G. Reason is that jpegs don't compress, because they're _already_ compressed. If you're backing up a lot of pre-compressed stuff (like JPEGs, or PDFs or zipfiles, or MPEGs, or MP3s) you're _not_ going to see the 2x compression those tape manufactures advertise. The fact that you managed to get 215G on a 200G tape tells me that compression is working, but you're just not getting the 2x you were hoping for. -- Bill Moran http://www.potentialtech.com - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] backup tapes at home
In response to Stéphane Lardier <[EMAIL PROTECTED]>: > I have an autoloader with 8 tapes. I am afraid of fire and I want have a > full backup of my computers at home. So, I can save on 6 tapes and have > a different job to save on my two dedicated tapes. But, may be it will > be not possible to restore the two tapes after fire event. Is it the > best way to prevent against a fire damage ? Write a _full_ backup to a tape that is stored offsite. This tape can be used to do a full restore and is far enough away to be safe from fire. You can use the other tapes in the changer for incrementals. How much data you're willing to lose determines how often you do full compared to incremental backups. -- Bill Moran http://www.potentialtech.com - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] backup tapes at home
In response to Ryan Novosielski <[EMAIL PROTECTED]>: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Stéphane Lardier wrote: > > I have an autoloader with 8 tapes. I am afraid of fire and I want have a > > full backup of my computers at home. So, I can save on 6 tapes and have > > a different job to save on my two dedicated tapes. But, may be it will > > be not possible to restore the two tapes after fire event. Is it the > > best way to prevent against a fire damage ? > > A fire safe might fit the bill here. I've always wondered about this. If I put _plastic_ tapes in a fire safe, how long does the safe actually insulate the tapes from the heat that will damage them. Keep in mind that they don't have to burn to be ruined. They don't even have to melt -- if they get hot enough, they'll be useless. Does anyone have any direct experience with how well a fire safe protects something like backup tapes? If it's well insulated, it will protect them for a while, but long enough for the fire to be extinguished? -- Bill Moran http://www.potentialtech.com - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] backup tapes at home
Alan Brown <[EMAIL PROTECTED]> wrote: > > On Fri, 2 Mar 2007, Bill Moran wrote: > > >> A fire safe might fit the bill here. > > > > I've always wondered about this. > > > > If I put _plastic_ tapes in a fire safe, how long does the safe actually > > insulate the tapes from the heat that will damage them. > > Not long. > > A standards-compliant "data safe" on the other hand will provide > protection agains at least 2 hours at 1500C and 24 hour coooling down > period. Most also provide protection against large drops (floor failure in > multistory buildings) and longer burn periods. > > Several data safes are simply firesafes fitted with inserts aimed at > keeping internal temperatures below 60C. > > > Does anyone have any direct experience with how well a fire safe protects > > something like backup tapes? If it's well insulated, it will protect them > > for a while, but long enough for the fire to be extinguished? > > Only once, and only a small safe on a customer site, but yes. > > Our current main fire/data safe is a Phoenx Data commander 4623, which is > capable of taking 720 LTO tapes in current configuration (39 per drawer, > cased, increasing to 45 uncased) > > See http://www.phoenixsafeusa.com/ > or http://www.phoenixsafeusa.com/us/viewproduct/4620_data_commander.html > > This cost a shade under US $10,000 with tax and delivery included. Interesting, but I suspect that's a little out of the price range for someone wanting to protect their data at home. I know I wouldn't even know where to put such a thing at my house. It'd be cheaper to just rent a safe-deposit box. -- Bill Moran http://www.potentialtech.com - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] director password...Urgent
In response to "Naufal Zamir" <[EMAIL PROTECTED]>: > yep true, but its encrypted and i don't know the password because it was set > by some one else.. So how do I recover the password now. It's not encrypted. > > Regards > > On 3/19/07, John Drescher <[EMAIL PROTECTED]> wrote: > > > > On 3/19/07, Naufal Zamir <[EMAIL PROTECTED]> wrote: > > > Hi, > > > > > > Please can anyone tell me how to reset or get the director password. I > > need > > > it for adding new clients and I am not sure what the password is. > > > > > This password is set in the bacula-dir.conf file on the director. > > > > John > > > -- Bill Moran http://www.potentialtech.com - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bogus/confusing warning? : Filesystem change prohibited. Will not descend into ...
In response to Marc Schiffbauer <[EMAIL PROTECTED]>:
> Hi all,
>
> I have a backup which finished with the following warning:
>
> ---
> bart-fd: Filesystem change prohibited. Will not descend into /home/mschiff
> ---
>
> "Filesystem change prohibited" is correct as I use default values
> here. But "Will not descend into /home/mschiff" is wrong because
> both "/home" and "/home/mschiff" are in the fileset which are two
> different filesystems. And a "list files jobid=xy" really shows me
> files on /home/mschiff.
>
> This is the FileSet:
>
> -
> FileSet {
> Name = "bart ImportantData FileSet"
> Include {
> Options {
> signature = MD5
> noatime = yes
> aclsupport = yes
> }
> Options {
> exclude = yes
> regex = ".*/[Cc]ache/.*"
> }
> File = "\\ }
>
> Exclude {
> # exclude all dirs containing a file called ".BACULA_NO_BACKUP"
> File = "\\|sh -c 'for D in /home; do find $D -xdev -name
> .BACULA_NO_BACKUP -type f -printf \"%h\\n\"; done | tee
> /root/bacula_excluded_dirs.log'"
> }
> }
> -
>
> The clients "/etc/bacula/backup-include.txt":
> --
> /etc
> /opt
> /root
> /usr/local
> /home
> /home/mschiff
> --
> (with /, /home and /home/mschiff being different filesystems)
>
>
> Any hints how I can get rid of the warning?
This is a known shortcoming at this time. The code that prevents filesystem
change does not check to see if the data will be caught via another rule.
There's no way to prevent the warning via the config. This would be a nice
feature to have added.
--
Bill Moran
http://www.potentialtech.com
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Connect to SD over OpenVPN
In response to Martijn de Munnik <[EMAIL PROTECTED]>: > Hi, > > I need to backup several laptops which can be in the office but also at home > or somewhere else. I trying to make this working using OpenVPN so the laptops > are connected to the backup server using a VPN network. > > > [backup server 172.16.0.10]< LAN 172.16.0.0/24 >[router with > portforwarding 1194]< INTERNET >[router]< LAN >[laptop] > > OpenVPN is successfully configured and continuously tries to make a > connection with the backup server. The laptop get an IP address based on > their certificate name so all VPN IPs are known. The Director on the backup > server can connect to the FD on the laptop and start a job but the laptop > tries to connect to the SD on IP 172.16.0.10. That's where the problem is > because the VPN IP of the backup server is 10.67.0.1 so it should connect to > the SD on 10.67.0.1. > How can I tell the client to connect to the SD on the right IP? Define the storage daemon that way. I'm confused, however. Is the SD on the same system as the dir? If so, why does it have 2 IPs? -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Connect to SD over OpenVPN
In response to Martijn de Munnik <[EMAIL PROTECTED]>: > > On Fri, 20 Apr 2007 08:20:36 -0400, Bill Moran <[EMAIL PROTECTED]> wrote: > > In response to Martijn de Munnik <[EMAIL PROTECTED]>: > > > >> Hi, > >> > >> I need to backup several laptops which can be in the office but also at > > home or somewhere else. I trying to make this working using OpenVPN so the > > laptops are connected to the backup server using a VPN network. > >> > >> > >> [backup server 172.16.0.10]< LAN 172.16.0.0/24 >[router with > > portforwarding 1194]< INTERNET >[router]< LAN > > >[laptop] > >> > >> OpenVPN is successfully configured and continuously tries to make a > > connection with the backup server. The laptop get an IP address based on > > their certificate name so all VPN IPs are known. The Director on the backup > > server can connect to the FD on the laptop and start a job but the laptop > > tries to connect to the SD on IP 172.16.0.10. That's where the problem is > > because the VPN IP of the backup server is 10.67.0.1 so it should connect > > to the SD on 10.67.0.1. > >> How can I tell the client to connect to the SD on the right IP? > > > > Define the storage daemon that way. > > > > I'm confused, however. Is the SD on the same system as the dir? If so, > > why > > does it have 2 IPs? > > > > Hi the SD is on the same machine as the DIR (172.16.0.10) but when a vpn > connection is made the SD and DIR are known as 10.67.0.1. One solution could > be to do all backups over a VPN connection (so also for the machines in > office). How about configuring routing on that system so attempts to contact 172.16.0.10 get routed to the correct interface? -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] High database load when migrating many small files
In response to Damian Lubosch <[EMAIL PROTECTED]>: > Hello! > > I am using Bacula 2.0.3 with Mysql5 on Debian Etch with a 2.8GHz > Celeron, 1.5GB RAM, LTO2 drive, 2x 500G (Backup) + 1x80 Disks (System/DB). > > I have a machine to backup with about 1-2 million small files (~1kb). > When I run a migration job for about 4 GB of such data the performance > is going down. The tape rewinds very often and the overall performance > is about 3MB/sec. I found out (with top) that the mysql is taking all > the processing power (together with bacula-sd) when migrating many files. > On the other hand, when migrating only few large files -backups is > running fine with 20-30MB/sec. > > How can I improve the performance? Are there any tricks I oversaw? This sounds like a database optimization problem to me. When Bacula is dealing with a lot of small files, performance generally bottlenecks at the datbase's ability to write new records, and everything else is held up waiting for the database (thus your tape drive runs out of data to write and has to shoe-shine). I'm not a MySQL expert, so I can't give any DB-specific advice, but you should look at what can be done to improve MySQL's performance. Is MySQL actually blocking waiting for free CPU, or is it waiting on disk IO, for example. I'm making a lot of guesses here, but what kind of HDD subsystem do you have? You may have to invest in faster (i.e. SCSI or SAS) drives to get enough IO throughput. But isolate the problem first. If MySQL is slow, it's slow doing _something_. Find out what that something is and you'll have a good hint as to what you need to do. -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] What does this error message mean?
In response to hal <[EMAIL PROTECTED]>: > 25-Apr 08:41 abe-sd: Job RestoreFiles.2007-04-24-08.41.05 waiting to > reserve a device. > > What does it mean? It means that no device suitable for that restore job is available. > What device? The previous messages should supply context. > What machine, server or client? Again -- previous log messages. > Why? I believe this is caused by another job using the device, and this job has to wait on it. Connect to the director and do "status dir" and "status storage". Between those two output, you should be able to ascertain what's going on. -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula-announce] Bacula version 2.2.x for Solaris, FreeBSD, and Windows
> On Sun, Apr 15, 2007 at 11:24:58AM +0200, Kern Sibbald wrote: > > > This email is a *BIG* urgent warning that unless the Bacula Solaris and > > FreeBSD users pull together and organize regression testing, you may find > > that version 2.2.x will not work on your platform, and it may be difficult > > to > > correct problems after production release in a timely fashion since once > > released, it is not possible to make any significant changes to mutex usage > > without destabilizing the code. What's your timeline, Kern. I'll do my damndest to get through all the FreeBSD regression tests with enough time for bug fixes to go through. -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula-devel] Goodbye and thanks for all the fish
PROTECTED] > > https://lists.sourceforge.net/lists/listinfo/bacula-devel > > > > > Jon Pounder > >_/_/_/ _/_/ _/ _/_/_/ _/_/ _/_/_/_/ > _/_/_/ _/ _/ _/_/_/ _/ _/_/ >_/_/ _/_/ _/ _/_/ _/_/ _/ > _/_/_/ _/_/ _/_/_/_/ _/_/_/ _/_/ _/_/_/_/ > > > Inline Internet Systems Inc. > Thorold, Ontario, Canada > > Tools to Power Your e-Business Solutions > www.inline.net > www.ihtml.com > www.ihtmlmerchant.com > www.opayc.com > > - > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Network Performance
In response to Jordan Desroches <[EMAIL PROTECTED]>: > Compression is not enabled in the FileSet. Can you isolate where the bottleneck is? IO? CPU? Network? There are frequently discussions about throughput being sub-optimal on the lists. In my experience, these fall into a few categories: *) The SQL server is the bottleneck. This kinda falls outside the discussion of Bacula, as it's usually specific to the particular SQL server you use. This problem shows up most often with filesets that contain lots of little files. However, there _has_ been work done to improve how Bacula writes to the database, so which version you're using is important. If you can make any suggestions on how to improve this further, or even provide test cases to show where it's slow, I expect it will be helpful to the developers. *) IO. Often, folks are saturating the IO of their hardware and don't realize it. This can show up when the database is on the same system as file volumes, as that system has to share the IO of database writes as well as file volume writes. *) Mysterious network problems. This is the one that it would be nice to get some real information on. Some folks have claimed that Bacula is unable to send data anywhere near the speed of the network, even when there is no other bottleneck. Unfortunately, this problem has been very difficult to diagnose, as it requires a high level of expertise in networking to track down the cause, and everyone who has report it has been unable and/or unwilling to isolate it well enough for anyone to really do anything to improve it. So, I hope that information is helpful in narrowing down your problem. > Michel Meyers wrote: > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 > > > > Jordan Desroches wrote: > >> Greetings! > >> > >> First, I apologize if this comes through multiple times. I'm having > >> trouble connecting to the list. > >> > >> I've been trying to bake off AMANDA and Bacula in our environment, and > >> have run up against a Bacula performance snag. Amanda is regularly able > >> to average ~50 MB/s over our network, while I'm getting ~30 MB/s out of > >> Bacula (spooling turned on over gig-e). I like the feature set and > >> usability of > >> Bacula much better than that which AMANDA provides, but the speed > >> difference is an issue. I think the difference may have to do with > >> AMANDA running multiple simultaneous dumpers on the client. I've bumped > >> "Maximum Network Buffer Size" to 65536 bytes in both the storage daemon > >> and file daemon configurations with little to no change from the 32K > >> buffer. A typical Bacula client status reads: > > [...] > >> Any ideas what I can do to eek out some more speed? > > > > Just a guess/question: Do you have compression enabled in your job? If > > the client's doing compression, that might throttle its throughput. > > > > Greetings, > > Michel > > -BEGIN PGP SIGNATURE- > > Version: GnuPG v1.4.7 (MingW32) > > > > iD8DBQFGSa8b2Vs+MkscAyURAuC+AKDnMf1RGfkBeq6qYmPZzEneCLVZxwCeNJYk > > YpXNPfBS5fQRAMS/rNEvgcE= > > =Adyp > > -END PGP SIGNATURE- > > > > - > > This SF.net email is sponsored by DB2 Express > > Download DB2 Express C - the FREE version of DB2 express and take > > control of your XML. No limits. Just data. Click to get it now. > > http://sourceforge.net/powerbar/db2/ > > ___ > > Bacula-users mailing list > > Bacula-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/bacula-users > > > > - > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Network Performance
On Mon, 21 May 2007 12:49:12 -0400 Jordan Desroches <[EMAIL PROTECTED]> wrote: > I've done a bunch of testing since my last post, and the slowdown > appears to be CPU usage on the client file daemon (bacula-fd). bacula-fd > consumes all the cpu resource it can get, and tends to transfers data > faster (40 MB/s) on to the director on a machine with better CPU, even > though the disk array on the slower machine (28 MB/s) was faster > (similar data sets). I'm using MD5 hashes and compression is not turned > on (also not explicitly turned off, if there is such a command). Is it > normal for the file daemon to consume 100% of a cpu, or am I doing > something wrong? Any time I've seen high CPU usage on the FD, it's been the result of compression being turned on and has been solved by either lowering the aggressiveness of compression or turning it off altogether. So my first advice would be to verify whether compression is on or not. Check your job defaults. Is this a POSIX system? Run top(1) and see where the CPU time is going: user? system? io? -- Bill Moran Potential Technologies http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] computer drags during backup, win32
"Dave" <[EMAIL PROTECTED]> wrote: > > Hello, > I'm running bacula 2.0.3 on windows xpsp2. When doing any kind of > backup, i have the entire system set to back up and if i'm using the box it > slows down to a crawl. I'm using vss if that matters. On my various Unix > boxes this does not happen. I was wondering if there was an equivalent to > unix nice that i could set for win32? If not, is there a way of finding out > where this slowdown is coming from, a resource limit, network performance, > cpu, and how best to deal with it? > Thanks. I've seen this problem as well. As best I can tell, Windows job scheduler bites. In my case, I was able to reduce the effect to an acceptable level by using gzip1 for the compression level instead of the default gzip6. Luckily, the extra space required was not a problem. I don't know if there's a feature request out for this, but maybe I should create one. It would be nice if the FD could be programmed with a delay loop during the compression process, configurable by the user. It's difficult to say whether this would actually meet the goal or not, though. -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] volume conflict between jobs
In response to Arno Lehmann <[EMAIL PROTECTED]>:
[snip]
> >>>
> >>> This is how my Pool is defined:
> >>>
> >>> Pool {
> >>>Name = dss07-pool
> >>>Pool Type = Backup
> >>>LabelFormat = dss07-
> >>>AutoPrune = yes
> >>>Volume Retention = 14 days
> >>>Maximum Volume Bytes = 21474836480
> >>>Recycle = yes
> >> You probably want to limit the number of volumes in that pool, because
> >> otherwise volumes will not be recycled but Bacula will forever try to
> >> create a new one.
> >>
> >
> > I thought that it would start recycling after the volume retention
> > period was over.
>
> No, volumes are recycled onle when there is no other legal way to find
> new space.
This is not the behaviour I observe. When a volume has been marked Purged,
it is then recycled the next time a volume from that pool is needed.
When a volume's retention time is exceeded, it is purged.
--
Bill Moran
http://www.potentialtech.com
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula memory usage
In response to Joseph Wright <[EMAIL PROTECTED]>: > It seems that the resources used by the database should be taken into > account when looking at Bacula's memory usage, because even though > the two are separate, the director is the one putting the records > there. Maybe I should re-ask the question as: How heavily does > Bacula tax the database with 50 or more clients being backed up > concurrently, and what kind of hardware is recommended for such a > situation? If you've got 50 jobs running simultaneously, you're liable to need a massive DB server. Unfortunately, however, that's not enough information to be sure. The real load on the DB server is (mostly) determined by two factors. 1) The number of files being backed up. 2) The amount of duplication in pathnames and filenames. If each backup job is 1000s of tiny files, you're going to be working the database pretty hard, as each of those files will require that a record be written to the database. If filenames and pathnames change often, that will require that additional database records be written for each file saved. For a file that has a name never used before in a directory never backed up before, you have 3 database records created. On the flip side, if you're filesets consist largely of the same files that are constantly being edited, and your average filesize is large, you'll find that the speed of your storage media will slow you down and the database will have no trouble keeping up. I suspect that's part of the reason that there are no published hardware requirements for Bacula. The hardware requirements aren't really for Bacula, their for your specific workload. Regardless of all that, if you really mean to run 50 jobs in parallel (i.e. at the same time, not just 50 jobs in series) I would recommend that you get a dedicated DB server with about 12 disks arranged in a RAID 10 and a battery-backed cache. If you can run the jobs in series, then just about any reasonably powered server hardware should suffice. -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backup rate decreasing
In response to "Support" <[EMAIL PROTECTED]>: > > I have Bacula-2.0.3 running on a dual P3 1 GHz linux system with a > 2.6.21.1 kernel. > > I have noticed that the backup rate of a client decreasing during the > backup it starts at 3,200 KB/s and slows down to 2,000 KB/s after 30 > minutes. I used Status of the Storage daemon to get the rate about every 5 > minutes. The backup was 6 GB to disc. > > I suspect the slow down may be due to the fact other jobs were waiting for > this job to finish since I ran another backup of another client and the > rate was constant during the whole backup, the difference being there were > no jobs queued to run ie waiting for the device. No. Queued jobs do nothing to affect the speed of a backup. > Could the slow down be caused by the director polling the system and so > impacting on the storage daemon? No. The "problem" is almost certainly caused by a change in the makeup of the data being backed up. Since "status client" gives you the speed at the time you run it, that can change over time if any factors change the speed. This could be something like activity on the client contending with Bacula, or network traffic. However, it's most likely (in my experience) that you start out backing up a lot of big files, then the backup moves into directories where a lot of small files are found. 2G composed of 2 1-G files will back up faster than if it's composed of 2000 1-meg files. This is because each file has some overhead associated with writing backup records to the database. If the speed difference is enough to be of concern, you should look at optimizing whatever database backend you're using to speed up inserts. -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] FreeBSD and Authochangers
In response to tomasz <[EMAIL PROTECTED]>: > Ken Gunderson wrote: > > Hello List: > > > > I've posted before regarding chio vs mtx-changer and got reply from Arno > > but I'd love to hear from some FreeBSD users who are using > > authochangers. So here goes again... > > > > Are you using chio or mtx-changer, and why? I've looked at the chio > > example scripts and seems that chio-bacula simulates barcodes and > > rc-chio-changer handles real barcodes but that chio-changer ignores > > them... The latter makes mention of FBSD-5.2 so that puts it a couple > > years old. rc-chio-changer has a cvs id of 2006-02-23 and looks to me > > most recent? Or are all of these abandoned in favor of mtx and > > mtx-changer? > > > i am using freebsd 6.2 with mtx-changer and barcodes and its all fine > (the only one problem was with access for bacula to autochenger node in > /dev) > autochanger is Quantum superloader 3 We're using chio-changer with FreeBSD 6 and a Dell LTO2 drive. Works just dandy, but we're not doing barcodes, so I can't comment on that. -- Bill Moran http://www.potentialtech.com - This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for mixed file and tape backups
In response to Jeff Tucker <[EMAIL PROTECTED]>: > > We'd like easy access to recent files for the case where someone deletes > the wrong file. We also want to take full backups (probably monthly) > offsite for long-term archiving and disaster recovery. > > My idea right now is to use a second machine with a TB of disk storage > as a near-line backup machine. I would do a monthly full backup and then > 30 days of incremental backups to disk. At the same time as the monthly > full disk backup, I could do a monthly full tape backup. That tape would > be taken offsite. > > Does this make sense? It seems like the disk backups would be more > convenient and faster to manage than rotating pools of tapes. Any other > ideas? We do this here. The disk storage is nice and fast compared to tapes. We do weekly offsites, though, so a little more intensive than what you're doing, and a single full backup is ~150G. > On the disk storage server, how would I go about setting up my volumes? I would recommend a full/differential/incremental rotation (which is pretty typical) Create a monthly pool, weekly pool, and daily pool. Have fulls got to monthly, differentials to weekly, incrementals to daily. Retention times: monthly 2 months, weekly, 4 weeks, daily, 7 days. The reason you want to keep at least 2 monthly backups is that if you have it shorter than that, and your full backup fails for some reason, it will have partially overwritten (i.e. corrupted) your only full backup! By having at least two full backup sets at any time, you avoid this. The differentials reduce the amount of storage space you need for backups between fulls, but the reduced the amount of resolution you have for restoring, so you need to consider your requirements carefully. The layout I describe is pretty typical. > I think I basically want a full backup with a volume retention period of > 25 days and incremental backup volumes with retention periods of 35 days > or so. So, if I do a full backup every month, it will overwrite the old > full backup. Yes. Again this creates a problem if the backup fails for some reason -- you lose you previous full backup. > And, if I do daily incrementals, then I will have a > rotating pool of 35 dailys. Yes. > Do the old dailys get deleted automatically? > Do they just get overwritten? The latter. After the retention period expires, the volume is marked "purged", but no data is destroyed. When the system needs a new volume, it take the oldest purged volume and truncates it and begins anew. > When I start out, using automatic volume > naming, will the system count up to 35, then start overwriting the older > volumes? Pretty much. The first month you'll see new volumes created every day, the second (and subsequent) month you'll see the system reuse them. > If someone could give me an idea what these volumes or pools should look > like in the config files, I'd appreciate it. Or, if I'm totally off > base, let me know. I believe the layout I describe above is documented in the Bacula docs, if not, it's typical enough that I'm sure there are examples out there. Just schedule two full jobs for the beginning of the month: one to tape and one to disk. Do the tape job first so future incrementals are based off the disk job. You could also do job cloning, but I don't think that's necessary to get what you want. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] WOW
Janco van der Merwe <[EMAIL PROTECTED]> wrote: > OH, I see, thank you for the speedy reply, just one thing I did not > turn on the FD compression but you know what it isn't that much of a > disater Compression is enabled by default. Unless you specifically do "compression = none", the data will be compressed. This is a tradeoff: compressed data takes less space on the network and on the tape. Uncompressed backups often go faster and don't slow down the clients as much. -- Bill Moran Microsoft: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming or what? - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula-dir and remote Postgres
Gabriele Bulfon <[EMAIL PROTECTED]> wrote:
> Hello,
> after discovering that Postgres is too slow on Sun 280R machines forBacula
> management, I decided to test one of them and have the PostgresDB run on
> another faster Solaris machine on the LAN.
> So I prepared the new DB from scratch, modified my original bacula-dir.conf
> to connect to the new DB host :
> Catalog {
> Name = MyCatalog
> dbname = bacula;
> user = bacula;
> password = "bacula"
> DB Address = myhost.mydomain.com
> DB Port = 5432
> }
> Now I see the Director starts on the new DB (the old one is down), but...
> if I run bconsole as before on the Director's machine, any command hangs
> for a while and then says:
> Could not open database "bacula".
> postgresql.c:168 Unable to connect to PostgreSQL server.
> Database=bacula User=bacula
> It is probably not running or your password is incorrect.
> I am absolutely sure that Director connects to the DB, because debug is
> enabled, and I can see the Director is ok at startup.
> Infact, if I shutdown the remote DB, Directors complains and does not start
> up.
> ...what is my problem?!?
Are you sure you're only running 1 director? Are you sure bconsole is
configured to connect to the director that's able to talk to the database?
Without more details, those are the best guesses I've got right now
Based on your narrative, I was confused as to where the director is now
running -- is it on the database machine or the 280R?
--
Bill Moran
Be calm.
Morpheus
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Multiple File Systems problem
In response to "Lautaro Di Martino" <[EMAIL PROTECTED]>: > hi all > > I have a problem with a partition mounted on the file system when > bacula doing a backup. For example: > > I have this directory: > /backup/nn1 > /backup/nn2 > /backup/nn3 > > on /backup/nn2 I have mount a lvm partition. When bacula do run job > in /backup for backup it and try read the directory /backup/nn2 a > warning error tell me this: > hostname-fd: Filesystem change prohibited. Will not descend into > /backup/nn2 > > Ok. I search in the bacula site and I find in File Set Resource this "onefs" > If I set "onefs=yes" I can do backups on multiple filesystems > technically, but this no work to me. > Any idea? onefs = yes tells Bacula _not_ to change filesystems. onefs = no will allow Bacula to switch to other mount points -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] New Admin
In response to "Keith Phipps" <[EMAIL PROTECTED]>: > > When I first logged in I had some messages, I typed 'messages' and got > them - sort of skimmed through to see what kinds of messages bacula gives. I > made the poor assumption that they'd still be there when I went back - I was > incorrect. Is there a log that these are logged to somewhere? There were > valid errors in there about labels that I would like to see. Console messages are gone after you view them. Hopefully, your admin configured the system to save those messages to a log file as well, which is pretty common. Look for the "messages" section in bacula-dir's config file to find out what it's doing. > Also, if it's stating: > > Archive "DATACENTER-TAPE1" is not open or does not exist. > > Is this a permissions issue perhaps? I'm currently going through the > tutorial I found looking for instances of this. Not sure. I get those messages when I check the status if a file device that isn't currently in use -- they don't indicate an error condition in that case. I've never seen that message occur in reference to a tape device. When there's no tape in the drive, I see: Device "Ultrium" (/dev/nsa0) open but no Bacula volume is mounted. > Any other advice you guys may have would be most appreciated. I really > need to get my head wrapped around this program quickly as it's priority > number one in my department. Well, your question-asking technique could use a little brushup. When providing an error message (such as the one above) it helps to say what you did that caused that message to be printed. Sure would be nice to know what version of Bacula you're running, as well as the OS and hardware involved. And when you ask questions, include the relevant parts of Bacula's config (sanitized: passwords removed and the like). Otherwise, my recommendation would be to grab a scratch machine, download the PDF version of the Bacula docs, and spend some time experimenting. Once you're more familiar with Bacula, it'll be easier to understand what the previous admin was doing. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Multiple File Systems problem
In response to "Lautaro Di Martino" <[EMAIL PROTECTED]>: > but this is not a solution for my problem. I have set onefs=yes in my > FileSet resource, anyway bacula can't read /backup/nn2 and tell me > will not descend into /backup/nn2 for backup it. I suspect there is a language barrier. onefs = yes _prevents_ Bacula from changing file systems and is the cause of those messages. If you change it to "no", Bacula will backup the data in those mount point. Be careful, if your fileset is organized poorly, this will result in data being backed up multiple times. What is your native language? Have the Bacula docs been translated to it yet? The English description of this setting is very good: http://www.bacula.org/rel-manual/Configuring_Director.html#SECTION000147000 > 2006/9/11, Bill Moran <[EMAIL PROTECTED]>: > > In response to "Lautaro Di Martino" <[EMAIL PROTECTED]>: > > > > > hi all > > > > > > I have a problem with a partition mounted on the file system when > > > bacula doing a backup. For example: > > > > > > I have this directory: > > > /backup/nn1 > > > /backup/nn2 > > > /backup/nn3 > > > > > > on /backup/nn2 I have mount a lvm partition. When bacula do run job > > > in /backup for backup it and try read the directory /backup/nn2 a > > > warning error tell me this: > > > hostname-fd: Filesystem change prohibited. Will not descend into > > > /backup/nn2 > > > > > > Ok. I search in the bacula site and I find in File Set Resource this > > > "onefs" > > > If I set "onefs=yes" I can do backups on multiple filesystems > > > technically, but this no work to me. > > > Any idea? > > > > onefs = yes tells Bacula _not_ to change filesystems. > > onefs = no will allow Bacula to switch to other mount points > > > > -- > > Bill Moran > > Collaborative Fusion Inc. > > > > - > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users -- Bill Moran Collaborative Fusion Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Productive Steps (was New Admin)
In response to "Keith Phipps" <[EMAIL PROTECTED]>: > > Now it may be that I just don't understand the steps of this software, > I've attempted the chapter "A Brief Tutorial" in the manual but since > this is a live environment and I don't have a test environment to hack > around in, I'm worried about 'breaking' something. There were critical > changes made to a server today and the backups must be done (lest we > have hundreds of customers angry with us should something go bad) > > What steps should I take from here, and in the future, to correct this issue? It looks as if you may have some networking problems between your director and file daemons. Specifically: "13-Sep 09:33 bacula-atl2-dir: tsbackend1.2006-07-26_00.00.08 Fatal error: Network error with FD during Backup: ERR=Connection reset by peer" I recommend using the "status client" and "estimate" commands to ensure that the communication between director and file daemons is set up correctly. These are both low-risk commands, as they don't back anything up or alter any data in any way. Note that "estimate" can take a long time on machines with lots of data. It looks as if you're confused about labelling as well, since you tried to write over an existing tape. This would not normally be what you'd want to do, but if you really do, you'll need to delete the media from the pool first. Can you simply find the tape that Bacula wants to use and ensure that it's in the tape drive? -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Issues concerning the production release of Bacula 1.39.x
In response to Kern Sibbald <[EMAIL PROTECTED]>: [snip] > 1. Make several more beta releases of 1.39.x before I leave on vacation (i.e. > between now and roughly 10 October) and hold the official production release > until mid-November after my return. > > 2. Release Bacula 1.39.x in the next couple of days even though it may be > less > tested and less stable in some areas than I would like. > > 3. Make the production release in a week or two -- say the end of September. Well, if you're just looking for an outright vote, I'd cast mine for #1. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem - Restore 500k files
In response to "Fernando Lima" <[EMAIL PROTECTED]>: > Hello, > > I'm performing tests of backup/restore using Bacula. > > The software is fantastic and was working better than my current solution > "Arcserve". > > Two days ago, I tried a test of backup/restore of one special server. This > server store half a million pdf files in one directory ! This isn't an answer to your question, but I thought I'd point out that it's considered a bad idea to put that many files in one directory for any reason. Many utilities will have difficulty working with directory trees that big (not just Bacula). Normally, this would be solved by hashing a directory tree, so each node of the tree had only a few hundred or few thousand leaves. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula and Postgres 8.1
In response to Gabriele Bulfon <[EMAIL PROTECTED]>: > Hello, > I open again the question about Bacula working on Postgres. > This time, I really need to have Director and Postgres run on two different > machines. > The reason: I only have one Solaris machine and one Windows machine, so I > cannot move the Director to the Windows machine together with Postgres (as > far as I know, the Director is still not supported on Windows). > What happens is that once I add the needed entries to the bacula-dir.conf to > connect to the new machine (DB Address and DB Port), Director starts (and > with log enabled to 500, I can see a lot of queries correctly handled), but > when I run bconsole on the Director, the Director initially responds with the > prompt, then any command that needs DB access (such as "list volumes") hangs > for some seconds and then fails saying that it cannot connect to the DB. > If I look at the log, I can see some timeout failures. > How can I investigate more this? Why the first queries go through correctly, > and then they fail with bconsole? > Please help...I really need to move the DB on the other machine Help: 1) Provide the logs you captured, not your interpretation of them. Nobody is going to be able to diagnose "I can see some timeout failures" 2) Bump up connection logging on the PostgreSQL server. You'll want to provide those logs as well. 3) Put some effort into formatting your emails so they're readable. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula-dir problems with -HUP? (was Re: [Bacula-devel] Backtrace Attached: bacula-dir lockups on FreeBSD)
In response to Bill Moran <[EMAIL PROTECTED]>: > > My plan at this point: > 1) Upgrading to RELENG_6_1 tonight > 2) See if the problem repeats this Saturday > 3) If it does, grab another backtrace and put it out on freebsd-hackers@ >to see if anyone has any suggestions. Followup posting. I upgraded the system to FreeBSD 6.1p5. There were some other circumstances that interfered with getting any reasonable data from the system, so it took a little longer than I expected to do the remaining steps. However, it occurred to me that the log rotation occurs at about the same time this freeze would occur. This involves sending Bacula a HUP signal. I moved the log rotation to later in the day (after the jobs are finished) and the whole set of backups ran without a hitch. It would appear as if sending bacula-dir a HUP signal in the middle of a backup causes this problem. Note that -HUPping the director while it is idle doesn't seem to cause any trouble. Note also that I experimented a bit, and hupping doesn't seem to cause the problem all the time. Can anyone verify this on another platform? I want to set up some test scenarios here, but I'm dogged for time, and it may be a while before I can do any more serious work on this issue. On a specific note: changing the time that logs rotate is only a workaround for us, and one with a dubious future. We're running backups more and more frequently, and a lot of them run rather dynamically (using reschedule on error and the like) so it's going to get increasingly more difficult to find a time when I can be sure that no backup is running. -- Bill Moran Collaborative Fusion Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula-dir problems with -HUP? (was Re: [Bacula-devel] Backtrace Attached: bacula-dir lockups on FreeBSD)
In response to Kern Sibbald <[EMAIL PROTECTED]>: > > However, it occurred to me that the log rotation occurs at about the same > > time this freeze would occur. This involves sending Bacula a HUP signal. > > Sending Bacula an HUP signal causes it to exit as rapidly as possible. It > attempts to cancel and cleanup jobs, but it does so in a rather brutal > fashion. > > Bottom line: make sure no root program sends HUP signals to Bacula while it > is > running a job, and preferably never send such signals unless you want to > shutdown Bacula. This is new to me. And not intuitive IMHO. Most programs interpret a HUP to mean "restart gracefully". I got the impression that it would be about the same as issuing "reload" from bconsole. Most daemons require a HUP signal to tell them to close and reopen their log files. I seem to remember reading a HOWTO that suggested hupping the director after log rotation, but I can't find it now -- perhaps my memory is flawed. > I don't understand why any process is sending HUP signals to Bacula when it > is > cycling logs. This seems to me to be a "configuration" error. That would be _my_ error. I put the command to HUP bacula-dir in newsyslog.conf. Personally, I find the director's handling of HUP atypical by comparison to other Unix daemons. From the docs, it's not clear whether the director requires a signal to tell it to start logging to a new file, I'm assuming (since HUP is a bad idea) that it does not? Additionally, this still seems to be a bug (even if it won't be affecting me any more) since the director does not exit when hupped, but instead freezes. I've done several searches in the past on the director's handling of signals, and found sparse data. I repeated those searches just now, and still haven't found any information. At least I know that HUP is a bad idea ... -- Bill Moran Collaborative Fusion Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula-devel] Bacula-dir problems with -HUP? (was Re: Backtrace Attached: bacula-dir lockups on FreeBSD)
In response to Kern Sibbald <[EMAIL PROTECTED]>: > On Tuesday 19 September 2006 17:53, Bill Moran wrote: [snip] > > This is new to me. And not intuitive IMHO. Most programs interpret a HUP > > to mean "restart gracefully". I got the impression that it would be about > > the same as issuing "reload" from bconsole. Most daemons require a HUP > > signal to tell them to close and reopen their log files. I seem to > > remember reading a HOWTO that suggested hupping the director after > > log rotation, but I can't find it now -- perhaps my memory is flawed. > > Yes, you are right, I was confusing a SIGHUP with a SIGTERM. > > SIGHUPing Bacula is never a very good idea. Depending on what version of > Bacula it is, it will most likely crash. If I am not mistaken, so far there > are no reported crashes against version 1.38.11. Well, that still tells me what to do to make the the problem go away, although you can consider this a report of lockup when sending the director a HUP ;) [snip] > > Personally, I find the director's handling of HUP atypical by comparison > > to other Unix daemons. From the docs, it's not clear whether the > > director requires a signal to tell it to start logging to a new file, > > I'm assuming (since HUP is a bad idea) that it does not? > > If you are talking about cycling a Job report log file as defined in the > Messages resource, then there is probably no need to SIGHUP Bacula as it > opens and closes the log file on a Job by Job basis, if I am not mistaken. That's what I meant. Although I'm still a bit concerned. What happens if the log file is rotated while a job is in progress? Will bacula attempt to append to the end of (the now compressed) previously opened file descriptor? Or will it reconnect to the new file with the correct filename. Job reports split between two log files is an annoyance, but not terrible. Corruption of old log data would be worse. [snip] > Yes, even if I did mistake SIGHUP and SIGTERM, it is not a good idea to > SIGHUP > Bacula. I've adjusted the newsyslog config. Thanks for the feedback, as always. -- Bill Moran Collaborative Fusion Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] useful indexes (was Re: dbcheck needs very long time...)
In response to Birger Blixt <[EMAIL PROTECTED]>: > Frank Sweetser wrote: > > On Thu, Sep 07, 2006 at 10:58:49AM +0200, Masopust, Christian wrote: > >> Hello Frank, > >> > >> thanks a lot for this info! but :-))) > >> > >> could anybody give the complete info (maybe also modify the wiki-page) how > >> these indexes should be created? > > > > The best place to find that is in the documentation for your database. That > > way you can make sure you're getting accurate information, as opposed to > > something for a previous version or the like. > > Amazing, my dbcheck at home was searching for orphaned Filename records > the whole day, and I finaly did mysaladmin kill on the query. > > Then I did: > > mysql bacula > > create index Job_FilesetId_idx on Job(FileSetId); > create index Job_ClientId_idx on Job(ClientId); I find it hard to believe that these indexes help much, unless you've got a boatload of jobs in the system. I doubt they'll hurt anything, though. > create index File_PathId_idx on File(PathId); > create index File_FilenameId_idx on File(FilenameId); One of these two is redundant. There's already an index: "file_fp_idx" btree (filenameid, pathid) CLUSTER which should be usable by queries searching on filenameid (this is PostgreSQL, but the idea is the same) Creating another index on filenameid is just bloat and will slow down inserts. Personally, I don't recommend that anyone blindly create these indexes. As you mentioned, it took over 30 minutes to create them, which seems to indicate that their existence will have a negative impact on inserts and updates. In my case, none of these indexes made a significant improvement to dbcheck time. dbcheck took 5-1/2 minutes to run with and without the above indexes (I have about 9,000,000 rows in the file table and 500,000 in the filename table). Try them out ... if they make a significant improvement, use them. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] 25-hour backup job
In response to "David Hatcher" <[EMAIL PROTECTED]>: > Hi folks, > > New user here. I find the following peculiar and wonder if "big" backup > jobs take longer to complete than running several consecutive smaller > jobs? Here's my story... > > Server = bacula-fd Version: 1.38.9, OS=Linux Fedora Core 4 Client = > labssrv-fd Version: 1.38.4, OS=Windows NT 4.0 > > For my first full backup on my client (labssrv), I setup the > bacula-dir.conf file (see attached) to backup everything on the C, F, G, > and H drives. Below is the summary, in particular, the job took 13 > hours to backup 173 GB and resulted in 430 non-fatal FD errors, most of > which were permission errors. This sounds fairly reasonable to me. > > JobId: 1150 > Job:Labssrv.2006-09-08_19.00.03 > Backup Level: Full > Client: "labssrv-fd" Windows NT 4.0,MVS,NT 4.0.1381 > FileSet:"Labssrv FileSet" 2006-09-08 22:52:20 > Pool: "Weekly" > Storage:"SDLT" > Scheduled time: 08-Sep-2006 19:00:02 > Start time: 08-Sep-2006 22:52:23 > End time: 09-Sep-2006 12:39:33 > Elapsed time: 13 hours 47 mins 10 secs > Priority: 11 > FD Files Written: 296,222 > SD Files Written: 296,222 > FD Bytes Written: 173,119,126,494 (173.1 GB) > SD Bytes Written: 173,182,590,769 (173.1 GB) > Rate: 3488.2 KB/s > Software Compression: None > Volume name(s): 000103|77 > Volume Session Id: 5 > Volume Session Time:1157757233 > Last Volume Bytes: 25,817,913,037 (25.81 GB) > Non-fatal FD errors:430 > SD Errors: 0 > FD termination status: OK > SD termination status: OK > Termination:Backup OK -- with warnings > > > Then I went through the details of the permission errors and granted > access to the respective files and directories on my labssrv machine. In > addition, I excluded some old archived data that doesn't need to be > backed up. Following is the new summary. The job took 25 hours to > backup 188 GB and resulted in 0 non-fatal FD errors I'm surprised it > took 12 additional hours to backup 15 GB of data, as if there's an > exponential problem somewhere. This doesn't make sense to me. I'm > thinking about splitting this job into two separate jobs (job one for > drives C and F, job two for drives G and H) to see if it will complete > in under 25 hours. I have other clients that I back up that run fairly > quick backup jobs, although the backups are typically less than 100 GB. > > JobId: 1225 > Job:Labssrv.2006-09-15_19.00.03 > Backup Level: Full > Client: "labssrv-fd" Windows NT 4.0,MVS,NT 4.0.1381 > FileSet:"Labssrv FileSet" 2006-09-15 22:50:24 > Pool: "Weekly" > Storage:"SDLT" > Scheduled time: 15-Sep-2006 19:00:02 > Start time: 15-Sep-2006 22:50:27 > End time: 16-Sep-2006 23:34:34 > Elapsed time: 1 day 44 mins 7 secs > Priority: 11 > FD Files Written: 309,962 > SD Files Written: 309,962 > FD Bytes Written: 187,971,236,795 (187.9 GB) > SD Bytes Written: 188,037,056,495 (188.0 GB) > Rate: 2110.9 KB/s > Software Compression: None > Volume name(s): 82|000100 > Volume Session Id: 5 > Volume Session Time:1158352255 > Last Volume Bytes: 39,196,049,239 (39.19 GB) > Non-fatal FD errors:0 > SD Errors: 0 > FD termination status: OK > SD termination status: OK > Termination:Backup OK Lots of information missing here -- difficult to help much without some additional diagnosis. What DB are you using? Is it possible that you've pushed the db server past some limit where performance starts to degrade? i.e. created enough records that inserts have become expensive? If you monitor CPU and IO usage during the backup, where is the holdup and which program (DBserver? director? storage daemon? file daemon?) is using that resource? Is it possible that you hit something on the DLT tape that caused it to have to rewind or do a bunch of seeking or something else that put the whole job in wait mode for a long time? What is the nature of the data in that directory? Is it possible that the FD is contested for access to those files and is spending a lot of time
Re: [Bacula-users] Bacula and Postgres 8.1
In response to Gabriele Bulfon <[EMAIL PROTECTED]>:
> Sorry for not resending the needed informations, I already sent them when I
> first encountered the problem. But then I could solve this by moving the
> Director together with the DB (I had another Solaris machine in that
> situation).
> I'm appending the original infos here.
> On postgres, I just find this debug, after getting the error from bconsole:
> LOG: unexpected EOF on client connection
This seems to indicate that you have a networking issue.
Any network control devices between these two systems? (firewalls, routers,
managed switches?) Ensure all your speed and duplexing match up. Grab
some tcpdumps off the network while this is happening, and see if anything
looks fishy. Do it on both ends of the connection and see if packets are
getting lost.
Can you use psql on the system running the director to connect manually
to the PostgreSQL database? Can you run a long running query without
getting disconnected ("SELECT COUNT(*) FROM " will
generally take a while)
> Hope you can help.
> Gabriele.
> ==
> I rised debug to 500.
> Before the I issue the "list volumes" on bconsole, I see a LOT LOT of
> debug about postgres selects going through very fine.
That's funny, because I don't see any in your attached logs.
> Once I run the bconsole command I find this:
> iserver-dir: scheduler.c:253 enter find_runs()
> iserver-dir: scheduler.c:289 Got job: Enterprise Backup
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:289 Got job: Backup Aliseo
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:289 Got job: Backup ZetaFax
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:289 Got job: Backup Centralino
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:289 Got job: BackupCatalog
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:332 [EMAIL PROTECTED]: run_now=0 run_nh=0
> iserver-dir: scheduler.c:355 Leave find_runs()
> iserver-dir: bsys.c:70 pthread_cond_timedwait sec=60 usec=0
> iserver-dir: ua_cmds.c:1615 Open database
> iserver-dir: postgresql.c:77 db_open first time
> iserver-dir: bsys.c:70 pthread_cond_timedwait sec=5 usec=0
> iserver-dir: bsys.c:77 pthread_cond_timedwait stat=145 ERR=Connection timed
> out
> Gabriele Bulfon - Sonicle S.r.l.
> Tel +39 028246016 Int. 30 - Fax +39 028243880
> Via Felice Cavallotti 16 - 20089, Rozzano - Milano - ITALY
> http://www.sonicle.com
--
Bill Moran
Collaborative Fusion Inc.
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] useful indexes (was Re: dbcheck needs very long time...)
In response to Birger Blixt <[EMAIL PROTECTED]>: > On 2006-09-20 15:44, Bill Moran wrote: > > In response to Birger Blixt <[EMAIL PROTECTED]>: > > > >> Frank Sweetser wrote: > >>> On Thu, Sep 07, 2006 at 10:58:49AM +0200, Masopust, Christian wrote: > >>>> Hello Frank, > >>>> > >>>> thanks a lot for this info! but :-))) > >>>> > >>>> could anybody give the complete info (maybe also modify the wiki-page) > >>>> how > >>>> these indexes should be created? > >>> The best place to find that is in the documentation for your database. > >>> That > >>> way you can make sure you're getting accurate information, as opposed to > >>> something for a previous version or the like. > >> Amazing, my dbcheck at home was searching for orphaned Filename records > >> the whole day, and I finaly did mysaladmin kill on the query. > >> > >> Then I did: > >> > >> mysql bacula > >> > >> create index Job_FilesetId_idx on Job(FileSetId); > >> create index Job_ClientId_idx on Job(ClientId); > > > > I find it hard to believe that these indexes help much, unless you've got > > a boatload of jobs in the system. I doubt they'll hurt anything, > > though. > > > >> create index File_PathId_idx on File(PathId); > >> create index File_FilenameId_idx on File(FilenameId); > > > > One of these two is redundant. There's already an index: > > "file_fp_idx" btree (filenameid, pathid) CLUSTER > > which should be usable by queries searching on filenameid (this is > > PostgreSQL, but the idea is the same) Creating another index on > > filenameid is just bloat and will slow down inserts. > > > > Personally, I don't recommend that anyone blindly create these > > indexes. As you mentioned, it took over 30 minutes to create them, > > which seems to indicate that their existence will have a negative > > impact on inserts and updates. > > > > In my case, none of these indexes made a significant improvement to > > dbcheck time. dbcheck took 5-1/2 minutes to run with and without the > > above indexes (I have about 9,000,000 rows in the file table and > > 500,000 in the filename table). > > > > Try them out ... if they make a significant improvement, use them. > > Well, I admit, I don't know what I'm doing in this case, > but ,,, the search query _did_ take forever, > and I canceled it at the end. > With new indexes it goes fast, so something did happen. > > I was checking the script http://www.aha.com/bacula/recover.pl > There I found: > "In order for this program to have a chance of not being painfully slow, > the following indexs should be added to your database. > CREATE INDEX file_pathid_idx on file(pathid); > CREATE INDEX file_filenameid_idx on file(filenameid); " > > I took that as an example, and added them, and the 2 Job indexes as a test, > there is always a drop index on table command to remove them if it don't > works. > > I should enjoy if someone that really knows how to tune mysql could > send an output from "show index from File" , so I can optimize the table > (that goes for the table Job too I guess ) > > File 0 PRIMARY 1 FileId 4541609 BTREE > File 1 JobId1 JobId 576 BTREE > File 1 JobId2 PathId 378467 BTREE > File 1 JobId3 FilenameId 4541609 BTREE > File 1 JobId_2 1 JobId 576 BTREE > File 1 File_PathId_idx 1 PathId 105618 BTREE > File 1 File_FilenameId_idx 1 FilenameId 504623 BTREE > > Maybe I can drop something here, who knows ? > > One source for information can be the make_mysql_tables script The problem is that depending on your data and usage of the database, YMMV. Optimizations that work for one person might not benefit another. For example: adding indexes _usually_ speeds query performance, but it also _usually_ slows insert/update performance, because every change to the table has to update all the indexes. If you're interested in very fast backups and rarely do restores, it might be a bad idea to add any indexes to improve looks in the database if it slows down the updates. On the other hand, if getting at your data quickly is important, it might be worth slower backups to be able to search the data quickly. People who have lots of jobs vs. folks with only a few jobs. A few large files vs. lots of small files. One big server vs. many servers. Each of t
Re: [Bacula-users] useful indexes (was Re: dbcheck needs very long time...)
In response to Frank Sweetser <[EMAIL PROTECTED]>:
> On Wed, Sep 20, 2006 at 09:44:24AM -0400, Bill Moran wrote:
> > Try them out ... if they make a significant improvement, use them.
>
> As an aid to experimenting with indexes, and to help people give more hard
> data, perhaps it might be usefull to add an option to dbcheck to output the
> results of running explain (mysql, I assume postgresql has an equivalent) on
> the queries used. This info is really useful to find hotspots that would
> benefit the most from adding an index or two.
Well, I have statement logging and query duration logging turned on
on PostgreSQL on our production system. Running dbcheck shows a bunch
of intense queries, Here are some of the killers:
duration: 153056.519 ms statement: SELECT Filename.FilenameId,
File.FilenameId FROM Filename LEFT OUTER JOIN File ON
(Filename.FilenameId=File.FilenameId) WHERE File.FilenameId IS NULL
bacula=> explain SELECT Filename.FilenameId,File.FilenameId FROM
Filename LEFT OUTER JOIN File ON (Filename.FilenameId=File.FilenameId)
WHERE File.FilenameId IS NULL;
QUERY PLAN
-
Merge Left Join (cost=0.00..824877.11 rows=506718 width=8)
Merge Cond: ("outer".filenameid = "inner".filenameid)
Filter: ("inner".filenameid IS NULL)
-> Index Scan using filename_pkey on filename (cost=0.00..23530.19
rows=506718 width=4)
-> Index Scan using file_fp_idx on file (cost=0.00..676093.57 rows=9918925
width=4)
(5 rows)
Notice that it's using existing indexes, it's the join that's rough.
This one is rough as well:
duration: 185632.013 ms statement: SELECT DISTINCT Path.PathId,
File.PathId FROM Path LEFT OUTER JOIN File ON (Path.PathId=File.PathId)
WHERE File.PathId IS NULL
QUERY PLAN
-
Unique (cost=1819887.33..1824163.31 rows=570131 width=8)
-> Sort (cost=1819887.33..1821312.65 rows=570131 width=8)
Sort Key: path.pathid, file.pathid
-> Merge Left Join (cost=1587495.88..1760367.99 rows=570131 width=8)
Merge Cond: ("outer".pathid = "inner".pathid)
Filter: ("inner".pathid IS NULL)
-> Index Scan using path_pkey on path (cost=0.00..22662.91
rows=570131 width=4)
-> Sort (cost=1587495.88..1612293.20 rows=9918925 width=4)
Sort Key: file.pathid
-> Seq Scan on file (cost=0.00..278755.25 rows=9918925
width=4)
The query plan on this seems to indicate that an index on pathid would
be helpful. It's interesting that I didn't see any measurable
improvement.
--
Bill Moran
Collaborative Fusion Inc.
IMPORTANT: This message contains confidential information and is
intended only for the individual named. If the reader of this
message is not an intended recipient (or the individual
responsible for the delivery of this message to an intended
recipient), please be advised that any re-use, dissemination,
distribution or copying of this message is prohibited. Please
notify the sender immediately by e-mail if you have received
this e-mail by mistake and delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or
error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The
sender therefore does not accept liability for any errors or
omissions in the contents of this message, which arise as a
result of e-mail transmission.
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula GZip question
In response to "Vadim A. Umanski" <[EMAIL PROTECTED]>: > Здравствуйте, bacula-users. > > Dear colleagues, an advice needed. > > I've got a mail server, the BackUp solution is Bacula 1.38.9. > > I've got a lot of data on Accounts disk slice, and the mail server's > CPU is not too strong (UltraSPARC II 450 MHz) and Full backup takes > 3-4 times more when GZip option is on than when it's off. It seems to > me that sending more data to network is easier for the server than > compressing them and then sending less data. > > But the mail server data can be compressed approx. 50% and I wouldn't > like to spend unneeded space ... Full backup would take about 80-90 GB > of raw data. > > Can I configure backup so that data compression would be performed by > Bacula server (CPU is quite strong and quite little loaded) and NOT by > Bacula client? > > Please give me an advice! Current versions of Bacula don't support this. I don't know whether or not the upcoming release does or not. Have you tried using GZIP1 instead of GZIP? In my own experiments, I've noticed that GZIP1 uses noticeably less CPU than GZIP (which is actually GZIP6) and still accomplishes good compression. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula and Postgres 8.1
In response to "Magnus Hagander" <[EMAIL PROTECTED]>: > > > Sorry for not resending the needed informations, I already sent > > them when I first encountered the problem. But then I could solve > > this by moving the Director together with the DB (I had another > > Solaris machine in that situation). > > > I'm appending the original infos here. > > > On postgres, I just find this debug, after getting the error from > > bconsole: > > > LOG: unexpected EOF on client connection > > > > This seems to indicate that you have a networking issue. > > That message alone, not necessarily. It will be logged by PosgreSQL > anytime a connection goes away without getting the close message. So a > client program exiting without doing PQfinish() will cause this, for > example (and this is way way common, unfortunately). Or a client program > crashing, of course. > > It *can* be a sign of network issues, absolutely. But it can also be > something else. Agreed. I gues the point I was trying to make was that, based on the messages from the server and the "timeout" messages on the client, my best guess at this point is that it's a network problem. But it wouldn't be the first time if I were wrong. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula and Postgres 8.1
This foray has gotten very frustrating for me. Please excuse if the following replies seem inflammatory, but it comes from an honest desire to _really_ help. In response to Gabriele Bulfon <[EMAIL PROTECTED]>: > That postgres db is being used on the network all day long. > - I use it from web apps through jdbc > - I use it from pgadmin on a windows laptop > - I use it from custom softwares on desktops > This problem arises only when I use bconsole on Bacula and the Director > is separated from the DB. > Everything works fine only if the Director works on localhost DB. > In the logs, I did not put the Director's startup traces because > they were a lot and they all went ok. Please put some effort in to formatting your emails. I'm tired of fixing your line wrapping. 72 chars is the standard. Now. I'm going to assume 1 of two things: 1) You're a super expert on both Bacula and PostgreSQL. This is evident because you can look at logs and _know_ that certain parts are not important because you _know_ that those sections show normal behaviour. You're so smart that you refuse to send those sections of the logs in spite of multiple requests for them. Obviously, there's absolutely no chance that you could be mistaken, since the system is running so smoothly. 2) There's something in the logs that you find embarrassing. If #1 is the case, then just fix the problem yourself. If #2 is the case, then I have no advice. > I really suspect this is something coming out only when I use bconsole. While this could be true, I find it unlikely and not indicated by the evidence you're providing. bconsole talks to the director and the director talks to the database. It appears as if bconsole is able to contact the director, and the errors seem to indicate a connection problem between the director and the database. The fact that this only occurs when you use the director is a silly benchmark, since the _only_ other thing you've managed to get the director to do is start. > I can't make a real test now, but because Director starts up very happy > with the DB, I'm pretty sure that it would also run the scheduled > backups happily. Those are some outlandish assumptions you're making there. If the director fails on a "status storage" then I find it difficult to imagine that it could succeed in backing anything up. > Because I cannot use bconsole, then I cannot create volumes, and > cannot verify this. Yes, you can. You could create autolabeled disk volumes in the config, create a test backup job, schedule it to run frequently and you'd find out very quickly. > Someone can explain me what happens when I use "list volumes" on > bconsole? there must be some difference in the way the DB is accessed > at startup and when I use bconsole No. bconsole -> director, then director -> db in the same way it does anything else. You've still yet to provide any kind of network debugging. For crying out loud, do a "tcpdump -s0 -w /some/file port 5432" during the failure and post the resultant file somewhere. And, quite frankly, the last several emails have amounted to you politely saying, "I don't agree with your advice, and therefore, won't take it." Based on this, I'm tempted to stop giving it. If you know so much about how bacula works, why are you still stuck? If you want any _real_ help, you're going to have to post some serious information somewhere. It's amazing that the open source community is generally willing to wade through such data and provide assistance for free. But I find it even more amazing that you repeated _refuse_ to take advantage of such free debugging. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] confused about Full vrs. Incremental backups & pools
In response to Arno Lehmann <[EMAIL PROTECTED]>: > > [Problem] > > Here's the problem...as incremental backups expire and are purged, bacula > > will > > often promote the next incremental to a "full" backup, since it correctly > > determines that there are no full backups for a particular client in the > > "Incremental" pool. Writing a full backup can be disruptive to both the > > client > > and backup server, as some backups are over 2TB, with clients on a slow > > network. > > I want to avoid unscheduled full backups as a result of promoted > > incrementals, > > and I don't want to be doing full backups every 2 weeks to satisfy the > > retention > > period of the "Incremental" pool. > > > > Is there anyway to avoid this behavior? > > Keep the incremental backups for more than a month. > > You need an unbroken chain of incrementals, i.e. from the last full > backup to the current date no incremental backup can be pruned. Not exactly true. Differentials can be used to "consolidate" incrementals. Assuming you make incrementals 6 days a week, and Sunday is for fulls and differentials, set retention on your incrementals to 6 days, differentials to 3 weeks. Then you'll always have enough data to perform an incremental without building a new full. That gives you the standard "decreasing granularity with increasing age" scheme that most people want. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem starting bacula-dir with Enable VSS
Just a "me too". I can verify that I'm having the same problem with
a 1.38.10 director on FreeBSD (installed from ports).
In response to Diego Rozzini Pires <[EMAIL PROTECTED]>:
>
> I' new with bacula and i'm having some difficults in enable VSS for
> Windows 2003.
> I'm using Gentoo, kernel 2.6.17-gentoo-r7 and Bacula Version 1.36.3 on
> server, and bacula-fd 1.38.10 on Windows 2003 Server.
>
> When I open the tray icon on Windows i have this:
> servername-fd Version: 1.38.10 (08 June 2006) VSS Windows Server 2003
> MVS NT 5.2.
>
> I already started VSS in the Windows server using: net start vss. And
> also restarted the FD.
>
> My FileSet are this:
> FileSet {
> Name = "Promedonserver2"
> Enable VSS = yes
> Include {
> Options {
> Compression=GZIP6
> signature = MD5
> }
> File = "d:/backup"
> }
> }
>
> When i try to start bacula-dir i got this message:
>
> backupserver bacula # /etc/init.d/bacula restart
> * Starting bacula storage daemon ...
> * Starting bacula file daemon ...
> * Starting bacula director ...
> 19-Sep 18:21 bacula-dir: ERROR TERMINATION at parse_conf.c:821
> Config error: Keyword "EnableVSS" not permitted in this resource.
> Perhaps you left the trailing brace off of the previous resource.
> : line 184, col 13 of file /etc/bacula/bacula-dir.conf
> Enable VSS = yes
>
> I alredy verified my conf looking for some trailing brace open and tried
> to use only one FileSet for test but i didn't found anything.
>
> I've looked at google and bacula site but i didn't find anything too. If
> i skipped something on google or bacula site, please let me know!
>
> Thanks for help!
>
> --
> __
>
> Diego Rozzini Pires
> Consultor de Tecnologia Trainee
> ZeniSys Tecnologia da Informação
> [EMAIL PROTECTED]
> Fone : 55 11 4425-2424
> Móvel: 55 11 82083168
> Visite : http://www.zenisys.com.br
> __
>
>
> -
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
--
Bill Moran
Collaborative Fusion Inc.
IMPORTANT: This message contains confidential information and is
intended only for the individual named. If the reader of this
message is not an intended recipient (or the individual
responsible for the delivery of this message to an intended
recipient), please be advised that any re-use, dissemination,
distribution or copying of this message is prohibited. Please
notify the sender immediately by e-mail if you have received
this e-mail by mistake and delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or
error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The
sender therefore does not accept liability for any errors or
omissions in the contents of this message, which arise as a
result of e-mail transmission.
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem starting bacula-dir with Enable VSS
In response to "Attila Fülöp" <[EMAIL PROTECTED]>:
> Enable VSS = "yes"
>
> works for me. FreeBSD / 1.38.5 from ports.
FileSet {
Name = "WindowsXP"
Include {
Options {
# Enable VSS = "Yes"
# compression = GZIP1
}
File = "C:/"
}
}
If I uncomment the VSS line, I get:
27-Sep 13:38 backup-dir: ERROR in inc_conf.c:330 Config error: Keyword
EnableVSS not permitted in this resource
: line 114, col 18 of file /usr/local/etc/bacula-dir.conf
Enable VSS = "Yes"
This config:
FileSet {
Name = "WindowsXP"
Include {
# Enable VSS = "Yes"
Options {
# compression = GZIP1
}
File = "C:/"
}
}
Generates the same error. The error is the same whether I use the " or
not.
> Bill Moran wrote:
> > Just a "me too". I can verify that I'm having the same problem with
> > a 1.38.10 director on FreeBSD (installed from ports).
> >
> > In response to Diego Rozzini Pires <[EMAIL PROTECTED]>:
> >> I' new with bacula and i'm having some difficults in enable VSS for
> >> Windows 2003.
> >> I'm using Gentoo, kernel 2.6.17-gentoo-r7 and Bacula Version 1.36.3 on
> >> server, and bacula-fd 1.38.10 on Windows 2003 Server.
> >>
> >> When I open the tray icon on Windows i have this:
> >> servername-fd Version: 1.38.10 (08 June 2006) VSS Windows Server 2003
> >> MVS NT 5.2.
> >>
> >> I already started VSS in the Windows server using: net start vss. And
> >> also restarted the FD.
> >>
> >> My FileSet are this:
> >> FileSet {
> >> Name = "Promedonserver2"
> >> Enable VSS = yes
> >> Include {
> >> Options {
> >> Compression=GZIP6
> >> signature = MD5
> >> }
> >> File = "d:/backup"
> >> }
> >> }
> >>
> >> When i try to start bacula-dir i got this message:
> >>
> >> backupserver bacula # /etc/init.d/bacula restart
> >> * Starting bacula storage daemon ...
> >> * Starting bacula file daemon ...
> >> * Starting bacula director ...
> >> 19-Sep 18:21 bacula-dir: ERROR TERMINATION at parse_conf.c:821
> >> Config error: Keyword "EnableVSS" not permitted in this resource.
> >> Perhaps you left the trailing brace off of the previous resource.
> >> : line 184, col 13 of file /etc/bacula/bacula-dir.conf
> >> Enable VSS = yes
> >>
> >> I alredy verified my conf looking for some trailing brace open and tried
> >> to use only one FileSet for test but i didn't found anything.
> >>
> >> I've looked at google and bacula site but i didn't find anything too. If
> >> i skipped something on google or bacula site, please let me know!
> >>
> >> Thanks for help!
> >>
> >> --
> >> __
> >>
> >> Diego Rozzini Pires
> >> Consultor de Tecnologia Trainee
> >> ZeniSys Tecnologia da Informação
> >> [EMAIL PROTECTED]
> >> Fone : 55 11 4425-2424
> >> Móvel: 55 11 82083168
> >> Visite : http://www.zenisys.com.br
> >> __
> >>
> >>
> >> -
> >> Take Surveys. Earn Cash. Influence the Future of IT
> >> Join SourceForge.net's Techsay panel and you'll get the chance to share
> >> your
> >> opinions on IT & business topics through brief surveys -- and earn cash
> >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> >> ___
> >> Bacula-users mailing list
> >> Bacula-users@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >
> >
>
>
>
> -
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
--
Bill Mo
Re: [Bacula-users] Problem starting bacula-dir with Enable VSS
In response to "Attila Fülöp" <[EMAIL PROTECTED]>:
> Bill Moran wrote:
> > In response to "Attila Fülöp" <[EMAIL PROTECTED]>:
> >
> >> Enable VSS = "yes"
> >>
> >> works for me. FreeBSD / 1.38.5 from ports.
>
> To be more precise
>
> FileSet {
>Name = "WinXP"
>Enable VSS = "yes"
>
>Include {
>
> Note, not in "Options" and not in "Include", just in "FileSet"
> The thread starter did it right though.
Yup. That fixed it for me. Don't know where I got the wrong idea
from. Just double-checked the docs, and they explain it correctly.
Thanks for straightening me out.
--
Bill Moran
Collaborative Fusion Inc.
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Binding to source IP address
In response to James Ray <[EMAIL PROTECTED]>: > We have a concept of a system address (the machine itself) and a service > address (one for each service, say .2 == an apache service, .3 == an > bacula service, .4 == a dns service, where as .1 is a _system_ address) > all the interfaces will be on the same machine. > > So any communications coming _from_ the bacula service need to come out > of .3 and any connections to the bacula service need to head onto .3 also. > > What I have seen (only on my breif testing) is that the listen address > is the .3 address (as I set in DirAddress) but the outcoming connections > from the bacula-dir to the bacula-fd across the network originate for > the system default address of .1... > > I shall re-test and post my outcomes. I just want to quickly chime in that this is of interest to us as well, since we take the same approach. (i.e. each machine has a "management" address and another address for each "service" it provides) The primary reason for this is to help simplify firewall rules. We're in a high-security situation here, so our firewalls default to deny (even from one server to another). Each communication must be explicitly allowed. A secondary reason for this is planning for expansion. Some servers run many services (DNS, NFS, LDAP, etc). In the event that we should split the aforementioned machine into two systems (i.e. put DNS & LDAP on one, and NFS on the second) we don't have to change any firewall rules or configurations on other machines, because the service IPs can follow the services. I haven't had time to investigate whether the [FD|SD|DIR]Address sets both the listening and the outgoing address, but a firewall audit is on the TODO list, and when I finally get to it, I'll have to address this for a number of services, not only Bacula. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula clients and OpenVPN?
[EMAIL PROTECTED] wrote: > Hi Frank, > > Thanks for sharing this with me as it is very useful and will allow me to > possibly remove the OpenVPN part from the design that we are considering > since Bacula seems to be able to take care of this matter for us as a > built in feature. > > I guess that I still have the major question of being able to traverse > firewalls and routers since as you have mentioned that the Bacula server > initiates the connections to the clients and the client software does not > initiate any connection to the server although I think that would be a > nice feature as well. If you're traversing firewalls, it sounds like you need to establish 1 VPN per firewall, then use routing to route the Bacula services through the VPN. Bacula's TLS isn't going to do that for you. Is there some reason why you can't just establish a persistent VPN between the Bacula server and the firewall and run the jobs across it? -- Bill Moran When I point out limitations of one technique as a motivation for another, I do so in the context of specific problems; for different problems or in other contexts, the first technique may indeed be the better choice. Useful software has been constructed using all of the techniques presented here. Bjarne Stroustrup, _The_C++_Programming_Language_ - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Poll - What operating systems do you run Bacula on?
In response to "Peter L. Buschman" <[EMAIL PROTECTED]>: > > All: > > If it isn't too much of an imposition, I'd like to survey the list > and ask the question > "what operating system are you running Bacula on?". I'm interested in which OS > distributions, versions and platforms are being deployed as Bacula servers. 4 systems running Bacula on FreeBSD 6.x using PostgreSQL as the backend. If you want to know where FDs are running, that's a bit of a larger topic. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula and Softlinks
In response to "Janco van der Merwe" <[EMAIL PROTECTED]>: > Hi, > > Before wasting valuable time in testing can someone tell me how Bacula > will react with softlinks? Is it worth going that route? To give you an > idea on the one server it has 3 partitions and the aim is to backup the > last 3 months of data but the twist is that the directories and > partitions of where the data is will change from month to month. What we > thought is creating a backup dir on the / partition and then a script > that creates softlinks to the required directories within the /backup > dir which brings me to the above question will Bacula follow the > softlinks? Please wrap your lines around 72 characters. Current versions of Bacula don't follow symlinks. There's been some discussion about how to add a "follow symlinks" config option, but I don't know what the current status is on that. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula: backup is slow
In response to Ryan Novosielski <[EMAIL PROTECTED]>: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > > Though there are probably 10-20 performance pitfalls, the two big problems > > of > > performance that I have seen are: > > > > - Poorly tuned Catalog database -- insertion of Bacula attributes in the > > database tends to be slow. There are probably 5 or ten reasons leading to > > poor DB performance. I'll be working on improving this and documenting it > > over the next 6-9 months. A good part of what you can do is written in the > > manual (Catalog Maintenance chapter). The rest appeared on this list within > > the last month. > > > > - A switch (mostly 3Com switches in my experience) that run in half-duplex > > mode, which slows network traffic down by about a factor of 10. > > Cisco does this just as often, if not more often. A little surprising to > find that the top 2 can't seem to compete on the same level as a D-Link > switch from Radio Shack. ;) If you read Cisco's docs, they make the claim that these problems are per-spec. My understanding of the argument is that if you manually set the speed and duplex, you have disabled auto-negotiation. If the other end tries to auto-negotiate, it will be able to detect the speed, just by dumb luck of how the protocol works, but it will _consistently_ mis- detect the duplex, again because of dumb luck of the protocol Their argument seems to be that this behaviour is per the specs. If a D-Link does it differently, then D-Link is doing it "wrong", even if it's doing it more intuitively. If Cisco is correct, then it would seem as if the spec were written poorly. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula: backup is slow
In response to "Anders Boström" <[EMAIL PROTECTED]>: > >>>>> "AL" == Arno Lehmann <[EMAIL PROTECTED]> writes: > > AL> Still the network is being used and that always involves latencies, > AL> syncronization times, etc. > >> > >> Yes, and that might be the problem. But if it is about latencies > >> and/or synchronization, then it is a bacula performance problem! > > AL> No, what I'm talking about is network fundamentals. Whenever you send > AL> data across a network that takes time, and it takes more time than > AL> dividing xMBit/s by the amount of data. Always. > > Are you talking about the long-fat-pipe problem? This isn't an issue > for us... Linux has a very good TCP-implementation, with > window-scaling, and we are running over a local GE-network with less > than 50us latency. It's not the LFP problem. First off, every TCP packet adds overhead to your data, which means 1G/sec is really only about 600mb/sec in practical measurements. Secondly, that's 50us per packet. Which means there's work to be done on both ends to normalize packet flow, ack, retransmit lossed packets, etc. I believe someone asked you to double-check that all speeds and duplexes are matched, as that's a _very_ common problem (especially, in my experience, with gig HW). If you responded, I missed it, but I recommend that you double-check. However, also read on ... > >> Is bacula limited in performance due to high latency? (Not that we > >> have that problem, but anyway...) > >> > >> Is bacula limited in performance due to synchronization? > > AL> Networks are limited by several factors. That's not something you can > AL> fix, and network throughput is not normally the most limiting factor in > AL> a Bacula setup. > > The specific issue here is about bacula. Has bacula any specific > limitations regarding the network? Please be specific, and don't just > generalize. Well, we're off on the wrong foot, in my opinion. There's no indication anywhere that networking is a problem. You even said that you were seeing indications in network traces that network traffic was flowing fine. As a result, I'm a bit confused as to why there's so much focus on the network, but I expect it's a result of the _huge_ number of people who assume that everything is fine with the network, then blame the application, only to later find out that there's something seriously hosed with their network. I've fallen into that trap, but I don't think you have in this case. I'm going to address what I _think_ is the problem in a response to your other email ... -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula: backup is slow
In response to "Anders Boström" <[EMAIL PROTECTED]>: > I did some new performance-tests: > > All operations are against a directory-tree with 7,255,659,224 bytes > data in 98,025 files. > > | test1 | test2 | test3 | > +---+---+---+ > bacula-fd, no compression, md5: | 10:25 | 10:42 | 10:15 | > bacula-fd, GZIP, md5: | 16:09 | 15:46 | 17:02 | > tar, local (1): | 8:37 | 8:53 | 8:54 | > tar + nc (2): | 9:48 | 9:52 | 9:43 | > tar + gzip + nc (3): | 14:11 | 14:26 | 15:03 | > +---+---+---+ OK. This indicates to me that bacula is doing a damn good job. Only 15% overhead to add checksumming and cataloging features to backup. If you ask me, that's a hell of a deal. > (1) time /bin/sh -c "tar cf - directory | cat >/dev/null" > (2) time /bin/sh -c "tar cf - directory | nc -q 0 backup_server 4711" > (3) time /bin/sh -c "tar czf - directory | nc -q 0 backup_server 4711" > > This round of tests is more in line with what I expected, and the > bacula performance is quite good. The only major difference compared > to my previous tests is that the file-server disc-performance is much > better. It seems like bacula suffers much more than tar from slow > disc-performance on the file-server. backup-server and network > performance don't seem to be an issue at all in the tests, even if > write to TCP is a bit slower than /dev/null. > > However, both tar and bacula suffers from quite large slow-down when > gzip is used. This is on an Athlon 64 X2 3800+ (2-core), running > >50% idle during backup, leading me to believe that there are room for > improvement. But part of the problem might be in the linux-kernel > (2.6.17.8). At least when tar was running, the gzip process seemed to > move from one CPU-core to the other very frequently. Improvement, maybe, but not for Bacula, as far as I can see. If a dual-core system is running at 50%, then 1 core is maxed out. Since the gzip process is serialized, it can only run on one core at a time, which means the CPU is the limiting factor at this time. If you really need more speed, try using GZIP1 or turning compression off. I suppose it'd be possible to make the FD parallelize the compression, but doing that is beyond my skills, and I'm not sure you'd want it anyway. I mean, do you want a backup system that can totally monopolize all your CPU and starve other applications? > Anyway the performance of bacula is good enough for us at the > moment. Well ... I'm so glad you're please. What was your performance goal anyway? If you actually thought you'd get backup throughput at wire speed on 1g network, that was your first mistake. I don't know of any disks that can feed data that fast. Hell, from your experiment above, those disks can feed data at about 13M/sec, which is closer to 100mb than gig, and that's the absolute fastest you're going to get. > I can share one experience with you regarding > disc-performance: > > Both our two Seagate ST3500641AS discs (500Gb Barracuda 7200.9, SATA) > never completes the SMART extended self-test. It worked fine for ~6 > months, and now both run forever. The drives report 30% remaining > of the self-test, then ~2 hours later 10% remaining. After that it > goes up to 40% remaining and the cycle repeats. Also when running the > SMART extended self-test, the IO-performance is more than 10 times > lower than normal (leading to very long backup-time). > > We have been in contact with Seagate about this, and have upgraded to > the latest firmware, without any success. So I guess we will have to > RMA the discs. Are these the disks you're using to test Bacula? Please tell me you're not using hardware known to be broken as a test bed. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula: backup is slow
In response to "Anders Boström" <[EMAIL PROTECTED]>: > >>>>> "BM" == Bill Moran <[EMAIL PROTECTED]> writes: > > BM> In response to "Anders Boström" <[EMAIL PROTECTED]>: > >> I did some new performance-tests: > >> > >> All operations are against a directory-tree with 7,255,659,224 bytes > >> data in 98,025 files. > >> > >> | test1 | test2 | test3 | > >> +---+---+---+ > >> bacula-fd, no compression, md5: | 10:25 | 10:42 | 10:15 | > >> bacula-fd, GZIP, md5: | 16:09 | 15:46 | 17:02 | > >> tar, local (1): | 8:37 | 8:53 | 8:54 | > >> tar + nc (2): | 9:48 | 9:52 | 9:43 | > >> tar + gzip + nc (3): | 14:11 | 14:26 | 15:03 | > >> +---+---+---+ > > BM> OK. This indicates to me that bacula is doing a damn good job. Only > BM> 15% overhead to add checksumming and cataloging features to backup. > BM> If you ask me, that's a hell of a deal. > > >> (1) time /bin/sh -c "tar cf - directory | cat >/dev/null" > >> (2) time /bin/sh -c "tar cf - directory | nc -q 0 backup_server 4711" > >> (3) time /bin/sh -c "tar czf - directory | nc -q 0 backup_server 4711" > >> > >> This round of tests is more in line with what I expected, and the > >> bacula performance is quite good. The only major difference compared > >> to my previous tests is that the file-server disc-performance is much > >> better. It seems like bacula suffers much more than tar from slow > >> disc-performance on the file-server. backup-server and network > >> performance don't seem to be an issue at all in the tests, even if > >> write to TCP is a bit slower than /dev/null. > >> > >> However, both tar and bacula suffers from quite large slow-down when > >> gzip is used. This is on an Athlon 64 X2 3800+ (2-core), running > >> >50% idle during backup, leading me to believe that there are room for > >> improvement. But part of the problem might be in the linux-kernel > >> (2.6.17.8). At least when tar was running, the gzip process seemed to > >> move from one CPU-core to the other very frequently. > > BM> Improvement, maybe, but not for Bacula, as far as I can see. If a > BM> dual-core system is running at 50%, then 1 core is maxed out. Since > > No, this was not the case. Most of the time, both CPU's was idle >30% > of the time, according to top. It's hard to help when your facts keep changing. Further up, you mention that the CPU is >50% idle, now you say that each independent core is >30% idle. This is not meant as an attack, but you will get absolutely nowhere in performance optimization if you can't get the details right. Details are terribly important. > BM> the gzip process is serialized, it can only run on one core at a time, > BM> which means the CPU is the limiting factor at this time. > > gzip on this computer, on one CPU, reach about 18 Mbyte/s. bacula with > gzip only reach ~7.7 Mbyte/s. This leads me to believe that there are > room for improvement. Again, the story changes. Above, you indicate that tar+gzip ran about 15% faster than bacula with gzip, which seems reasonable. Now you're saying that gzip is ~twice as fast as Bacula + gzip. Where did this new number come from? Are you taking in to account networking on this new test? One thing to consider is that network has an impact, even when the physical speed of the network is faster than the gzip process. There is extra processing involved in packing up the packets, and delays (even if minimal) introduced by networking add latency to the process that isn't always intuitive. > BM> What was your performance goal anyway? If you actually thought you'd > BM> get backup throughput at wire speed on 1g network, that was your first > BM> mistake. I don't know of any disks that can feed data that fast. Hell, > BM> from your experiment above, those disks can feed data at about 13M/sec, > BM> which is closer to 100mb than gig, and that's the absolute fastest > BM> you're going to get. > > No, I don't expect better performance than the disc-performance. In > fact, a lot lower than disc-performance is acceptable. The 7.7 Mbyte/s > we reach now is OK. However, full backup times are long, but as long > as we can run a full backup in less than 48 hours, it's OK. Again, I'm glad it's working for you.
Re: [Bacula-users] bacula: backup is slow
In response to "Anders Boström" <[EMAIL PROTECTED]>: > >> gzip on this computer, on one CPU, reach about 18 Mbyte/s. bacula with > >> gzip only reach ~7.7 Mbyte/s. This leads me to believe that there are > >> room for improvement. > > BM> Again, the story changes. Above, you indicate that tar+gzip ran about > BM> 15% faster than bacula with gzip, which seems reasonable. Now you're > BM> saying that gzip is ~twice as fast as Bacula + gzip. Where did this new > BM> number come from? Are you taking in to account networking on this new > BM> test? > > If I state that "gzip on this computer, on one CPU, reach about 18 > Mbyte/s", I mean just that, nothing else. To clarify, this means that > pure gzip-performance on this computer, using just one gzip-process, > is 18 Mbyte/s. If you would be kind enough to humor me ... Please create a file (or use an existing one) of notable size: few hundred meg. Put the file on the disk and time gzipping it. Run it 5 times. Create a memory filesystem and repeate the gzip tests with the file living on the mfs and the gzipped target existing on the mfs. I have a suspicion that your drives are the limiting factor in this. The above tests should confirm or deny that theory. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula: backup is slow
In response to "Anders Boström" <[EMAIL PROTECTED]>: > >>>>> "BM" == Bill Moran <[EMAIL PROTECTED]> writes: > > BM> In response to "Anders Boström" <[EMAIL PROTECTED]>: > >> >> gzip on this computer, on one CPU, reach about 18 Mbyte/s. bacula with > >> >> gzip only reach ~7.7 Mbyte/s. This leads me to believe that there are > >> >> room for improvement. > >> > BM> Again, the story changes. Above, you indicate that tar+gzip ran about > BM> 15% faster than bacula with gzip, which seems reasonable. Now you're > BM> saying that gzip is ~twice as fast as Bacula + gzip. Where did this new > BM> number come from? Are you taking in to account networking on this new > BM> test? > >> > >> If I state that "gzip on this computer, on one CPU, reach about 18 > >> Mbyte/s", I mean just that, nothing else. To clarify, this means that > >> pure gzip-performance on this computer, using just one gzip-process, > >> is 18 Mbyte/s. > > BM> If you would be kind enough to humor me ... > > BM> Please create a file (or use an existing one) of notable size: few > BM> hundred meg. > > BM> Put the file on the disk and time gzipping it. Run it 5 times. > > BM> Create a memory filesystem and repeate the gzip tests with the > BM> file living on the mfs and the gzipped target existing on the mfs. > > BM> I have a suspicion that your drives are the limiting factor in this. > BM> The above tests should confirm or deny that theory. > > I've already done this, but did it again. The results are the same, > disc and tmpfs gives the same results, about 18 Mbyte/s. And I would > have been *very* surprised if they had differed, as the disc-tests > are running out of memory in test 2-5 (due to caching). If the system is caching the entire write operation in RAM, then you're set up for a major disaster some day. Even if there's enough RAM to cache the read op, the write op will ALWAYS require disk writes. If those two operations are taking the same amount of time, then either you're testing wrong, or something is really weird with your setup, or you have the fastest hard drives on this planet. Additionally, the point was to _not_ allow it to use cache for the disk ops, which I didn't communicate clearly -- my apologies. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Recycle a volume by deleting the datas ?
In response to ctobini <[EMAIL PROTECTED]>: > > Hello, > > It seems that a volume life time in bacula is in fact its life time in the > catalog database. So, I would like to know is there would be possible to > fully recycle a volume (restart to the beginning of the volume by deleting > the datas). You can set retention times and configure the volumes to automatically recycle and Bacula will take care of it for you. Or you can use the purge or prune commands from bconsole, depending on your config. Some reading on these topics in the Bacula manual should help. If you get confused, ask. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Irregular backups
In response to Paul Constable <[EMAIL PROTECTED]>: > Hi alll, > I have the need to backup a laptop that is connected to the network at > irregular intervals, which precludes any form of scheduling. > > Does anyone have a means of backing up at, say an hour after logging into the > network, or of connecting to the server. > > I have a user who is frequently away from the network and never leaves the > laptop connected to the network overnight. Therefore I need to stimulate the > backup automatically without the users intervention or mine. > > The laptop runs windoze, so I need the means of triggering a script, etc. I've managed to work around this with clever scheduling. By setting reschedule on error to 1 hour, setting the backup to start at 9:00 AM and setting the system to reschedule 9 times, we get most of our mobile users most of the time that they are in. The upshot is that it tries to initiate a backup every hour between 9:00 AM and 6:00 PM. Not the perfect solution, but it's working OK for us. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Can't write to the tape
In response to "Greg Little" <[EMAIL PROTECTED]>: > Here is where I am at now. I can use mt and tar and dump to the tape, > but bconsole shows: > > Device status: > Autochanger "turing" with devices: >"LTO-2" (/dev/nst0) > Device "FileStorage" (/bacula/FileVol) is not open or does not exist. > Device "LTO-2" (/dev/nst0) open but no Bacula volume is mounted. > Slot 6 is loaded in drive 0. > Total Bytes Read=0 Blocks Read=0 Bytes/block=0 > Positioned at File=0 Block=0 > > > > When I tried to spool to tape, it asks to mount the volume, although the > tape is already in there and I can write to it with mt and tar. If you wrote to the tape with tar _after_ labelling it, then you destroyed the Bacula label. You'll have to relabel the media before you can use it. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Why does bacula not believe that it's week 44, and how do I figure out what's wrong?
Bacula seem not to understand that today is week 44 of the year. It seemed to understand that last week was week 43, as it ran the job that was scheduled for that week. Here's the schedule that applies: Run = Level=Full Pool=OffSite Storage=Ultrium thursday w03,w07,w11,w15,w19,w23,w27,w31,w35,w39,w43,w47,w51 at 23:05 Run = Level=Full Pool=OffSiteRotation Storage=Ultrium thursday w01,w02,w04,w05,w06,w08,w09,w10,w12,w13,w14,w16,w17,w18,w20,w21,w22,w24,w25,w26,w28,w29,w30,w32,w33,w34,w36,w37,w38,w40,w41,w42,w44,w45,w46,w48,w49,w50,w52,w53 at 23:05 Server date/time seems correct: *time 03-Nov-2006 09:16:47 Bacula insists that it needs a volume from the "OffSite" pool. As a result, 240G of backups failed to run last night, and I'm having to manually run/modify them this morning (because the tape I need to use is part part of the OffSiteRotation pool, in accordance with our schedule). Is there something wrong with my schedule that I'm simply not seeing? A much more important question to me is "How do I diagnose this?" How can I peek in to Bacula and see what week it thinks it is? bconsole's time command isn't very helpful on this count. -- Bill Moran Collaborative Fusion Inc. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Is Bacula for me: Videography business backups
On Wed, 08 Nov 2006 18:49:30 -0600 [EMAIL PROTECTED] wrote: > Arno Lehmann <[EMAIL PROTECTED]> writes: > > > Currently, the windows FD is running stable and reliable IMO. > > Unfortunately, I have some problems with VSS backups (windows' name for > > volume snapshots) where files can't be accessed when I do a snapshot but > > can be accessed without VSS. > > > > For backups of data files not in use during backup operations, this > > should not be a problem. > > I'm not sure I'm following you here. Are you saying I would not be > able to backup files that were in use? Or just not in some special > circumstances...? > > I'm not sure what role VSS plays or when it might be used but I would > need to routinely be able to backup files that are in use. Depends on your definition of "in use". If the system has the file locked for writing, you'll have to enable VSS to get a backup of it while it's write locked. If the application has it open read-only, you'll get a good backup either way. On the flip side, I've been using VSS to back up Windows workstations for a few months now with no problems, and all files are able to be backed up no matter how they're opened. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Job not appearing in console
In response to luket <[EMAIL PROTECTED]>: > > Good morning > > I have installed bacula-1.38.11 onto a CentOS server. > > It is installed and appears to be running fine and backing up to disk with > no problems. I have added 3 clients: A Windows client, and 2 linux clients > one of which is on a seperate (internal) network with firewall permissions > granted for ports 9001, 9002 & 9003. > > We I enter bconsole and type run I get the list of clients but one is > missing. I have checked through my configuration files and all looks OK to > me. When I take out the working linux job and restart bacula the other one > appears and runs through fine so I know it isn't a problem with the clients > setup. Sounds like those two jobs are cancelling each other out. They don't share the same name, do they? > I have looked around and can't find anything but was wondering if there was > a setting which specifies how many will appear on the list. I did have > Bacula installed on another server for testing purposes, this was on CentOS > also with the same version of Bacula, same ./configure command, and > everything I can see is fine. Does anybody know of any settings that need > to be changed in order for all the jobs to appear on the list? I'm pretty sure bconsole always lists all jobs, but I can guarantee that it will show 26 minimum, as that's how many I have on one server here. I would guess that you've got some obscure error in your config file. Strip the passwords out of it and paste it into a reply to this thread. We probably only need to the director's config to diagnose this. -- Bill Moran Collaborative Fusion Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Job not appearing in console
In response to "Luke Taylor" <[EMAIL PROTECTED]>:
>
> Thanks for replying, FYI the jobs don't have the same name but below is the
> directors configuration file excluding passwords.
>
> FYI I have since addded another job resource which also doesn't appear on
> the list.the jobs in question are hignlighted in boldbut just incase
> it doesn't get displayed I have put the job resource just below.
Please don't rely on HTML to communicate for you. The bold doesn't
display on my MUA, and I had to do a good bit of fscking around to
figure out which jobs you were having trouble with. Do I have the
correct two?
> Job {
> Name = "Twiki Server"
> JobDefs = "Twiki"
> Type = Backup
> Level = Full
> Client = twiki-fd
> FileSet = "Twiki Set"
> Schedule = "twiki-fd"
> Write Bootstrap = "/usr/local/bacula/bin/working/twiki.bsr
> }
>
> Job {
> Name = "Autonet Server"
> JobDefs = "Autonet"
> Type = Backup
> Level = Full
> Client = jspautonet-fd
> FileSet = "Webserver Set"
> Schedule = "jspautonet-fd"
> Write Bootstrap = "/usr/local/bacula/bin/working/jspautonet.bsr
> }
I looked at this yesterday and couldn't find anything wrong, then looked
at it again this morning and still can see anything obviously wrong.
So I'm guessing. I don't believe that spaces in job names would cause
any problems, but it's the only thing I can see that stands out. Trying
removing the spaces from your various names (JobDefs, Job, FileSet) and
see if the problem persists. Hope this isn't a wild goose chase for
you, but I don't have any other ideas.
Another good idea would be to start the director in the foreground with
debugging turned on and see if it reports anything helpful. See the
docs for the details on how to do this.
--
Bill Moran
Collaborative Fusion Inc.
IMPORTANT: This message contains confidential information and is intended only
for the individual named. If the reader of this message is not an intended
recipient (or the individual responsible for the delivery of this message to an
intended recipient), please be advised that any re-use, dissemination,
distribution or copying of this message is prohibited. Please notify the
sender immediately by e-mail if you have received this e-mail by mistake and
delete this e-mail from your system.
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Job not appearing in console
In response to "Luke Taylor" <[EMAIL PROTECTED]>:
> Hi I have made the amendments but still no joy... The file now contains the
> following for the two clients not showing up.
Have you tried starting the director in the foreground with debugging?
>
> JobDefs {
> Name = "Autonet"
> Type = Backup
> Level = Full
> Client = jspautonet-fd
> FileSet = "WebserverSet"
> Schedule = "jspautonet-fd"
> Storage = File
> Messages = Standard
> Pool = autonetpool
> Priority = 10
> }
>
> Job {
> Name = "AutonetServer"
> JobDefs = "Autonet"
> Type = Backup
> Level = Full
> Client = jspautonet-fd
> FileSet = "WebserverSet"
> Schedule = "jspautonet-fd"
> Write Bootstrap = "/usr/local/bacula/bin/working/jspautonet.bsr
> }
>
> FileSet {
> Name = "WebserverSet"
> Include {
> Options {
> }
> File = /usr/local/apache/conf/
> File = /usr/local/apache/conf/ssl/
> File = /usr/local/tomcat/webapps/
> File = /usr/local/tomcat/conf/Catalina/localhost/
> File = /usr/local/etc/tomcat/
> }
> }
>
> Schedule {
> Name = "jspautonet-fd"
> Run = Level=Incremental Pool=autonetpool Monday at 20:30
> Run = Level=Incremental Pool=autonetpool Tuesday at 20:30
> Run = Level=Incremental Pool=autonetpool Wednesday at 20:30
> Run = Level=Incremental Pool=autonetpool Thursday at 20:30
> Run = Level=Incremental Pool=autonetpool Friday at 20:30
> }
>
> Client {
> Name = jspautonet-fd
> Address = 10.210.17.177
> FDPort = 9102
> Catalog = MyCatalog
> Password = ""
>
> File Retention = 30 days
> Job Retention = 6 months
> AutoPrune = yes
> }
>
> Pool {
> Name = autonetpool
> Pool Type = Backup
> Recycle = yes
> AutoPrune = yes
> Volume Retention = 365 days
> Accept Any volume = yes
> }
>
> And
>
> JobDefs {
> Name = "MXdev"
> Type = Backup
> Level = Full
> Client = mxdev-fd
> FileSet = "Mxdev Set"
> Schedule = "mxdev-fd"
> Storage = File
> Messages = Standard
> Pool = mxdevpool
> Priority = 10
> }
>
> Job {
> Name = "Backupmxdev"
> JobDefs = "MXdev"
> Type = Backup
> Level = Full
> Client = mxdev-fd
> FileSet = "Mxdev Set" # Luke desktop
> Schedule = "mxdev-fd"
> Write Bootstrap = "/usr/local/bacula/bin/working/mxdev.bsr"
> }
>
> FileSet {
> Name = "Mxdev Set"
> Include {
> Options {
> }
> File = /usr/local/luke
> }
> }
>
> Schedule {
> Name = "mxdev-fd"
> Run = Level=Incremental Pool=mxdevpool Monday at 20:15
> Run = Level=Incremental Pool=mxdevpool Tuesday at 20:15
> Run = Level=Incremental Pool=mxdevpool Wednesday at 20:15
> Run = Level=Incremental Pool=mxdevpool Thursday at 20:15
> Run = Level=Incremental Pool=mxdevpool Friday at 20:15
> }
>
> Client {
> Name = mxdev-fd
> Address = 10.26.13.112
> FDPort = 9102
> Catalog = MyCatalog
> Password = ""
> File Retention = 30 days# 30 days
> Job Retention = 6 months# six months
> AutoPrune = yes # Prune expired Jobs/Files
> }
>
>
> Pool {
> Name = mxdevpool
> Pool Type = Backup
> Recycle = yes
> AutoPrune = yes
> Volume Retention = 365 days
> Accept Any Volume = yes
> }
>
> All that appears in the bconcole is:
>
> *run
> Using default Catalog name=MyCatalog DB=bacula
> A job name must be specified.
> The defined Job resources are:
> 1: Client1
> 2: Twiki Server
> 3: Backupwindows
> 4: BackupCatalog
> 5: RestoreFiles
> Select Job resource (1-5):
>
> HOWEVER, if I remove the "Twiki Server" job frrom the director configuration
> file the Autonet job appears (there is still no sign of the mxdev job) and
> completes with no errors. If I remover the Autonet and Twiki jobs the mxdev
> job appears on the list.it looks to me like there is a setting somewhere
> on the system that defines how many jobs can be displayed on that list but I
> can't find anything anywhere.
>
> Any further assistance would be much appreciated.
>
> Regards
> Luke
>
> -Original Message-
> From: Bill Moran [mailto:[EMAIL PROTECTED]
> Sent: 15 November 2006 14:38
> To: [EMAIL PROTECTED]
> Cc: bacula-users@lists.sourceforge.net
> Subject: Re: [Bacula-users] Job not appearing in console
>
> In r
Re: [Bacula-users] Job not appearing in console
In response to "Luke Taylor" <[EMAIL PROTECTED]>:
> I tried with ./bacula start -f -100 but don't get anything, I am having a
> look around to find out how to do it.
>
> I will update further if/when I manage to find instructions.
bacula-dir -f -d100
(You may have to give -c if it doesn't find the config file)
If you do bacula-dir --help, it'll give you a list of options. Not
because it understands --help, but because it does that when it doesn't
understand its options.
> -Original Message-
> From: Bill Moran [mailto:[EMAIL PROTECTED]
> Sent: 16 November 2006 13:58
> To: [EMAIL PROTECTED]
> Cc: bacula-users@lists.sourceforge.net
> Subject: Re: [Bacula-users] Job not appearing in console
>
> In response to "Luke Taylor" <[EMAIL PROTECTED]>:
>
> > Hi I have made the amendments but still no joy... The file now
> > contains the following for the two clients not showing up.
>
> Have you tried starting the director in the foreground with debugging?
>
> >
> > JobDefs {
> > Name = "Autonet"
> > Type = Backup
> > Level = Full
> > Client = jspautonet-fd
> > FileSet = "WebserverSet"
> > Schedule = "jspautonet-fd"
> > Storage = File
> > Messages = Standard
> > Pool = autonetpool
> > Priority = 10
> > }
> >
> > Job {
> > Name = "AutonetServer"
> > JobDefs = "Autonet"
> > Type = Backup
> > Level = Full
> > Client = jspautonet-fd
> > FileSet = "WebserverSet"
> > Schedule = "jspautonet-fd"
> > Write Bootstrap = "/usr/local/bacula/bin/working/jspautonet.bsr
> > }
> >
> > FileSet {
> > Name = "WebserverSet"
> > Include {
> > Options {
> > }
> > File = /usr/local/apache/conf/
> > File = /usr/local/apache/conf/ssl/
> > File = /usr/local/tomcat/webapps/
> > File = /usr/local/tomcat/conf/Catalina/localhost/
> > File = /usr/local/etc/tomcat/
> > }
> > }
> >
> > Schedule {
> > Name = "jspautonet-fd"
> > Run = Level=Incremental Pool=autonetpool Monday at 20:30
> > Run = Level=Incremental Pool=autonetpool Tuesday at 20:30
> > Run = Level=Incremental Pool=autonetpool Wednesday at 20:30
> > Run = Level=Incremental Pool=autonetpool Thursday at 20:30
> > Run = Level=Incremental Pool=autonetpool Friday at 20:30 }
> >
> > Client {
> > Name = jspautonet-fd
> > Address = 10.210.17.177
> > FDPort = 9102
> > Catalog = MyCatalog
> > Password = ""
> >
> > File Retention = 30 days
> > Job Retention = 6 months
> > AutoPrune = yes
> > }
> >
> > Pool {
> > Name = autonetpool
> > Pool Type = Backup
> > Recycle = yes
> > AutoPrune = yes
> > Volume Retention = 365 days
> > Accept Any volume = yes
> > }
> >
> > And
> >
> > JobDefs {
> > Name = "MXdev"
> > Type = Backup
> > Level = Full
> > Client = mxdev-fd
> > FileSet = "Mxdev Set"
> > Schedule = "mxdev-fd"
> > Storage = File
> > Messages = Standard
> > Pool = mxdevpool
> > Priority = 10
> > }
> >
> > Job {
> > Name = "Backupmxdev"
> > JobDefs = "MXdev"
> > Type = Backup
> > Level = Full
> > Client = mxdev-fd
> > FileSet = "Mxdev Set" # Luke desktop
> > Schedule = "mxdev-fd"
> > Write Bootstrap = "/usr/local/bacula/bin/working/mxdev.bsr"
> > }
> >
> > FileSet {
> > Name = "Mxdev Set"
> > Include {
> > Options {
> > }
> > File = /usr/local/luke
> > }
> > }
> >
> > Schedule {
> > Name = "mxdev-fd"
> > Run = Level=Incremental Pool=mxdevpool Monday at 20:15
> > Run = Level=Incremental Pool=mxdevpool Tuesday at 20:15
> > Run = Level=Incremental Pool=mxdevpool Wednesday at 20:15
> > Run = Level=Incremental Pool=mxdevpool Thursday at 20:15
> > Run = Level=Incremental Pool=mxdevpool Friday at 20:15 }
> >
> > Client {
> > Name = mxdev-fd
> > Address = 10.26.13.112
> > FDPort = 9102
> > Catalog = MyCatalog
> > Password = ""
> > File Retention = 30 days# 30 days
> > Job Retention = 6 months# six months
> > AutoPrune = yes
Re: [Bacula-users] Job not appearing in console
In response to "Luke Taylor" <[EMAIL PROTECTED]>: > OK I have run the command you gave me (stopped bacula first) and get the > following output: > [EMAIL PROTECTED] bin]# ./bacula-dir -f -d100 > bacula-dir: dird.c:129 Debug level = 100 > bacula-dir: inc_conf.c:487 set wildfile 0x81c4850 size=1 *.obj > bacula-dir: inc_conf.c:487 set wildfile 0x81c4850 size=2 *.exe > bacula-dir: mysql.c:75 db_open first time > bacula-dir: mysql.c:135 mysql_init done > bacula-dir: mysql.c:156 mysql_real_connect done > bacula-dir: mysql.c:158 db_user=bacula db_name=bacula db_password= > unixutil-dir: jcr.c:116 read_last_jobs seek to 188 > unixutil-dir: jcr.c:123 Read num_items=10 > unixutil-dir: bnet_server.c:83 Addresses host[ipv4:0.0.0.0:9101] > > It then just hangs at this point (I have left it for about 10 mins to make > sure) and won't start Bacula.I have also tried doing this with bacula-sd > & bacula-fd still running and get the same result. Not really sure if this > makes things any clearer to be honestI am a bit lost :( Doesn't seem to have any problems loading the config. From another console, log in with bconsole and try to run a job and see what messages it generates. -- Bill Moran Collaborative Fusion Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Preventing windows bacula-fd from accessing some files
In response to Jaime Ventura <[EMAIL PROTECTED]>: > Hello. >I need to backup some directories on windows client used by a > user AA. >The problem is that the user AA have some confidential files > which cant be accessible by bacula. >I thought I could create a user BB having permissions to access > specific directories(the ones that should by backed up) and having > bacula-fd running as user BB. The problem with this solution is that > bacula-fd runnig as BB cant have access to user AA desktop, and so i > cant get the icon on AA' systray. > How can I solve this? >As far as i can see, there isn't a bacula monitor for Windows? >Is there any client side ACL's. >Thanks for you help Are you sure? Have you tried? When I installed Bacula on W$ machines, it runs as some meta-user that has read rights to the entire system. Are you doing some sort of custom install? Why not just use the meta-user built into Windows? -- Bill Moran Collaborative Fusion Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Preventing windows bacula-fd from accessing some files
In response to Jaime Ventura <[EMAIL PROTECTED]>: > Hi Bill, > thank you for your reply. > I know that if install the windows client using default options, it > will run as a service using the "system" user. > Since bacula-fd is running using the "system" user, it will be able > to access every file on the pc, rigth? > I need to limit the bacula-fd to access to just a few files ( the > ones I need to backup). The thing is, the pc user (AAA in my previous > email) have personal information on it and he doesn't want bacula to > able to access it. > So I thought of running bacula-fd as a BBB user having permissions > to access only files I need to backup. That way, bacula-fd doesnt have > access to all files on the system, but just the ones I need to backup. > This should work just fine. > But, having bacula-fd running as BBB user, it wont have permissions > to put the icon tray on the AAA systemTray when he is logged in. > I could solve this problem if bacula had a "bacula-monitor" for > windows, which it does not. > So, i don't really know how to solve this :( > Once again, thanks I have a feeling that you're on the wrong road for the wrong reasons. First, you need to clearly establish the status of those files: 1) Are they important data? 2) Are they confidential data? If the answer to #1 is "yes", then you need to back them up. It sounds to me that the answer to #2 is "yes" and you're using that as a reason _not_ to back the data up, which is a _VERY_ bad idea. I could go on and on about foolish assumptions such as the assumption that your Windows client is more secure than the Bacula server ... However, I'll cut to the chase: 1) If the data is just "personal", then simply exclude those files from the backup fileset using a wildcard if necessary. 2) If the data is "confidential", then you're putting the cart before the horse by trying to teach bacula not to back it up. Instead, use some sort of file-level encryption, such as one of pgp's tools. Then, you can even back up the encrypted files if it makes sense to do so, without sacrificing their confidentiality. The approach you're taking is like trying to fix a flat tire by changing the spark plugs. -- Bill Moran Collaborative Fusion Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Restore: Building directory tree takes forever
In response to Frank Mueller <[EMAIL PROTECTED]>: > > I'm having trouble restoring files from a backup job that contains > 1,933,009 files. Building the directory tree seems to take forever... at > least I had to interupt it after 30 hours of 100% CPU-usage :-( Wow ... Building directory tree for JobId 4868 ... +++ 1 Job, 1,316,538 files inserted into the tree. Took about 1 minute. I didn't see any excessive CPU usage during the build, but disks ran at 100% the whole time (loading the database data off disk). Fast hardware helps. This system is a Dell 1850 with 2G of RAM and SCSI 10,000 RPM disks. Are you sure it was CPU-bound during the build, and if so, what process was CPU bound? The DB server? The director? Also, I'm using PostgreSQL as the DB backend. I've got a bit of experience tuning Postgres, so I've made sure that it uses all the system RAM for caching. Is it possible your DB server would benefit from some tuning? -- Bill Moran Collaborative Fusion Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Restore: Building directory tree takes forever
In response to Frank Mueller <[EMAIL PROTECTED]>: > Hi there, > > ok, my machine here is not comparable to your hardware, but it still > should work at least within minutes, not hours or even days I think. > Some additional information: > > CPU: P4 1,7 GHZ > RAM: 1GB > ATA-Harddisks > OS: FreeBSD > DB: Mysql 4.0.27 > bacula: 1.38.11_1 > > While building the directory tree, top tells me > > CPU states: 98.9% user, 0.0% nice, 1.1% system, 0.0% interrupt, 0.0% idle > > 5463 bacula 3 118 0 14540K 12124K RUN 1:14 96.68% bacula-dir > 457 mysql 6 20 0 57580K 32720K kserel 5:15 0.29% mysqld > > Looks like the director is using all of CPU, not the database, while it > only takes about 60MB of RAM. > > I already checked the db indexes and found em all in place... > > I have no idea what the hell's going wrong on that machine. I'm no MySQL expert, but isn't there something that needs to go in the my.cnf or whatever in order for MySQL to operate efficiently on large data sets? -- Bill Moran Collaborative Fusion Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Restore: Building directory tree takes forever
On Sat, 18 Nov 2006 11:47:53 +0100 Frank Mueller <[EMAIL PROTECTED]> wrote: > Hi there, > > memory usage is at 60MB out of 1GB. No swapping at all. > > By the way. Shouldn't it be possible to edit the database to delete some > File entries to make things work? > I already took a look at the db with phpmyadmin, but I do not get the db > structure, coz I do not know much about mysql and bacula. > I know that there's a very huge directory in that contains over 2 Mio > SPAM-Mails and I think completely deleteing that directory might be the > solution for my prob. > Can anyone lead me to the correct entries to delete from the db? Absolutely do a dump of the database before you try anything! However, you may be on to something. Long term you really need to find the performance problem and get it fixed, but you may be able to speed things up some short term. Bacula keeps the pathnames and filenames in seperate tables from the records of what files were backed up. The following procedure should allow you to clean up the excess files: 1) SELECT pathid FROM path WHERE path='/path/to/files/you/want/to/delete' 2) DELETE FROM file WHERE pathid= I take no responsibility if this deletes more than you want. Take a dump of the database before you try this. Restore the dump when you're done. The table structure is documented: http://bacula.org/developers/Catalog_Services.html#SECTION000101600 Good luck. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] restore takes "forever"
On Wed, 22 Nov 2006 23:11:59 +0100 Andras Horvai <[EMAIL PROTECTED]> wrote: > Hi, > > Well thanks for your answer. I will change my volumes size useing the > Max Volume Bytes setting. What settings do you reccomend if you used > this feature? If I issue the list volumes command in console I got > this: > > +-+---+---++--+--+-+--+---+-+-+ > | MediaId | VolumeName| VolStatus | VolBytes | VolFiles | > VolRetention | Recycle | Slot | InChanger | MediaType | LastWritten > | > +-+---+---++--+--+-+--+---+-+-+ > | 3 | ServersDiff01 | Used | 51,590,350,000 | 12 | > 432,000 | 1 |0 | 1 | ServersFile | 2006-11-17 00:24:09 | > | 4 | ServersDiff02 | Append| 33,334,244,961 |7 | > 432,000 | 1 |0 | 1 | ServersFile | 2006-11-22 23:01:41 | > +-+---+---++--+--+-+--+---+-+-+ > > What does VolFiles mean? I didn't find it in the documentation. I've been wondering about that. I wish one of the developers would chime in here. I seem to remember that a trick used on tape drives is to write an EOF marker every so often, then when restoring, the drive can quickly seek X EOF markers ahead before it has to slow down to read through the data. If I'm understanding this correctly, there's no reason Bacula can't do the same thing with file volumes. If it writes an EOF marker every 4G (which it seems to, based on your output) it can seek() to the within 4G of the data it needs, then it only needs to read() through a maximum of 4G to get the data. Again, I wish a developer would chime in and comment on whether I'm correct or not. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Filesystem change prohibited. Will not descend into
On Thu, 23 Nov 2006 09:09:13 +0100 Manuel Staechele <[EMAIL PROTECTED]> wrote: > Hello List, > > i do not understand the message: > > server-fd: Filesystem change prohibited. Will not descend into /home > > did bacula save this directory or not? It came only this message, the > job it self is OK without warnings, so i think it doesn't matter. > > If bacula did not save the directory, how can i say to do so? No, bacula did not save that directory. Despite the fact that you did not include any config files, I can assume that your backup job includes a "File = /". Bacual does not cross mount barriers by default. See the relevent docs on the onefs parameter: http://bacula.org/rel-manual/Configuring_Director.html#SECTION000147000 The quick answer is: 1) Either set onefs = no 2) or add File = /home/ to your FileSet 3) Do _not_ do both or you'll backup /home twice - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] restore takes "forever"
On Thu, 23 Nov 2006 10:38:41 GMT Martin Simmons <[EMAIL PROTECTED]> wrote: > >>>>> On Wed, 22 Nov 2006 17:26:47 -0500, Bill Moran said: > > > > On Wed, 22 Nov 2006 23:11:59 +0100 > > Andras Horvai <[EMAIL PROTECTED]> wrote: > > > > > What does VolFiles mean? I didn't find it in the documentation. > > > > I've been wondering about that. I wish one of the developers would chime > > in here. > > > > I seem to remember that a trick used on tape drives is to write an EOF > > marker every > > so often, then when restoring, the drive can quickly seek X EOF markers > > ahead before > > it has to slow down to read through the data. > > > > If I'm understanding this correctly, there's no reason Bacula can't do the > > same > > thing with file volumes. If it writes an EOF marker every 4G (which it > > seems to, > > based on your output) it can seek() to the within 4G of the data it needs, > > then > > it only needs to read() through a maximum of 4G to get the data. > > Using EOF markers like that in a file won't work, because there is no fast way > of seeking to an EOF marker (unlike on a tape). ?? If Bacula knows it's writing a file marker every 4G, why can't it just use fseek() to skip forward? > However, it is easier than that: Bacula could seek directly to the right place > in the file. IIRC, there is some code to do that but it was disabled because > it didn't quite work. Well, that's an obvious problem ... - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] windows xp client backup issue
-temp-sd
> SDPort = 9103 # Director's port
> WorkingDirectory = "/usr/local/bacula/working"
> Pid Directory = "/usr/local/bacula/var/run"
> Maximum Concurrent Jobs = 20
> }
>
> #
> # List Directors who are permitted to contact Storage daemon
> #
> Director {
> Name = backup-temp-dir
> Password = "x"
> }
>
> #
> # Restricted Director, used by tray-monitor to get the
> # status of the storage daemon
> #
> Director {
> Name = backup-temp-mon
> Password = "x"
> Monitor = yes
> }
>
> #
> # Devices supported by this Storage daemon
> # To connect, the Director's bacula-dir.conf must have the
> # same Name and MediaType.
> #
>
> Device {
> Name = FileStorage
> Media Type = File
> Archive Device = /usr/local/bacula/bacula-vol
> LabelMedia = yes; # lets Bacula label unlabeled media
> Random Access = Yes;
> AutomaticMount = yes; # when device opened, read it
> RemovableMedia = no;
> AlwaysOpen = no;
> }
>
> #
> # An autochanger device with two drives
> #
> #Autochanger {
> # Name = Autochanger
> # Device = Drive-1
> # Device = Drive-2
> # Changer Command = "/home/kern/bacula/bin/mtx-changer %c %o %S %a %d"
> # Changer Device = /dev/sg0
> #}
>
> #Device {
> # Name = Drive-1 #
> # Drive Index = 0
> # Media Type = DLT-8000
> # Archive Device = /dev/nst0
> # AutomaticMount = yes; # when device opened, read it
> # AlwaysOpen = yes;
> # RemovableMedia = yes;
> # RandomAccess = no;
> # AutoChanger = yes
> # # Enable the Alert command only if you have the mtx package loaded
> # Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
> #}
>
> #Device {
> # Name = Drive-2 #
> # Drive Index = 1
> # Media Type = DLT-8000
> # Archive Device = /dev/nst1
> # AutomaticMount = yes; # when device opened, read it
> # AlwaysOpen = yes;
> # RemovableMedia = yes;
> # RandomAccess = no;
> # AutoChanger = yes
> # # Enable the Alert command only if you have the mtx package loaded
> # Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
> #}
>
> #
> # A Linux or Solaris tape drive
> #
> #Device {
> # Name = DDS-4#
> # Media Type = DDS-4
> # Archive Device = /dev/nst0
> # AutomaticMount = yes; # when device opened, read it
> # AlwaysOpen = yes;
> # RemovableMedia = yes;
> # RandomAccess = no;
> ## Changer Command = "/usr/local/bacula/etc/mtx-changer %c %o %S %a %d"
> ## Changer Device = /dev/sg0
> ## AutoChanger = yes
> # # Enable the Alert command only if you have the mtx package loaded
> ## Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
> #}
>
> #
> # A FreeBSD tape drive
> #
> #Device {
> # Name = DDS-4
> # Description = "DDS-4 for FreeBSD"
> # Media Type = DDS-4
> # Archive Device = /dev/nsa1
> # AutomaticMount = yes; # when device opened, read it
> # AlwaysOpen = yes
> # Offline On Unmount = no
> # Hardware End of Medium = no
> # BSF at EOM = yes
> # Backward Space Record = no
> # Fast Forward Space File = no
> # TWO EOF = yes
> #}
>
> #
> # A OnStream tape drive.
> # You need the kernel osst driver 0.9.14 or later, and
> # do "mt -f /dev/nosst0 defblksize 32768" once as root.
> #
> #Device {
> # Name = OnStream
> # Description = "OnStream drive on Linux"
> # Media Type = OnStream
> # Archive Device = /dev/nst0
> # AutomaticMount = yes; # when device opened, read it
> # AlwaysOpen = yes
> # Offline On Unmount = no
> ## The min/max blocksizes of 32768 are *required*
> # Minimum Block Size = 32768
> # Maximum Block Size = 32768
> #}
>
> #
> # A DVD device
> #
> #Device {
> # Name = "DVD-Writer"
> # Media Type = DVD
> # Archive Device = /dev/hdc
> # LabelMedia = yes; # lets Bacula label unlabeled media
> # Random Access = Yes;
> # AutomaticMount = yes; # when device opened, read it
> # RemovableMedia = yes;
> # AlwaysOpen = no;
> # MaximumPartSize = 800M;
> # RequiresMount = yes;
> # MountPoint = /mnt/cdrom;
> # MountCommand = "/bin/mount -t iso9660 -o ro %a %m";
> # UnmountCommand = "/bin/umount %m";
> # SpoolDirectory = /tmp/backup;
> # WritePartCommand = "/etc/bacula/dvd-handler %a write %e %v"
> # FreeSpaceCommand = "/etc/bacula/dvd-handler %a free"
> #}
>
> #
> # For OpenBSD OS >= 3.6
> #
> #Device {
> # Name = DDS-3
> # Media Type = DDS-3
> # Archive Device = /dev/nrst0
> # Use MTIOCGET= no
> # BSF at EOM = yes
> # TWO EOF = no
> # AutomaticMount = yes;
> # AlwaysOpen = yes;
> # RemovableMedia = yes;
> # RandomAccess = no;
> #}
>
> #
> # A very old Exabyte with no end of media detection
> #
> #Device {
> # Name = "Exabyte 8mm"
> # Media Type = "8mm"
> # Archive Device = /dev/nst0
> # Hardware end of medium = No;
> # AutomaticMount = yes; # when device opened, read it
> # AlwaysOpen = Yes;
> # RemovableMedia = yes;
> # RandomAccess = no;
> #}
>
> #
> # Send all messages to the Director,
> # mount messages also are sent to the email address
> #
> Messages {
> Name = Standard
> director = backup-temp-dir = all
> }
>
> thanks
>
>
>
>
>
>
>
>
--
Bill Moran
Collaborative Fusion Inc.
[EMAIL PROTECTED]
Phone: 412-422-3463x4023
IMPORTANT: This message contains confidential information and is
intended only for the individual named. If the reader of this
message is not an intended recipient (or the individual
responsible for the delivery of this message to an intended
recipient), please be advised that any re-use, dissemination,
distribution or copying of this message is prohibited. Please
notify the sender immediately by e-mail if you have received
this e-mail by mistake and delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or
error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The
sender therefore does not accept liability for any errors or
omissions in the contents of this message, which arise as a
result of e-mail transmission.
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Source Forge abusive commercial ads
In response to Chris Hoogendyk <[EMAIL PROTECTED]>: > Marketing claims always tend to be a bit sketchy, and getting marketing > departments to follow academic practices of source citation is rather > unlikely. However, it is possible that they do have a larger installed > base for historical reasons, aside from what you might see listed in > download statistics from Source Forge. > > I guess a broader question for Source Forge would be whether they have > any stated policy or practice regarding targeting ads at competitors > pages within Source Forge. Unfortunately, there are rules in marketing. One of the more important rules is: "Say anything that will get you customers, as long as the profit from the customers exceeds the cost of any lawsuits." (Oddly enough, if you replace "customers" with "votes", politicians have a similar rule.) I've noticed that this practice has been getting worse in recent years, at least in the U.S. It used to be, if you had a "doctor" on a commercial, there was fine print at the bottom of the screen that said something like "not a real doctor" or otherwise identified the person as an actor with no medical training. That doesn't occur anymore. TV ads (in particular) lie outright -- they just do it in a way that their lawyers think it's unlikely they'll ever have a sizable lawsuit occur as a result. The biggest problem, IMHO, is that the American public has ceased to notice and/or do anything about it. I mean, to bring this back to the original topic, if Kern complains (hell, if the entire Bacula community complains) it will probably cause very little to happen aside from a polite apology and explanation of why it is this way from Sourceforge. If, however, Kern were to start publicly researching alternatives to Sourceforge, they might take notice, as Bacula is a pretty important project with a lot of draw. I'm not going to suggest that you _should_ do that, I'm just saying it might be more effective. If Sourceforge is unwilling/unable to force their advertisers to be honest, perhaps some other project hosting service would be a better fit anyway? -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula & FreeBSD 6.x
In response to "Shaun T. Erickson" <[EMAIL PROTECTED]>: > Does the latest stable release of Bacula work ok on FreeBSD 6.x? Is > there anything special I need to know about bacula and this version of > FreeBSD? Nothing that I've come across. I've got four separate directors running on 6.X and a whole messa FDs. Dan has done an outstanding job of keeping the port up to date. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backs up Windows, but can't restore
In response to "Richard White" <[EMAIL PROTECTED]>: > This is Bacula 1.36.3. I have been backing up Linux servers for more > than a year. I have done a test backup and restore of my Windows 2000 > Pro workstation with no problems (it has, however, FAT32 partitions). As > a part of planning for the future (migrating our NetWare servers to OES > Linux and adding the inevitable Winders servers), my boss wants me to be > sure I could back up and restore a W2K server. > > Well, it backs up fine -- nearly 3 GB. When I try to restore, Bacula > informs me that 14,694 files have been added to the tree and my cwd is > /. However, ls shows nothing at all, I can't cd to any known directory > and can't mark any file for restore. > > The manual's caveats for Windows refer to the (ab)use of '\' and to be > mindful of the treatment of NTFS naming conventions. > > Any ideas on where I am going worng? You might want to try some things and cut/paste them into a response email so we can get a better idea of what's going on. The initial problem I had was that Bacula created a c: and d: directory at the root to handle the problem of Windows having "drives". I also seem to remember the c: and d: being case-sensitive (perhaps they were C: and D:) ... anyway, that confused me at first, and I don't remember them showing up in an ls ... but I could be wrong. HTH -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backs up Windows, but can't restore
In response to "Richard White" <[EMAIL PROTECTED]>:
> >>> On 12/12/2006 at 9:08 AM, in message
> <[EMAIL PROTECTED]>, Bill Moran
> <[EMAIL PROTECTED]> wrote:
> > In response to "Richard White" <[EMAIL PROTECTED]>:
> >
> >> This is Bacula 1.36.3. I have been backing up Linux servers for
> more
> >> than a year. I have done a test backup and restore of my Windows
> 2000
> >> Pro workstation with no problems (it has, however, FAT32
> partitions). As
> >> a part of planning for the future (migrating our NetWare servers to
> OES
> >> Linux and adding the inevitable Winders servers), my boss wants me
> to be
> >> sure I could back up and restore a W2K server.
Please make an effort to fix your mail client so it doesn't mangle quoted
messages. The above should look like this:
> On 12/12/2006 at 9:08 AM, in message
> <[EMAIL PROTECTED]>, Bill Moran
> <[EMAIL PROTECTED]> wrote:
> > In response to "Richard White" <[EMAIL PROTECTED]>:
> >
> >> This is Bacula 1.36.3. I have been backing up Linux servers for more
> >> than a year. I have done a test backup and restore of my Windows 2000
> >> Pro workstation with no problems (it has, however, FAT32 partitions). As
> >> a part of planning for the future (migrating our NetWare servers to OES
> >> Linux and adding the inevitable Winders servers), my boss wants me to be
> >> sure I could back up and restore a W2K server.
> >> Well, it backs up fine -- nearly 3 GB. When I try to restore,> Bacula
> >> informs me that 14,694 files have been added to the tree and my cwd is
> >> /. However, ls shows nothing at all, I can't cd to any known directory
> >> and can't mark any file for restore.
> >>
> >> The manual's caveats for Windows refer to the (ab)use of '\' and to be
> >> mindful of the treatment of NTFS naming conventions.
> >>
> >> Any ideas on where I am going worng?
> >
> > You might want to try some things and cut/paste them into a response
> > email so we can get a better idea of what's going on.
> >
> > The initial problem I had was that Bacula created a c: and d: directory
> > at the root to handle the problem of Windows having "drives". I also
> > seem to remember the c: and d: being case-sensitive (perhaps they were
> > C: and D:) ... anyway, that confused me at first, and I don't remember
> > them showing up in an ls ... but I could be wrong.
>
> Here are the pertinent parts of bacula-dir.conf:
>
> Job {
> Name = "Sophos"
> Type = backup
> Pool = Daily
> Full Backup Pool = Weekend
> Schedule = "Cycle_S"
> Client = sophos-fd
> Level = differential
> Storage = Internal
> Fileset = "Sophos_One"
> Messages = Standard
> Priority = 10
> SpoolData = yes
> Write Bootstrap = "/bacula/bin/working/BackupCatalog.bsr"
> }
>
> Job {
> Name = "RestoreSophos"
> Type = Restore
> Client = sophos-fd
> FileSet = "Sophos_One"
> Storage = Internal
> Pool = Daily
> Messages = Standard
> }
>
> FileSet {
> Name = "Sophos_One"
> Include {
> Options {
> signature = MD5
> }
> File = c:/
> }
> }
>
> Schedule {
> Name = "Cycle_S"
> Run = Differential mon-thu at 17:15
> }
>
> Client {
> Name = sophos-fd
> Address = 10.1.0.5
> FDPort = 9102
> Catalog = MyCatalog
> Password = "filedaemon"
> File Retention = 30 days
> Job Retention = 6 months
> AutoPrune = yes
> }
>
> Here is the bacula-fd.conf on the Windows server:
>
> FileDaemon { # this is me
> Name = sophos-fd
> FDport = 9102 # where we listen for the director
> WorkingDirectory = c:/bacula/working
> Pid Directory = c:/bacula/working
> }
>
>
> The only difference between this and my exercise is that the backup job
> was defined as full with Weekend being the default pool.
>
> When I run Restore and choose the correct job number, it declares that
> there are 14,694 files, as I mentioned, but I can neither see nor mark
> them.
Please show this part. The configs aren't likely to help, as it seems
your config is OK. I'm suspecting that the process you're trying to
go through to restore is somehow incorrect. If you show us the
process, we might be able to pick out what's wrong.
--
Bill Moran
Collaborative Fusion Inc.
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backs up Windows, but can't restore
In response to "Richard White" <[EMAIL PROTECTED]>:
>
> > Please show this part. The configs aren't likely to help, as it
> seems
> > your config is OK. I'm suspecting that the process you're trying to
> > go through to restore is somehow incorrect. If you show us the
> > process, we might be able to pick out what's wrong.
>
> OK, here goes.
>
> In bconsole, type 'restore'.
> Select 2 ('Enter list of comma separated JobIDs to select').
> Type in the job number.
>
> At this time I see this:
> .
> .
> .
> 1 Job, 14,694 files inserted into the tree.
>
> You are now entering file selection mode where you add (mark) and
> remove (unmark) files to be restored. No files are initially added,
> unless
> you used the "all" keyword on the command line.
> Enter "done" to leave this mode.
>
> cwd is: /
> $
>
> Now I type
>
> ls
>
> and get this:
>
> c:/
>
> I know there is a directory called sec20, so I type this:
>
> cd sec20
>
> and get this:
>
> Invalid path given.
> cwd is: /
>
> And so on. It has been a couple of months since I did the testing on my
> W2K Pro desktop, but I know I restored files in the "usual" way.
As I mentioned in my earlier response, Bacula has to work around the
broken Windows concept of "drives". Bacula doesn't "do" drives.
In order to work around it, Bacula creates a directory for each
drive, as you can see above in your ls, it created a c: directory
in the backup volume.
So, try "cd c:" then do another ls. An example for one of my systems:
cwd is: /
$ ls
e:/
$ cd e:
cwd is: e:/
$ ls
RECYCLER/
System Volume Information/
Virtual Machines/
[...]
--
Bill Moran
Collaborative Fusion Inc.
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Postgres slow because of autocommit
In response to Gabriele Bulfon <[EMAIL PROTECTED]>: > Hello, > I stumbled upon some pages that confirm a suspect I had about using > postgres with bacula: > - Postgres has autocommit by default, slowing down a lot any Bacula > operation because Bacula does not use any transaction during write. > If this is truedo I have any way to set up just the Bacula database > (not the other ones I have) to have no autocommit? > Is there any "pre-script" to set the bacula connection without auto commit? There is no way (that I'm aware of) to disable "autocommit". By definition, any database that's not being fed transactions is in "autocommit" mode (PostgreSQL actually calls this "unchained" mode). It makes sense: without an explicit transaction, how would the DB server know when to commit the data? I don't know of any DB that does it any differently. That being said, the Bacula team has been working hard at adding transaction support and other performance improvements -- if you search the list archives, you'll see lots of discussions. I think some of the improvements are slated to appear in the next version. That being said, PostgreSQL runs just fine even without the transactions. Everything I run uses PostgreSQL (I manage 4 directors). Two of them manage large data sets using beefy hardware (of 25 servers, 1 of them has 1,500,000 files/12G compressed) Another one is my laptop, and the fourth is a medium-grade machine backing up about 20G of servers and desktops. None of them are experiencing any performance problems. -- Bill Moran Collaborative Fusion Inc. [EMAIL PROTECTED] Phone: 412-422-3463x4023 - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Questions regarding bacula from a potential user
In response to "Terry Zink" <[EMAIL PROTECTED]>: > Currently my company uses a custom backup solution. While effective, we > are growing out of the point where it is scalable for what we need and > becoming unmanageable. It is also starting to lack features necessary. > > Essentially, it backs up both linux and windows systems. It does > standard dumps (A level 0, then level 1's compared against the level 0 > until the level 1 reaches 50% the level0 size) for linux, and for > windows we use smbclient to grab and tar up specific windows shares > (usually the c$ drive.. yes.. There are many limitations of this heh.) > > I stumbled across bacula awhile back, and it looks great! Looks like it > may do everything I need. (I need to read the documentation a bit more > clearly to determine if it does the 0 and 1's as needed, but I'm sure it > can.) > > The question I'm wondering is.. how well does it work in large scale > environments? > > Our current system, is disk(client server) to disk (backup server) to > disk (big array to back up the backup servers) and then to tape. The > first level of backup servers (what the client server sends the data to) > contains about 22 servers. Currently our system means managing each of > those individually. You see my dilemma. Now, granted bacula could > easily probably just handle them each as 20 or so different SD machines > and the director could manage all 20 or so. At least that's my > understanding. 22 storage devices? Well we're backing up around 800 > servers at least like this. > > My question is... aside from anything being blatantly wrong in the above > statements, can bacula reliably be used to backup 800 different systems > or should I look for a different solution? Every testimonial I see > mentions 30 or 50 clients max... Wow. That's a pretty big setup. I've never heard of anyone using Bacula for something that size. In theory, it should scale up that big, but you may encounter some scaling issues that nobody else has seen yet. However, Bacula is open source, and (from your email) I get the feeling that you're the type who can help out by submitting helpful bug reports and working with the community -- maybe even submitting patches. The devel team is great. I think if you hit scaling problems putting Bacula into such a huge environment, you'll get great response from the Bacula community to help make it work. > Also, I noticed in the current "limitations" that 4 billion files is > the limit per database? (this would I assume be per director then in > reality if they used different databases?) Actually, it's per-database catalog. This is because most databases use 32 bit int for IDs. You can have multiple databases (what Bacula calls a catalot) per Bacula director, which allows you to get around this. Each catalog can only have 4 billion files. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Feature Request - Source address control
Item 1: Cause daemons to use a specific IP address to source communications Origin: Bill Moran <[EMAIL PROTECTED]> Date: 18 Dec 2006 Status: What: Cause Bacula daemons (dir, fd, sd) to always use the ip address specified in the [DIR|DF|SD]Addr directive as the source IP for initiating communication. Why:On complex networks, as well as extremely secure networks, it's not unusual to have multiple possible routes through the network. Often, each of these routes is secured by different policies (effectively, firewalls allow or deny different traffic depending on the source address) Unfortunately, it can sometimes be difficult or impossible to represent this in a system routing table, as the result is excessive subnetting that quickly exhausts available IP space. The best available workaround is to provide multiple IPs to a single machine that are all on the same subnet. In order for this to work properly, applications must support the ability to bind outgoing connections to a specified address, otherwise the operating system will always choose the first IP that matches the required route. Notes: Many other programs support this. For example, the following can be configured in BIND: query-source address 10.0.0.1; transfer-source 10.0.0.2; Which means queries from this server will always come from 10.0.0.1 and zone transfers will always originate from 10.0.0.2. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Questions regarding bacula from a potential user
In response to Eric Bollengier <[EMAIL PROTECTED]>: > >> Also, I noticed in the current "limitations" that 4 billion files is > >> the limit per database? (this would I assume be per director then in > >> reality if they used different databases?) > > > > Actually, it's per-database catalog. This is because most databases > > use 32 bit int for IDs. > > You can use bigint instead of integer and bigserial instead of serial in your > make_postgresql_tables. > > > You can have multiple databases (what Bacula calls a catalot) per Bacula > > director, which allows you to get around this. Each catalog can only > > have 4 billion files. > > You can use 64bit FileId_t..., It works with our postgresql database > > - cats/cats.h:typedef uint32_t FileId_t; > - jcr.h: uint32_t FileId; /* Last file id inserted */ > + cats/cats.h:typedef uint64_t FileId_t; > + jcr.h: uint64_t FileId; /* Last file id inserted */ I'm surprised, but delighted, to find it's that simple. That's a sign of a well-written application. Would this work with MySQL and SQLLite as well? (Don't know if MySQL has a 64-bit int or not?) If not, any reason why this couldn't be implemented with an ifdef for PostgreSQL -- when SQLLite and MySQL catch up, the ifdef can be removed. Personally, I don't think we're in danger of exceeding this too soon: bacula=> select max(fileid) from file; max --- 111205419 (1 row) Been running about 6 months. We should probably plan to upgrade to 64-bit when we upgrade to the latest version in the spring. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] looking at bacula
In response to Arno Lehmann <[EMAIL PROTECTED]>: > > That's not too much for Bacula, although OpenBSD as a client is not as > easy to set up as other OSes. At least that's what I remember from list > mail - there seem to be some difficulties getting the client to compile. > AFAIK, it works, though. http://people.collaborativefusion.com/~seklecki/obsd_bacula.html -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] looking at bacula
In response to Oliver Lehmann <[EMAIL PROTECTED]>: > Bill Moran wrote: > > In response to Arno Lehmann <[EMAIL PROTECTED]>: > > > > > > That's not too much for Bacula, although OpenBSD as a client is not as > > > easy to set up as other OSes. At least that's what I remember from list > > > mail - there seem to be some difficulties getting the client to compile. > > > AFAIK, it works, though. > > > > http://people.collaborativefusion.com/~seklecki/obsd_bacula.html > > somehow, the tar is broken here: > > [EMAIL PROTECTED] tmp> gtar -xf ../bacula_openbsd_port.tar > gtar: Unexpected EOF in archive > gtar: Unexpected EOF in archive > gtar: Error is not recoverable: exiting now > Exit 2 > [EMAIL PROTECTED] tmp> tar -xf ../bacula_openbsd_port.tar > bacula/w-bacula-clientonly/bacula-1.39.30/autoconf/gnome-macros/autogen.sh: > (Empty error message) > tar: Truncated input file (need to skip 7312 bytes) > Exit 1 > [EMAIL PROTECTED] tmp> Hmmm ... I get a similar error: tar xf bacula_openbsd_port.tar bacula/w-bacula-clientonly/bacula-1.39.30/autoconf/gnome-macros/autogen.sh: (Empty error message) tar: (Empty error message) Although it seems to extract everything in spite of the message. Brian, can you have a look at this? -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula in Windows XP
In response to "Rogerio" <[EMAIL PROTECTED]>: > I´m testing Bacula in a Windows XP, when a run job i have the following > error: > > Job started. JobId=10 > > 27-Dec 12:01 Debian-dir: No prior Full backup Job record found. > > 27-Dec 12:01 Debian-dir: No prior or suitable Full backup found. Doing FULL > backup. > > 27-Dec 12:01 Debian-dir: Start Backup JobId 10, > Job=Suporte.2006-12-27_12.01.07 > > 27-Dec 15:01 suporte-fd: Suporte.2006-12-27_12.01.07 Fatal error: Failed to > connect to Storage daemon: Debian:9103 > > 27-Dec 15:01 suporte-fd: Suporte.2006-12-27_12.01.07 Error: > c:\cygwin\home\kern\bacula\k\src\win32\lib\../../lib/bnet.c:768 > gethostbyname() for host "Debian" failed: ERR=Authoritative answer for host > not found. > > 27-Dec 12:01 Debian-dir: Suporte.2006-12-27_12.01.07 Fatal error: Socket > error from Filed on Storage command: ERR=No data available > > 27-Dec 12:01 Debian-dir: Suporte.2006-12-27_12.01.07 Error: Bacula 1.36.2 > (28Feb05): 27-Dec-2006 12:01:13 > > JobId: 10 > > Job:Suporte.2006-12-27_12.01.07 > > Backup Level: Full (upgraded from Incremental) > > Client: suporte-fd > > FileSet:"Suporte" 2006-12-27 11:46:42 > > Pool: "Default" > > Storage:"File" > > Start time: 27-Dec-2006 12:01:09 > > End time: 27-Dec-2006 12:01:13 > > FD Files Written: 0 > > SD Files Written: 0 > > FD Bytes Written: 0 > > SD Bytes Written: 0 > > Rate: 0.0 KB/s > > Software Compression: None > > Volume name(s): > > Volume Session Id: 4 > > Volume Session Time:1167227172 > > Last Volume Bytes: 0 > > Non-fatal FD errors:0 > > SD Errors: 0 > > FD termination status: > > SD termination status: Waiting on FD > > Termination:*** Backup Error *** > > > > Any help??? Yes. Use host names that actually resolve to IP addresses. >From your error messages, the XP machine is being told to connect to a Storage Daemon on host "debian", but there is no such host in DNS. Either add the host to DNS, or correct the config so it uses a proper hostname or IP. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] FOSDEM slide presentation
In response to Kern Sibbald <[EMAIL PROTECTED]>: > > I've posted to the web site a preliminary copy of the slide show that I plan > to present at FOSDEM in Febrary, and would appreciate your feedback on it. > I haven't timed the talk yet, but it should be about 30-40 minutes. > > I would like to have better graphics for the one graphics slide I have, but > then I'm not very good as an artist. About the only other thing I am > thinking about doing is showing some actual commands and output much like > what is in the Brief Tutorial chapter of the manual -- the problem is that > such text either is too small to read on a projected screen or requires 3 > slides to get it up ... You can view it at: > > http://www.bacula.org/presentations/Bacula-FOSDEM-talk-24Feb07.pdf > > Best regards, * Slide 21: I found the indenting a bit confusing. At first it seems as if you had multiple filesets on the slide. Also, I gave a presentation at our local LUG last month on Bacula, and I left myself time at the end to actually show them what it was like to manually kick off backups and restores in bconsole. I run a full Bacula setup on my laptop so it was pretty convenient. The point being that the demonstration at the end was more popular than the presentation itself -- you might want to consider doing the same. Other than that, it looks really good. I didn't think the graphic slide was particularly bad -- it looks better than the OOo graphics I did for my presentation. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula-devel] [CROSSML] Can Bacula fulfil the following requirement?
In response to "Eric Leung" <[EMAIL PROTECTED]>: > Dear Bill > > Thank you very much for your reply, it was so helpful, as I am new to > Bacula. However, there are few more question I would like to ask. It's generally considered bad form to pester someone personally about a mailing-list response. I've added the list back into the CC. It's also rather difficult to follow the conversation when you top post. > Regarding to the following requirements: > > 05) "backup notification email with html template - NOT SURE" > > Your answer: Email notification works. Not sure about HTML templates. > > Do you think I can use script to reproduce the email notification > instead of using the default email notification from Bacula, in order to > use a HTML template for email notification, do I have to modify the > source code in Bacula, etc. I'm sure that you could do either of those to get what you want. > 07) "Quick Restore (backup not split into 2G files, i.e. one single > file) - NOT SURE" > > As the current backup system I am using need to split the backup to > multiple "2G files", that is, I would like to ask how the backup is > stored by Bacula? In a "tar" format or some other format? In addition to > this question, is there any limit to the compression file size, as I > know some backup systems limit the size of the files in order to be > compressed. So I would like to ask is there any limit in Bacula for > compression large files (e.g. few hundred GB files) Yes, I have a system with an external drive array that doesn't support files larger than 2G. I just set the max volume size at 2G and Bacula creates new files when needed. > 09) "Shadow filesystem backups - NOT SURE" > > Your answer: Yes. VSS in Windows, and other systems can be scripted. > > So you mean if I backup a Windows machine, I need to use VSS, and other > systems, I need to write script? The client I need to backup mainly are > Window and Linux machine, so what should I do? Window: use VSS. Linux: it depends on the filesystem you're using, and whether that filesystem supports snapshots. For example, I believe Reiser does, but I don't think that ext2 does. For how to write the script, do a little googling. > -Original Message- > From: Bill Moran [mailto:[EMAIL PROTECTED] > Sent: Thursday, 4 January 2007 12:14 AM > To: Eric Leung > Cc: [EMAIL PROTECTED] > Subject: Re: [Bacula-devel] [CROSSML] Can Bacula fulfil the following > requirement? > > In response to "Eric Leung" <[EMAIL PROTECTED]>: > > > > 01) Able to split backups across multiple disks - OK > > 02) Incremental backups - OK > > 03) Improved flexibility for configuring which folders are backed > > up > > 04) Encrypted backup support - OK > > 05) backup notification email with html template - NOT SURE > > Email notification works. Not sure about HTML templates. > > > 06) ACL support - NOT SURE > > Yes. > > > 07) Quick Restore (backup not split into 2G files, i.e. one > single > > file) - NOT SURE > > I don't understand what this means. > > > 08) Scripts to allow specified files to be restored from the > backup > > - OK: base on the catelog in DB > > 09) Shadow filesystem backups - NOT SURE > > Yes. VSS in Windows, and other systems can be scripted. > > > 10) Incremental backups to remote fileservers rather than disks - > > OK > > > > I am looking forward to your reply and using Bacula. It will be great > > if Bacula can fulfil all of the above requirements, as these are the > > major factor for my decision. > > > -- > Bill Moran > Collaborative Fusion Inc. > > > > > > -- Bill Moran Collaborative Fusion Inc. [EMAIL PROTECTED] Phone: 412-422-3463x4023 IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the conte
Re: [Bacula-users] as my Bacula project progresses...
Jonathan Horne <[EMAIL PROTECTED]> wrote: > > as i mentioned in another thread, i am migrating from netbackup to bacula. > my > bacula dev server: > > FreeBSD fbsd62-1.dev.dfwlp.com 6.1-SECURITY FreeBSD 6.1-SECURITY #0: Mon Dec > 4 07:53:55 UTC 2006 > [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC i386 > > my clients will range from FreeBSD, OpenBSD, OpenSuse (both i386 and ppc), > Mac > OSX 10.4.8, Solaris 10 (x86), and a windows xp. my current netbackup is > WS2003, but it will migrate to FreeBSD when im ready to deploy bacula. i > will be using backup-to-disk only at this time (maybe i can talk my office > out of a fully depreciated and forgotten 9 slot DLT drive, and maybe that > will change *wink*) > > on my FreeBSD dev box, ive installed sysutils/bacula-server, built with mysql > and ssl. i am following dan langille's diary document and the man page, and > the first stumbling block ive come to is that i cannot locate bconsole. > > currently retracing all my steps to find out where i went wrong. can anyone > offer any guidance? The FreeBSD port installs bconsole in /usr/local/sbin. It's possible that's not in your path. Try providing the full path. -Bill - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] as my Bacula project progresses...
Jonathan Horne <[EMAIL PROTECTED]> wrote: > > On Saturday 06 January 2007 14:14, Jonathan Horne wrote: > > ok, im backing up, and starting over. looks like 2.0 was just committed to > > freebsd ports. > > > > question: is it best to use those random generated passwords, or to put in > > my own? > > > > cheers, > > jonathan > > well, ive made some pretty fair progress. ive got to where i can get the job > to start... but it wont move. for my first test example, im trying to get my > dev server to back itself up. > > this is the message im getting: > > *messages > 06-Jan 15:38 fbsd62-1-dir: Start Backup JobId 2, > Job=Client1.2007-01-06_15.38.53 > 06-Jan 15:38 fbsd62-1-sd: Job Client1.2007-01-06_15.38.53 waiting. Cannot > find > any appendable volumes. > Please use the "label" command to create a new Volume for: > Storage: "FileStorage" (/usr/backups) > Media type: File > Pool: Default > > in the file definition, ive specified the /, and added /usr/backups to the > omitted files. as ive mentioned before, im trying to set up bacula to only > backup to files. my current backup server has: You may want to provide your pool defintions, as it looks like the problem lies therein. Based on the error you're getting, you don't have any volumes in existence to back up to. You either need to manually create volumes in bconsole with the label command, or enable automatic volume creation in the pools. -Bill - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] speeding up bacula?
Oliver Lehmann <[EMAIL PROTECTED]> wrote: > > I wonder how I can speed up bacula? > > FD Bytes Written: 54,663,990,797 (54.66 GB) > SD Bytes Written: 54,667,075,813 (54.66 GB) > Rate: 2687.1 KB/s > > the directories to backup are on the same 3ware-RAID-5 as the storage is > located (Media Type = File). I know that this is not the optimum > settings, but the 4*250GB SATA drives are taking all the place in the 4HE > case so there is no space left for other disks ;) > The MySQL DB is located on the same "disk". > > Even with this "not optimum" setup I wonder why it is so slow. I can read > +write a file from and to the same disk with a throughput of ~25MB/sec > (writing only is about 50MB/sec) First off, throw your assumptions away. You're assuming that Bacula should be able to run at the same speed as the disk. Turn off compression and see if it comes close. Then watch top and iostat while a backup is running and see what kind of resources the whole process is utilizing. Check your MySQL insert performance and see if that's holding things up. I give it a 50% chance that the CPU is the bottleneck, not the disk. -Bill - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Volume use duration
In response to Amiche <[EMAIL PROTECTED]>: > Whan bacula start the backup of the night on a tape volume, I want the status > of > the volume pass immediatly as Used when the last job is terminated. > So I defined "Volume Use Duration = 10h" but Bacula wait for the next job > using > this volume to mark it as Used,and not immediatly after the last job. I need > to > have the check "Max configured use duration exceeded" just after my last job. > Because my jobs start at 4am, and the status is marked as used because Max > configured use duration exceeded, and bacula wait for a new volume until an > operator come and change it in the morning. > Is it possible to check the Volume use duration on a volume without starting a > new job on this volume? Configure an admin job. See: http://bacula.org/rel-manual/Configuring_Director.html#JobResource -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Building directory tree is very slowly
In response to Ondrej PLANKA <[EMAIL PROTECTED]>:
> Alan Brown napsal(a):
> > On Tue, 9 Jan 2007, Ondrej PLANKA wrote:
> >
> >>> It sounds silly, but add individual indexes for JobId, PathId and
> >>> FilnameId as well as the 3-way one.
> >>>
> >> Ok, do you have some tip, how can I add individual indexes? Because I
> >> think, my indexes are OK or not?
> >
> > ALTER TABLE File ADD INDEX [name] {column} ;
> >
> > so...
> >
> > ALTER TABLE File ADD INDEX file_jobid_idx JobId ;
> > ALTER TABLE File ADD INDEX file_pathid_idx PathId ;
> > ALTER TABLE File ADD INDEX file_filenameid_idx FilenameId ;
> >
> I really done this individual indexes, but building directory tree for
> Jobs are still very slowly.
Index problems are probably the most common reason for this, but they're
not the only possible cause.
I didn't see any hardware details -- perhaps you posted it earlier and
I missed them. Bacula will need a lot of RAM to store that large of a
directory tree. Monitor top, iostat, and any other sys monitoring programs
you can during the tree build.
In our case, we decided to add another gig of RAM to backup server and things
are nice and fast now.
--
Bill Moran
Collaborative Fusion Inc.
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Don't understand error
In response to "Richard White" <[EMAIL PROTECTED]>:
> I have added a Winders server to our Bacula backup jobs. It is located
> in a DMZ. Our web server (running on SLES 10) is also in the DMZ and we
> have been backing it up with Bacula for a month now. I created rules in
> the firewall for the Windows server that are identical to those for the
> web server, just with a different IP address. The Bacula client is
> installed and the bacula-fd.conf file has been edited. The bacula-fd
> service starts automatically.
>
> Here is the bacula-fd.conf:
>
> #
> # Default Bacula File Daemon Configuration file
> #
> # For Bacula release 1.38.10 (08 June 2006) -- cygwin
> 1.5.18(0.132/4/2)
> #
> # There is not much to change here except perhaps the
> # File daemon Name to
> #
>
> #
> # List Directors who are permitted to contact this File daemon
> #
> Director {
> Name = lbackup-dir
> Password = "filedaemon"
> }
>
> #
> # "Global" File daemon configuration specifications
> #
> FileDaemon { # this is me
> Name = ArcIMS-fd
> FDport = 9102 # where we listen for the director
> WorkingDirectory = c:/bacula/working
> Pid Directory = c:/bacula/working
> }
>
> # Send all messages except skipped files back to Director
> Messages {
> Name = Standard
> director = lbackup-dir = all, !skipped
> }
>
> This is identical to the conf file on systems that work, except for the
> filedaemon name.
> When I run a backup job for this server, the job runs for two hours
> (almost exactly), with the tape drive indicating data being written,
> then terminates with an error. The job summary looks like this:
Try turning keepalive on. Sounds like the system runs for a while without
exchanging any data over the network, and your firewall is dropping the
state from the state table.
Stateful firewalls are a stupid idea.
> 29-Jan 18:07 lbackup-dir: ArcIMS.2007-01-29_16.07.46 Fatal error:
> Network error with FD during Backup: ERR=Connection reset by peer
> 29-Jan 18:07 lbackup-dir: ArcIMS.2007-01-29_16.07.46 Fatal error: No
> Job status returned from FD.
> 29-Jan 18:07 lbackup-dir: ArcIMS.2007-01-29_16.07.46 Error: Bacula
> 1.36.3 (22Apr05): 29-Jan-2007 18:07:51
> JobId: 1462
> Job:ArcIMS.2007-01-29_16.07.46
> Backup Level: Full (upgraded from Differential)
> Client: ArcIMS-fd
> FileSet:"ArcIMS" 2007-01-19 15:50:02
> Pool: "Weekend"
> Storage:"Internal"
> Start time: 29-Jan-2007 16:07:48
> End time: 29-Jan-2007 18:07:51
> FD Files Written: 0
> SD Files Written: 13,339
> FD Bytes Written: 0
> SD Bytes Written: 9,959,280,437
> Rate: 0.0 KB/s
> Software Compression: None
> Volume name(s): Week_2D
> Volume Session Id: 3
> Volume Session Time:1170100801
> Last Volume Bytes: 19,948,336,363
> Non-fatal FD errors:0
> SD Errors: 0
> FD termination status: Error
> SD termination status: OK
> Termination:*** Backup Error ***
>
> Status dir looks like this:
>
> Terminated Jobs:
> JobId Level Files Bytes Status FinishedName
>
> [other jobs]
> 1462 Full 0 0 Error29-Jan-07 18:07 ArcIMS
>
> I am puzzled why there are 13,339 SD files and 9,959,280,437 bytes
> written, but zero FD files and bytes written.
Tape files are different than actual files. But the fact that you see
neither files nor bytes indicates (to me) that your doing spooling?
--
Bill Moran
Collaborative Fusion Inc.
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] slow postgres, index/table does not exist
In response to Bacula User <[EMAIL PROTECTED]>: > > I am using freebsd 6.1, bacula 1.38.11_1, and postgresql-server-7.4.13_1. > Each night I dump 913,157,728,765 bytes of 5,395,281 files to tape. I > spool data and attributes. I will need to almost triple this amount to > deploy bacula. Data goes to tape in under 16.5 hours, but attributes > continue going into the catalog for over 13 hours afterwards. In other > words, I'm still writing the previous night's data to the database when > the new day's backup cycle starts. I never get a chance to get an ascii > dump of the db before the next backup cycle begins. > > My untuned postgres is too slow. I'm researching ways to speed it up. If > you have pointers, please share. Can you replace 7.4 with 8.2? There are a LOT of performance improvements in 8.2 over 7.4. You should probably join the PostgreSQL performance mailing list: http://www.postgresql.org/community/lists/subscribe There is a good guide here: http://www.varlena.com/GeneralBits/Tidbits/perf.html Generally speaking, you're dealing with lots of inserts. To speed them up you're going to need very fast disks. You'll probably also benefit from increasing checkpoint_segments and checkpoint_timeout (see http://www.postgresql.org/docs/8.2/static/wal-configuration.html) Make sure you vacuum and analyze frequently. Your scenario may benefit from using the autovacuum daemon. > Furthermore, each night has many errors > like the following. What do these mean and how do I fix them? They are normal operation. Bacula creates a number of temp tables and deletes them when it's done with them. To ensure sane recovery from a crash, Bacula tries to delete them before using them, just in case they were left over from a crash. There are ways to do this in PostgreSQL without producing these errors, but I don't think there's a database- independent way of doing this without causing these errors. The short answer is they're nothing to worry about. > > postgres[19278]: [324-1] ERROR: index "delinx1" does not exist > postgres[19278]: [325-1] ERROR: index "delinx1" does not exist > postgres[19278]: [326-1] ERROR: table "delcandidates" does not exist > postgres[19278]: [327-1] ERROR: index "delinx1" does not exist > postgres[19278]: [328-1] ERROR: index "delinx1" does not exist > postgres[19278]: [329-1] ERROR: table "delcandidates" does not exist > postgres[19278]: [330-1] ERROR: index "delinx1" does not exist > postgres[19278]: [331-1] ERROR: index "delinx1" does not exist > postgres[19278]: [332-1] ERROR: table "delcandidates" does not exist > postgres[19278]: [333-1] ERROR: index "delinx1" does not exist > postgres[19278]: [334-1] ERROR: index "delinx1" does not exist > postgres[19278]: [335-1] ERROR: table "delcandidates" does not exist > postgres[19278]: [336-1] ERROR: index "delinx1" does not exist > postgres[19278]: [337-1] ERROR: index "delinx1" does not exist > postgres[19278]: [338-1] ERROR: table "delcandidates" does not exist > postgres[19278]: [339-1] ERROR: index "delinx1" does not exist > postgres[19278]: [340-1] ERROR: index "delinx1" does not exist > postgres[19278]: [341-1] ERROR: table "delcandidates" does not exist > postgres[19278]: [342-1] ERROR: index "delinx1" does not exist > postgres[19278]: [343-1] ERROR: index "delinx1" does not exist > postgres[19278]: [344-1] ERROR: table "delcandidates" does not exist > postgres[19278]: [345-1] ERROR: index "delinx1" does not exist > postgres[19278]: [346-1] ERROR: index "delinx1" does not exist > postgres[19278]: [347-1] ERROR: table "delcandidates" does not exist > postgres[19278]: [348-1] ERROR: index "delinx1" does not exist > postgres[19278]: [349-1] ERROR: index "delinx1" does not exist > > Thank you for your help. Bacula is well on its way to replacing our > current method of madness. I just need to sort a few more things out. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula and HP JetDirect (was: (2.0.1) 5 minute 5 seconds problem)
In response to Chris Hoogendyk <[EMAIL PROTECTED]>: [snip] > I passed an earlier message from this thread along to my network expert, > because we have had some complaints about recent HP stuff. I'm only > casually up on that stuff. > > I was surprised by his answer, but not by his depth of knowledge. So, > I'm passing it along just so you all have it. Please note that lprng and > cups are two of the most widely used printing systems. Mac OS X uses > cups, and lprng is frequently recommended as a replacement system for > people in linux/unix environments who want some added capability. > > > > Chris, I would hope someone would point out to the list that ONLY ports < > 1024 historically > > are "well known ports", and those < 4096 are sort of registered. His information is dated. "Historically" we only use 2 digits for the year as well ... > > So far as I am aware there is > > no binding registration for ports > 4096. But far worse is the fact that > not just HP printers > > use port 9100. So do almost all printers which use the idea of a "service > port", and this is > > supported by default in lprng and in cups. His facts are wrong here. IPP (i.e. CUPS) uses port 631 and lpd/lpr uses 515. Those 910x ports are registered to Bacula officially. There are no "internet police" and there's no fine or anything that means that the Jetdirect systems are doing anything "illegal", but it's _only_ jetdirect cards that use those ports outside of the IANA registration. Facts: *) Bacula is registered with IANA to use the 910x ports. It's "official" *) _only_ jetdirect cards use the 910x ports. CUPS and LPD do not. If they are, it's because you're using some sort of CUPS->jetdirect driver (which is actually pretty common) but it's not CUPS, it's the jetdirect driver. > > If bacula is on port 9100 that is a totally bad idea. Well, he's welcome to his opinion, but he doesn't seem to have any facts to back it up. > > Bacula ought to have gotten a port registered by IANA in the reserved > range rather than just > > grabbing ports that have long been in use for printing. Again, he's working off 10-year old data here. The ports _are_ registered, the registered port range has been expanded in recent years because there's no room left for new applications below 1024. -- Bill Moran Collaborative Fusion Inc. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
