On Tuesday 30 March 2010 13:34:12 Ralf Gross wrote: > Kern Sibbald schrieb: > > > The memory bacula-sd uses seems to be growing constantly over time (500 > > > MB atm). > > > > > > http://img163.imageshack.us/img163/7928/baculasdmem.png > > > > > > This is no problem right now, but I just started an restore of 13 files > > > and server started swapping again. bacula-sd's memory usage began to > > > boost over 2,5 GB RAM. I can reproduce this ever time I try to restore > > > these files. > > > > Normally, there is no reason why a SD would ever use at a maximum more > > than a few hundred megabytes of memory. In fact, the data below > > indicates that the SD is using 19MB, which is quite normal and > > reasonable. So there is a big difference between what your system is > > reporting and what the SD thinks it is using. > > err, I meant bacula-dir, but I see in your other mail that you've cached > this. > > Last time I had problems it was definitly the sd. This is the sd status > right now. > > VUMEM004-sd Version: 3.0.3 (18 October 2009) x86_64-pc-linux-gnu debian > 5.0.3 Daemon started 17-Mär010 14:45, 321 Jobs run since started. > Heap: heap=308,527,104 smbytes=309,185,337 max_bytes=309,318,227 > bufs=22,688 max_bufs=22,921 Sizes: boffset_t=8 size_t=8 int32_t=4 int64_t=8 > > Not that critical.
OK. 300MB is a lot, but if the SD is busy, it could be OK. > > > Can you tell me how big the .bsr file was that the Director sent to the > > SD? > > Run Restore job > JobName: RestoreFiles > Bootstrap: /var/lib/bacula//VUMEM004-dir.restore.5.bsr > Where: /tmp/bacula-restores > Replace: always > FileSet: Full Set > Backup Client: VU0EM003 > Restore Client: VU0EM003 > Storage: Neo4100-LTO4-D1_B-Net > When: 2010-03-30 13:30:47 > Catalog: MyCatalog > Priority: 5 > Plugin Options: *None* > OK to run? (yes/mod/no): > > > The bsr file is 56 lines long. > > -rw-r----- 1 bacula bacula 1,1K 30. Mär 13:30 > /var/lib/bacula/VUMEM004-dir.restore.5.bsr OK, that is normal nothing unusual. > > > > > [...] > > > > My question: is this memory usage expected for restore jobs of this > > > size? I've never noticed it before 3.x. Accurate Backups are not active > > > for this client/backup. > > > > The memory usage for restoring from a 3TB backup could be rather large in > > the Director if you have not configured your Storage daemon properly. > > > > Can you tell me how many JobMedia records there are for JobId 20161? > > list jobmedia jobid=20161 > +--------+------------+------------+-----------+ > > | jobid | volumename | firstindex | lastindex | > > +--------+------------+------------+-----------+ > > | 20,161 | A00206L4 | 7,022,481 | 7,022,547 | > | 20,161 | A00206L4 | 7,022,547 | 7,022,593 | > > [...] > > | 20,161 | A00165L4 | 1 | 284,312 | > | 20,161 | A00165L4 | 284,312 | 297,233 | > > [...] > > | 20,161 | A00205L4 | 7,016,946 | 7,017,353 | > | 20,161 | A00205L4 | 7,017,353 | 7,017,564 | > > +--------+------------+------------+-----------+ > > > The output seems not to be sorted, but I get a total of 752 lines (Maximum > File Size = 5G for the LTO-4 drives) OK, 752 is a lot, but given the size of your backup, and the fact that you have properly set the Maximum File Size to 5G, this is OK. > > > What is the output device? Tape, disk, ??? > > Hm, what do you mean with "output device"? The full and differential > backups were done on tape (LTO-4 / LTO-3), the incremental on disk. The > restore should happen on the bacula-fd where the backup was taken from. OK. > > > This will give an idea if your SD is properly configured. As far as I can see everything is reasonably well configured. There is no reason for the SD to consume a lot of memory. The Dir, can consume a lot of memory, possibly 100-200 bytes (just a guess) for each file which means 200 X 7,000,000 which is only 1.4GB if I count right, so it can get big when it is creating the in memory list, and perhaps it will be double that if your database backend "caches" the files in memory. So, anything up to about 3GB is not terribly unusual, but the memory load should go down once the bsr file has been built. Best regards, Kern ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
