Hi Heitor/All,
I found set "synchronous_commit = off" has very much improvement of rebuilding
catalog using bscan command.
Do you have any idea of why mark a huge number of files take so long? After
marked the files, I also found restoring data to local harddisk took a long
time. I could see the speed took around 20MBps? It seems to be the bottle neck
at bacula-fd and bacula-sd process since the harddisk (SATA) speed tested is
greater than 100MBps. Many thanks!
Regards,Keith
Show original message On Wednesday, August 24, 2016 2:17 AM, Heitor Faria
<hei...@bacula.com.br> wrote:
Hi there,
Hello, Keith,
I was trying to restore a huge data to local harddisk from an offline backup
harddisk. There are 36,225,746 files and the data size is 1.7TB.
Steps to restore data:
1. Rebuild catalog using bscan
2. bconsole > restore > option 3 "Enter list of comma separated JobIds to
select" > enter {jobid} > mark folder_to_be_restored
After entered "mark folder_to_be_restored", it took about 28 hours to show $
prompt and let me proceed to enter "done". Not sure why took so long. Is it
possible to do some tuning? I could see there is a process "bacula-dir" which
occupied 4GB memory only. While postgres processes are below 2GB memory. There
are 7.7GB is free and 3.5GB for cache.
System configuration:
OS: 2.6.35.14-106.fc14.x86_64
Mem: 16G
Partition1(for OS): RAID1
Partition2(for data restoration): RAID1Bacula: 5.0.3Postgresql:
8.4.9-1.fc14.x86_64
#postgresql.conf
shared:2GB
effective_cache_size = 8GB
maintenance_work_mem=512MB
work_mem:128MB
wal_buffers = 8MB
checkpoint_segments = 64
checkpoint_timeout = 20min
checkpoint_completion_target = 0.9
synchronous_commit = off
I would use Postgresql "synchronous_commit = on". It's the single option (I
tested) that most improves T/s, but other people from this list have deep
database knowledge and may suggest other modifications.
#sysctl.conf
kernel.shmmax = 6442450944
kernel.shmall = 4194304
Best,Keith
Regards,
--
===========================================================================
Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified
Administrator IIDo you need Bacula training? http://bacula.us/video-classes/+55
61 8268-4220
Site: http://bacula.us FB: heitor.faria
===========================================================================
------------------------------------------------------------------------------
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
