-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 mehma sarja wrote: > MY SITUATION > > I can take your Megabytes and shame you with my 9,868,868 mostly Maildir > files and 690.8 GB space they take up. Take that! It took 25 hours to > transfer and is currently "indexing." Before I ramble on, here is some > confguration info:
Indexing? Sorry, what do you mean? Do you have some output of some command? > CONFIGURATION > > dir Version: 2.4.2 (26 July 2008), amd64 freebsd7.1, PG version 8.2 I don't know if 2.4.4 helps, or if PostgreSQL 8.3.7 is better. > The OS is a stripe of mirrors (2x2) running on FreeBSD ZFS filesystem > and a spare (in a pear tree). Hey, it is still almost winter. I would try plain old UFS as a comparison. > WHAT?! > > Yes, this humongo transfer has been done TWICE before and the indexing > just runs for days till I get sick of it and kill PG. What is PG doing? Have you looked at that? > WHAT HAVE YOU DONE?! > > I have tweaked FreeBSD to run on all 4 CPU cores Number of cores may not help if everything is running in serial. Have you considered dividing the backup into four and running four concurrent jobs? Is PostgreSQL, bacula-sd, and bacula-dir running on the same box? > and tweaked PG > (postgresql.conf) thus: > > # - Memory - > > shared_buffers = 10MB # min 128kB or max_connections*16kB > temp_buffers = 10MB # min 800kB > work_mem = 10MB # min 64kB > maintenance_work_mem = 40MB # min 1MB > max_stack_depth = 525kB # min 100kB > > # - Free Space Map - > max_fsm_pages = 30000 # min max_fsm_relations*16, 6 bytes each > max_fsm_relations = 1000 # min 100, ~70 bytes each The experts there would be the PostgreSQL mailing lists. > WHAT DID THIS GET YOU? > > Well, the transfers for one finished 22% faster (25 instead of 32 > hours). And "top" does not show a high number for IVCSW - involuntary > context switching. This means it is using more cores and maybe more > memory. I was hoping it would also speed up indexing but I am now 20 > hours into indexing after the transfer completed. vmstat and iostat output may help you diagnose. > CAN SOMEONE HELP ME? > > I suspect the PG tweaks are not right and I am dealing with a large > number of files are my main problems. It could also have something to do > with the two previous indexing which I stopped. Perhaps. Have you run any vacuums on PostgreSQL? - -- Dan Langille BSDCan - The Technical BSD Conference : http://www.bsdcan.org/ PGCon - The PostgreSQL Conference: http://www.pgcon.org/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAknP9ZoACgkQCgsXFM/7nTw5BwCgyHG3s4odNe7ClQCw+Bgbqz5c klIAoKnRt+0IO9+j05cS8HmgC/TBINtK =PEgb -----END PGP SIGNATURE----- ------------------------------------------------------------------------------ _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
