On Fri, 28 Feb 2003 16:30, Emile van Bergen wrote: > > database ldbm > > cachesize 100000 > > dbcachesize 10000000 > > I suspect the law of diminishing returns here. Cachesize (in entries) is > a factor 100 above slapd's default, the same for dbcachesize. I have no > idea what their in-memory data structures look like, but they may be > inefficient for larger sizes. > > You may also be suffering from excessive lock contention if you're > pounding it on an SMP box. OpenLDAP isn't known to handle that > particularly well.
I don't think that slapadd has any concurrancy issues as it only has one process. > Try setting threads and concurrency to 1, and enable dbnolocking. > > Lastly, even though the CPU time indicates no I/O seek latency problems, > you may want to experiment enabling dbnosync as well. It can make a > *huge* write throughput difference if you trust your backend storage > enough. Here's the initial time for a load (NB it's a bit longer than before because I am now doing sub-string indexing on the uid attribute): real 3m25.101s user 2m47.120s sys 0m23.950s Here's the time with dbnolocking and dbnosync: real 3m25.184s user 2m47.750s sys 0m23.980s So I removed them as they obviously did no good. Now I tried the default cachesize and dbcachesize values which isn't good: real 3m55.394s user 2m54.050s sys 0m50.570s default cachesize and the dbcachesize of 10M which shows that a large cachesize is bad: real 3m19.413s user 2m46.840s sys 0m23.650s default cachesize and a dbcachesize of 40M, increasing the dbcachesize helps: real 2m56.051s user 2m39.360s sys 0m6.940s cachesize of 300 and a dbcachesize of 40M, so a really small cachesize doesn't help: real 2m56.204s user 2m40.300s sys 0m6.160s default cachesize and a dbcachesize of 100M, we're getting towards the law of diminishing returns but I've got plenty of RAM: real 2m50.340s user 2m38.430s sys 0m5.020s default cachesize and dbcachesize of 500M: real 2m47.355s user 2m36.690s sys 0m4.070s I doubt that I would get any benefit from specifying a dbcachesize >500M. Also I think it would be unwise as 500M is 13% of the RAM in the machine and with future growth that risks having slapadd starting to thrash. Thanks for the suggestions, this resulted in a performance increase of almost 20%! The speed still sucks though, I think it should be at least twice as fast. I'll have to try out OpenLDAP 2.1.x with BDB. -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/ Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
