> Some interesting observations from these tests... With the generic > 2.3.21 code, throwing 26 SLAMD clients at a SunFire T2000 with a 1GHz > processor, we get around 6400 authentications per second on a 1 million > entry database. (Which is pretty respectable, really. SunOne only got > 6233 for the same configuration) > http://blogs.sun.com/roller/page/DirectoryManager/20060110 > > What's interesting is that we only see about 19% CPU usage on slapd (13% > user, 6% kernel), even though there are 32 virtual processors here and > 16 worker threads (LWPs) and the database is 100% cached in RAM. Clearly > slapd isn't making as good a use of the available CPUs as it could; if > it were keeping the threads busy we should see right about 50% CPU usage. > > With the lightweight dispatcher and multi-conn array code, we see CPU > utilization go up to 45% (20% user, 25% kernel) and throughput goes up > to about 8800 authentications per second. With the lightweight > dispatcher and the new code with no connections_mutex we get about 8900 > authentications per second, and CPU use is at 49% (23% user, 26% kernel). > > (We didn't collect numbers for just the lightweight dispatcher by itself > yet.) At this point it seems we've eliminated a bottleneck in the > frontend, only to find that there's probably a bottleneck in the backend > as well. (Otherwise, performance ought to keep improving as we add > worker threads, but it doesn't.) I'm puzzled about why we're seeing so > much more kernel time vs user time. Hopefully spending some time with > dtrace will tell us what's going on there. Ideally we should see a tiny > percentage of kernel time, and we should see performance continue to > improve as more worker threads are added. (Why stop at just 42% faster > than The Other Guys, when we can potentially be 100% faster?) > > It's worth noting that this SLAMD authentication job consists of a > search for a uid, followed by a Simple Bind on the returned entry's DN, > so it's really the equivalent of almost 18,000 searches per second. And > since we processed 6400 authentications/second using only 19% of the > CPU, we really ought to be able to get 12,800 auths/second at 40%, and > so on. (OK, it won't scale perfectly linearly because slapd only has one > dispatcher thread, and at some point we will probably saturate the > network interfaces on the server, or the gigabit switch. Also, we're > struggling to get any more requests out of the clients. I really wish > the SLAMD clients were written in C with libldap, this Java stuff is way > too slow. It shouldn't take 26 clients to generate this workload; it > should take at most 16 clients to keep the 16 server threads busy. Who > ever thought that writing performance-critical code in Java was a good > idea?)
Your numbers are quite interesting. About the workload tool issue, we could do some work to improve the existing test suite in tests/progs. The current versions in HEAD look much more flexible than ever: now they allow random search/bind on entries looked up from the database, we're using them to benchmork stock and custom slapd on different architectures. Maybe a more consistent and coordinated development would turn them into something more valuable?. p. Ing. Pierangelo Masarati Responsabile Open Solution OpenLDAP Core Team SysNet s.n.c. Via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ------------------------------------------ Office: +39.02.23998309 Mobile: +39.333.4963172 Email: [EMAIL PROTECTED] ------------------------------------------
