Go and grab your share at
https://tortoiseanalyze.svn.sourceforge.net/svnroot/tortoiseanalyze/OptimizedSVN/branches/prototype
It is SVN 1.7 + APR 1.3 + ZLIB 1.2.5 tuned for svnadmin and
threaded svnserve performance with FSFS. All tests pass on Linux.
-- Stefan^2.
What is this all about?
-----------------------
A couple of weeks ago, I started working on SVN's server performance.
Without access to a development repository, I found it hard to
develop a compelling solution to the underlying problems. Thus,
the baby-step patches I sent to this and other lists got often
rejected.
So, I decided to set up a project in my own repository and to
see how my ideas would turn out in the end. For instance, the
"cache item pinning" feature got dropped and replaced by an
even more effective solution.
Now, I got a working code base that already served many TBytes
without complaint and many code fragments can directly be
copied into the SVN /trunk - after due review, of course.
What has been changed?
----------------------
* APR: improve performance when scanning a file backward
* APR: defer (and often eliminate) seek operations
* FSFS only: file handle cache to keep files open as long as
feasible and to reuse their buffer content.
* FSFS only: in-process shared fulltext and txdelta caches
(similar to memcached but without the latency penalty)
* zlib: major deflate() and adler32() speedup, minor inflate() speedup
* simple and fast serialization "framework"
* let stream_readline() fetch data chunks instead of single bytes
* partial object access in svncache_*
* tons of local optimizations (buffer sizes, copy elimination,
loop unrolling etc.)
* svnserve: wire-compression level is now a command line option
* svnserve, svnadmin: data and file handle cache sizes can now
be set from the command line
* back-ported client-relevant changes to 1.6.x to get a baseline
for c/o performance
What was accomplished?
----------------------
* sustained real-world data throughput > 140MByte/sec per thread
(wire-compression off)
* > 800 MBytes/sec threaded throughput (wire-compression off)
@25% server CPU load, client limited
* > 120 MBytes/sec threaded throughput (wire-compression on)
@50% server CPU load, client limited
* e.g. svn trunk export in 0.34 sec, c/o in 1.3sec
kde trunk export in 58 sec (8.4GB, ~400.000 files & folders)
* drastic reduction in OS overhead, especially file open & close
* e.g. 15x I/O reduction in svnadmin verify, 200x file reduction
* client runtime dominated by wire-(de-)compression and MD5 checksumming
What is left to do?
-------------------
* code cleanup (e.g. move new files to more appropriate places)
* commenting
* finalize zlib (first version submitted a while ago was well
received) and APR patches (dito, but no feedback from the APR
guys whatsoever)
* help people get bits and pieces of the code into SVN /trunk
* test and analyze on Windows
* turn the built-in performance data collection code into
a "framework"
* use that to tune caches and their usage for various commands
* use the new serialization code in more places
* add a shared mmaped file backed to the membuffer_cache to
facilitate pre-fork servers
* tune svn_mod_dav + httpd where appropriate
Performance measurements
----------------------
see attachment
(for comparison: http://svn.haxx.se/dev/archive-2010-05/0180.shtml)
Test system
-----------
2x XEON 5550 (8 cores total), 2.66GHz, hyper-threading disabled, turbo boost
disabled
24GB RAM
4x128GB cheapo ssd on RAID-0 controller w/ 256 MB cache
LINUX 2.6.28-18-generic SMP
64 bits
repositories on ext4
export to tempfs
Repository mirrors used
-----------------------
criterion tsvn tsvn_packed apache.org kde.org
repo size 5.3G 4.8G 30.4G 58.4G
revisions 18,937 18,937 943,322 1,125,690
export items 3,840 3,840 1,941 399,419
export size 81M 81M 38M 8.4G
Repository verification (technically a dump discarding the output)
------------------------------------------------------------------
1.7.trunk: ~/1.7/svnadmin verify -q $repos
prototype: ~/prototype/svnadmin verify -q -F 512 -M 16000 $repos
* tsvn (hot OS file cache because this repo is quite small)
1.7.trunk 4m02.246s real, 3m36.450s user, 0m25.086s sys
prototype 0m56.497s real, 0m54.879s user, 0m01.620s sys
1.7.trunk 4802975 r/o files, 4858443 seeks, 21449142982 bytes in
5278236 reads
prototype 21805 r/o files, 272935 seeks, 1406509016 bytes in
353616 reads
* apache (cold OS caches)
1.7.trunk 473m44.992s real, 426m41.080s user, 34m33.394s sys
prototype 87m24.859s real, 74m44.800s user, 3m02.275s sys
1.7.trunk 392465497 r/o files, 377499431 seeks, 1719424827656 bytes in
454170506 reads
prototype 1695138 r/o files, 28943613 seeks, 136101585284 bytes in
33321548 reads
* kde (cold OS caches)
1.7.trunk 2129m03.571s real, 2025m11.194s user, 87m44.689s sys
prototype 480m38.854s real, 454m33.124s user, 6m33.397s sys
1.7.trunk 960393120 r/o files, 871510017 seeks, 4836969421866 bytes in
1353804552 reads
prototype 4406426 r/o files, 74136903 seeks, 352105428412 bytes in
85352822 reads
I/O for typical client operations (pre-1.7 repository format)
-------------------------------------------------------------
./svn export -q --ignore-externals
file:///mnt/archive/svnroot/tsvn_mirror_packed/trunk /dev/shm/t
1.7.trunk 22365 r/o 13488 r/w files, 16151 seeks, 165077368 bytes in 49250
reads, 139533633 bytes in 17475 writes
prototype 3678 r/o 11433 r/w files, 14654 seeks, 154009260 bytes in 26784
reads, 139533634 bytes in 10906 writes
./svn log -v file:///mnt/archive/svnroot/tsvn_mirror_packed/trunk/www
1.7.trunk 6522 r/o files, 8438 seeks, 29087119 bytes in 7916 reads
prototype 706 r/o files, 5712 seeks, 21052707 bytes in 6492 reads
./svn log -q file:///mnt/archive/svnroot/tsvn_mirror_packed/trunk/www
1.7.trunk 5224 r/o files, 5843 seeks, 21132519 bytes in 5932 reads
prototype 706 r/o files, 860 seeks, 3530970 bytes in 2211 reads
Export using the new server, wire-compression off
-------------------------------------------------
(all run with the same svnserve instance)
~/prototype/svnserve -d -T -c 0 -F 512 -M 12000 $repos_parent
~/prototype/svn export --ignore-externals -q svn://localhost/$TOEXPORT
/dev/shm/t
1st run tsvn tsvn_packed apache.org kde.org
real 0m5.534s 0m4.803s 0m3.852s 7m27.693s
user 0m0.636s 0m0.700s 0m0.376s 0m57.980s
sys 0m0.444s 0m0.472s 0m0.192s 0m39.298s
2nd run tsvn tsvn_packed apache.org kde.org
real 0m0.747s 0m0.661s 0m0.408s 1m06.505s
user 0m0.372s 0m0.328s 0m0.224s 0m34.462s
sys 0m0.368s 0m0.324s 0m0.172s 0m23.537s
3rd run tsvn tsvn_packed apache.org kde.org
real 0m0.639s 0m0.642s 0m0.342s 0m58.237s
user 0m0.388s 0m0.368s 0m0.208s 0m33.518s
sys 0m0.248s 0m0.268s 0m0.128s 0m24.358s
MByte/s 126 126 110 148
Export using the new server, wire-compression on
-------------------------------------------------
(all run with the same svnserve instance)
~/prototype/svnserve -d -T -c 1 -F 512 -M 12000 $repos_parent
~/prototype/svn export --ignore-externals -q svn://localhost/$TOEXPORT
/dev/shm/t
kde.org 1st run 2nd run 3rd run
real 11m41.616s 4m26.154s 3m56.603s
user 01m42.998s 1m13.365s 1m24.585s
sys 00m36.098s 0m24.602s 0m25.774s
MByte/s 12.2 32.3 36.3
