On Fri, Mar 25, 2011 at 1:33 PM, Mark Phippard <markp...@gmail.com> wrote:
> I have been working on a framework for writing tests to record > performance. I have something good enough to share: > > https://ctf.open.collab.net/sf/projects/csvn > > It is pretty easy to add new tests if you have ideas on more tests you > think we should add. I think I have pretty good coverage of the major > functions. The wiki on the site I linked to above has details on how > I have constructed the current tests. I am going to put out a call to > users for feedback and try to get more people to run the tests and > record results. > > I am not claiming these are anything definitive or even that we will > use them to help us make the release decision, but I think it is a > start on coming up with some reproducible tests that people can run > easily. If after people look at and run the tests they think they are > useful or can be tweaked to be useful, then great. If not, then at > least I got to write some code for a change :) > > The tests are written in Java because that is what I know and it gives > me good cross platform coverage. However, the Java just drives the > command line so all you need to do is have the svn command line in > your PATH and that is what it uses for all the work. These tests are showing some interesting results. * There are a number of cases where 1.7 shows large performance gains over 1.6. For example, a WC with a lot of folders on Windows: https://ctf.open.collab.net/sf/wiki/do/viewPage/projects.csvn/wiki/FolderTests * Checkout seems to be the biggest remaining hotspot where 1.7 tends to be slower, although commit seems to still be slower too. * Delete and move are slower than I would have expected. Is this because we can actually delete the folder now, as opposed to waiting until commit time? I actually thought that made would make it faster as I thought the folder used to get deleted and then put back (maybe just in IDE's). * There is something seriously bad going on with Anti-Virus on my Windows laptop (Symantec Endpoint Protection). I noticed the times for commit were way out of whack so I configured my A/V to ignore the folder where the benchmarks were running. When I ran the tests again, the performance of 1.7 was waaay better. I am about 95% certain the A/V is the problem but I need to make sure. While it was great to see the good performance, it is still a big concern to see that A/V might be having an even bigger impact than it has before. 1.6 also showed performance improvements but they were less dramatic than some of the 1.7 commands. Stefan Küng asked me to add svnversion to the benchmark and I have done so in the latest release. I would love to see someone do some tests with the WC on local disk vs network mount (1.6 and 1.7). I tried to do it using some virtual machines I have access to at CollabNet. The problem is that the connection of these boxes to the NetApp with our home folders is too slow. Some of the checkouts (even using 1.6) were running for an hour and I finally killed the test. Anyway, overall I am encouraged by the results I am seeing so far. I look forward to more people running the tests and sharing the results. -- Thanks Mark Phippard http://markphip.blogspot.com/
