darcs [1] is slow in a few places, and I'm working on benchmarking tool in Perl to help monitor the performance. I'm got some questions about the best way to proceed.
1. http://www.darcs.net/ So far: I've divided the task into a couple specific problems: A. What repos to use for testing? B. Actually timing various darcs binaries running the same command against the test repos. A. Creating test repos --------------------- My first attempt at creating 'test repos' was to generate them randomly. As I got further into that, I decided it would be incredibly difficult to randomly create a patch history that approximates real world development. Now I'm thinking: Why not use a few real world open source repos as starting points? We could keep a few static 'read only' copies around, modifying them just enough to create the condition we need to to test. This would mean a lot of megabytes to distribute the whole benchmarking suite, but I'm OK wit that. B. How to test benchmark system calls from Perl ------------------------------------------------- My first stop was the Benchmark module. I was surprised it didn't seem to work for this-- it would report "0" times, although the command clearly ran for 10 seconds. It appeared there was a newer version to use, but it's bundled with Perl and I didn't want to go through that hassle. My solution? my $out = `time $bin diff 1/1 2>&1`; # XXX Parsing of time output may be fragile $out =~ m/\s*([\d\.]*\s+real.*)/; Ouch. Perhaps my whole approach is wrong. Am I overlooking a good open source tool to benchmark binaries with? What better ways are there to benchmark system calls from Perl? Mark -- http://mark.stosberg.com/
