Greetings, * David Rowley (dgrowle...@gmail.com) wrote: > 1. We could quickly identify when someone adds some overly complex > test and slows down the regression tests too much.
Sure, makes sense to me. We do track the individual 'stage_duration' but we don't track things down to a per-regression-test basis. To do that, I think we'd need the regression system to spit that kind of detailed information out somewhere (in a structured format) that the buildfarm client would then be able to pick it up and send to the server to write into an appropriate table. > 2. We might get some faster insight into performance regressions. There's some effort going into continuing to build out a "performance" farm, whose specific goal is to try and help exactly this issue. Trying to do that with the buildfarm has the challenge that many of those systems aren't dedicated and therefore the timing could vary wildly between runs due to entirely independent things than our code. Would certainly be great to have more people working on that. Currently it's primarily Ilaria (GSoC student), Ads and I. Current repo is https://github.com/PGPerfFarm/pgperffarm if folks want to look at it, but we're in the process of making some pretty serious changes, so now might not be the best time to look at the code. We're coordinating on the 'Postgresteam' slack in #perffarm for anyone interested tho. > I only thought of this after reading [1]. If we went ahead with that, > as of now, it feels like someone could quite easily break that > optimisation and nobody would notice for a long time. One of the goals with the perffarm is to be able to support different types of benchmarks, beyond just pgbench, so that we'd be able to add a benchmark for "numeric", perhaps, or maybe create a script with pgbench that ends up being heavy on numerics or such. > I admit to not having looked at the buildfarm code to determine how > practical such a change would be. I've assumed there is a central > database that stores all the results. Yes, there's a central database where results are pushed and that's what you see when you go to buildfarm.postgresql.org, there's also an archive server which has the logs going much farther back (and is quite a bit larger, of course..). Thanks, Stephen
signature.asc
Description: PGP signature
