Hi, I have some additional comments to a couple areas...
On Wed, Mar 14, 2018 at 05:33:00PM -0400, Dave Page wrote: > On Tue, Mar 13, 2018 at 11:31 PM, Hongyuan Ma <cs_maleica...@163.com> wrote: > > At the same time I hope you can help me understand the functional > > requirements of this project more clearly. Here are some of my thoughts > > on PerfFarm: > > > > - I see this comment in the client folder (In line 15 of the "pgperffarm > > \ client \ benchmarks \ pgbench.py" file): > > ''' > > # TODO allow running custom scripts, not just the default > > ''' > > Will PerfFarm have many test items so that the test item search function > > or even the test item sort function is expected to be provided? > > > > I don't know - Tomas or Mark are probably the best folks to ask about that. > I spent some time on the initial web app work, but then ran out of spare > cycles. Yes, there is potential for the number of tests to change over time. (Maybe a test is less relevant, maybe there is a more useful test down he road.) I'm haven't thought too much about sorting and searching functions. But right now there is just pgbench with 6 basic configurations: read/write, read-only, and a low, medium, and high data set size for each of those. > > - What value will be used to determine if a machine's performance is > > improving or declining? > > > > Good question - I think that needs discussion, as it's not necessarily > clear-cut when you think that performance may vary at different numbers of > clients. My thought here is that each test should provide a single metric, defined by the test itself, so expect this to be define in the test results sent to the web site. I do expect that off of the test results should be archived somehow and retrievable. But the idea I have is to have a generic high level view that all of the different tests can share. > > - I see BuildFarm assigning an animal name to each registered machine. Will > > PerfFarm also have this interesting feature? > > > > It was going to, but I forget what we decided on for a naming scheme! > Another discussion item I think - in the code, we can just use "Name" or > similar. Since the buildfarm is composed of animals, I thought plants would be a complimentary scheme? I'm also not sure if this was discussed previously... Regards, Mark
