=?utf-8?B?6YKx5a6H6Iiq?= <iam...@gmail.com> writes:
> I wrote a script and test on branch REL_[10-16]_STABLE, and do see
> performance drop in REL_13_STABLE, which is about 1~2%.
I'm really skeptical that we should pay much attention to these numbers.
You've made several of the mistakes that we typically tell people not to
make when using pgbench:
* scale <= number of sessions means you're measuring a lot of
row-update contention
* once you crank up the scale enough to avoid that problem, running
with the default shared_buffers seems like a pretty poor choice
* 10-second runtime is probably an order of magnitude too small
to get useful, reliable numbers
On top of all that, discrepancies on the order of a percent or two
commonly arise from hard-to-control-for effects like the cache
alignment of hot spots in different parts of the code. That means
that you can see changes of that size from nothing more than
day-to-day changes in completely unrelated parts of the code.
I'd get excited about say a 10% performance drop, because that's
probably more than noise; but I'm not convinced that any of the
differences you show here are more than noise.
regards, tom lane
