On Thu, 18 Jun 2026 11:24:04 +0800 Chao Li <[email protected]> wrote:
> Hi > > While testing “[0ab208fa5] pgbench: Add --continue-on-error option”, I got a > question about the doc: > ``` > <para> > This option is useful when your custom script may raise errors > such as unique constraint violations, but you want the benchmark > to continue and measure performance including those failures. > </para> > ``` > > The phrase “measure performance including those failures” gives the > impression that failed transactions might be counted in TPS, but they are not. I agree with your point. The original phrasing "including those failures" is indeed ambiguous and could mislead users into thinking failed transactions are included in the TPS calculation. > In my test, all transactions failed, and TPS was reported as 0: > ``` > % pgbench -d evantest -n -t 5 --continue-on-error --failures-detailed -f > pgbench-coe.sql > pgbench (19beta1) > transaction type: pgbench-coe.sql > scaling factor: 1 > query mode: simple > number of clients: 1 > number of threads: 1 > maximum number of tries: 1 > number of transactions per client: 5 > number of transactions actually processed: 0/5 > number of failed transactions: 5 (100.000%) > number of serialization failures: 0 (0.000%) > number of deadlock failures: 0 (0.000%) > number of other failures: 5 (100.000%) > latency average = 0.389 ms (including failures) > initial connection time = 4.172 ms > tps = 0.000000 (without initial connection time) > ``` > > If this behavior is intended, I wonder if we should clarify the TPS counting > logic more explicitly. For example: > ``` > <para> > This option is useful when your custom script may raise errors > such as unique constraint violations, but you want the benchmark > to continue despite individual statement failures. Failed transactions > are reported separately and included in latency statistics, but are > not > counted as transactions actually processed for TPS. > </para> > ``` Your proposed documentation change is much clearer and accurately reflects the actual behavior. I prefer this version as well. Regards, Yugo Nagata -- Yugo Nagata <[email protected]>
