> On Jun 18, 2026, at 13:36, Yugo Nagata <[email protected]> wrote: > > On Thu, 18 Jun 2026 11:24:04 +0800 > Chao Li <[email protected]> wrote: > >> Hi >> >> While testing “[0ab208fa5] pgbench: Add --continue-on-error option”, I got a >> question about the doc: >> ``` >> <para> >> This option is useful when your custom script may raise errors >> such as unique constraint violations, but you want the benchmark >> to continue and measure performance including those failures. >> </para> >> ``` >> >> The phrase “measure performance including those failures” gives the >> impression that failed transactions might be counted in TPS, but they are >> not. > > I agree with your point. The original phrasing "including those failures" is > indeed ambiguous and could mislead users into thinking failed transactions are > included in the TPS calculation. > >> In my test, all transactions failed, and TPS was reported as 0: >> ``` >> % pgbench -d evantest -n -t 5 --continue-on-error --failures-detailed -f >> pgbench-coe.sql >> pgbench (19beta1) >> transaction type: pgbench-coe.sql >> scaling factor: 1 >> query mode: simple >> number of clients: 1 >> number of threads: 1 >> maximum number of tries: 1 >> number of transactions per client: 5 >> number of transactions actually processed: 0/5 >> number of failed transactions: 5 (100.000%) >> number of serialization failures: 0 (0.000%) >> number of deadlock failures: 0 (0.000%) >> number of other failures: 5 (100.000%) >> latency average = 0.389 ms (including failures) >> initial connection time = 4.172 ms >> tps = 0.000000 (without initial connection time) >> ``` >> >> If this behavior is intended, I wonder if we should clarify the TPS counting >> logic more explicitly. For example: >> ``` >> <para> >> This option is useful when your custom script may raise errors >> such as unique constraint violations, but you want the benchmark >> to continue despite individual statement failures. Failed transactions >> are reported separately and included in latency statistics, but are >> not >> counted as transactions actually processed for TPS. >> </para> >> ``` > > Your proposed documentation change is much clearer and accurately reflects > the actual behavior. I prefer this version as well. > > Regards, > Yugo Nagata > > -- > Yugo Nagata <[email protected]>
Thanks for confirming the doc issue. The here comes a small doc patch. I also noticed there is a later paragraph about “latency” that also needs an update, it’s included in the patch as well. Best regards, -- Chao Li (Evan) HighGo Software Co., Ltd. https://www.highgo.com/
v1-0001-doc-Clarify-pgbench-reporting-with-continue-on-er.patch
Description: Binary data
