On Sat, Oct 5, 2013 at 6:10 PM, Noah Misch wrote:
The sum of the squares of the latencies wraps after 2^63/(10^12 * avg_latency
* nclients) seconds. That's unlikely to come up with the ordinary pgbench
script, but one can reach it in a few hours when benchmarking a command that
runs for many
On Sat, Oct 5, 2013 at 6:10 PM, Noah Misch wrote:
> The sum of the squares of the latencies wraps after 2^63/(10^12 * avg_latency
> * nclients) seconds. That's unlikely to come up with the ordinary pgbench
> script, but one can reach it in a few hours when benchmarking a command that
> runs for m
pgbench already offers two schedules of "pgbench --initialize" messaging,
message-per-100k-rows and message-per-5s. A user too picky to find
satisfaction in either option can filter the messages through grep, sed et al.
We patched pgbench on two occasions during the 9.3 cycle to arrive at that
On Sun, Oct 06, 2013 at 06:44:11PM +0200, Fabien COELHO wrote:
> >>>Patch (2): Make --initialize mode respect --progress.
> >>>Rejected
> >>
> >>I missed this one...
> >
> >See the second half of this message, including quoted material:
> >http://www.postgresql.org/message-id/CA+TgmoZNXkm-EtszHX=kw
Patch (2): Make --initialize mode respect --progress.
Rejected
I missed this one...
See the second half of this message, including quoted material:
http://www.postgresql.org/message-id/CA+TgmoZNXkm-EtszHX=kwq34h5ni4cs8dg31no86cmdryaq...@mail.gmail.com
I did not read Peter Haas comments as
On Sun, Oct 06, 2013 at 05:04:56PM +0200, Fabien COELHO wrote:
> > Patch (2): Make --initialize mode respect --progress.
> >Rejected
>
> I missed this one...
See the second half of this message, including quoted material:
http://www.postgresql.org/message-id/CA+TgmoZNXkm-EtszHX=kwq34h5ni4cs8dg31n
Hello Noah,
Patch (1): Change the default from --progress=0 to --progress=5
Rejected
Yep. Too bad, esp as the default is meaningless and does not scale.
Patch (2): Make --initialize mode respect --progress.
Rejected
I missed this one...
This is just about having the same option (--prog
On Sun, Oct 06, 2013 at 09:40:40AM +0200, Fabien COELHO wrote:
> Which part do you want as a next step?
I think we're done; here are the distinct changes in your original patch,
along with their dispositions:
Patch (1): Change the default from --progress=0 to --progress=5
Rejected
Patch (2)
Hello Noah,
Patch (4): Redefine "latency" as reported by pgbench and report "lag" more.
Here is a first partial patch, which focusses on measuring latency
and reporting the measure under --progress.
This patch contains the features pertaining to both hypothetical patches (3)
and (4), not ju
On Thu, Sep 26, 2013 at 08:50:15PM +0200, Fabien COELHO wrote:
> > Patch (4): Redefine "latency" as reported by pgbench and report "lag" more.
>
> Here is a first partial patch, which focusses on measuring latency
> and reporting the measure under --progress.
This patch contains the features pert
Patch (4): Redefine "latency" as reported by pgbench and report "lag" more.
Here is a first partial patch, which focusses on measuring latency and
reporting the measure under --progress.
**
Improve pgbench measurements & progress report
Measure transaction latency instead under --rate an
My feelings on the patch split haven't changed; your three bullet points call
for four separate patches. Conflicting patches are bad, but dependent patches
are okay;
Indeed, this is the only solution if you do not want one patch. Note that
it will not possible to backtrack one of the patch b
On Tue, Sep 24, 2013 at 10:41:17PM +0200, Fabien COELHO wrote:
> These changes are coupled because measures are changed, and their
> reporting as well. Submitting separate patches for these different
> features would result in conflicting or dependent patches, so I
> wish to avoid that if possible.
On Tue, Sep 24, 2013 at 08:42:15PM +0200, Fabien COELHO wrote:
>> meet all those goals simultaneously with simpler code, can we not?
>>
>> int64 wait = (int64) (throttle_delay *
>> Min(7.0, -log(1 - pg_erand48(thread->random_state;
>
> If you truncate roughly t
Split 3 of the initial submission, which actually deal with data measured and
reported on stderr under various options.
This version currently takes into account many comments by Noah Mish. In
particular, the default "no report" behavior under benchmarking is not
changed, although I really t
On 09/22/2013 10:44 PM, Fabien COELHO wrote:
> Due to the possibly repetitive table structure of the data, maybe CSV
> would make sense as well. It is less extensible, but it is directly
> usable by sqlite or pg.
I'd be OK with CSV. At least I wouldn't be regexing the output.
--
Josh Berkus
Pos
Dear Robert,
(1) ...
I really don't think it's a good idea to change the default behavior.
We've had many discussions about how the overhead of gettimeofday() can
sometimes be surprisingly large; I don't think we should assume it's
guaranteed to be small in this case.
Also, changing establ
Hello Noah,
meet all those goals simultaneously with simpler code, can we not?
int64 wait = (int64) (throttle_delay *
Min(7.0, -log(1 - pg_erand48(thread->random_state;
If you truncate roughly the multipler, as it is done here with "min", you
nece
On Sun, Sep 22, 2013 at 08:46:55PM +0200, Fabien wrote:
> pgbench: reduce and compensate throttling underestimation bias.
>
> This is a consequence of relying on an integer random generator,
> which allow to ensure that delays inserted stay reasonably in
> range of the target average delay.
>
> The
On Sun, Sep 22, 2013 at 08:44:08PM +0200, Fabien COELHO wrote:
> pgbench: minor update of documentation & help message.
>
> Use NUM in help message for homogeneity with other options. The target
> *start* time of the transaction is set by the stochastic process which is
> doing the throttling (-
On Sat, Sep 21, 2013 at 4:55 AM, Fabien COELHO wrote:
>>> - Use progress option both under init & bench.
>>>
>>>Activate progress report by default, every 5 seconds.
>>>When initializing, --quiet reverts to the old every 100,000 insertions
>>>behavior...
>>
>> Patch (1): Change the d
Dear Peter,
Split 3 of the initial submission, which actually deal with data
measured and reported on stderr under various options.
It seems this patch doesn't apply. Does it need the first two applied
first?
Oops. Indeed. The patch is fully independent of the two others. It was
generated
On 9/22/13 2:58 PM, Fabien wrote:
> Split 3 of the initial submission, which actually deal with data
> measured and reported on stderr under various options.
It seems this patch doesn't apply. Does it need the first two applied
first?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postg
Hello Josh,
As long as you're hacking pgbench output, what about offering a JSON
option instead of the text output? I'm working on automating pgbench
performance testing, and having the output in a proper delimited format
would be really helpful.
I'm more a "grep | cut | ..." person, but I d
Fabien,
As long as you're hacking pgbench output, what about offering a JSON
option instead of the text output? I'm working on automating pgbench
performance testing, and having the output in a proper delimited format
would be really helpful.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexpe
Split 3 of the initial submission, which actually deal with data measured
and reported on stderr under various options.
This version currently takes into account many comments by Noah Mish. In
particular, the default "no report" behavior under benchmarking is not
changed, although I really t
Split 2 of the initial submission
pgbench: reduce and compensate throttling underestimation bias.
This is a consequence of relying on an integer random generator,
which allow to ensure that delays inserted stay reasonably in
range of the target average delay.
The bias was about 0.5% with 1000
Split 1 of the initial submission.
pgbench: minor update of documentation & help message.
Use NUM in help message for homogeneity with other options. The target
*start* time of the transaction is set by the stochastic process which is
doing the throttling (--rate), not the end time.
--
Fabi
It is also printed without --rate. There is a "if" above because there is
one report with "lag" (under --rate), and one without.
The code I quoted is for the final report in printResults(), and that only
shows latency mean/stddev when using --rate. The progress reporting in
threadRun() does h
Dear Noah,
Thanks for your answers and remarks,
[...]
I'll split some part of the patch where there is no coupling, but I do
not want to submit conflicting patches.
Those benefits aren't compelling enough to counterbalance the risk of
gettimeofday() overhead affecting results. (Other opin
On Sat, Sep 21, 2013 at 10:55:54AM +0200, Fabien COELHO wrote:
>>> Improve pgbench measurements & progress report
>>
>> These changes are loosely coupled; please separate them into several patch
>> files:
>
> I thought about this. I submitted a bunch of very small pgbench patches
> to the previous
Moreover, using 'xxx=figure" breaks simple "cut" pipelining to extract the
figures, so I would prefer to stick to spaces.
Maybe:
progress: 36.0 s, 115.2 tps, lat avg 9.678 ms stddev 1.792, lag 0.143 ms
but I liked my "+-" approach:-)
100 +- 3 implies a range of 97 to 103 and no values are
On Sat, Sep 21, 2013 at 4:55 AM, Fabien COELHO wrote:
>
>
> I suggest getting the term "stddev" in there somehow, maybe like this:
>>
>> progress: 37.0 s, 115.2 tps, latency avg=8.678 ms stddev=1.792
>>
>
> My issue is to try to keep the line width under control so as to avoid
> line breaks in
Hello Noah,
Thanks a lot for all these comments!
I'm not planning to apply all of them directly, especially removing
features that I think really desirable. Please find a defense against some
of your suggestions. I wish to wait for some more feedback about these
arguments before spending tim
For others following along, Pavel Stehule reviewed this on a different thread:
http://www.postgresql.org/message-id/cafj8pravyuhv8x3feewasggvdcijogvlzsavd7rmwn9mw8r...@mail.gmail.com
On Tue, Aug 06, 2013 at 10:47:14AM +0200, Fabien wrote:
> Improve pgbench measurements & progress report
These cha
Here is a patch submission for reference to the next commitfest.
Please replace the tabs in the SGML files with spaces.
Attached a v2 with tabs replaced by spaces.
--
Fabien.diff --git a/contrib/pgbench/pgbench.c b/contrib/pgbench/pgbench.c
index ad8e272..a90b188 100644
--- a/contrib/pgbenc
On Tue, 2013-08-06 at 10:47 +0200, Fabien wrote:
> Here is a patch submission for reference to the next commitfest.
Please replace the tabs in the SGML files with spaces.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.post
Here is a patch submission for reference to the next commitfest.
Improve pgbench measurements & progress report
- Use progress option both under init & bench.
Activate progress report by default, every 5 seconds.
When initializing, --quiet reverts to the old every 100,000 insertions
38 matches
Mail list logo
