Hi hackers,
On Tue, 10 Mar 2020 09:48:23 +1300
Thomas Munro wrote:
> On Tue, Mar 10, 2020 at 8:43 AM Fabien COELHO wrote:
> > >> Thank you very much! I'm going to send a new patch set until the end of
> > >> this week (I'm sorry I was very busy in the release of Postgres Pro
> > >> 11...).
> >
On Tue, Mar 10, 2020 at 8:43 AM Fabien COELHO wrote:
> >> Thank you very much! I'm going to send a new patch set until the end of
> >> this week (I'm sorry I was very busy in the release of Postgres Pro
> >> 11...).
> >
> > Is anyone interested in rebasing this, and summarising what needs to
> > b
Hello Thomas,
Thank you very much! I'm going to send a new patch set until the end of
this week (I'm sorry I was very busy in the release of Postgres Pro
11...).
Is anyone interested in rebasing this, and summarising what needs to
be done to get it in? It's arguably a bug or at least quite
On Mon, Mar 9, 2020 at 10:00 AM Marina Polyakova
wrote:
> On 2018-11-16 22:59, Alvaro Herrera wrote:
> > On 2018-Sep-05, Marina Polyakova wrote:
> >
> >> v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch
> >> - a patch for the RandomState structure (this is used to reset a
> >> clien
Feel free to update a patch status to "needs review" yourself after
submitting a new version that in your opinion respond to a reviewer's
comments.
Sure, I do that. But I will not switch any of my patch to "Ready". AFAICR
the concerns where mostly about imprecise comments in the code, and a
On 2018-Nov-19, Fabien COELHO wrote:
>
> Hello Alvaro,
>
> > I also think that the pgbench_error() patch should go in before the main
> > one. It seems a bit pointless to introduce code using a bad API only to
> > fix the API together with all the new callers immediately afterwards.
>
> I'm no
Hello Alvaro,
I also think that the pgbench_error() patch should go in before the main
one. It seems a bit pointless to introduce code using a bad API only to
fix the API together with all the new callers immediately afterwards.
I'm not that keen on this part of the patch, because ISTM that
On 2018-Nov-19, Marina Polyakova wrote:
> On 2018-11-16 22:59, Alvaro Herrera wrote:
> > On 2018-Sep-05, Marina Polyakova wrote:
> >
> > > v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch
> > > - a patch for the RandomState structure (this is used to reset a
> > > client's
> > > ra
On 2018-11-16 22:59, Alvaro Herrera wrote:
On 2018-Sep-05, Marina Polyakova wrote:
v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch
- a patch for the RandomState structure (this is used to reset a
client's
random seed during the repeating of transactions after
serialization/dea
On 2018-Sep-05, Marina Polyakova wrote:
> v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch
> - a patch for the RandomState structure (this is used to reset a client's
> random seed during the repeating of transactions after
> serialization/deadlock failures).
Pushed this one with m
On Wed, Sep 12, 2018 at 06:12:29PM +0300, Marina Polyakova wrote:
> The discussion about this has become entangled from the beginning, because
> as I wrote in [1] at first I misread your original proposal...
The last emails are about the last reviews of Fabien, which has remained
unanswered for th
On 12-09-2018 17:04, Fabien COELHO wrote:
Hello Marina,
You can get other errors that cannot happen for only one client if you
use shell commands in meta commands:
Or if you use untrusted procedural languages in SQL expressions (see
the used file in the attachments):
Or if you try to creat
Hello Marina,
You can get other errors that cannot happen for only one client if you use
shell commands in meta commands:
Or if you use untrusted procedural languages in SQL expressions (see the used
file in the attachments):
Or if you try to create a function and perhaps replace an exis
On 11-09-2018 18:29, Fabien COELHO wrote:
Hello Marina,
Hmm, but we can say the same for serialization or deadlock errors that
were not retried (the client test code itself could not run correctly
or the SQL sent was somehow wrong, which is also the client's fault),
can't we?
I think not.
Hello Marina,
Hmm, but we can say the same for serialization or deadlock errors that were
not retried (the client test code itself could not run correctly or the SQL
sent was somehow wrong, which is also the client's fault), can't we?
I think not.
If a client asks for something "legal", bu
On 11-09-2018 16:47, Marina Polyakova wrote:
On 08-09-2018 16:03, Fabien COELHO wrote:
Hello Marina,
I'd insist in a comment that "cnt" does not include "skipped"
transactions
(anymore).
If you mean CState.cnt I'm not sure if this is practically useful
because the code uses only the sum of a
On 08-09-2018 16:03, Fabien COELHO wrote:
Hello Marina,
v11-0003-Pgbench-errors-and-serialization-deadlock-retrie.patch
- the main patch for handling client errors and repetition of
transactions with serialization/deadlock failures (see the detailed
description in the file).
About patch v11
On 08-09-2018 10:17, Fabien COELHO wrote:
Hello Marina,
Hello, Fabien!
About the two first preparatory patches.
v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch
- a patch for the RandomState structure (this is used to reset a
client's random seed during the repeating of tran
Hello Marina,
v11-0003-Pgbench-errors-and-serialization-deadlock-retrie.patch
- the main patch for handling client errors and repetition of transactions
with serialization/deadlock failures (see the detailed description in the
file).
About patch v11-3.
Patch applies cleanly on top of the
Hello Marina,
About the two first preparatory patches.
v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch
- a patch for the RandomState structure (this is used to reset a client's
random seed during the repeating of transactions after serialization/deadlock
failures).
Same ve
On 17-08-2018 14:04, Fabien COELHO wrote:
...
Or perhaps we can use a more detailed failure status so for each type
of failure we always know the command name (argument "cmd") and
whether the client is aborted. Something like this (but in comparison
with the first variant ISTM overly complicat
commandFailed: I'm not thrilled by the added boolean, which is partially
redundant with the second argument.
Do you mean that it is partially redundant with the argument "cmd" and,
for example, the meta commands errors always do not cause the abortions of
the client?
Yes. And also I'm not
On 17-08-2018 10:49, Fabien COELHO wrote:
Hello Marina,
Detailed -r report. I understand from the doc that the retry number
on the detailed per-statement report is to identify at what point
errors occur? Probably this is more or less always at the same point
on a given script, so that the mos
Hello Marina,
Detailed -r report. I understand from the doc that the retry number on
the detailed per-statement report is to identify at what point errors
occur? Probably this is more or less always at the same point on a
given script, so that the most interesting feature is to report the
n
On 15-08-2018 11:50, Fabien COELHO wrote:
Hello Marina,
Hello!
v10-0004-Pgbench-errors-and-serialization-deadlock-retrie.patch
- the main patch for handling client errors and repetition of
transactions with serialization/deadlock failures (see the detailed
description in the file).
Patch
Hello Marina,
v10-0004-Pgbench-errors-and-serialization-deadlock-retrie.patch
- the main patch for handling client errors and repetition of transactions
with serialization/deadlock failures (see the detailed description in the
file).
Patch applies cleanly.
It allows retrying a script (con
On 12-08-2018 12:14, Fabien COELHO wrote:
HEllo Marina,
Hello, Fabien!
v10-0003-Pgbench-errors-use-the-Variables-structure-for-c.patch
- a patch for the Variables structure (this is used to reset client
variables during the repeating of transactions after
serialization/deadlock failures).
About part 3:
Patch applies cleanly,
I forgot: compiles, global & local "make check" are ok.
--
Fabien.
HEllo Marina,
v10-0003-Pgbench-errors-use-the-Variables-structure-for-c.patch
- a patch for the Variables structure (this is used to reset client variables
during the repeating of transactions after serialization/deadlock failures).
This patch adds an explicit structure to manage Variables,
On 10-08-2018 17:19, Arthur Zakirov wrote:
On Fri, Aug 10, 2018 at 04:46:04PM +0300, Marina Polyakova wrote:
> +1 from me to keep initial name "pgbench_error". "pgbench_log" for new
> function looks nice to me. I think it is better than just "log",
> because "log" may conflict with natural logar
On Fri, Aug 10, 2018 at 04:46:04PM +0300, Marina Polyakova wrote:
> > +1 from me to keep initial name "pgbench_error". "pgbench_log" for new
> > function looks nice to me. I think it is better than just "log",
> > because "log" may conflict with natural logarithmic function (see "man 3
> > log").
>
On 10-08-2018 15:53, Arthur Zakirov wrote:
On Thu, Aug 09, 2018 at 06:17:22PM +0300, Marina Polyakova wrote:
> * ErrorLevel
>
> If ErrorLevel is used for things which are not errors, its name should
> not include "Error"? Maybe "LogLevel"?
On the one hand, this sounds better for me too. On the
On Thu, Aug 09, 2018 at 06:17:22PM +0300, Marina Polyakova wrote:
> > * ErrorLevel
> >
> > If ErrorLevel is used for things which are not errors, its name should
> > not include "Error"? Maybe "LogLevel"?
>
> On the one hand, this sounds better for me too. On the other hand, will not
> this be in
On 10-08-2018 11:33, Fabien COELHO wrote:
Hello Marina,
I'd suggest to let lookupCreateVariable, putVariable* as they are,
call pgbench_error with a level which does not stop the execution,
and
abort if necessary from the callers with a "aborted because of
putVariable/eval/... error" message,
Hello Marina,
I'd suggest to let lookupCreateVariable, putVariable* as they are,
call pgbench_error with a level which does not stop the execution, and
abort if necessary from the callers with a "aborted because of
putVariable/eval/... error" message, as it was done before.
There's one more
On 09-08-2018 12:28, Fabien COELHO wrote:
Hello Marina,
Hello!
v10-0002-Pgbench-errors-use-a-separate-function-to-report.patch
- a patch for a separate error reporting function (this is used to
report client failures that do not cause an aborts and this depends on
the level of debugging).
Hello Marina,
v10-0002-Pgbench-errors-use-a-separate-function-to-report.patch
- a patch for a separate error reporting function (this is used to report
client failures that do not cause an aborts and this depends on the level of
debugging).
Patch applies cleanly, compiles, global & local m
On 07-08-2018 19:21, Fabien COELHO wrote:
Hello Marina,
Hello, Fabien!
v10-0001-Pgbench-errors-use-the-RandomState-structure-for.patch
- a patch for the RandomState structure (this is used to reset a
client's random seed during the repeating of transactions after
serialization/deadlock fail
Hello Marina,
v10-0001-Pgbench-errors-use-the-RandomState-structure-for.patch
- a patch for the RandomState structure (this is used to reset a client's
random seed during the repeating of transactions after serialization/deadlock
failures).
About this v10 part 1:
Patch applies cleanly, co
The point is to avoid building the message with dynamic allocation and so
if in the end it is not used.
Ok! About avoidance - I'm afraid there's one more piece of debugging code
with the same problem:
Indeed. I'd like to avoid all instances, so that PQExpBufferData is not
needed anywhere,
On 11-07-2018 22:34, Fabien COELHO wrote:
can we try something like this?
PGBENCH_ERROR_START(DEBUG_FAIL)
{
PGBENCH_ERROR("client %d repeats the failed transaction (try %d",
Argh, no? I was thinking of something much more trivial:
pgbench_error(DEBUG, "message format %d %s...", 12,
On 11-07-2018 21:04, Alvaro Herrera wrote:
Just a quick skim while refreshing what were those error reporting API
changes about ...
Thank you!
On 2018-May-21, Marina Polyakova wrote:
v9-0001-Pgbench-errors-use-the-RandomState-structure-for-.patch
- a patch for the RandomState structure (thi
On 11-07-2018 20:49, Alvaro Herrera wrote:
On 2018-Jul-11, Marina Polyakova wrote:
can we try something like this?
PGBENCH_ERROR_START(DEBUG_FAIL)
{
PGBENCH_ERROR("client %d repeats the failed transaction (try %d",
st->id, st->retries + 1);
if
can we try something like this?
PGBENCH_ERROR_START(DEBUG_FAIL)
{
PGBENCH_ERROR("client %d repeats the failed transaction (try %d",
Argh, no? I was thinking of something much more trivial:
pgbench_error(DEBUG, "message format %d %s...", 12, "hello world");
If you really need so
Just a quick skim while refreshing what were those error reporting API
changes about ...
On 2018-May-21, Marina Polyakova wrote:
> v9-0001-Pgbench-errors-use-the-RandomState-structure-for-.patch
> - a patch for the RandomState structure (this is used to reset a client's
> random seed during the r
On 2018-Jul-11, Marina Polyakova wrote:
> can we try something like this?
>
> PGBENCH_ERROR_START(DEBUG_FAIL)
> {
> PGBENCH_ERROR("client %d repeats the failed transaction (try %d",
> st->id, st->retries + 1);
> if (max_tries)
> PGBENCH_ER
On 11-07-2018 16:24, Fabien COELHO wrote:
Hello Marina,
* -d/--debug: I'm not in favor in requiring a mandatory text argument
on this option.
As you wrote in [1], adding an additional option is also a bad idea:
Hey, I'm entitled to some internal contradictions:-)
... and discussions will
Hello Marina,
* -d/--debug: I'm not in favor in requiring a mandatory text argument on
this option.
As you wrote in [1], adding an additional option is also a bad idea:
Hey, I'm entitled to some internal contradictions:-)
I'm sceptical of the "--debug-fails" options. ISTM that --debug is
On 09-07-2018 16:05, Fabien COELHO wrote:
Hello Marina,
Hello, Fabien!
Here is a review for the last part of your v9 version.
Thank you very much for this!
Patch does not "git apply" (may anymore):
error: patch failed: doc/src/sgml/ref/pgbench.sgml:513
error: doc/src/sgml/ref/pgbench.
Hello Marina,
v9-0004-Pgbench-errors-and-serialization-deadlock-retries.patch
- the main patch for handling client errors and repetition of transactions
with serialization/deadlock failures (see the detailed description in the
file).
Here is a review for the last part of your v9 version.
Hello Alvaro,
For context: in the backend, elog() is only used for internal messages
(i.e. "can't-happen" conditions), and ereport() is used for user-facing
messages. There are many things ereport() has that elog() doesn't, such
as additional message fields (HINT, DETAIL, etc) that I think co
On 13-06-2018 22:44, Fabien COELHO wrote:
Hello Marina,
I suppose that this is related; because of my patch there may be a lot
of such code (see v7 in [1]):
- fprintf(stderr,
- "malformed variable \"%s\" value:
\"%s\"\n",
-
On 13-06-2018 22:59, Alvaro Herrera wrote:
For context: in the backend, elog() is only used for internal messages
(i.e. "can't-happen" conditions), and ereport() is used for user-facing
messages. There are many things ereport() has that elog() doesn't,
such
as additional message fields (HINT, D
On 2018-Jun-13, Fabien COELHO wrote:
> > > > With that, is there a need for elog()? In the backend we have
> > > > it because $HISTORY but there's no need for that here -- I
> > > > propose to lose elog() and use only ereport everywhere.
>
> See commit 8a07ebb3c172 which turns some ereport into
Hello Marina,
I suppose that this is related; because of my patch there may be a lot of
such code (see v7 in [1]):
- fprintf(stderr,
- "malformed variable \"%s\" value:
\"%s\"\n",
- var->name, var->svalue);
+
On 10-06-2018 10:38, Fabien COELHO wrote:
Hello Marina,
Hello!
v9-0003-Pgbench-errors-use-the-ereport-macro-to-report-de.patch
- a patch for the ereport() macro (this is used to report client
failures that do not cause an aborts and this depends on the level of
debugging).
ISTM that abort
Hello Marina,
v9-0003-Pgbench-errors-use-the-ereport-macro-to-report-de.patch
- a patch for the ereport() macro (this is used to report client failures
that do not cause an aborts and this depends on the level of debugging).
ISTM that abort() is called under FATAL.
- implementation: if poss
On 09-06-2018 16:31, Fabien COELHO wrote:
Hello Marina,
Hello!
v9-0002-Pgbench-errors-use-the-Variables-structure-for-cl.patch
- a patch for the Variables structure (this is used to reset client
variables during the repeating of transactions after
serialization/deadlock failures).
About t
On 09-06-2018 9:55, Fabien COELHO wrote:
Hello Marina,
Hello!
v9-0001-Pgbench-errors-use-the-RandomState-structure-for-.patch
- a patch for the RandomState structure (this is used to reset a
client's random seed during the repeating of transactions after
serialization/deadlock failures).
Hello Marina,
v9-0002-Pgbench-errors-use-the-Variables-structure-for-cl.patch
- a patch for the Variables structure (this is used to reset client variables
during the repeating of transactions after serialization/deadlock failures).
About this second patch:
This extracts the variable holdi
Hello Marina,
v9-0001-Pgbench-errors-use-the-RandomState-structure-for-.patch
- a patch for the RandomState structure (this is used to reset a client's
random seed during the repeating of transactions after serialization/deadlock
failures).
A few comments about this first patch.
Patch app
Hello,
Fabien COELHO wrote:
> > Looking over the diff, I find that this patch tries to do too much and
> > needs to be split up.
>
> Yep, I agree that it would help the reviewing process. On the other hand I
> have bad memories about maintaining dependent patches which interfere
> significantly.
Hello Alvaro,
I think that I'll have time for a round of review in the first half of July.
Providing a rebased patch before then would be nice.
Note that even in the absence of a rebased patch, you can apply to an
older checkout if you have some limited window of time for a review.
Yes, su
Fabien COELHO wrote:
> I think that I'll have time for a round of review in the first half of July.
> Providing a rebased patch before then would be nice.
Note that even in the absence of a rebased patch, you can apply to an
older checkout if you have some limited window of time for a review.
Lo
Hello Marina,
FYI the v8 patch does not apply anymore, mostly because of a recent perl
reindentation.
I think that I'll have time for a round of review in the first half of
July. Providing a rebased patch before then would be nice.
--
Fabien.
On Wed, 04 Apr 2018 16:07:25 +0300
Marina Polyakova wrote:
> Hello, hackers!
>
> Here there's a seventh version of the patch for error handling and
> retrying of transactions with serialization/deadlock failures in
> pgbench (based on the commit
> a08dc711952081d63577fc182fcf955958f70add). I ad
Conception of max-retry option seems strange for me. if number of retries
reaches max-retry option, then we just increment counter of failed
transaction and try again (possibly, with different random numbers). At the
end we should distinguish number of error transaction and failed transaction,
Conception of max-retry option seems strange for me. if number of retries
reaches max-retry option, then we just increment counter of failed transaction
and try again (possibly, with different random numbers). At the end we should
distinguish number of error transaction and failed transaction, t
Hello Marina,
Many thanks to both of you! I'm working on a patch in this direction..
I think that the best approach for now is simply to reset (command
zero, random generator) and start over the whole script, without
attempting to be more intelligent. The limitations should be clearly
documen
On 25-03-2018 15:23, Fabien COELHO wrote:
Hm, I took a look on both thread about patch and it seems to me now
it's overcomplicated. With recently committed enhancements of pgbench
(\if, \when) it becomes close to impossible to retry transation in
case of failure. So, initial approach just to ro
Hm, I took a look on both thread about patch and it seems to me now it's
overcomplicated. With recently committed enhancements of pgbench (\if,
\when) it becomes close to impossible to retry transation in case of
failure. So, initial approach just to rollback such transaction looks
more attra
I suggest a patch where pgbench client sessions are not disconnected because of
serialization or deadlock failures and these failures are mentioned in reports.
In details:
- transaction with one of these failures continue run normally, but its result
is rolled back;
- if there were these failur
Hello,
Hi! I'm very sorry that I did not answer for so long, I was very busy in
the release of Postgres Pro 10 :(
Here is the third version of the patch for pgbench thanks to Fabien
Coelho comments. As in the previous one, transactions with
serialization and deadlock failures are rolled bac
73 matches
Mail list logo
