Re: pgbench - refactor init functions with buffers

2020-10-02 Thread Fabien COELHO
Can you elaborate what you meant by the new "print overheads should probably be avoided" comment? Because printf is slow and this is on the critical path of data generation. Printf has to interpret the format each time just to print three ints, specialized functions could be used which woul

Re: pgbench - refactor init functions with buffers

2020-09-30 Thread Heikki Linnakangas
On 09/07/2020 10:05, Fabien COELHO wrote: in favor of *PQExpBuffer(). Attached v7 is rebased v5 which uses PQExpBuffer, per cfbot. Thanks! I pushed this with small changes: - I left out the changes to executeStatement(). I'm not quite convinced it's a good idea or worth it, and it's unrelat

Re: pgbench - refactor init functions with buffers

2020-07-09 Thread Fabien COELHO
in favor of *PQExpBuffer(). Attached v7 is rebased v5 which uses PQExpBuffer, per cfbot. -- Fabien.diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index 08a5947a9e..3abc41954a 100644 --- a/src/bin/pgbench/pgbench.c +++ b/src/bin/pgbench/pgbench.c @@ -602,7 +602,9 @@ static

Re: pgbench - refactor init functions with buffers

2020-03-28 Thread Fabien COELHO
Hello Andres, That being the case, I'd think a better design principle is "make your new code look like the code around it", which would tend to weigh against introducing StringInfo uses into pgbench when there's none there now and a bunch of PQExpBuffer instead. So I can't help thinking the a

Re: pgbench - refactor init functions with buffers

2020-03-28 Thread Andres Freund
On 2020-03-28 15:16:21 -0400, Tom Lane wrote: > Andres Freund writes: > > - both stringinfo and pqexpbuffer are performance relevant in some uses, > > needing to optimize both is wasted effort > > I'm not aware that anybody is trying to micro-optimize either. https://postgr.es/m/5450.157879703

Re: pgbench - refactor init functions with buffers

2020-03-28 Thread Tom Lane
Andres Freund writes: > On 2020-03-28 14:49:31 -0400, Tom Lane wrote: >> Why? I'm not aware of any intention to deprecate/remove PQExpBuffer, >> and I doubt it'd be a good thing to try. It does some things that >> StringInfo won't, notably cope with OOM without crashing. > - code using it canno

Re: pgbench - refactor init functions with buffers

2020-03-28 Thread Andres Freund
On 2020-03-28 14:49:31 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2020-03-27 19:57:12 -0400, Tom Lane wrote: > >> That being the case, I'd think a better design principle is "make your > >> new code look like the code around it", which would tend to weigh against > >> introducing String

Re: pgbench - refactor init functions with buffers

2020-03-28 Thread Tom Lane
Andres Freund writes: > On 2020-03-27 19:57:12 -0400, Tom Lane wrote: >> That being the case, I'd think a better design principle is "make your >> new code look like the code around it", which would tend to weigh against >> introducing StringInfo uses into pgbench when there's none there now and >

Re: pgbench - refactor init functions with buffers

2020-03-28 Thread Andres Freund
Hi, On 2020-03-27 19:57:12 -0400, Tom Lane wrote: > Fabien COELHO writes: > >>> Ok. I find it strange to mix PQExpBuffer & StringInfo in the same file. > > >> Agreed, but we'd rather use StringInfo going forward. However, I don't > >> think > >> that puts you on the hook for updating all the

Re: pgbench - refactor init functions with buffers

2020-03-28 Thread David Steele
On 3/27/20 9:52 PM, Alvaro Herrera wrote: On 2020-Mar-27, Tom Lane wrote: That being the case, I'd think a better design principle is "make your new code look like the code around it", which would tend to weigh against introducing StringInfo uses into pgbench when there's none there now and a b

Re: pgbench - refactor init functions with buffers

2020-03-28 Thread Fabien COELHO
Hello Tom, I cannot say that I "want" to fix something which already works the same way, because it is against my coding principles. [...] I counted nearly 3500 calls under src/bin. Yeah, that's the problem. If someone does come forward with a patch to do that, I think it'd be summarily rej

Re: pgbench - refactor init functions with buffers

2020-03-27 Thread Alvaro Herrera
On 2020-Mar-27, Tom Lane wrote: > That being the case, I'd think a better design principle is "make your > new code look like the code around it", which would tend to weigh against > introducing StringInfo uses into pgbench when there's none there now and > a bunch of PQExpBuffer instead. So I ca

Re: pgbench - refactor init functions with buffers

2020-03-27 Thread Tom Lane
Fabien COELHO writes: >>> Ok. I find it strange to mix PQExpBuffer & StringInfo in the same file. >> Agreed, but we'd rather use StringInfo going forward. However, I don't >> think >> that puts you on the hook for updating all the PQExpBuffer references. >> Unless you want to... > I cannot sa

Re: pgbench - refactor init functions with buffers

2020-03-27 Thread Fabien COELHO
Ok. I find it strange to mix PQExpBuffer & StringInfo in the same file. Agreed, but we'd rather use StringInfo going forward. However, I don't think that puts you on the hook for updating all the PQExpBuffer references. Unless you want to... I cannot say that I "want" to fix something w

Re: pgbench - refactor init functions with buffers

2020-03-27 Thread David Steele
On 3/27/20 6:13 PM, Fabien COELHO wrote: Hello David, I'd prefer not to expand the use of pqexpbuffer in more places, and instead rather see this use StringInfo, now that's also available to frontend programs. Franckly, one or the other does not matter much to me. FWIW, I agree with Andre

Re: pgbench - refactor init functions with buffers

2020-03-27 Thread Fabien COELHO
Hello David, I'd prefer not to expand the use of pqexpbuffer in more places, and instead rather see this use StringInfo, now that's also available to frontend programs. Franckly, one or the other does not matter much to me. FWIW, I agree with Andres with regard to using StringInfo. Ok. I

Re: pgbench - refactor init functions with buffers

2020-03-27 Thread David Steele
On 11/6/19 12:48 AM, Fabien COELHO wrote: Hello Andres, Attached v3 shorten some lines and adds "append_tablespace". A v4 which just extends the patch to newly added 'G'. I'd prefer not to expand the use of pqexpbuffer in more places, and instead rather see this use StringInfo, now that's

Re: pgbench - refactor init functions with buffers

2020-01-09 Thread Fabien COELHO
Attached v3 shorten some lines and adds "append_tablespace". A v4 which just extends the patch to newly added 'G'. v5 is a rebase after 30a3e772. -- Fabien.diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index ee1134aea2..9666c644b3 100644 --- a/src/bin/pgbench/pgbench.c

Re: pgbench - refactor init functions with buffers

2019-11-05 Thread Fabien COELHO
Hello Andres, Attached v3 shorten some lines and adds "append_tablespace". A v4 which just extends the patch to newly added 'G'. I'd prefer not to expand the use of pqexpbuffer in more places, and instead rather see this use StringInfo, now that's also available to frontend programs. Fran

Re: pgbench - refactor init functions with buffers

2019-11-05 Thread Andres Freund
Hi, On 2019-10-24 08:33:06 +0200, Fabien COELHO wrote: > Attached v3 shorten some lines and adds "append_tablespace". I'd prefer not to expand the use of pqexpbuffer in more places, and instead rather see this use StringInfo, now that's also available to frontend programs. Greetings, Andres Fre

Re: pgbench - refactor init functions with buffers

2019-10-23 Thread Fabien COELHO
Hello Jeevan, +static void +executeStatementExpect(PGconn *con, const char *sql, const ExecStatusType expected, bool errorOK) +{ I think some instances like this need 80 column alignment? Yep. Applying the pgindent is kind-of a pain, so I tend to do a reasonable job by hand and rely on the

Re: pgbench - refactor init functions with buffers

2019-10-23 Thread Jeevan Ladhe
I am able to apply the v2 patch with "patch -p1 " - +static void +executeStatementExpect(PGconn *con, const char *sql, const ExecStatusType expected, bool errorOK) +{ I think some instances like this need 80 column alignment? - in initCreatePKeys(): + for (int i = 0; i < lengthof(DDLIN

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Fabien COELHO
The patch does not apply on master, needs rebase. Hmmm. "git apply pgbench-buffer-1.patch" works for me on current master. Also, I got some whitespace errors. It possible, but I cannot see any. Could you be more specific? For me it failing, see below: $ git log -1 commit ad4b7aeb84434c9

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Jeevan Ladhe
On Tue, Oct 22, 2019 at 4:36 PM Fabien COELHO wrote: > > Hello Jeevan, > > >> I haven't read the complete patch. But, I have noticed that many > >> places you changed the variable declaration from c to c++ style (i.e > >> moved the declaration in the for loop). IMHO, generally in PG, we > >> do

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Dilip Kumar
On Tue, Oct 22, 2019 at 3:30 PM Fabien COELHO wrote: > > > Hello Dilip, > > > - for (i = 0; i < nbranches * scale; i++) > > + for (int i = 0; i < nbranches * scale; i++) > > ... > > - for (i = 0; i < ntellers * scale; i++) > > + for (int i = 0; i < ntellers * scale; i++) > > { > > > > I haven't r

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Fabien COELHO
Hello Jeevan, I haven't read the complete patch. But, I have noticed that many places you changed the variable declaration from c to c++ style (i.e moved the declaration in the for loop). IMHO, generally in PG, we don't follow this convention. Is there any specific reason to do this? +1.

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Fabien COELHO
Hello Dilip, - for (i = 0; i < nbranches * scale; i++) + for (int i = 0; i < nbranches * scale; i++) ... - for (i = 0; i < ntellers * scale; i++) + for (int i = 0; i < ntellers * scale; i++) { I haven't read the complete patch. But, I have noticed that many places you changed the variable d

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Jeevan Ladhe
> > > I haven't read the complete patch. But, I have noticed that many > places you changed the variable declaration from c to c++ style (i.e > moved the declaration in the for loop). IMHO, generally in PG, we > don't follow this convention. Is there any specific reason to do > this? > +1. The

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Dilip Kumar
On Tue, Oct 22, 2019 at 12:03 PM Fabien COELHO wrote: > > > Hello, > > While developing pgbench to allow partitioned tabled, I reproduced the > string management style used in the corresponding functions, but was > pretty unhappy with that kind of pattern: > > snprintf(buf + strlen(buf), s

pgbench - refactor init functions with buffers

2019-10-21 Thread Fabien COELHO
Hello, While developing pgbench to allow partitioned tabled, I reproduced the string management style used in the corresponding functions, but was pretty unhappy with that kind of pattern: snprintf(buf + strlen(buf), sizeof(buf) - strlen(buf), ...) However adding a feature is not th