Re: fix psql \conninfo & \connect when using hostaddr

2018-11-17 Thread Fabien COELHO
I'm working on improving the patch. Cool. Here is the updated v2 - libpq internal function getHostaddr get a length, and I added an assert about it. - added a few braces on if/if/else/if/else/else - added an UNKNOWN_HOST macro to hide "???" - moved host_addr[] declaration earlier to

Re: [HACKERS] pgbench - allow to store select results into variables

2018-11-18 Thread Fabien COELHO
Hello Alvaro, I think this patch's Command->lines would benefit from using PQExpBuffer (or maybe StringInfo?) for the command string instead of open-coding string manipulation and allocation. [...] Ok. I'm not sure that Command->first_line is really all that useful. It seems we go to a l

Re: doc - improve description of default privileges

2018-11-18 Thread Fabien COELHO
Hello Tom, Thanks for this precise feedback. Progress on this patch seems to be blocked on the question of whether we want to keep enlarging the amount of psql-specific information in the GRANT reference page, or move that all somewhere else. Yep. FWIW, I think I agree with Peter's positio

Re: fix psql \conninfo & \connect when using hostaddr

2018-11-19 Thread Fabien COELHO
Hello Alvaro, - libpq internal function getHostaddr get a length, and I added an assert about it. - added a few braces on if/if/else/if/else/else - added an UNKNOWN_HOST macro to hide "???" - moved host_addr[] declaration earlier to avoid some braces You forgot to free(conn->connip) d

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-11-19 Thread Fabien COELHO
Hello Alvaro, I also think that the pgbench_error() patch should go in before the main one. It seems a bit pointless to introduce code using a bad API only to fix the API together with all the new callers immediately afterwards. I'm not that keen on this part of the patch, because ISTM that

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-11-19 Thread Fabien COELHO
Feel free to update a patch status to "needs review" yourself after submitting a new version that in your opinion respond to a reviewer's comments. Sure, I do that. But I will not switch any of my patch to "Ready". AFAICR the concerns where mostly about imprecise comments in the code, and a

Re: pgbench - doCustom cleanup

2018-11-20 Thread Fabien COELHO
Hello Alvaro, Thanks for the review and improvements. Attached v4 is a rebase after 409231919443984635b7ae9b7e2e261ab984eb1e Attached v5. I thought that separating the part that executes the command was an obvious readability improvement. Hmm. It is somehow, but the aim of the refactoring

Re: pgbench - doCustom cleanup

2018-11-20 Thread Fabien COELHO
I didn't quite understand this hunk. Why does it remove the is_latencies conditional? (The preceding comment shown here should be updated obviously if this change is correct, but I'm not sure it is.) Pgbench runs benches a collects performance data about it. I simplified the code to alway

Re: [HACKERS] pgbench - allow to store select results into variables

2018-11-23 Thread Fabien COELHO
Hello Alvaro, Please when you rebase, consider these (minor) changes. Here it is. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index b5e3a62a33..246944ea79 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@ -954,6 +954,87

Re: pgbench - doCustom cleanup

2018-11-24 Thread Fabien COELHO
Hello Alvaro, I just pushed this. I hope not to have upset you too much with the subroutine thing. Sorry for the feedback delay, my week was kind of overloaded… Thanks for the push. About the patch you committed, a post-commit review: - the state and function renamings are indeed a good

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-11-24 Thread Fabien COELHO
Unfortunately, there was no activity over the last few commitfests and the proposed patch pgbench-tap-progress-6 can't be applied anymore without conflicts. Fabien, what are your plans about it, could you please post a rebased version? Here it is. -- Fabien.diff --git a/src/bin/pgbench/pgbenc

Re: [HACKERS] pgbench - allow to store select results into variables

2018-11-24 Thread Fabien COELHO
FWIW I think the terminology in this patch is wrong. You use the term "compound" to mean "one query within a string containing multiple queries", but that's not what compound means. Compound is the whole thing, comprised of the multiple queries. Indeed. Compounded query? Maybe "query" is

Re: pgbench doc fix

2018-11-30 Thread Fabien COELHO
So I do not think a more precise wording harms. Maybe: "prepared: use extended query protocol with REUSED named prepared statements" would be even less slightly ambiguous. I like this. But maybe we can remove "named"? I also think it makes sense to adjust wording a bit here, and this versio

pgbench - allow to specify scale as a size

2018-02-17 Thread Fabien COELHO
After Karel Moppel piece on pgbench scale/size conversion, it occured to me that having this as an option would be nice. https://www.cybertec-postgresql.com/en/a-formula-to-calculate-pgbench-scaling-factor-for-target-db-size/ Here is a attempt at extending --scale so that it can be given a si

Re: pgbench - allow to specify scale as a size

2018-02-17 Thread Fabien COELHO
Seem a nice addition but something isn't quite right; with '-s 50' (no unit) I get: 'scale 50 too small': Sigh. Indeed, it seems that I forgot to test some cases... Thanks for the debug. Here is an hopefully better attempt. I also upgraded the regression test to test more that "-s 1". I als

Re: pgbench - allow to specify scale as a size

2018-02-17 Thread Fabien COELHO
Hello Tom, Here is a attempt at extending --scale so that it can be given a size. I do not actually find this to be a good idea. It's going to be platform-dependent, or not very accurate, or both, and thereby contribute to confusion by making results less reproducible. I have often wanted

Re: pgbench - allow to specify scale as a size

2018-02-17 Thread Fabien COELHO
    Why not then insert a "few" rows, measure size, truncate the table, compute the formula and then insert to the desired user requested size? (or insert what should be the minimum, scale 1, measure, and extrapolate what's missing). It doesn't sound too complicated to me, and targeting a size

Re: pgbench - allow to specify scale as a size

2018-02-18 Thread Fabien COELHO
Hello Alvaro & Tom, Why not then insert a "few" rows, measure size, truncate the table, compute the formula and then insert to the desired user requested size? (or insert what should be the minimum, scale 1, measure, and extrapolate what's missing). It doesn't sound too complicated to me, an

Re: pgbench - allow to specify scale as a size

2018-02-19 Thread Fabien COELHO
Hello Mark, What if we consider using ascii (utf8?) text file sizes as a reference point, something independent from the database? Why not. TPC-B basically specifies that rows (accounts, tellers, branches) are all padded to 100 bytes, thus we could consider (i.e. document) that --scale=SIZ

pgbench - test whether a variable exists

2018-02-19 Thread Fabien COELHO
Hello devs, While investigating moving pgbench expressions to fe_utils so that they can be shared with psql (\if ..., \let ?), I figure out that psql's \if has a syntax to test whether a variable exists, which is not yet available to pgbench. This patch adds the same syntax to pgbench expre

Re: pgbench - allow to specify scale as a size

2018-02-20 Thread Fabien COELHO
Now the overhead is really 60-65%. Although the specification is unambiguous, but we still need some maths to know whether it fits in buffers or memory... The point of Karel regression is to take this into account. Also, whether this option would be more admissible to Tom is still an open qu

Re: PATCH: pgbench - break out timing data for initialization phases

2018-02-21 Thread Fabien COELHO
Hello Doug, Doing the "in progress" way suffers from everything before 'generating data' possibly scrolling off the screen/window. Yeah, that is a point. I tend to "| less" when I want to see a long output in details, so it is not an issue for me. Also, I like to have an information when

Re: csv format for psql

2018-02-26 Thread Fabien COELHO
Hello Daniel, This patch implements csv as an output format in psql (\pset format csv). It's quite similar to the unaligned format, except that it applies CSV quoting rules (obviously!) and that it prints no footer and no title. As with unaligned, a header with column names is output unless tup

Re: [HACKERS] pgbench randomness initialization

2018-02-28 Thread Fabien COELHO
Hello Tom, Fabien COELHO writes: This is a simple patch that does what it says on the tin. I ran into trouble with the pgbench TAP test *even before applying the patch*, but only because I was doing a VPATH build as a user without 'write' on the source tree (001_pgbench_with_serve

Re: pgbench - test whether a variable exists

2018-03-01 Thread Fabien COELHO
Note that it is not really that useful for benchmarking, although it does not harm. That seems plenty reason to plainly reject this patch? If we end up unifying it'll be added via that, no? On the contrary, my point is to add the feature beforehand so that it is not intermixed in the large

Re: pgbench - add \if support

2018-03-01 Thread Fabien COELHO
Hello Andres, Here is a very minor rebase. Your recent patches all seem to have windows line-endings. There does not seem to be any on my side: sh> hexdump pgbench-if-6.patch | grep '0[ad]' | head -3 040 6562 636e 2e68 6773 6c6d 690a 646e 7865 060 2031 3031 3630 3434 2d0a

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2018-03-01 Thread Fabien COELHO
-#ifdef HAVE_SYS_SELECT_H +#ifdef PGBENCH_USE_SELECT /* force use of select(2)? */ +#undef HAVE_PPOLL +#endif +#ifdef HAVE_PPOLL +#include +#elif defined(HAVE_SYS_SELECT_H) +#define POLL_USING_SELECT (random thing noticed while going through patches) It strikes me as a b

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-01 Thread Fabien COELHO
Hello Andres, Thanks for managing this CF. - pgbench - allow to store query results into variables RFC, has been worked on for a while. Patch doesn't look 100% there, but not that far away from it either. What would be missing to look 100% there? A bit concerned that we're turning pgb

Re: pgbench - add \if support

2018-03-01 Thread Fabien COELHO
0040 62 65 6e 63 68 2e 73 67 6d 6c 0d 0a 69 6e 64 65 Hmmm. "0d 0a" looks bad. I've meddled into "/etc/mime.types" to change "text/x-diff" to "text/plain" used by some other mailers. Here is the v6 version again, size is 38424 and md5sum is 63d79c0d5a93294f002edc640a3f525b. -- Fabien

Re: zheap: a new storage format for PostgreSQL

2018-03-02 Thread Fabien COELHO
Hello Amit, At EnterpriseDB, we (me and some of my colleagues) are working from more than a year on the new storage format in which only the latest version of the data is kept in main storage and the old versions are moved to an undo log. [...] This looks more than great! *We’ve shown the

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-02 Thread Fabien COELHO
Hello andres & Tom, A bit concerned that we're turning pgbench into a kitchen sink. I do not understand "kitchen sink" expression in this context, and your general concerns about pgbench in various comments in your message. We're adding a lot of stuff to pgbench that only a few people use.

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-02 Thread Fabien COELHO
Hello Tom, FWIW, I share Andres' concern that pgbench is being extended far past what anyone has shown a need for. If we had infinite resources this wouldn't be a big problem, but it's eating into limited committer hours and I'm not really convinced that we're getting adequate return. Anot

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-03 Thread Fabien COELHO
Hello, [...] The consequence of this appears to be that we should integrate everything that anybody decided worthy enough to review. That just doesn't make sense, we can't maintain the project that way, nor will the design be even remotely coherent. Sure. The pgbench capabilities we are dis

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-03 Thread Fabien COELHO
Hello Peter, On the "adequate return" point, my opinion is that currently pgbench is just below the feature set needed to be generally usable, so not improving it is a self-fullfilling ensurance that it will not be used further. Once the "right" feature set is reached (for me, at least extracti

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-03 Thread Fabien COELHO
Hello Andres, For instance, I used extensively tps throttling, latencies and timeouts measures when developping and testing the checkpointer sorting & throttling patch. That doesn't say that much about proposed feature additions, we didn't say that feature isn't useful? Sure. The point I a

Desirability of client-side expressions in psql?

2018-03-03 Thread Fabien COELHO
Hello devs, This is a discussion without actual patch intended for pg12, to be added to CF 2018-09. The expected end result is either "returned with feedback", meaning proceed to send some big patch(es), or "rejected", meaning the project does not want this, no point in submitting something.

Re: pgbench - allow to specify scale as a size

2018-03-04 Thread Fabien COELHO
Now the overhead is really 60-65%. Although the specification is unambiguous, but we still need some maths to know whether it fits in buffers or memory... The point of Karel regression is to take this into account. Also, whether this option would be more admissible to Tom is still an open quest

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-04 Thread Fabien COELHO
Hello Andres, [...] I find that is a fairly absurd, and frankly insulting, ascription of motives. I do not thought that I was insulting anyone, and I apologise if anyone felt insulted by my message. I didn't "veto" the patch or anything, nor did Tom. Good. My past experience is that if

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-04 Thread Fabien COELHO
Hello Craig, I still haven't really grasped why this isn't done by embedding a client-side scripting language interpreter, giving us vastly greater capabilities with only the maintenance of the glue instead of writing our own ad-hoc scripting tool. Something confine-able like JavaScript or Lua.

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-04 Thread Fabien COELHO
Fine by me so long as it remains a manageable scope, rather than incrementally turning into some horror scripting language. Pavel's description of what is needed is really the deal. The target is "cpp" level simple scripting: expressions, variables, conditions, echoing, quitting on error...

Re: pgbench randomness initialization

2018-03-05 Thread Fabien COELHO
Hello Chapman, Here is v9. It needs s/explicitely/explicitly/ in the docs. Done. The parsing of the seed involves matters of taste, I guess: if it were a signed int, then sscanf's built-in %i would do everything those three explicit hex/octal/decimal branches do, but there's no unsigned v

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-03-05 Thread Fabien COELHO
Minor rebase after vpath fix (e94f2bc809a0c684185666f19d81f6496e732a3a). -- Fabien.diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c index 5c07dd9..1fd2451 100644 --- a/src/bin/pgbench/pgbench.c +++ b/src/bin/pgbench/pgbench.c @@ -5163,6 +5163,96 @@ main(int argc, char **argv)

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-06 Thread Fabien COELHO
Hello, Please check your numbers before criticising someone unduly. I did. I filtered emails by threads, and counted the number of messages. I do not see how this is related to the number of patch submissions or the number of reviews posted, but it is certainly counting something. The C

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-07 Thread Fabien COELHO
because pgbench isn't overflow safe. I reported that, but you didn't follow up with fixes. Indeed. AFAICR you did it before, I think that I reviewed it, it was not a period for which I had a lot of available time, and I did not feel it was something that urgent to fix because there was no prac

Re: csv format for psql

2018-03-07 Thread Fabien COELHO
Hello Daniel, Attached is a v2 fixing the bugs you mentioned, and adding ---csv/-C as discussed upthread. I'll add some regression tests shortly. Basically I'm waiting for the version with regression tests before reviewing. It is unclear whether committer will like it. From my point of vi

Re: csv format for psql

2018-03-07 Thread Fabien COELHO
Hello Pavel, psql --csv 'TABLE Stuff;' > stuff.csv There is commad -c and it should be used. The --csv options should not to have a parameter. I don't like a idea to have more options for query execution. Yes, I agree and that is indeed what I meant, sorry for the typo. The cleaner

Re: pgbench's expression parsing & negative numbers

2018-03-07 Thread Fabien COELHO
Hello Andres, working on overflow correctness in pg I noticed that pgbench isn't quite there. Indeed. I assume it won't matter terribly often, but the way it parses integers makes it incorrect for, at least, the negativemost number. [...] but that unfortunately means that the sign is no in

Re: csv format for psql

2018-03-07 Thread Fabien COELHO
psql --csv -c 'TABLE foo' > foo.csv With a -c to introduce the command. This seems pretty specialized. If we're adding something new, how about psql --format=csv -o foo.csv -c 'TABLE foo' Or we could stick with: psql -P format=csv -o foo.csv -c 'TABLE foo' Currently "-P format=c

Re: csv format for psql

2018-03-08 Thread Fabien COELHO
I.e. really generate some csv from the data in just one option, not many. But this is obviously debatable. I suspect we'll get requests for an all-JSON option, HTML tables, etc., assuming we don't have them already. I would definitely be fine with --html (which indeed already exists) & --j

Re: csv format for psql

2018-03-08 Thread Fabien COELHO
Hello Daniel, PFA a v3 patch that implements that, along with regression tests this time. My 0.02 €: Patch applies cleanly, compiles, make check ok, doc generation ok. I'm in favor of having a simple psql way to generate a convenient and compliant CSV output for export/import. I also thi

Re: csv format for psql

2018-03-08 Thread Fabien COELHO
About "fieldsep_csv", I do not like much the principle of having different output variables to represent the same concept depending on the format. I would rather have reused fieldsep as in your previous submission and set it to "," when under --csv. yes how will be possible to set differe

Re: csv format for psql

2018-03-09 Thread Fabien COELHO
Hello Daniel, About "-C", I'm fine if it is used and if it is not used, really. psql begins to be quite full of one letter options, currently 34 of them, with upper & lower cases and numbers included. The solution to set fieldsep automatically to ',' with \pset format csv is problematic.

Re: csv format for psql

2018-03-09 Thread Fabien COELHO
recordsep in the unaligned mode doesn't play the role of a line ending because the last line is not finished by recordsep. According to the source code, this is intended, see print_unaligned_text() in print.c: Something else comes to mind: CSV allows linefeeds inside fields, and we don't want

Re: Re: pgbench randomness initialization

2018-03-16 Thread Fabien COELHO
But now the documentation is back to its original state of silence on what base or how many bases might be allowed. Could it just say "or an unsigned decimal integer value"? Then no one will wonder. Done in the attached. Thanks for the reviews. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench

Re: pgbench - add \if support

2018-03-21 Thread Fabien COELHO
Hello Teodor, Patch seems usefull and commitable except comments in conditional.[ch]. I'd like to top/header comment in each file more detailed and descriptive. As for now it mentions only psql usage without explaining how it is basic or common. Indeed, it was not updated. I've fixed the fi

Re: pgbench randomness initialization

2018-03-22 Thread Fabien COELHO
Patch isn't applyed cleanly anymore. Indeed. Here is a rebase. All pgbench patches conflict about test cases. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index f07ddf1..e4582bf 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.s

Re: pg_stat_statements HLD for futur developments

2018-03-22 Thread Fabien COELHO
Hello, As a new user of PostgreSQL, I have started using pg_stat_statements, and was pleased but a little surprised: First of all, the normalized form of the query string makes it impossible to be used in EXPLAIN commands. Yes, because of the normalization. Second, normalized constants and

RE: pg_stat_statements HLD for futur developments

2018-03-22 Thread Fabien COELHO
Hello, My question is more as there are so many developments arround pg_stat_statements (see the list at the end of the initial post): What is the roadmap for its design ? None? As for any feature, some people may have some plans, that they may end up developing or not. If developed, it ma

Re: [PATCH] random_normal function

2022-12-17 Thread Fabien COELHO
Hello Paul, My 0.02€ about the patch: Patch did not apply with "git apply", I had to "patch -p1 <" and there was a bunch of warnings. Patches compile and make check is okay. The first patch adds empty lines at the end of "sql/random.sql", I think that it should not. # About random_normal

Re: [PATCH] random_normal function

2022-12-20 Thread Fabien COELHO
Bonjour Michaël, Overall, I think that there should be a clearer discussion and plan about which random functionS postgres should provide to complement the standard instead of going there… randomly:-) So, what does the specification tells about seeds, normal and random functions? A bunch of

Re: pgbench - adding pl/pgsql versions of tests

2023-01-10 Thread Fabien COELHO
of as "inverted lambda symbol" :) :-) If there are no strong objections, I'll add it to the commitfest as well Please do that. -- Fabien Coelho.

Re: [PATCH] pgbench: add multiconnect option

2023-01-10 Thread Fabien COELHO
Hello Jelte, This patch seems to have quite some use case overlap with my patch which adds load balancing to libpq itself: https://www.postgresql.org/message-id/flat/pr3pr83mb04768e2ff04818eeb2179949f7...@pr3pr83mb0476.eurprd83.prod.outlook.com Thanks for the pointer. The end purpose of the

Re: [PATCH] pgbench: add multiconnect option

2022-03-25 Thread Fabien COELHO
Pgbench is a simple benchmark tool by design, and I wonder if adding a multiconnect feature will cause pgbench to be used incorrectly. Maybe, but I do not see how it would be worse that what pgbench already allows. I agree that pgbench is simple; perhaps really too simple when it comes to

Re: psql - add SHOW_ALL_RESULTS option

2022-03-25 Thread Fabien COELHO
As Tom said earlier, wasn't this fixed by 618c16707? If not, is there any other discussion on the specifics of this issue? I'm not aware of one. Hmmm… I'll try to understand why the doubled message seems to be still there. -- Fabien.

Re: psql - add SHOW_ALL_RESULTS option

2022-03-26 Thread Fabien COELHO
Hello Peter, As Tom said earlier, wasn't this fixed by 618c16707? If not, is there any other discussion on the specifics of this issue? I'm not aware of one. This answer is that I had kept psql's calls to PQerrorMessage which reports errors from the connection, whereas it needed to change

Re: psql - add SHOW_ALL_RESULTS option

2022-03-31 Thread Fabien COELHO
Attached a rebase. -- Fabien.diff --git a/contrib/pg_stat_statements/expected/pg_stat_statements.out b/contrib/pg_stat_statements/expected/pg_stat_statements.out index e0abe34bb6..8f7f93172a 100644 --- a/contrib/pg_stat_statements/expected/pg_stat_statements.out +++ b/contrib/pg_stat_statements/

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-31 Thread Fabien COELHO
Or those three columns always, sum_lag sum_lag_2, min_lag max_lag, skipped, retried retries? Anyway now that current CF is closing, it will not be possible to change those logging design soon. Or can we change the logging design even after CF is closed? My 0.02€: I'm not sure how the offici

Re: psql - add SHOW_ALL_RESULTS option

2022-03-31 Thread Fabien COELHO
Attached a rebase. Again, after the SendQuery refactoring extraction. -- Fabien.diff --git a/contrib/pg_stat_statements/expected/pg_stat_statements.out b/contrib/pg_stat_statements/expected/pg_stat_statements.out index e0abe34bb6..8f7f93172a 100644 --- a/contrib/pg_stat_statements/expected/pg

Re: psql - add SHOW_ALL_RESULTS option

2022-04-02 Thread Fabien COELHO
Again, after the SendQuery refactoring extraction. I'm doing this locally, so don't feel obliged to send more of these. ;-) Good for me :-) -- Fabien.

Re: [PATCH] pgbench: add multiconnect option

2022-04-02 Thread Fabien COELHO
According to the cfbot this patch needs a rebase Indeed. v4 attached. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index ebdb4b3f46..d96d2d291d 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@ -29,7 +29,7 @@ PostgreSQL d

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-03 Thread Fabien COELHO
Or those three columns always, sum_lag sum_lag_2, min_lag max_lag, skipped, retried retries? What about this? (a log line is not actually folded) interval_start num_transactions sum_latency sum_latency_2 min_latency max_latency failures serialization_failures deadlock_failures retried retries

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-04 Thread Fabien COELHO
serfail dlfail because I intend to add connection failures handling at some point, and it would make more sense to add the corresponding count at the end with other fails. -- Fabien Coelho - CRI, MINES ParisTech

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-10 Thread Fabien COELHO
Hello Tom, The buildfarm is still complaining about the synopsis being too wide for PDF format. I think what we ought to do is give up on using a for log lines at all, and instead convert the documentation into a tabular list of fields. Proposal attached, which also fixes a couple of outrigh

Re: random() function documentation

2022-04-12 Thread Fabien COELHO
How about we just say "uses a linear-feedback shift register algorithm"? I think it'd be sufficient to just say that it's a deterministic pseudorandom number generator. I don't see much value in documenting the internal algorithm used. Hmmm… I'm not so sure. ISTM that people interested in us

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-19 Thread Fabien COELHO
Ok. Fine with me. Possibly at some point there was the idea that there could be other failures counted, but there are none. Also, there has been questions about the failures detailed option, or whether the reports should always be detailed, and the result may be some kind of not convincing comp

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-19 Thread Fabien COELHO
It find it a little annoying that there is no change in tests, it means that the format is not checked at all:-( Yeah. Perhaps it's a little bit hard to perform this kind of tests in the TAP test? Not really. I'll look into it. -- Fabien.

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-15 Thread Fabien COELHO
Hello Justin, Some feedback on v10: All patches apply cleanly, one on top of the previous. I really wish there would be less than 9 patches… * v10.1 doc change: ok * v10.2 doc change: ok, not sure why it is not merged with previous * v10.3 test add: could be merge with both previous Tests

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-16 Thread Fabien COELHO
About v11, ISTM that the recursive function should check for symbolic links and possibly avoid them: sh> cd data/base sh> ln -s .. foo psql> SELECT * FROM pg_ls_dir_recurse('.'); ERROR: could not stat file "./base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-16 Thread Fabien COELHO
Hello Justin, psql> SELECT * FROM pg_ls_dir_recurse('.'); ERROR: could not stat file "./base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-17 Thread Fabien COELHO
About v13, seens as one patch: Function "pg_ls_dir_metadata" documentation suggests a variable number of arguments with brackets, but parameters are really mandatory. postgres=# SELECT pg_ls_dir_metadata('.'); ERROR: function pg_ls_dir_metadata(unknown) does not exist LINE 1: SELECT pg_ls

Allow continuations in "pg_hba.conf" files

2020-03-25 Thread Fabien COELHO
Hello, After writing an unreadable and stupidly long line for ldap authentification in a "pg_hba.conf" file, I figured out that allowing continuations looked simple enough and should just be done. Patch attached. -- Fabien.diff --git a/doc/src/sgml/client-auth.sgml b/doc/src/sgml/client-aut

Re: Allow continuations in "pg_hba.conf" files

2020-03-25 Thread Fabien COELHO
Hello Justin, thanks for the feedback. - Records cannot be continued across lines. + Records can be backslash-continued across lines. Maybe say: "lines ending with a backslash are logically continued on the next line", or similar. I tried to change it along that. Since it puts a blan

Re: pgbench - add \aset to store results of a combined query

2020-03-26 Thread Fabien COELHO
Bonjour Michaël, [...] Still sounds strange to me to invent a new variable to this structure if it is possible to track the exact same thing with an existing part of a Command, or it would make sense to split Command into two different structures with an extra structure used after the parsing

Re: pgbench - refactor init functions with buffers

2020-03-27 Thread Fabien COELHO
Hello David, I'd prefer not to expand the use of pqexpbuffer in more places, and instead rather see this use StringInfo, now that's also available to frontend programs. Franckly, one or the other does not matter much to me. FWIW, I agree with Andres with regard to using StringInfo. Ok. I

Re: pgbench - rework variable management

2020-03-27 Thread Fabien COELHO
Patch v5 is a rebase with some adjustements. This patch is failing on the Windows build: https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.85698 I'm not sure if this had been fixed in v3 and this is a new issue or if it has been failing all along. Either way, it should b

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2020-03-27 Thread Fabien COELHO
This patch is registered in 2020-01, but the last message in the thread seems to be from 2019/05/23. The patch itself seems to be OK (it applies fine etc.) What do we need to get it over the line, instead of just moving it to the next one CF over and over? It does not look like the remainder

Re: psql FETCH_COUNT feature does not work with combined queries

2020-03-27 Thread Fabien COELHO
Assuming there's no one willing to fix the behavior (and that seems unlikely given we're in the middle of a 2020-01 commitfest) it makes sense to at least document the behavior. That being said, I think the proposed patch may be a tad too brief. I don't think we need to describe the exact behav

Re: pgbench - refactor init functions with buffers

2020-03-27 Thread Fabien COELHO
Ok. I find it strange to mix PQExpBuffer & StringInfo in the same file. Agreed, but we'd rather use StringInfo going forward. However, I don't think that puts you on the hook for updating all the PQExpBuffer references. Unless you want to... I cannot say that I "want" to fix something w

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2020-03-28 Thread Fabien COELHO
Hello Tom, Thanks for your feedback, I'd be rather unclear about what the actual feedback is, though. I'd interpret it as "pg does not care much about code coverage". Most clients are in the red on coverage.postgresql.org. I'd like pgbench at least to be in the green, but it does not look that

Re: pgbench - refactor init functions with buffers

2020-03-28 Thread Fabien COELHO
Hello Tom, I cannot say that I "want" to fix something which already works the same way, because it is against my coding principles. [...] I counted nearly 3500 calls under src/bin. Yeah, that's the problem. If someone does come forward with a patch to do that, I think it'd be summarily rej

Re: pgbench - refactor init functions with buffers

2020-03-28 Thread Fabien COELHO
Hello Andres, That being the case, I'd think a better design principle is "make your new code look like the code around it", which would tend to weigh against introducing StringInfo uses into pgbench when there's none there now and a bunch of PQExpBuffer instead. So I can't help thinking the a

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

2020-03-29 Thread Fabien COELHO
Hello Justin, Well, the following comment says "ignore anything but regular files", so I'm supposing that that is the behavior that we actually want here and failed to implement correctly. There might be scope for additional directory-reading functions, but I'd think you'd want more informati

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

2020-03-30 Thread Fabien COELHO
Hello, As I wrote about an earlier version of the patch, ISTM that instead of reinventing, extending, adapting various ls variants (with/without metadata, which show only files, which shows target of links, which shows directory, etc.) we would just need *one* postgres "ls" implementation which

Re: Allow continuations in "pg_hba.conf" files

2020-04-01 Thread Fabien COELHO
Hello, FWIW, I don't think so. Generally a trailing backspace is an escape character for the following newline. And '\ ' is a escaped space, which is usualy menas a space itself. In this case escape character doesn't work generally and I think it is natural that a backslash in the middle of

Re: pgbench - add \aset to store results of a combined query

2020-04-01 Thread Fabien COELHO
Bonjour Michaël, Except for the addition of a test case to skip empty results when \aset is used, I think that we are pretty good here. While hacking on the patch more by myself, I found that mixing tests for \gset and \aset was rather messy. A test for an empty result leads also to a failur

Re: pgbench - add pseudo-random permutation function

2020-04-01 Thread Fabien COELHO
Hello David, Attached is an attempt at improving things. I have added a explicit note and hijacked an existing example to better illustrate the purpose of the function. This patch does not build on Linux due to some unused functions and variables: http://cfbot.cputube.org/patch_27_1712.log

Re: Allow continuations in "pg_hba.conf" files

2020-04-01 Thread Fabien COELHO
Hi Justin, There are no assumption about backslash escaping, quotes and such, which seems reasonable given the lexing structure of the files, i.e. records of space-separated words, and # line comments. Quoting does allow words containing spaces: Ok. I meant that the continuation handling

Re: pgbench - add \aset to store results of a combined query

2020-04-02 Thread Fabien COELHO
Michaël, ISTM that I submitted a patch to test whether a variable exists in pgbench, like available in psql (:{?var} I think), Not sure if improving the readability of the tests is a reason for this patch. So I would suggest to just live with relying on debug() for now to check that a variab

Re: pgbench - add \aset to store results of a combined query

2020-04-02 Thread Fabien COELHO
Bonjour Michaël, Sure. I meant that the feature would make sense to write benchmark scripts which would use aset and be able to act on the success or not of this aset, not to resurrect it for a hidden coverage test. This could always be discussed for v14. We'll see. Or v15, or never, who k

Re: Allow continuations in "pg_hba.conf" files

2020-04-02 Thread Fabien COELHO
Hello David, +Agree. However, it would nice to update the sentence below if I understand it correctly. "+   Comments, whitespace and continuations are handled in the same way as in" pg_hba.conf In the attached v3, I've tried to clarify comments and doc about tokenization rules relating to

<    2   3   4   5   6   7   8   9   10   11   >