Re: Typo in pgbench messages.

2022-02-24 Thread Fabien COELHO
Bonjour Michaël, I think it's better to back-patch this to stable branches if there's no objection. Thought? That's only cosmetic, so I would just bother about HEAD if I were to change something like that (I would not bother at all, personally). One argument against a backpatch is that this

Re: Typo in pgbench messages.

2022-02-25 Thread Fabien COELHO
Hello Daniel, I think that the break of typographical rules is intentional to allow such simplistic low-level stream handling through pipes or scripts. I'd prefer that the format is not changed. Maybe a comment could be added to explain the reason behind it. That doesn't sound like an over

Re: Typo in pgbench messages.

2022-03-01 Thread Fabien COELHO
So fine with me wrt having a more homogeneous report. So are you fine with Kawamoto-san's patch? Yes. Patch applies cleanly (hmmm, it would have been better to have it as an attachement). Make & make check ok. Fine with me. -- Fabien.

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-02 Thread Fabien COELHO
Hello Tatsuo-san, It seems the patch is ready for committer except below. Do you guys want to do more on below? I'm planning a new review of this significant patch, possibly over the next week-end, or the next. -- Fabien.

Re: Commitfest 2022-03 Patch Triage Part 1b

2022-03-02 Thread Fabien COELHO
Hello Greg, Peter posted an updated version of Fabiens patch about a month ago (which at this point no longer applies) Attached a v15 which is a rebase, after some minor changes in the source and some new test cases added (good!). fixing a few issues, but also point at old review comments

Re: psql - add SHOW_ALL_RESULTS option

2022-03-04 Thread Fabien COELHO
Are you planning to send a rebased patch for this commit fest? Argh, I did it in a reply in another thread:-( Attached v15. So as to help moves things forward, I'd suggest that we should not to care too much about corner case repetition of some error messages which are due to libpq interna

Re: Commitfest 2022-03 Patch Triage Part 1b

2022-03-04 Thread Fabien COELHO
Just FYI. Better to follow up to the thread for the patch that's already in the CF. Otherwise your patch will missed by someone who looks at the CF entry to see the latest patch. Indeed. Done. -- Fabien.

Re: cpluspluscheck complains about use of register

2022-03-09 Thread Fabien COELHO
It seems we should just remove the use of register? I have a vague idea that it was once important to say "register" if you are going to use the variable in an asm snippet that requires it to be in a register. That might be wrong, or it might be obsolete even if once true. We could try taki

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-10 Thread Fabien COELHO
Hello Justin, I hope to look at it over the week-end. -- Fabien Coelho - CRI, MINES ParisTech

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-12 Thread Fabien COELHO
Hello Justin, Review about v34, up from v32 which I reviewed in January. v34 is an updated version of v32, without the part about lstat at the end of the series. All 7 patches apply cleanly. # part 01 One liner doc improvement about the directory flag. OK. # part 02 Add tests for vario

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-03-12 Thread Fabien COELHO
Hello Yugo-san, About Pgbench error handling v16: This patch set needs a minor rebase because of 506035b0. Otherwise, patch compiles, global and local "make check" are ok. Doc generation is ok. This patch is in good shape, the code and comments are clear. Some minor remarks below, including

Re: psql - add SHOW_ALL_RESULTS option

2022-03-12 Thread Fabien COELHO
Are you planning to send a rebased patch for this commit fest? Argh, I did it in a reply in another thread:-( Attached v15. So as to help moves things forward, I'd suggest that we should not to care too much about corner case repetition of some error messages which are due to libpq internal

Re: [PATCH] pgbench: add multiconnect option

2022-03-16 Thread Fabien COELHO
Hello Greg, It looks like David sent a patch and Fabien sent a followup patch. But there hasn't been a whole lot of discussion or further patches. It sounds like there are some basic questions about what the right interface should be. Are there specific questions that would be helpful for mov

Re: psql - add SHOW_ALL_RESULTS option

2022-03-17 Thread Fabien COELHO
Hello Peter, See attached v16 which removes the libpq workaround. I suppose this depends on https://www.postgresql.org/message-id/flat/ab4288f8-be5c-57fb-2400-e3e857f53e46%40enterprisedb.com getting committed, because right now this makes the psql TAP tests fail because of the duplicate er

Re: [PATCH] pgbench: add multiconnect option

2022-03-19 Thread Fabien COELHO
Hi Sami, Pgbench is a simple benchmark tool by design, and I wonder if adding a multiconnect feature will cause pgbench to be used incorrectly. Maybe, but I do not see how it would be worse that what pgbench already allows. A real world use-case will be helpful for this thread. Basicall

Re: psql - add SHOW_ALL_RESULTS option

2022-03-23 Thread Fabien COELHO
Hello Peter, Attached v17 is another try. The point is to record the current status, whatever it is, buggy or not, and to update the test when libpq fixes things, whenever this is done. [...] The expected output (which passes) contains this line twice: psql::2: FATAL: terminating connect

Re: PG vs LLVM 12 on seawasp, next round

2020-12-01 Thread Fabien COELHO
Hello Thomas, Since seawasp's bleeding edge LLVM installation moved to "trunk 20201114 c8f4e06b 12.0.0" ~16 days ago, it has been red. Further updates didn't help it and it's now on "trunk 20201127 6ee22ca6 12.0.0". I wonder if there is something in Fabien's scripting that needs to be tweaked

Re: Add table access method as an option to pgbench

2020-12-02 Thread Fabien COELHO
Hello David, Some feedback about v4. It looks that the option is *silently* ignored when creating a partitionned table, which currently results in an error, and not passed to partitions, which would accept them. This is pretty weird. The input check is added with an error message when both p

Re: PG vs LLVM 12 on seawasp, next round

2020-12-11 Thread Fabien COELHO
Hello Andres, I hadn't checked that before, but for the last few days there's been a different failure than the one I saw earlier: +ERROR: could not load library "/home/fabien/pg/build-farm-11/buildroot/HEAD/pgsql.build/tmp_install/home/fabien/pg/build-farm-11/buildroot/HEAD/inst/lib/postgr

Re: \gsetenv

2020-12-20 Thread Fabien COELHO
Hello David, We have \gset to set some parameters, but not ones in the environment, so I fixed this with a new analogous command, \gsetenv. I considered refactoring SetVariable to include environment variables, but for a first cut, I just made a separate function and an extra if. My 0.02€: IS

Re: pgbench failed when -f option contains a char '@'

2020-12-20 Thread Fabien COELHO
Hello, pgbench use -f filename[@weight] to receive a sql script file with a weight, ISTM that I thought of this: "pgbench -f filen@me@1" does work. sh> touch foo@bla sh> pgbench -f foo@bla@1 pgbench: fatal: empty command list for script "foo@bla" The documentation could point this ou

Re: pgbench failed when -f option contains a char '@'

2020-12-20 Thread Fabien COELHO
Hello Tom, I think we should just leave this as it is. The user can simply rename the file. Yeah. The assumption when we defined the script-weight syntax was that there's no particular reason to use "@" in a script file name, and I don't see why that's a bad assumption. The "parser" looks

Re: [PATCH] Automatic HASH and LIST partition creation

2020-12-22 Thread Fabien COELHO
CREATE TABLE foo(a int) PARTITION BY LIST(a) CONFIGURATION (FOR VALUES IN (1,2),(3,4) DEFAULT PARTITION foo_def); I would like to disagree with this syntactic approach because it would very specific to each partition method. IMHO the syntax should be as generic as possible. I'd suggest (pro

Re: [PATCH] Automatic HASH and LIST partition creation

2020-12-22 Thread Fabien COELHO
HEllo. CREATE TABLE foo(a int) PARTITION BY LIST(a) CONFIGURATION (FOR VALUES IN (1,2),(3,4) DEFAULT PARTITION foo_def); I would like to disagree with this syntactic approach because it would very specific to each partition method. IMHO the syntax should be as generic as possible. I'd sugge

Re: [PATCH] Automatic HASH and LIST partition creation

2020-12-22 Thread Fabien COELHO
BTW could you tell me a couple of words about pros and cons of c-code syntax parsing comparing to parsing using gram.y trees? I'd rather use an automatic tool (lexer/parser) if possible instead of doing it by hand if I can. If you want a really nice syntax with clever tricks, then you may n

Re: [PATCH] Automatic HASH and LIST partition creation

2020-12-23 Thread Fabien COELHO
Fabien, do you consider it possible to change the syntax of declarative partitioning too? My 0.02 €: What I think does not matter much, what committers think is the way to pass something. However, I do not think that such an idea would pass a committer:-) It is problematic as it is already

timestamp bogus parser?

2020-12-23 Thread Fabien COELHO
Hello, I tried: psql> SELECT TIMESTAMP '2020-12-23Z19:28:45'; The result of which is: 2020-12-23 00:00:00 This is disappointing. Ok, my fault, I should have written TIMESTAMPTZ, or use T instead of Z, or whatever. Anyway, is there a rational for this behavior? I would have expected ei

Re: Add table access method as an option to pgbench

2020-12-27 Thread Fabien COELHO
Hello Justin, src/test/regress/sql/create_am.sql:CREATE ACCESS METHOD heap2 TYPE TABLE HANDLER heap_tableam_handler; ... src/test/regress/sql/create_am.sql:DROP ACCESS METHOD heap2; Or maybe using SET default_tablespace instead of modifying the CREATE sql. That'd need to be done separately

Re: pgsql: Add pg_alterckey utility to change the cluster key

2020-12-28 Thread Fabien COELHO
Hello Bruce, I put the thread back on hackers. The first two keys are stored in pg_cryptokeys/ in the data directory, while the third one is retrieved using a GUC for validation at server startup for the other two. Do we necessarily have to store the first level keys within the data directo

Re: Proposed patch for key managment

2020-12-28 Thread Fabien COELHO
I want to repeat here what I said in another thread: I think ultimately we will need three commands to control the keys. First, there is the cluster_key_command, which we have now. Second, I think we will need an optional command which returns random bytes --- this would allow users to get ra

Re: trailing junk in numeric literals

2020-12-29 Thread Fabien COELHO
Hello Peter, My 0.02€: So strictly speaking this SQL code is nonstandard anyway. But our lexer has always been forgiving about not requiring space if it's not logically necessary to separate tokens. I doubt trying to change that would improve matters. Well, the idea is to diagnose potentia

Re: Proposed patch for key management

2020-12-31 Thread Fabien COELHO
Hello Bruce, The API should NOT make assumptions about the cryptographic design, what depends about what, where things are stored… ISTM that Pg should only care about naming keys, holding them when created/retrieved (but not create them), basically interacting with the key manager, passing the

Re: Proposed patch for key managment

2020-12-31 Thread Fabien COELHO
Hello Stephen, The implementations should not have to be in any particular language: Shell, Perl, Python, C should be possible. I disagree that it makes any sense to pass actual encryption out to a shell script. Yes, sure. I'm talking about key management. For actual crypto functions, ISTM

Re: Proposed patch for key managment

2020-12-31 Thread Fabien COELHO
Hello, The API to fetch the KEK doesn't care at all about where it's stored or how it's derived or anything like that. There's a relatively small change which could be made to have PG request all of the keys that it'll need on startup, if we want to go there as has been suggested elsewhere, b

Re: pgbench: option delaying queries till connections establishment?

2021-01-01 Thread Fabien COELHO
It looks like macOS doesn't have pthread barriers (via cfbot 2021, now with more operating systems): Indeed:-( I'll look into that. -- Fabien.

Re: Proposed patch for key management

2021-01-02 Thread Fabien COELHO
Hello Stephen, I'm unsure about what is the "common use case". ISTM that having the postgres process holding the master key looks like a "no" for me in any use case with actual security concern: as the database must be remotely accessible, a reasonable security assumption is that a pg account c

Re: pgbench: option delaying queries till connections establishment?

2021-01-02 Thread Fabien COELHO
It looks like macOS doesn't have pthread barriers (via cfbot 2021, now with more operating systems): Indeed:-( I'll look into that. Just for fun, the attached 0002 patch is a quick prototype of a replacement for that stuff that seems to work OK on a Mac here. (I'm not sure if the Windows

Re: PG vs LLVM 12 on seawasp, next round

2021-02-15 Thread Fabien COELHO
would manage the link path automatically, which may be quite naïve, indeed. PS Could you try blowing away the accache directory so we can rule out bad cached configure stuff? Hmmm. I've tried that before. I can do it again. I've added an explicit LD_LIBRARY_PATH, which will be trigg

Re: PG vs LLVM 12 on seawasp, next round

2021-02-15 Thread Fabien COELHO
I've added an explicit LD_LIBRARY_PATH, which will be triggered at some point later. This seems to have fixed the issue. I'm sorry for the noise and quite baffled anyway, because according to my change logs it does not seem that I modified anything from my side about the dynamic library pa

Re: Using COPY FREEZE in pgbench

2021-03-08 Thread Fabien COELHO
Hello Tatsuo-san, Currently pgbench uses plain COPY to populate pgbench_accounts table. With adding FREEZE option to COPY, the time to perform "pgbench -i" will be significantly reduced. Curent master: pgbench -i -s 100 done in 70.78 s (drop tables 0.21 s, create tables 0.02 s, client-side ge

Re: pgbench - add pseudo-random permutation function

2021-03-08 Thread Fabien COELHO
What are your current thoughts? Thanks for prodding. I still think it's a useful feature. However I don't think I'll have to time to get it done on the current commitfest. I suggest to let it sit in the commitfest to see if somebody else will pick it up -- and if not, we move it to the next

Re: pgbench - add pseudo-random permutation function

2021-03-12 Thread Fabien COELHO
Hello Dean, The implementation looks plausible too, though it adds quite a large amount of new code. A significant part of this new code the the multiply-modulo implementation, which can be dropped if we assume that the target has int128 available, and accept that the feature is not availab

Re: pgbench: option delaying queries till connections establishment?

2021-03-13 Thread Fabien COELHO
Hello Thomas, David Rowley kindly tested this for me on Windows and told me how to fix one of the macros that had incorrect error checking on that OS. So here's a new version. I'm planning to commit 0001 and 0002 soon, if there are no objections. 0003 needs some more review. I made a few mo

Re: pgbench - add pseudo-random permutation function

2021-03-13 Thread Fabien COELHO
Hello Alvaro, That doesn't sound like a bad option to me, if it makes this much simpler. The main modern system without it seems to be MSVC. The Linux, BSD, Apple, illumos, AIX systems using Clang/GCC with Intel/AMD/ARM/PowerPC CPUs have it, and the Windows systems using open source compilers

Re: pgbench: option delaying queries till connections establishment?

2021-03-13 Thread Fabien COELHO
Hello Thomas, I must say that I'm not a big fan of the macro-based all-in-capitals API for threads because it exposes some platform specific uglyness (eg THREAD_FUNC_CC) and it does not look much like clean C code when used. I liked the previous partial pthread implementation better, even if it

Re: pgbench - add pseudo-random permutation function

2021-03-13 Thread Fabien COELHO
Hello Alvaro, Clearly we got rid of a lot of code. About the tests, maybe the easiest thing to do is "skip ... if $Config{'osname'} eq 'MSWin32'". I tried that. One comment in pseudorandom_perm talks about "whitening" while the other talks about "scramble". I think they should both use th

Re: Using COPY FREEZE in pgbench

2021-03-13 Thread Fabien COELHO
Hello Tatsuo-san, So I think adding "freeze" to the copy statement should only happen in PostgreSQL 14 or later. Probably the test should be "PQserverVersion() >= 14" I think. Attached is the patch doing what you suggest. I have created a CommitFest entry for this. https://commitfest.pos

Re: Using COPY FREEZE in pgbench

2021-03-14 Thread Fabien COELHO
Hello, After giving it some thought, ISTM that there could also be a performance improvement with copy freeze from older version, so I'd suggest to add it after 9.3 where the option was added, i.e. 90300. You misunderstand about COPY FREEZE. Pre-13 COPY FREEZE does not contribute a performan

Re: psql - add SHOW_ALL_RESULTS option

2021-03-14 Thread Fabien COELHO
Hello Peter, This reply was two months ago, and nothing has happened, so I have marked the patch as RwF. Given the ongoing work on returning multiple result sets from stored procedures[0], I went to dust off this patch. Based on the feedback, I put back the titular SHOW_ALL_RESULTS option,

Re: pgbench - add pseudo-random permutation function

2021-03-14 Thread Fabien COELHO
See attached v22. v23: - replaces remaining occurences of "pr_perm" in the documentation - removes duplicated includes -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 299d93b241..9f49a6a78d 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/s

Re: pgbench - add pseudo-random permutation function

2021-03-14 Thread Fabien COELHO
Hello Alvaro, + /*- +* Apply 4 rounds of bijective transformations using key updated +* at each stage: +* +* (1) whiten: partial xors on overlapping power-of-2 subsets +* for instance with v in 0 .. 14 (i.e. with size == 15): +*

RE: pgbench: option delaying queries till connections establishment?

2020-11-02 Thread Fabien COELHO
Hello, Please complete fixes for the documentation. At least the following sentence should be fixed: ``` The last two lines report the number of transactions per second, figured with and without counting the time to start database sessions. ``` Indeed. I scanned the file but did not find o

Re: pgbench stopped supporting large number of client connections on Windows

2020-11-06 Thread Fabien COELHO
Hello Marina, While trying to test a patch that adds a synchronization barrier in pgbench [1] on Windows, Thanks for trying that, I do not have a windows setup for testing, and the sync code I wrote for Windows is basically blind coding:-( I found that since the commit "Use ppoll(2), if a

RE: pgbench: option delaying queries till connections establishment?

2020-11-07 Thread Fabien COELHO
Hello, Indeed. I took your next patch with an added explanation. I'm unclear whether proceeding makes much sense though, that is some thread would run the test and other would have aborted. Hmmm. Your comment looks good, thanks. In the previous version pgbench starts benchmarking even if so

Re: pgbench stopped supporting large number of client connections on Windows

2020-11-08 Thread Fabien COELHO
Hello Tom, Use ppoll, and start more threads but not too many? Does ppoll exist on Windows? Some g*gling suggest that the answer is no. There was a prior thread on this topic, which seems to have drifted off into the sunset: Indeed. I may have contributed to this dwindling by not adding

Re: Useless string ouput in error message

2020-11-09 Thread Fabien COELHO
I think I found a typo for the output of an error message which may cause building warning. Please refer to the attachment for the detail. Indeed. Thanks for the fix! -- Fabien.

RE: pgbench: option delaying queries till connections establishment?

2020-11-11 Thread Fabien COELHO
Hello, I can remove the line, but I strongly believe that reporting performance figures if some client connection failed thus the bench could not run as prescribed is a bad behavior. The good news is that it is probably quite unlikely. So I'd prefer to keep it and possibly submit a patch to ch

Re: pgbench: option delaying queries till connections establishment?

2020-11-14 Thread Fabien COELHO
Hello Marina, 1) It looks like pgbench will no longer support Windows XP due to the function DeleteSynchronizationBarrier. From https://docs.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-deletesynchronizationbarrier Minimum supported client: Windows 8 [desktop apps only] Minimum s

Re: pgbench: option delaying queries till connections establishment?

2020-11-16 Thread Fabien COELHO
I think the issue really is that, independent of PG lock contention, it'll take a while to establish all connections, and that starting to benchmark with only some connections established will create pretty pointless numbers. Yes. This is why I think that if we have some way to synchronize it

Re: parsing pg_ident.conf

2020-11-19 Thread Fabien COELHO
Hello Andrew, I noticed somewhat to my surprise as I was prepping the tests for the "match the whole DN" patch that pg_ident.conf is parsed using the same routines used for pg_hba.conf, and as a result the DN almost always needs to be quoted, because they almost all contain a comma e.g. "O=PGDG

Re: parsing pg_ident.conf

2020-11-20 Thread Fabien COELHO
I noticed somewhat to my surprise as I was prepping the tests for the "match the whole DN" patch that pg_ident.conf is parsed using the same routines used for pg_hba.conf, and as a result the DN almost always needs to be quoted, because they almost all contain a comma e.g. "O=PGDG,OU=Testing".

Re: Add table access method as an option to pgbench

2020-11-25 Thread Fabien COELHO
Hello David, The previous patch was based on branch "REL_13_STABLE". Now, the attached new patch v2 is based on master branch. I followed the new code structure using appendPQExpBuffer to append the new clause "using TABLEAM" in a proper position and tested. In the meantime, I also updated th

Re: pgbench and timestamps (bounced)

2020-11-26 Thread Fabien COELHO
CFM reminder. Hi, this entry is "Waiting on Author" and the thread was inactive for a while. I see this discussion still has some open questions. Are you going to continue working on it, or should I mark it as "returned with feedback" until a better time? IMHO the proposed fix is reasonab

Re: Add table access method as an option to pgbench

2020-11-27 Thread Fabien COELHO
Hello David, Some feedback about v3: In the doc I find TABLEACCESSMETHOD quite hard to read. Use TABLEAM instead? Also, the next entry uses lowercase (tablespace), why change the style? Remove space after comma in help string. I'd use the more readable TABLEAM in the help string rather th

Re: pgbench - test whether a variable exists

2020-11-30 Thread Fabien COELHO
The new status of this patch is: Waiting on Author This patch was inactive during the commitfest, so I am going to mark it as "Returned with Feedback". Fabien, are you planning to continue working on it? Not in the short term, but probably for the next CF. Can you park it there? -- Fab

Re: pgbench - test whether a variable exists

2020-11-30 Thread Fabien COELHO
Sure, I'll move it to the next CF then. I also noticed, that the first message mentions the idea of refactoring to use some code it in both pgbench and psql code. Can you, please, share a link to the thread, if it exists? Unsure. I'll try to find it if it exists. -- Fabien.

Re: pgbench - test whether a variable exists

2020-11-30 Thread Fabien COELHO
Bonjour Michaël, On Mon, Nov 30, 2020 at 12:53:20PM +0100, Fabien COELHO wrote: Not in the short term, but probably for the next CF. Can you park it there? IMO, I think that it would be better to leave this item marked as RwF for now if you are not sure, and register it again in the CF once

Re: pgbench and timestamps (bounced)

2021-01-13 Thread Fabien COELHO
thing as portability-fraught as introducing mutexes. Sure. I'm unable to do much pg work at the moment, but this should be eased quite soon. [1] https://www.postgresql.org/message-id/flat/20200227180100.zyvjwzcpiokfsqm2%40alap3.anarazel.de -- Fabien Coelho - CRI, MINES ParisTech

Re: PG vs LLVM 12 on seawasp, next round

2021-01-18 Thread Fabien COELHO
Hello Alvaro, The "no such file" error seems more like a machine local issue to me. I'll look into it when I have time, which make take some time. Hopefully over the holidays. This is still happening ... Any chance you can have a look at it? Indeed. I'll try to look (again) into it soon.

Re: pgbench: option delaying queries till connections establishment?

2021-01-30 Thread Fabien COELHO
Hello Thomas, 3 . Decide if it's sane for the Windows-based emulation to be in here too, or if it should stay in pgbench.c. Or alternatively, if we're emulating pthread stuff on Windows, why not also put the other pthread emulation stuff from pgbench.c into a "ports" file; that seems premature

Re: psql - add SHOW_ALL_RESULTS option

2021-12-23 Thread Fabien COELHO
Hello Peter, I finally took some time to look at this. Attached v11 is a rebase. This patch still has a few of the problems reported earlier this year. In [0], it was reported that certain replication commands result in infinite loops because of faulty error handling. This still happens.

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-12-23 Thread Fabien COELHO
Hello Justin, It seems that the v31 patch does not apply anymore: postgresql> git apply ~/v31-0001-Document-historic-behavior-of-links-to-directori.patch error: patch failed: doc/src/sgml/func.sgml:27410 error: doc/src/sgml/func.sgml: patch does not apply -- Fabien.

Re: psql - add SHOW_ALL_RESULTS option

2021-12-28 Thread Fabien COELHO
[...] I agree that these two behaviors in libpq are dubious, especially the second one. I want to spend some time analyzing this more and see if fixes in libpq might be appropriate. Ok. My analysis is that fixing libpq behavior is not in the scope of a psql patch, and that if I was to

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-01-02 Thread Fabien COELHO
Hello Justin, Happy new year! I think the 2nd half of the patches are still waiting for fixes to lstat() on windows. Not your problem? Here is my review about v32: All patches apply cleanly. # part 01 One liner doc improvement to tell that creation time is only available on windows. It

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-01-02 Thread Fabien COELHO
Here is my review about v32: I forgot to tell that doc generation for the cumulated changes also works. -- Fabien.

Re: psql - add SHOW_ALL_RESULTS option

2022-01-03 Thread Fabien COELHO
Hello Peter, With this "voluntary crash", the regression test output is now psql ... ok (test process exited with exit code 2) 281 ms Normally, I'd expect this during development if there was a crash somewhere, but showing this during a normal run now,

Re: psql - add SHOW_ALL_RESULTS option

2022-01-08 Thread Fabien COELHO
Hello Peter, quite weird and confusing. Maybe this type of test should be done in the TAP framework instead. Attached v13 where the crash test is moved to tap. -- Fabien.diff --git a/contrib/pg_stat_statements/expected/pg_stat_statements.out b/contrib/pg_stat_statements/expected/pg_stat_sta

Re: psql - add SHOW_ALL_RESULTS option

2022-01-15 Thread Fabien COELHO
Hello Andres, The reason this test constantly fails on cfbot windows is a use-after-free bug. Indeed! Thanks a lot for the catch and the debug! The ClearOrSaveResult function is quite annoying because it may or may not clear the result as a side effect. Attached v14 moves the status extra

Re: pgsql: Add libpq pipeline mode support to pgbench

2021-03-17 Thread Fabien COELHO
Bonjour Daniel, Ola Alvaro, Add libpq pipeline mode support to pgbench New metacommands \startpipeline and \endpipeline allow the user to run queries in libpq pipeline mode. Author: Daniel Vérité Reviewed-by: Álvaro Herrera Discussion: https://postgr.es/m/b4e34135-2bd9-4b8a-94ca-27d760da2.

Re: Using COPY FREEZE in pgbench

2021-03-20 Thread Fabien COELHO
Hello Tatsuo-san, I have looked in the code of PQprotocolVersion. The only case in which it returns 0 is there's no connection. Yes, you are right. Once the connection has been successfuly established, there's no chance it fails. So I agree with you. Attached v3 patch addresses this. The "

Re: Using COPY FREEZE in pgbench

2021-03-21 Thread Fabien COELHO
V3 works for me and looks ok. I changed it to ready in the CF app. Thank you for your review! Unfortunately it seems cfbot is not happy with the patch. Argh. Indeed, I did not thought of testing on a partitioned table:-( ISTM I did "make check" in pgbench to trigger tap tests, but possib

Re: Using COPY FREEZE in pgbench

2021-03-21 Thread Fabien COELHO
Attached is the v5 patch. About v5: doc gen ok, global and local make check ok. I did a few tests on my laptop. Is seems that copying takes a little more time, say about 10%, but vacuum is indeed very significantly reduced, so that the total time for copying and vacuuming is reduced by 10%

Re: Using COPY FREEZE in pgbench

2021-03-22 Thread Fabien COELHO
Hello Tatsuo-san, 13.2 pgbench + master branch server: done in 15.47 s (drop tables 0.19 s, create tables 0.01 s, client-side generate 9.07 s, vacuum 2.07 s, primary keys 4.13 s). With patch on master branch: done in 13.38 s (drop tables 0.19 s, create tables 0.01 s, client-side generate 9.

Re: pgbench - add pseudo-random permutation function

2021-03-30 Thread Fabien COELHO
Hello Dean, Thanks a lot for this work. This version looks much better than the previous one you sent, and has significant advantages over the one I sent, in particular avoiding the prime number stuff and large modular multiply. So this looks good! I'm happy that halves-xoring is back becau

Re: pgbench - add pseudo-random permutation function

2021-03-31 Thread Fabien COELHO
Hello Dean, First, I have a thing against erand48. Yeah, that's probably a fair point. However, all the existing pgbench random functions are using it, so I think it's fair enough for permute() to do the same (and actually 2^48 is pretty huge). Switching to a 64-bit PRNG might not be a ba

Re: pgbench - add pseudo-random permutation function

2021-03-31 Thread Fabien COELHO
Hello Dean, OK, attached is an update making this change and simplifying the rotate code, which hopefully just leaves the question of what (if anything) to do with pg_erand48(). Yep. While looking at it, I have some doubts on this part: m = (uint64) (pg_erand48(random_state.xseed) * (mask

Re: pgbench - add pseudo-random permutation function

2021-04-01 Thread Fabien COELHO
r = (uint64) (pg_erand48(random_state.xseed) * size); I do not understand why the random values are multiplied by anything in the first place… These are just random integers in the range [0,mask] and [0,size-1], formed in exactly the same way as getrand(). Indeed, erand returns a double,

Re: pgbench - add pseudo-random permutation function

2021-04-01 Thread Fabien COELHO
See attached v27 proposal. As usual, it is easier to see with the actual attachement:-) -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 50cf22ba6b..84d9566f49 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@ -1057,7 +

Re: [PATCH] Implement motd for PostgreSQL

2021-04-03 Thread Fabien COELHO
Hello Joel, This patch is one day late, my apologies for missing the deadline this year. PostgreSQL has since long been suffering from the lack of a proper UNIX style motd (message of the day). My 0.02€: apart from the Fool's day joke dimension, I'd admit that I would not mind actually hav

Re: [PATCH] Implement motd for PostgreSQL

2021-04-03 Thread Fabien COELHO
Perhaps the configuration-file parser has been fixed since to support embedded newlines? If so, then maybe it would actually be an idea to support newlines by escaping them? Dunno. If such a feature gets considered, I'm not sure I'd like to actually edit pg configuration file to change th

Re: [PATCH] Implement motd for PostgreSQL

2021-04-04 Thread Fabien COELHO
Hello Joel, My 0.02€: If such a feature gets considered, I'm not sure I'd like to actually edit pg configuration file to change the message. For the ALTER SYSTEM case, the value would be written to postgresql.auto.conf, and that file we shouldn't edit manually anyway, right? Sure. I meant

Re: pgbench - add pseudo-random permutation function

2021-04-05 Thread Fabien COELHO
Hello Dean, I think that permute should only use integer operations. I'd suggest to use one of the integer variants instead of going through a double computation and casting back to int. The internal state is based on integers, I do not see the added value of going through floats, possibly endu

Re: psql - add SHOW_ALL_RESULTS option

2021-04-06 Thread Fabien COELHO
Hello Peter, My 0.02€: I tested this updated version and do not have any comment on this version. From my point of view it could be committed. I would not bother to separate the test style ajustments. Committed. The last thing I fixed was the diff in the copy2.out regression test. The ord

Re: pgbench - add pseudo-random permutation function

2021-04-06 Thread Fabien COELHO
Hello Dean, Pushed. I decided not to go with the "joke" randu64() function, but instead used getrand() directly. This at least removes any *direct* use of doubles in permute() (though of course they're still used indirectly), and means that if getrand() is improved in the future, permute() wi

RE: psql - add SHOW_ALL_RESULTS option

2021-04-07 Thread Fabien COELHO
Hello, I met a problem after commit 3a51306722. While executing a SQL statement with psql, I can't interrupt it by pressing ctrl+c. For example: postgres=# insert into test select generate_series(1,1000); ^C^CINSERT 0 1000 Press ctrl+c before finishing INSERT, and psql still conti

Re: psql - add SHOW_ALL_RESULTS option

2021-04-08 Thread Fabien COELHO
Bonjour Julien, Attached a patch which attempts to fix this by moving the cancellation cancelling request after processing results. Thank you for your fixing. I tested and the problem has been solved after applying your patch. Thanks for the patch Fabien. I've hit this issue multiple time

Re: psql - add SHOW_ALL_RESULTS option

2021-04-08 Thread Fabien COELHO
Bonjour Michaël, I was running a long query this morning and wondered why the cancellation was suddenly broken. So I am not alone, and here you are with already a solution :) So, studying through 3a51306, this stuff has changed the query execution from a sync PQexec() to an async PQsendQuer

Re: psql - add SHOW_ALL_RESULTS option

2021-04-08 Thread Fabien COELHO
Attached v2 does as you suggest. There is not a single test of "ctrl-c" which would have caught this trivial and irritating regression. ISTM that a TAP test is doable. Should one be added? -- Fabien.

Re: psql - add SHOW_ALL_RESULTS option

2021-04-09 Thread Fabien COELHO
Yep, it looks much better. I found it strange that the later did a reset but was not doing the set. Attached v2 does as you suggest. Close enough. I was thinking about this position of the attached, which is more consistent with the rest. Given the structural complexity of the function, th

Re: psql - add SHOW_ALL_RESULTS option

2021-04-12 Thread Fabien COELHO
Hello Tom, It's right: this is dead code because all paths through the if-nest starting at line 1373 now leave results = NULL. Hence, this patch has broken the autocommit logic; Do you mean yet another feature without a single non-regression test? :-( I tend to rely on non regression tests

  1   2   3   4   5   6   7   8   9   10   >