Re: A qsort template

2021-03-13 Thread Zhihong Yu
Hi, For 0001-Add-bsearch-and-unique-templates-to-sort_template.h.patch : + * Remove duplicates from an array. Return the new size. + */ +ST_SCOPE size_t +ST_UNIQUE(ST_ELEMENT_TYPE *array, The array is supposed to be sorted, right ? The comment should mention this. Cheers On Sat, Mar 13, 2021

Re: New IndexAM API controlling index vacuum strategies

2021-03-13 Thread Peter Geoghegan
On Fri, Mar 12, 2021 at 9:34 PM Masahiko Sawada wrote: > I agreed that when we're close to overrunning the > maintnenance_work_mem space, the situation changes. If we skip it in > even that case, the next vacuum will be likely to use up > maintenance_work_mem, leading to a second index scan. Which

RE: [HACKERS] logical decoding of two-phase transactions

2021-03-13 Thread wangsh.f...@fujitsu.com
Hi, I noticed in patch v58-0001-Add-support-for-apply-at-prepare-time-to-built-i.patch > +static void > +prepare_spoolfile_name(char *path, int szpath, Oid subid, char *gid) > +{ > + PsfHashEntry *hentry; > + > + /* > + * This name is used as the key in the psf_hash HTAB. Therefore

Re: A qsort template

2021-03-13 Thread Thomas Munro
On Sat, Mar 13, 2021 at 3:49 PM Thomas Munro wrote: > On Fri, Mar 12, 2021 at 7:58 AM Andres Freund wrote: > > I wish we had the same for bsearch... :) > > Glibc already has the definition of the traditional void-based > function in /usr/include/bits/stdlib-bsearch.h, so the generated code > when

Re: Using COPY FREEZE in pgbench

2021-03-13 Thread Tatsuo Ishii
>> After giving it some thought, ISTM that there could also be a >> performance improvement with copy freeze from older version, so I'd >> suggest to add it after 9.3 where the option was added, i.e. 90300. > > You misunderstand about COPY FREEZE. Pre-13 COPY FREEZE does not Oops. I meant Pre-14,

Re: Using COPY FREEZE in pgbench

2021-03-13 Thread Tatsuo Ishii
>> I have created a CommitFest entry for this. >> https://commitfest.postgresql.org/33/3034/ > > My 0.02 € > > After giving it some thought, ISTM that there could also be a > performance improvement with copy freeze from older version, so I'd > suggest to add it after 9.3 where the option was add

Re: Using COPY FREEZE in pgbench

2021-03-13 Thread Fabien COELHO
Hello Tatsuo-san, So I think adding "freeze" to the copy statement should only happen in PostgreSQL 14 or later. Probably the test should be "PQserverVersion() >= 14" I think. Attached is the patch doing what you suggest. I have created a CommitFest entry for this. https://commitfest.pos

Re: PITR promote bug: Checkpointer writes to older timeline

2021-03-13 Thread Soumyadeep Chakraborty
Hello, PFA version 2 of the TAP test. I removed the non-deterministic sleep and introduced retries until the WAL segment is archived and promotion is complete. Some additional tidying up too. Regards, Soumyadeep (VMware) diff --git a/src/test/recovery/t/022_pitr_prepared_xact.pl b/src/test/recove

Re: amcheck hardening

2021-03-13 Thread Mark Dilger
> On Mar 13, 2021, at 11:06 AM, Peter Geoghegan wrote: > > On Sat, Mar 13, 2021 at 10:35 AM Mark Dilger > wrote: >> The testing strategy I'm using is to corrupt heap and btree pages in schema >> "public" of the "regression" database created by `make installcheck`, by >> overwriting random b

Re: pg_amcheck contrib application

2021-03-13 Thread Noah Misch
On Sat, Mar 13, 2021 at 10:51:27AM -0800, Mark Dilger wrote: > > On Mar 13, 2021, at 10:46 AM, Noah Misch wrote: > > On Fri, Mar 12, 2021 at 05:04:09PM -0800, Mark Dilger wrote: > >>> On Mar 12, 2021, at 3:24 PM, Mark Dilger > >>> wrote: > >>> and the second deals with an apparent problem with I

Re: amcheck hardening

2021-03-13 Thread Peter Geoghegan
On Sat, Mar 13, 2021 at 10:35 AM Mark Dilger wrote: > The testing strategy I'm using is to corrupt heap and btree pages in schema > "public" of the "regression" database created by `make installcheck`, by > overwriting random bytes in randomly selected locations on pages after the > page header

Re: pg_amcheck contrib application

2021-03-13 Thread Mark Dilger
> On Mar 13, 2021, at 10:46 AM, Noah Misch wrote: > > On Fri, Mar 12, 2021 at 05:04:09PM -0800, Mark Dilger wrote: >>> On Mar 12, 2021, at 3:24 PM, Mark Dilger >>> wrote: >>> >>> and the second deals with an apparent problem with IPC::Run shell expanding >>> an asterisk on some platforms b

Re: pg_amcheck contrib application

2021-03-13 Thread Noah Misch
On Fri, Mar 12, 2021 at 05:04:09PM -0800, Mark Dilger wrote: > > On Mar 12, 2021, at 3:24 PM, Mark Dilger > > wrote: > > > > and the second deals with an apparent problem with IPC::Run shell expanding > > an asterisk on some platforms but not others. That second one, if true, > > seems like a

amcheck hardening

2021-03-13 Thread Mark Dilger
Peter, I'd like your advice on the following observations, if you'd be so kind: Using the pg_amcheck command committed yesterday (thanks, Robert! thanks Tom!), I have begun investigating segfaults that sometimes occur when running the amcheck routines on intentionally corrupted databases. We'

Re: shared-memory based stats collector

2021-03-13 Thread Andres Freund
Hi, On 2021-03-13 12:53:30 +0100, Magnus Hagander wrote: > On Sat, Mar 13, 2021 at 7:20 AM Andres Freund wrote: > >I think before making things differently complicated with this patch, > >we need to clean this up, unfortunately. I think we should initially have > >- src/backend/postma

Re: pg_amcheck contrib application

2021-03-13 Thread Robert Haas
On Sat, Mar 13, 2021 at 10:20 AM Mark Dilger wrote: > We want to use values that don't look like any of the others. The complete > set of nibbles in the values above is [012345678B], leaving the set [9ACDEF] > unused. The attached patch uses the value DEADF9F9 as it seems a little > easier to

Re: Different compression methods for FPI

2021-03-13 Thread Andrey Borodin
> 13 марта 2021 г., в 06:28, Justin Pryzby написал(а): >> > Updated patch with a minor fix to configure.ac to avoid warnings on OSX. > And 2ndary patches from another thread to allow passing recovery tests. > Renamed to WAL_COMPRESSION_* > Split LZ4 support to a separate patch and support zstd

Re: pgbench - add pseudo-random permutation function

2021-03-13 Thread Fabien COELHO
Hello Alvaro, Clearly we got rid of a lot of code. About the tests, maybe the easiest thing to do is "skip ... if $Config{'osname'} eq 'MSWin32'". I tried that. One comment in pseudorandom_perm talks about "whitening" while the other talks about "scramble". I think they should both use th

Re: pg_amcheck contrib application

2021-03-13 Thread Mark Dilger
> On Mar 13, 2021, at 6:50 AM, Tom Lane wrote: > > Robert Haas writes: >> I think it would be good to use a non-zero value here. We're doing a >> lot of poking into raw bytes here, and if something goes wrong, a zero >> value is more likely to look like something normal than whatever else. >>

Re: pg_amcheck contrib application

2021-03-13 Thread Tom Lane
Robert Haas writes: > I think it would be good to use a non-zero value here. We're doing a > lot of poking into raw bytes here, and if something goes wrong, a zero > value is more likely to look like something normal than whatever else. > I suggest picking a value where all 8 bytes are the same, b

Re: pgbench - add pseudo-random permutation function

2021-03-13 Thread Alvaro Herrera
Hello, On 2021-Mar-13, Fabien COELHO wrote: > This looks like a good compromise, which can even be a little better because > we still have 64 bits ints. Yeah, we rely on those being available elsewhere. > Attached a simplified patch which does that, removing 1/3 of the code. What > do you think

Re: pg_amcheck contrib application

2021-03-13 Thread Robert Haas
On Sat, Mar 13, 2021 at 1:55 AM Mark Dilger wrote: > I thought about that, but I'm not sure that it proves much more than just > using zero. The test doesn't really do much of interest with this value, and > it doesn't seem worth complicating the test. The idea originally was that > perl's "q

Re: shared-memory based stats collector

2021-03-13 Thread Magnus Hagander
On Sat, Mar 13, 2021 at 7:20 AM Andres Freund wrote: > > Hi, > > On 2021-03-11 19:22:57 -0800, Andres Freund wrote: > > I started changing the patch to address my complaints. I'll try to do > > it as an incremental patch ontop of your 0004, but it might become too > > unwieldy. Not planning to tou

Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-13 Thread Julien Rouhaud
On Sat, Mar 13, 2021 at 12:10:29PM +0100, Pavel Stehule wrote: > > so 13. 3. 2021 v 9:53 odesílatel Julien Rouhaud napsal: > > > > I don't think that it makes sense to have multiple occurences of this > > command, > > and we should simply error out if plpgsql_curr_compile->root_ns->itemno is > >

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-03-13 Thread Mark Rofail
> > > This still fails for CI (windows) and me (linux) > > Can you provide more information about your Linux platform? So I may test > the errors for myself? > Seems the new tests fails every CI. That’s good honestly, that we found this problem. The `arrays` regression test extensively test the ne

Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-13 Thread Pavel Stehule
Hi so 13. 3. 2021 v 9:53 odesílatel Julien Rouhaud napsal: > I think that this can be a useful feature in many occasions, especially > since > the overhead is almost inexistent. > > The implementation is sensible and there are regression tests to cover the > new > feature. > > There's a problem

Re: pgbench: option delaying queries till connections establishment?

2021-03-13 Thread Fabien COELHO
Hello Thomas, I must say that I'm not a big fan of the macro-based all-in-capitals API for threads because it exposes some platform specific uglyness (eg THREAD_FUNC_CC) and it does not look much like clean C code when used. I liked the previous partial pthread implementation better, even if it

Re: [PATCH] Provide more information to filter_prepare

2021-03-13 Thread Amit Kapila
On Thu, Mar 11, 2021 at 2:44 PM Markus Wanner wrote: > > On 11.03.21 04:58, Amit Kapila wrote: > > But this happens when we are decoding prepare, so it is clear that the > > transaction is prepared, why any additional check? > > An output plugin cannot assume the transaction is still prepared and

Re: TRUNCATE on foreign table

2021-03-13 Thread Kazutaka Onishi
I have fixed the patch to pass check-world test. :D 2021年3月13日(土) 12:35 Kazutaka Onishi : > To Ibrar, > Thank you for updating the patch! > > To Amit, > Thank you for checking the patch, and I have confirmed the failure. > Now I'm trying to fix it. > > > > 2021年3月9日(火) 11:54 Amit Langote : > >> O

Re: pgbench: option delaying queries till connections establishment?

2021-03-13 Thread Thomas Munro
On Sat, Mar 13, 2021 at 9:08 PM Fabien COELHO wrote: > I must say that I'm not a big fan of the macro-based all-in-capitals API > for threads because it exposes some platform specific uglyness (eg > THREAD_FUNC_CC) and it does not look much like clean C code when used. I > liked the previous parti

Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-13 Thread Julien Rouhaud
I think that this can be a useful feature in many occasions, especially since the overhead is almost inexistent. The implementation is sensible and there are regression tests to cover the new feature. There's a problem however if you try to add multiple #routine_label commands in the same functio

Re: pgbench - add pseudo-random permutation function

2021-03-13 Thread Fabien COELHO
Hello Alvaro, That doesn't sound like a bad option to me, if it makes this much simpler. The main modern system without it seems to be MSVC. The Linux, BSD, Apple, illumos, AIX systems using Clang/GCC with Intel/AMD/ARM/PowerPC CPUs have it, and the Windows systems using open source compilers

Re: pgbench: option delaying queries till connections establishment?

2021-03-13 Thread Fabien COELHO
Hello Thomas, David Rowley kindly tested this for me on Windows and told me how to fix one of the macros that had incorrect error checking on that OS. So here's a new version. I'm planning to commit 0001 and 0002 soon, if there are no objections. 0003 needs some more review. I made a few mo

RE: [HACKERS] logical decoding of two-phase transactions

2021-03-13 Thread osumi.takami...@fujitsu.com
Hi On Friday, March 12, 2021 5:40 PM Peter Smith > Please find attached the latest patch set v58* Thank you for updating those. I'm testing the patchset and I think it's preferable that you add simple two types of more tests in 020_twophase.pl because those aren't checked by v58. (1) execute s