RE: Transactions involving multiple postgres foreign servers, take 2

2020-09-30 Thread tsunakawa.ta...@fujitsu.com
From: Masahiko Sawada > To avoid misunderstanding, I didn't mean to disregard the performance. > I mean especially for the transaction management feature it's > essential to work fine even in failure cases. So I hope we have a > safe, robust, and probably simple design for the first version that >

Re: [PATCH] Remove useless distinct clauses

2020-09-30 Thread Michael Paquier
On Tue, Sep 15, 2020 at 10:57:04PM +1200, David Rowley wrote: > Unfortunately, there are quite a few issues with what you have: This review has not been answered after two weeks, so this is marked as RwF. -- Michael signature.asc Description: PGP signature

Re: track_planning causing performance regression

2020-09-30 Thread Michael Paquier
On Fri, Sep 11, 2020 at 03:32:54PM -0700, Andres Freund wrote: > The piece about a single shared lwlocks is/was about protecting the set > of entries that are currently in-memory - which can't easily be > implemented just using atomics (at least without the risk of increasing > the counters of an e

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-09-30 Thread Noah Misch
On Wed, Sep 30, 2020 at 03:06:40PM +0900, Michael Paquier wrote: > On Sun, Sep 06, 2020 at 08:14:21PM -0700, Noah Misch wrote: > > http://cfbot.cputube.org/patch_29_2026.log applies the two patches in the > > wrong order. For this CF entry, ignore it. > > OK, thanks. This is a bug fix, so I have

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2020-09-30 Thread Michael Paquier
On Thu, Sep 17, 2020 at 02:53:27PM +0900, Michael Paquier wrote: > As far as I can see, patches 0001 and 0002 have been already applied, > but not 0003. Could you send a rebase to allow the CF bot to run, at > least? This was two weeks ago. Looking at 0003, the thing is not really complicated, b

Re: Add timeline to partial WAL segments

2020-09-30 Thread Michael Paquier
On Sat, Jan 05, 2019 at 08:32:55AM +0200, David Steele wrote: > Excellent. This now requires work on my part so I have marked the patch > "Waiting on Author". As nothing has happened here, I have marked the patch as RwF. -- Michael signature.asc Description: PGP signature

Re: Why does PostgresNode.pm set such a low value of max_wal_senders?

2020-09-30 Thread Michael Paquier
On Tue, Sep 29, 2020 at 07:04:22PM -0400, Tom Lane wrote: > Alvaro Herrera writes: >> I suggest to remove that line. max_wal_senders used to default to 0 >> when PostgresNode was touched to have this line in commit 89ac7004dad; >> the global default was raised in f6d6d2920d2c. > > Hm. We could

Re: Change a constraint's index - ALTER TABLE ... ALTER CONSTRAINT ... USING INDEX ...

2020-09-30 Thread Michael Paquier
On Fri, Sep 04, 2020 at 01:59:47PM +0200, Laurenz Albe wrote: > I'll set the commitfest entry to "waiting for author". This review, as well as any of the follow-up emails, have not been answered by the author, so I have marked the patch as returned with feedback. -- Michael signature.asc Descrip

Re: Asynchronous Append on postgres_fdw nodes.

2020-09-30 Thread Etsuro Fujita
On Mon, Sep 28, 2020 at 10:35 AM Kyotaro Horiguchi wrote: > At Sat, 26 Sep 2020 19:45:39 +0900, Etsuro Fujita > wrote in > > Your patch (and the original patch by Robert [1]) modified > > ExecAppend() so that it can process local plan nodes while waiting for > > the results from remote queries,

Re: [PATCH] Allow queries in WHEN expression of FOR EACH STATEMENT triggers

2020-09-30 Thread Michael Paquier
On Thu, Sep 03, 2020 at 09:22:31PM +0300, Surafel Temesgen wrote: > server crashed That's a problem. As this feedback has not been answered after two weeks, I am marking the patch as returned with feedback. -- Michael signature.asc Description: PGP signature

Re: SQL:2011 PERIODS vs Postgres Ranges?

2020-09-30 Thread Michael Paquier
On Thu, Sep 17, 2020 at 04:51:01PM +0900, Michael Paquier wrote: > This patch had no reviews, unfortunately it cannot be applied > cleanly. Could you send a rebase please? This had no replies after two weeks, so I have marked the patch as RwF. Please feel free to resubmit if you are planning to

Re: Adding Support for Copy callback functionality on COPY TO api

2020-09-30 Thread Michael Paquier
On Mon, Sep 14, 2020 at 04:28:12PM -0700, Soumyadeep Chakraborty wrote: > I think a similar change should also be applied to BeginCopyFrom() and > CopyFrom(). Or even better, such code could be refactored to have a > separate destination type COPY_PIPE. This of course, will be a separate > patch. I

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-30 Thread Dilip Kumar
On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila wrote: > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila wrote: > > > > On Sat, Sep 19, 2020 at 1:48 PM Amit Kapila wrote: > > > > > > On Tue, Sep 8, 2020 at 7:02 PM Amit Kapila > > > wrote: > > > > > > > > On Tue, Sep 8, 2020 at 7:53 AM Masahiko Sawad

Re: Implement for window functions

2020-09-30 Thread Michael Paquier
On Wed, Jul 01, 2020 at 02:27:45PM +0200, Daniel Gustafsson wrote: > This was with GCC in the Travis build, the Windows build passed and so does > clang locally for me. This was two months ago, so this patch has been marked as returned with feedback. Please feel free to resubmit once you have a n

Re: Ltree syntax improvement

2020-09-30 Thread Michael Paquier
On Wed, Jul 01, 2020 at 03:23:30PM +0200, Daniel Gustafsson wrote: > Please submit a rebased version of the patch. Which has not happened after two months, so I have marked the patch as returned with feedback. -- Michael signature.asc Description: PGP signature

Re: Libpq support to connect to standby server as priority

2020-09-30 Thread Greg Nancarrow
On Sun, Sep 27, 2020 at 4:34 AM Tom Lane wrote: > > > Thoughts? > Thanks for your thoughts, patches and all the pointers. I'll be looking at all of them. (And yes, the comma instead of bitwise OR is of course an error, somehow made and gone unnoticed; the next field in the struct is an enum, so a

Re: pgbench - refactor init functions with buffers

2020-09-30 Thread Heikki Linnakangas
On 09/07/2020 10:05, Fabien COELHO wrote: in favor of *PQExpBuffer(). Attached v7 is rebased v5 which uses PQExpBuffer, per cfbot. Thanks! I pushed this with small changes: - I left out the changes to executeStatement(). I'm not quite convinced it's a good idea or worth it, and it's unrelat

PoC patch: expose TCP socket stats for walsenders

2020-09-30 Thread Craig Ringer
Hi all I've attached a PoC patch demonstrating how to use the linux ioctls SIOCINQ and SIOCOUTQ and its getsockopt option TCP_INFO to expose a lot of useful network socket info directly in system views. Sample output from pg_stat_replication, \x format, trimmed, from a test run where I deliberate

Re: Adding Support for Copy callback functionality on COPY TO api

2020-09-30 Thread Andrey V. Lepikhov
On 7/2/20 2:41 AM, Sanaba, Bilva wrote: Hi hackers, Currently, the COPY TO api does not support callback functions, while the COPY FROM api does. The COPY TO code does, however, include placeholders for supporting callbacks in the future. Rounding out the support of callback functions to bot

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Amit Kapila
On Tue, Sep 29, 2020 at 8:04 PM Dilip Kumar wrote: > > I have started looking into you latest patches, as of now I have a > few comments. > > v6-0001 > > @@ -1987,7 +2072,7 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, > ReorderBufferTXN *txn, > prev_lsn = change->lsn; > > /* Set the current

Re: [DOC] Document concurrent index builds waiting on each other

2020-09-30 Thread Michael Paquier
On Tue, Sep 08, 2020 at 01:25:21PM -0400, James Coleman wrote: > Álvaro's patch confused the current state of this thread, so I'm > reattaching (rebased) v2 as v3. + + CREATE INDEX (including the CONCURRENTLY + option) commands are included when VACUUM calculates what + dead tuples are sa

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-30 Thread Amit Kapila
On Wed, Sep 30, 2020 at 1:12 PM Dilip Kumar wrote: > > On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila wrote: > > > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila wrote: > > > > I have done some more testing of this patch especially for the case > > where we spill before streaming the transaction and

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Dilip Kumar
On Wed, Sep 30, 2020 at 2:36 PM Amit Kapila wrote: > > On Tue, Sep 29, 2020 at 8:04 PM Dilip Kumar wrote: > > > > I have started looking into you latest patches, as of now I have a > > few comments. > > > > v6-0001 > > > > @@ -1987,7 +2072,7 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, > > Reor

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-09-30 Thread Amit Kapila
On Wed, Sep 30, 2020 at 12:16 PM Dilip Kumar wrote: > > On Mon, Sep 28, 2020 at 2:22 PM Amit Kapila wrote: > > > > > > I don't have a patch for this idea but you can refer [1] > > (v25-0002-Issue-individual-invalidations-with-wal_level-lo) to see > > what I was trying to say about registering the

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Amit Kapila
On Wed, Sep 30, 2020 at 2:46 PM Dilip Kumar wrote: > > On Wed, Sep 30, 2020 at 2:36 PM Amit Kapila wrote: > > > > On Tue, Sep 29, 2020 at 8:04 PM Dilip Kumar wrote: > > > > > > I have started looking into you latest patches, as of now I have a > > > few comments. > > > > > > v6-0001 > > > > > >

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Dilip Kumar
On Wed, Sep 30, 2020 at 3:08 PM Amit Kapila wrote: > > On Wed, Sep 30, 2020 at 2:46 PM Dilip Kumar wrote: > > > > On Wed, Sep 30, 2020 at 2:36 PM Amit Kapila wrote: > > > > > > On Tue, Sep 29, 2020 at 8:04 PM Dilip Kumar wrote: > > > > > > > > I have started looking into you latest patches, as

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Amit Kapila
On Wed, Sep 30, 2020 at 3:12 PM Dilip Kumar wrote: > > On Wed, Sep 30, 2020 at 3:08 PM Amit Kapila wrote: > > > > On Wed, Sep 30, 2020 at 2:46 PM Dilip Kumar wrote: > > > > > > On Wed, Sep 30, 2020 at 2:36 PM Amit Kapila > > > wrote: > > > > > > > > On Tue, Sep 29, 2020 at 8:04 PM Dilip Kumar

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-30 Thread Dilip Kumar
On Wed, Sep 30, 2020 at 3:27 PM Amit Kapila wrote: > > On Wed, Sep 30, 2020 at 3:12 PM Dilip Kumar wrote: > > > > On Wed, Sep 30, 2020 at 3:08 PM Amit Kapila wrote: > > > > > > On Wed, Sep 30, 2020 at 2:46 PM Dilip Kumar wrote: > > > > > > > > On Wed, Sep 30, 2020 at 2:36 PM Amit Kapila > > >

Re: Use PG_FINALLY to simplify code

2020-09-30 Thread Kyotaro Horiguchi
At Tue, 29 Sep 2020 23:10:52 -0400, Tom Lane wrote in tgl> Kyotaro Horiguchi writes: tgl> > At Tue, 29 Sep 2020 01:03:13 +, "Hou, Zhijie" wrote in tgl> >> Since PG_FINALLY can be used now, I think we can use PG_FINALLY to simplify code here. tgl> tgl> > The patch removes PG_RETHROW(), w

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-30 Thread Dilip Kumar
On Wed, Sep 30, 2020 at 2:40 PM Amit Kapila wrote: > > On Wed, Sep 30, 2020 at 1:12 PM Dilip Kumar wrote: > > > > On Fri, Sep 25, 2020 at 4:33 PM Amit Kapila wrote: > > > > > > On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila > > > wrote: > > > > > > I have done some more testing of this patch espe

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Amit Kapila
On Tue, Sep 29, 2020 at 9:23 PM Fujii Masao wrote: > > On 2020/09/29 11:51, Masahiro Ikeda wrote: > > On 2020-09-29 11:43, Amit Kapila wrote: > >> On Tue, Sep 29, 2020 at 7:39 AM Masahiro Ikeda > >> wrote: > >>> > >>> On 2020-09-28 12:43, Amit Kapila wrote: > >>> > On Mon, Sep 28, 2020 at 8:24 A

Re: [PATCH] Add section headings to index types doc

2020-09-30 Thread Dagfinn Ilmari Mannsåker
Michael Paquier writes: > On Mon, Aug 10, 2020 at 12:52:17PM +, Jürgen Purtz wrote: >> The new status of this patch is: Waiting on Author > > This has not been answered yet, so I have marked the patch as returned > with feedback. Updated patch attached, wich reformats the operator lists as r

Re: Change a constraint's index - ALTER TABLE ... ALTER CONSTRAINT ... USING INDEX ...

2020-09-30 Thread Laurenz Albe
On Wed, 2020-09-30 at 16:27 +0900, Michael Paquier wrote: > On Fri, Sep 04, 2020 at 01:59:47PM +0200, Laurenz Albe wrote: > > I'll set the commitfest entry to "waiting for author". > > This review, as well as any of the follow-up emails, have not been > answered by the author, so I have marked the

Re: WIP: BRIN multi-range indexes

2020-09-30 Thread John Naylor
On Mon, Sep 28, 2020 at 10:12 PM Tomas Vondra wrote: > Is it actually all that different from the existing BRIN indexes? > Consider this example: > > create table x (a text, b text, c text); > > create index on x using brin (a,b,c); > > create or replace function random_str(p_len int) returns tex

Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away

2020-09-30 Thread Bharath Rupireddy
On Tue, Sep 29, 2020 at 10:01 PM Fujii Masao wrote: > > > I think this is okay, because pg_terminate_backend() sends SIGTERM to > > the backend, and upon receiving SIGTERM the signal handler die() will > > be called and since there is no query being executed on the backend by > > the time SIGTERM

Re: [PATCH] Add section headings to index types doc

2020-09-30 Thread Heikki Linnakangas
On 30/09/2020 14:25, Dagfinn Ilmari Mannsåker wrote: Michael Paquier writes: On Mon, Aug 10, 2020 at 12:52:17PM +, Jürgen Purtz wrote: The new status of this patch is: Waiting on Author This has not been answered yet, so I have marked the patch as returned with feedback. Updated patch

Re: VACUUM PARALLEL option vs. max_parallel_maintenance_workers

2020-09-30 Thread Amit Kapila
On Tue, Sep 29, 2020 at 3:13 PM Peter Eisentraut wrote: > > On 2020-09-26 07:32, Amit Kapila wrote: > > This is exactly my feeling too. But how about changing documentation a > > bit as proposed above [1] to make it precise. > > > > [1] - > > https://www.postgresql.org/message-id/CAA4eK1LQWXS_4Rw

Re: Residual cpluspluscheck issues

2020-09-30 Thread Jesse Zhang
Hi Tom, So it's been 17 months since you sent this email, so I'm not sure that nothing has happened (off list or in the code base), but... On Sun, Jun 2, 2019 at 9:53 AM Tom Lane wrote: > * On FreeBSD 12, I get > > /home/tgl/pgsql/src/include/utils/hashutils.h:23:23: warning: 'register' > storag

Re: Residual cpluspluscheck issues

2020-09-30 Thread Tom Lane
Jesse Zhang writes: > So it's been 17 months since you sent this email, so I'm not sure that > nothing has happened (off list or in the code base), but... Well, we fixed the case that was discussed at the time [1]. I'm not exactly convinced about removing the register keywords in s_lock.h. Thos

Re: VACUUM PARALLEL option vs. max_parallel_maintenance_workers

2020-09-30 Thread Robert Haas
On Tue, Sep 22, 2020 at 3:20 AM David Rowley wrote: > It would be good if we were consistent with these parallel options. > Right now max_parallel_workers_per_gather will restrict the > parallel_workers reloption. I'd say this > max_parallel_workers_per_gather is similar to > max_parallel_mainten

Re: [PATCH] audo-detect and use -moutline-atomics compilation flag for aarch64

2020-09-30 Thread Zidenberg, Tsahi
On 29/09/2020, 10:21, "Heikki Linnakangas" wrote: > If it's a good idea to use -moutline-atomics, I would expect the > compiler or distribution to enable it by default. And as you pointed > out, many have. -moutline-atomics is only enabled by default on the gcc-10 branch where it w

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Tue, Sep 29, 2020 at 01:26:29PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Sep 4, 2020 at 12:45:36PM -0700, David G. Johnston wrote: > >>> Because we already have the to_date/make_date inconsistency, and the -1 > >>> to -2 BC mapping is confusing, and doesn't match Oracle, I th

Re: [PATCH] audo-detect and use -moutline-atomics compilation flag for aarch64

2020-09-30 Thread Heikki Linnakangas
On 30/09/2020 19:04, Zidenberg, Tsahi wrote: Ubuntu 20.04 even turned it off by default for gcc-10, which seems like a compatibility step with the main gcc-9 compiler. Ok, I definitely don't want to override that decision. I'm marking this as Rejected in the commitfest. But thanks for the benc

Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away

2020-09-30 Thread Fujii Masao
On 2020/09/30 21:02, Bharath Rupireddy wrote: On Tue, Sep 29, 2020 at 10:01 PM Fujii Masao wrote: I think this is okay, because pg_terminate_backend() sends SIGTERM to the backend, and upon receiving SIGTERM the signal handler die() will be called and since there is no query being executed

Re: [PATCH] audo-detect and use -moutline-atomics compilation flag for aarch64

2020-09-30 Thread Tom Lane
"Zidenberg, Tsahi" writes: > On 29/09/2020, 10:21, "Heikki Linnakangas" wrote: >> If it's a good idea to use -moutline-atomics, I would expect the >> compiler or distribution to enable it by default. And as you pointed >> out, many have. > -moutline-atomics is only enabled by defaul

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Bruce Momjian writes: > On Tue, Sep 29, 2020 at 01:26:29PM -0400, Tom Lane wrote: >> Bruce Momjian writes: >>> On Fri, Sep 4, 2020 at 12:45:36PM -0700, David G. Johnston wrote: Because we already have the to_date/make_date inconsistency, and the -1 to -2 BC mapping is confusing, and do

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Wed, Sep 30, 2020 at 02:11:54PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Tue, Sep 29, 2020 at 01:26:29PM -0400, Tom Lane wrote: > >> Bruce Momjian writes: > >>> On Fri, Sep 4, 2020 at 12:45:36PM -0700, David G. Johnston wrote: > Because we already have the to_date/make_date

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Bruce Momjian writes: > On Wed, Sep 30, 2020 at 02:11:54PM -0400, Tom Lane wrote: >> Hm, I read your reference to "the release notes" as suggesting that >> we should change it only in a major release, ie HEAD only (and it >> looks like David read it the same). If you meant minor release notes, >>

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Wed, Sep 30, 2020 at 02:50:31PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Sep 30, 2020 at 02:11:54PM -0400, Tom Lane wrote: > >> Hm, I read your reference to "the release notes" as suggesting that > >> we should change it only in a major release, ie HEAD only (and it > >> looks

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Bruce Momjian writes: > On Wed, Sep 30, 2020 at 02:50:31PM -0400, Tom Lane wrote: >> Actually, I was just finishing up back-patching the patch I posted >> yesterday. I think we should just fix it, not document that it's >> broken. > Agreed, that's what I wanted. You stated in a later email you

Re: __pg_log_level in anonynous enum should be initialized? (Was: pgsql: Change SHA2 implementation based on OpenSSL to use EVP digest ro)

2020-09-30 Thread Tom Lane
Michael Paquier writes: > However I would prefer the style of the attached. LGTM. regards, tom lane

Re: [PATCH] Automatic HASH and LIST partition creation

2020-09-30 Thread Rahila Syed
Hi Anastasia, I tested the syntax with some basic commands and it works fine, regression tests also pass. Couple of comments: 1. The syntax used omits the { IMMEDIATE | DEFERRED} keywords suggested in the earlier discussions. I think it is intuitive to include IMMEDIATE with the current implement

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Wed, Sep 30, 2020 at 03:11:06PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Sep 30, 2020 at 02:50:31PM -0400, Tom Lane wrote: > >> Actually, I was just finishing up back-patching the patch I posted > >> yesterday. I think we should just fix it, not document that it's > >> broken

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Robert Haas
On Tue, Sep 29, 2020 at 1:26 PM Tom Lane wrote: > I think this is nuts. The current behavior is obviously broken; > we should just treat it as a bug and fix it, including back-patching. > I do not think there is a compatibility problem of any significance. > Who out there is going to have an appl

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Robert Haas writes: > On Tue, Sep 29, 2020 at 1:26 PM Tom Lane wrote: >> I think this is nuts. The current behavior is obviously broken; >> we should just treat it as a bug and fix it, including back-patching. >> I do not think there is a compatibility problem of any significance. >> Who out the

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Robert Haas
On Wed, Sep 30, 2020 at 5:35 PM Tom Lane wrote: > By that logic, we should never fix any bug in a back branch. No, by that logic, we should not change any behavior in a back-branch upon which a customer is plausibly relying. No one relies on a certain query causing a server crash, for example, or

Re: Error on failed COMMIT

2020-09-30 Thread Andrew Dunstan
On 8/4/20 12:19 PM, Dave Cramer wrote: > Attached is the rebased patch for consideration. > > It's a bit sad this has been hanging around so long without attention. The previous discussion seems to give the patch a clean bill of health for most/all of the native drivers. Are there any implica

CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2020-09-30 Thread Justin Pryzby
CREATE TABLE t(i int) PARTITION BY RANGE(i); CREATE TABLE t1 PARTITION OF t FOR VALUES FROM (1) TO (10); CREATE OR REPLACE FUNCTION tgf() RETURNS trigger LANGUAGE plpgsql AS $$ begin raise exception 'except'; end $$; CREATE TRIGGER tg AFTER INSERT ON t FOR EACH ROW EXECUTE FUNCTION tgf(); ALTER TA

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Robert Haas writes: > On Wed, Sep 30, 2020 at 5:35 PM Tom Lane wrote: >> By that logic, we should never fix any bug in a back branch. > No, by that logic, we should not change any behavior in a back-branch > upon which a customer is plausibly relying. I guess where we differ here is on the idea

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Wed, Sep 30, 2020 at 06:10:38PM -0400, Robert Haas wrote: > On Wed, Sep 30, 2020 at 5:35 PM Tom Lane wrote: > > By that logic, we should never fix any bug in a back branch. > > No, by that logic, we should not change any behavior in a back-branch > upon which a customer is plausibly relying. N

Re: Improving connection scalability: GetSnapshotData()

2020-09-30 Thread Andres Freund
Hi, On 2020-09-14 16:17:18 -0700, Andres Freund wrote: > I've also included a quite heavily revised version of your test. Instead > of using dblink I went for having a long-running psql that I feed over > stdin. The main reason for not liking the previous version is that it > seems fragile, with t

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Bruce Momjian writes: > On Wed, Sep 30, 2020 at 06:10:38PM -0400, Robert Haas wrote: >> On Wed, Sep 30, 2020 at 5:35 PM Tom Lane wrote: >>> By that logic, we should never fix any bug in a back branch. >> No, by that logic, we should not change any behavior in a back-branch >> upon which a custom

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Bruce Momjian
On Wed, Sep 30, 2020 at 07:26:55PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Sep 30, 2020 at 06:10:38PM -0400, Robert Haas wrote: > >> On Wed, Sep 30, 2020 at 5:35 PM Tom Lane wrote: > >>> By that logic, we should never fix any bug in a back branch. > > >> No, by that logic, we

Re: avoid bitmapOR-ing indexes with scan condition inconsistent with partition constraint

2020-09-30 Thread Soumyadeep Chakraborty
Hi Justin, Attached is a minimal patch that is rebased and removes the dependency on Konstantin's earlier patch[1], making it enough to remove the extraneous index scans as Justin brought up. Is this the direction we want to head in? Tagging Konstantin in the email to hear his input on his old pa

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Fujii Masao
On 2020/09/30 20:21, Amit Kapila wrote: On Tue, Sep 29, 2020 at 9:23 PM Fujii Masao wrote: On 2020/09/29 11:51, Masahiro Ikeda wrote: On 2020-09-29 11:43, Amit Kapila wrote: On Tue, Sep 29, 2020 at 7:39 AM Masahiro Ikeda wrote: On 2020-09-28 12:43, Amit Kapila wrote: On Mon, Sep 28, 2

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Robert Haas
On Wed, Sep 30, 2020 at 7:26 PM Tom Lane wrote: > We do not have, and never have had, a project policy against > back-patching non-crash-related behavioral changes. If we did, > we would not for example put timezone database updates into the > back branches. It's not terribly hard to imagine suc

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread David G. Johnston
On Wed, Sep 30, 2020 at 5:24 PM Robert Haas wrote: > The > fact that we've suddenly discovered that this is not what Oracle does > doesn't mean that no users have discovered that it is what PostgreSQL > does. > Presently I cannot seem to make up my mind so I'm going to go with my original opinio

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-30 Thread Tom Lane
Robert Haas writes: > Right. Ultimately, this comes down to a judgement call about what you > think people are likely to rely on, and what you think they are > unlikely to rely on. Good, so at least we agree on that principle. > But the present case does not seem to me to be comparable. If someo

Re: Disable WAL logging to speed up data loading

2020-09-30 Thread Fujii Masao
On 2020/09/30 12:10, osumi.takami...@fujitsu.com wrote: Hello, Ashutosh Can they use a database with all unlogged tables? Unfortunately, no. They want to switch a cluster condition to "without WAL logging" only when they execute night bulk loading for their data warehouse. In other words,

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Kyotaro Horiguchi
At Thu, 1 Oct 2020 09:05:19 +0900, Fujii Masao wrote in > > > On 2020/09/30 20:21, Amit Kapila wrote: > > On Tue, Sep 29, 2020 at 9:23 PM Fujii Masao > > wrote: > >> > >> On 2020/09/29 11:51, Masahiro Ikeda wrote: > >>> On 2020-09-29 11:43, Amit Kapila wrote: > On Tue, Sep 29, 2020 at 7:

Re: __pg_log_level in anonynous enum should be initialized? (Was: pgsql: Change SHA2 implementation based on OpenSSL to use EVP digest ro)

2020-09-30 Thread Michael Paquier
On Wed, Sep 30, 2020 at 03:47:26PM -0400, Tom Lane wrote: > LGTM. Thanks, I have applied this version then. I am curious to see what the buildfarm thinks about it. -- Michael signature.asc Description: PGP signature

RE: New statistics for tuning WAL buffer size

2020-09-30 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > Another reason that I mildly want to object to subdivided functions is > I was annoyed that a stats view makes many individual calls to > functions that internally share the same statistics entry. That > behavior required me to provide an entry-caching feature to my > sh

terminate called after throwing an instance of 'std::bad_alloc'

2020-09-30 Thread Justin Pryzby
A VM crashed which is now running PG13.0 on centos7: Sep 30 19:40:08 database7 abrt-hook-ccpp: Process 17905 (postgres) of user 26 killed by SIGABRT - dumping core Core was generated by `postgres: telsasoft ts 192.168.122.11(34608) SELECT'. Unfortunately, the filesystem wasn't large enough a

Re: Asynchronous Append on postgres_fdw nodes.

2020-09-30 Thread Kyotaro Horiguchi
At Wed, 30 Sep 2020 16:30:41 +0900, Etsuro Fujita wrote in > On Mon, Sep 28, 2020 at 10:35 AM Kyotaro Horiguchi > wrote: > > At Sat, 26 Sep 2020 19:45:39 +0900, Etsuro Fujita > > wrote in > > > Your patch (and the original patch by Robert [1]) modified > > > ExecAppend() so that it can proces

Re: terminate called after throwing an instance of 'std::bad_alloc'

2020-09-30 Thread Tom Lane
Justin Pryzby writes: > A VM crashed which is now running PG13.0 on centos7: > Sep 30 19:40:08 database7 abrt-hook-ccpp: Process 17905 (postgres) of user 26 > killed by SIGABRT - dumping core > Core was generated by `postgres: telsasoft ts 192.168.122.11(34608) SELECT > '. > Unfortunately, t

Re: enable_incremental_sort changes query behavior

2020-09-30 Thread James Coleman
On Sat, Sep 26, 2020 at 2:49 PM Jaime Casanova wrote: > > Hi, > > With sqlsmith I found a query that gives this error: > ERROR: ORDER/GROUP BY expression not found in targetlist > > I noted the query (sql query below, sorry it uses custom tables i > couldn't replicate with regression tables) beca

Re: terminate called after throwing an instance of 'std::bad_alloc'

2020-09-30 Thread Andres Freund
Hi, On 2020-09-30 21:16:10 -0500, Justin Pryzby wrote: > Sep 30 19:40:08 database7 abrt-hook-ccpp: Process 17905 (postgres) of user 26 > killed by SIGABRT - dumping core > Core was generated by `postgres: telsasoft ts 192.168.122.11(34608) SELECT > '. > > Unfortunately, the filesystem wasn't

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Amit Kapila
On Thu, Oct 1, 2020 at 6:53 AM Kyotaro Horiguchi wrote: > > At Thu, 1 Oct 2020 09:05:19 +0900, Fujii Masao > wrote in > > > > > > On 2020/09/30 20:21, Amit Kapila wrote: > > > On Tue, Sep 29, 2020 at 9:23 PM Fujii Masao > > > wrote: > > >> > > >> On 2020/09/29 11:51, Masahiro Ikeda wrote: > > >

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread tsunakawa.ta...@fujitsu.com
From: Jamison, Kirk/ジャミソン カーク > Recovery performance measurement results below. > But it seems there are overhead even with large shared buffers. > > | s_b | master | patched | %reg | > |---||-|---| > | 128MB | 36.052 | 39.451 | 8.62% | > | 1GB | 21.731 | 21.73 | 0

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread Amit Kapila
On Thu, Oct 1, 2020 at 8:11 AM tsunakawa.ta...@fujitsu.com wrote: > > From: Jamison, Kirk/ジャミソン カーク > > Recovery performance measurement results below. > > But it seems there are overhead even with large shared buffers. > > > > | s_b | master | patched | %reg | > > |---||-|

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-09-30 Thread Keisuke Kuroda
Hi Dilip, Amit, Thank you for the patch! I test the patch on the master HEAD(9796f455) and it works fine. * make installcheck-world: passed * walsender process continues to use 100% of the CPU 1core when TRUNCATE 1000 partition: 1s or less ** before patch : 230s There is "ReorderBufferAddInvalid

Re: Disable WAL logging to speed up data loading

2020-09-30 Thread Kyotaro Horiguchi
At Thu, 1 Oct 2020 10:01:56 +0900, Fujii Masao wrote in > > > On 2020/09/30 12:10, osumi.takami...@fujitsu.com wrote: > > Hello, Ashutosh > > > >> Can they use a database with all unlogged tables? > > Unfortunately, no. They want to switch a cluster condition to "without > > WAL logging" > >

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread tsunakawa.ta...@fujitsu.com
From: Amit Kapila > I have one idea for performance testing. We can even test this for > non-recovery paths by removing the recovery-related check like only > use it when there are cached blocks. You can do this if testing via > recovery path is difficult because at the end performance should be >

RE: Disable WAL logging to speed up data loading

2020-09-30 Thread osumi.takami...@fujitsu.com
Hello. > >> Can they use a database with all unlogged tables? > > Unfortunately, no. They want to switch a cluster condition to "without WAL > logging" > > only when they execute night bulk loading for their data warehouse. > > In other words, they would like to keep their other usual operations w

Re: Why does PostgresNode.pm set such a low value of max_wal_senders?

2020-09-30 Thread Michael Paquier
On Tue, Sep 29, 2020 at 07:04:22PM -0400, Tom Lane wrote: > Hm. We could do so back to v10 where that came in, and there are no > src/test/subscription tests before v10, so that should be sufficient. > Sold. Since this stuff has been committed, thorntail has showed a very interesting failure with

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread Kyotaro Horiguchi
At Thu, 1 Oct 2020 02:40:52 +, "tsunakawa.ta...@fujitsu.com" wrote in > With the following code, when the main fork does not meet the > optimization criteria, other forks are not optimized as well. You > want to determine each fork's optimization separately, don't you? In more detail, if s

Re: VACUUM (INTERRUPTIBLE)?

2020-09-30 Thread Andres Freund
Hi, On 2020-09-08 21:11:47 -0700, Andres Freund wrote: > > Not yet sure about what a suitable way to test this for analyze would > > be, unless we'd also accept VacuumDelay as a wait condition :(. I guess > > one fairly easy way would be to include an expression index, and have > > the expression

Re: Implementing Incremental View Maintenance

2020-09-30 Thread Michael Paquier
On Thu, Sep 17, 2020 at 09:42:45AM +0900, Tatsuo Ishii wrote: > I am asking because these patch sets are now getting closer to > committable state in my opinion, and if there's someting wrong, it > should be fixed soon so that these patches are getting into the master > branch. > > I think this fe

Re: proposal: schema variables

2020-09-30 Thread Michael Paquier
On Thu, Sep 24, 2020 at 08:49:50PM +0200, Pavel Stehule wrote: > fixed patch attached It looks like there are again conflicts within setrefs.c. -- Michael signature.asc Description: PGP signature

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-09-30 Thread Amit Kapila
On Thu, Oct 1, 2020 at 8:22 AM Keisuke Kuroda wrote: > > Hi Dilip, Amit, > > Thank you for the patch! > > I test the patch on the master HEAD(9796f455) and it works fine. > * make installcheck-world: passed > * walsender process continues to use 100% of the CPU 1core when > TRUNCATE 1000 partition

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-09-30 Thread Michael Paquier
On Thu, Sep 24, 2020 at 12:51:52PM +0900, Amit Langote wrote: > Also, I noticed that looking up a parent's partitions via > RelationBuildPartitionDesc or directly will inspect inhdetached to > include or exclude partitions, but checking if a child table is a > partition of a given parent table via

Re: Asynchronous Append on postgres_fdw nodes.

2020-09-30 Thread Michael Paquier
On Thu, Oct 01, 2020 at 11:16:53AM +0900, Kyotaro Horiguchi wrote: > Thanks. Since it starts all remote nodes before local ones, the > startup gain would be the shorter of the startup time of the fastest > remote and the time required for all local nodes. Plus remote > transfer gets asynchronous

Re: New statistics for tuning WAL buffer size

2020-09-30 Thread Masahiro Ikeda
On 2020-10-01 11:33, Amit Kapila wrote: On Thu, Oct 1, 2020 at 6:53 AM Kyotaro Horiguchi wrote: At Thu, 1 Oct 2020 09:05:19 +0900, Fujii Masao wrote in > > > On 2020/09/30 20:21, Amit Kapila wrote: > > On Tue, Sep 29, 2020 at 9:23 PM Fujii Masao > > wrote: > >> > >> On 2020/09/29 11:51, Ma

Re: Disable WAL logging to speed up data loading

2020-09-30 Thread Fujii Masao
On 2020/10/01 12:04, osumi.takami...@fujitsu.com wrote: Hello. Can they use a database with all unlogged tables? Unfortunately, no. They want to switch a cluster condition to "without WAL logging" only when they execute night bulk loading for their data warehouse. In other words, they wou

Re: Implementing Incremental View Maintenance

2020-09-30 Thread Tatsuo Ishii
> On Thu, Sep 17, 2020 at 09:42:45AM +0900, Tatsuo Ishii wrote: >> I am asking because these patch sets are now getting closer to >> committable state in my opinion, and if there's someting wrong, it >> should be fixed soon so that these patches are getting into the master >> branch. >> >> I think

Re: POC: postgres_fdw insert batching

2020-09-30 Thread Michael Paquier
On Sun, Jul 12, 2020 at 02:11:01AM +0200, Tomas Vondra wrote: > Because I was focused on speeding-up inserts, and that is not using > CopyMultiInsertBuffer I think. I agree the way the tuples are stored > may be improved, of course. The CF bot is telling that the regression tests of postgres_fdw a

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-09-30 Thread Dilip Kumar
On Thu, Oct 1, 2020 at 9:19 AM Amit Kapila wrote: > > On Thu, Oct 1, 2020 at 8:22 AM Keisuke Kuroda > wrote: > > > > Hi Dilip, Amit, > > > > Thank you for the patch! > > > > I test the patch on the master HEAD(9796f455) and it works fine. > > * make installcheck-world: passed > > * walsender proc

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-30 Thread tsunakawa.ta...@fujitsu.com
From: Kyotaro Horiguchi > In more detail, if smgrcachednblocks() returned InvalidBlockNumber for > any of the forks, we should give up the optimization at all since we > need to run a full scan anyway. On the other hand, if any of the > forks is smaller than the threshold, we still can use the op

Re: making update/delete of inheritance trees scale better

2020-09-30 Thread Michael Paquier
On Fri, Sep 11, 2020 at 07:20:56PM +0900, Amit Langote wrote: > I have been working away at this and have updated the patches for many > cosmetic and some functional improvements. Please note that this patch set fails to apply. Could you provide a rebase please? -- Michael signature.asc Descrip

Re: PATCH: Batch/pipelining support for libpq

2020-09-30 Thread Michael Paquier
On Mon, Sep 21, 2020 at 07:55:03PM +0200, Matthieu Garrigues wrote: > I merged PQbatchProcessQueue into PQgetResult. > One first init call to PQbatchProcessQueue was also required in > PQsendQueue to have > PQgetResult ready to read the first batch query. > > Tests and documentation are updated ac

  1   2   >