Re: partition routing layering in nodeModifyTable.c

2020-10-22 Thread Heikki Linnakangas
On 23/10/2020 05:56, Amit Langote wrote: On Thu, Oct 22, 2020 at 11:25 PM Alvaro Herrera wrote: On 2020-Oct-22, Amit Langote wrote: 0001 fixes a thinko of the recent commit 1375422c782 that I discovered when debugging a problem with 0003. Hmm, how hard is it to produce a test case that fai

Re: Re: parallel distinct union and aggregate support patch

2020-10-22 Thread bu...@sohu.com
> Interesting idea. So IIUC, whenever a worker is scanning the tuple it > will directly put it into the respective batch(shared tuple store), > based on the hash on grouping column and once all the workers are > doing preparing the batch then each worker will pick those baches one > by one, perfor

RE: ECPG: proposal for new DECLARE STATEMENT

2020-10-22 Thread kuroda.hay...@fujitsu.com
Dear Tomas, Daniel, Michael, I missed your e-mails, and I apologize the very late reply. I want you to thank keeping the thread. > I'm not an ecpg expert (in fact I've never even used it), so my review > is pretty superficial, but I only found a couple of minor whitespace > issues (adding/remov

Re: Would it be helpful for share the patch merge result from cfbot

2020-10-22 Thread Kyotaro Horiguchi
At Fri, 23 Oct 2020 10:01:14 +0800, Andy Fan wrote in > On Fri, Oct 23, 2020 at 9:58 AM Thomas Munro wrote: > > > > Try this: > > > > > > git remote add cfbot https://github.com/postgresql-cfbot/postgresql.git > > > git fetch cfbot commitfest/30/2785 > > > git checkout commitfest/30/2785 > > >

Re: Enumize logical replication message actions

2020-10-22 Thread Kyotaro Horiguchi
At Thu, 22 Oct 2020 22:31:41 -0300, Alvaro Herrera wrote in > On 2020-Oct-22, Ashutosh Bapat wrote: > > > On Thu, 22 Oct 2020 at 14:46, Kyotaro Horiguchi > > wrote: > > > > pg_send_logicalrep_msg_type() looks somewhat too-much. If we need > > > something like that we shouldn't do this refact

Re: Autovacuum on partitioned table (autoanalyze)

2020-10-22 Thread yuzuko
Hello, I reconsidered a way based on the v5 patch in line with Horiguchi-san's comment. This approach is as follows: - A partitioned table is checked whether it needs analyze like a plain table in relation_needs_vacanalyze(). To do this, we should store partitioned table's stats (changes_si

Re: allow partial union-all and improve parallel subquery costing

2020-10-22 Thread Luc Vlaming
On 14.10.20 09:38, Luc Vlaming wrote: Hi, It seems I ran the wrong make checks to verify everything is correct (make check instead of make installcheck-world) and this uncovered another regress test change. I also noticed the statistics are sometimes giving different row count results so I inc

pg_dump, ATTACH, and independently restorable child partitions

2020-10-22 Thread Justin Pryzby
Since this commit, pg_dump CREATEs tables and then ATTACHes them: |commit 33a53130a89447e171a8268ae0b221bb48af6468 |Author: Alvaro Herrera |Date: Mon Jun 10 18:56:23 2019 -0400 | |Make pg_dump emit ATTACH PARTITION instead of PARTITION OF (reprise) |... |This change also has the advanta

Re: Mop-up around psql's \connect behavior

2020-10-22 Thread Kyotaro Horiguchi
At Thu, 22 Oct 2020 15:23:04 -0400, Tom Lane wrote in > I wrote: > > I did actually look into saving the active connection's PQconninfo > > immediately at connection establishment and then referring to it in any > > subsequent \connect. Then things could work the same even if the original > > co

Tab complete for alter table rls

2020-10-22 Thread Li Japin
Sorry, I forgot add the subject. -- Best regards Japin Li On Oct 23, 2020, at 1:19 PM, Li Japin mailto:japi...@hotmail.com>> wrote: Hi, hackers I find that ALTER TABLE xxx FORCE/NO FORCE ROW LEVEL SECURITY cannot support tab complete. The attached add the tab complete for rls. diff --git a/s

[no subject]

2020-10-22 Thread Li Japin
Hi, hackers I find that ALTER TABLE xxx FORCE/NO FORCE ROW LEVEL SECURITY cannot support tab complete. The attached add the tab complete for rls. diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c index 561fe1dff9..b2b4f1fd4d 100644 --- a/src/bin/psql/tab-complete.c +++ b/src

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-22 Thread Amit Kapila
On Fri, Oct 23, 2020 at 8:59 AM Amit Kapila wrote: > > On Fri, Oct 23, 2020 at 7:42 AM Masahiko Sawada > wrote: > > > > On Thu, 22 Oct 2020 at 20:34, Amit Kapila wrote: > > > > > > > > I have modified the description of spill_count and spill_txns to make > > > > > things clear. Any suggestions?

Re: Track statistics for streaming of in-progress transactions

2020-10-22 Thread Amit Kapila
On Thu, Oct 22, 2020 at 2:09 PM Amit Kapila wrote: > > On Thu, Oct 22, 2020 at 11:52 AM Dilip Kumar wrote: > > > > On Wed, Oct 14, 2020 at 9:09 AM Amit Kapila wrote: > > > > > > Commit 464824323e has added the support of the streaming of > > > in-progress transactions into the built-in logical r

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-22 Thread Tom Lane
I wrote: > * unix_socket_directories should have been marked GUC_LIST_INPUT | > * GUC_LIST_QUOTE, but it's too late to change it without creating > * big compatibility problems for pg_dump. Although ... just to argue against myself for a moment, how likely is it that pg_dump is going to be face

Re: Get memory contexts of an arbitrary backend process

2020-10-22 Thread Kyotaro Horiguchi
Wait... > Attachments: 0003-Enabled-pg_get_backend_memory_contexts-to-collect.patch For a moment I thought that the number is patch number but the predecessors are 0002-Enabled..collect.patch and 0001-(same name). It's not mandatory but we usually do as the follows and it's the way of git. v1-00

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-22 Thread Tom Lane
Ian Lawrence Barwick writes: > How about adding a comment along the lines of A comment seems reasonable, but I'd probably write it more like /* * unix_socket_directories should have been marked GUC_LIST_INPUT | * GUC_LIST_QUOTE, but it's too late to change it without creating * big compatibil

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 7:01 PM, Tom Lane wrote: > > Mark Dilger writes: >> Ahh, crud. It's because >> syswrite($fh, '\x77\x77\x77\x77', 500) >> is wrong twice. The 500 was wrong, but the string there isn't the bit >> pattern we want -- it's just a string literal with backslashes and suc

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-22 Thread Ian Lawrence Barwick
2020年10月23日(金) 12:56 Tom Lane : > > Michael Paquier writes: > > I'll look again at that in the next couple of days and double-check > > the relevant areas of the code, just in case. It is Friday afternoon > > here, and I suspect that my mind is missing something obvious. > > Indeed. The patch fa

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2020-10-22 Thread Craig Ringer
On Wed, Oct 7, 2020 at 8:39 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Wed, Oct 7, 2020 at 8:00 AM Bharath Rupireddy > wrote: > > > > On Tue, Oct 6, 2020 at 11:41 AM Bharath Rupireddy > > wrote: > > > > > > On Tue, Oct 6, 2020 at 11:20 AM Fujii Masao < > masao.fu.

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-22 Thread Tom Lane
Michael Paquier writes: > I'll look again at that in the next couple of days and double-check > the relevant areas of the code, just in case. It is Friday afternoon > here, and I suspect that my mind is missing something obvious. Indeed. The patch fails to update pg_dump.c's variable_is_guc_lis

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-22 Thread Michael Paquier
On Fri, Oct 23, 2020 at 12:23:28PM +0900, Ian Lawrence Barwick wrote: > Updated version attached. LGTM. Looking at c9b0cbe and the relevant thread it looks like this point was not really covered, so my guess is that this was just forgotten: https://www.postgresql.org/message-id/4fcf6040.5030...@r

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-22 Thread Amit Kapila
On Fri, Oct 23, 2020 at 7:42 AM Masahiko Sawada wrote: > > On Thu, 22 Oct 2020 at 20:34, Amit Kapila wrote: > > > > > > I have modified the description of spill_count and spill_txns to make > > > > things clear. Any suggestions? > > > > > > Thank you for the patch. > > > > > > -logical de

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-22 Thread Ian Lawrence Barwick
2020年10月23日(金) 12:12 Michael Paquier : > > On Fri, Oct 23, 2020 at 11:34:06AM +0900, Ian Lawrence Barwick wrote: > > Not that I've ever had to do this (or would want to do it on a production > > system), but this error message seems incorrect: > > > > postgres=# ALTER SYSTEM SET unix_socket_dir

Re: Re: parallel distinct union and aggregate support patch

2020-10-22 Thread bu...@sohu.com
> If I understood correctly, the tuples emitted by Parallel Batch Sort > in each process are ordered by (hash(key, ...) % npartitions, key, > ...), but the path is claiming to be ordered by (key, ...), no? > That's enough for Unique and Aggregate to give the correct answer, > because they really on

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-22 Thread Michael Paquier
On Fri, Oct 23, 2020 at 11:34:06AM +0900, Ian Lawrence Barwick wrote: > Not that I've ever had to do this (or would want to do it on a production > system), but this error message seems incorrect: > > postgres=# ALTER SYSTEM SET unix_socket_directories = > '/tmp/sock1','/tmp/sock2'; > ERR

Re: Global snapshots

2020-10-22 Thread Masahiko Sawada
On Thu, 15 Oct 2020 at 01:41, Fujii Masao wrote: > > > > On 2020/09/17 15:56, Amit Kapila wrote: > > On Thu, Sep 10, 2020 at 4:20 PM Fujii Masao > > wrote: > >> > One alternative is to add only hooks into PostgreSQL core so that we can > implement the global transaction management outs

Re: partition routing layering in nodeModifyTable.c

2020-10-22 Thread Amit Langote
On Thu, Oct 22, 2020 at 11:25 PM Alvaro Herrera wrote: > > On 2020-Oct-22, Amit Langote wrote: > > > 0001 fixes a thinko of the recent commit 1375422c782 that I discovered > > when debugging a problem with 0003. > > Hmm, how hard is it to produce a test case that fails because of this > problem?

"unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-22 Thread Ian Lawrence Barwick
Hi Not that I've ever had to do this (or would want to do it on a production system), but this error message seems incorrect: postgres=# ALTER SYSTEM SET unix_socket_directories = '/tmp/sock1','/tmp/sock2'; ERROR: SET unix_socket_directories takes only one argument Trivial patch attach

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-22 Thread Masahiko Sawada
On Thu, 22 Oct 2020 at 20:34, Amit Kapila wrote: > > On Thu, Oct 22, 2020 at 4:09 PM Masahiko Sawada > wrote: > > > > On Wed, 21 Oct 2020 at 12:56, Amit Kapila wrote: > > > > > > On Tue, Oct 13, 2020 at 10:33 AM Amit Kapila > > > wrote: > > > > > > > > On Tue, Oct 13, 2020 at 10:21 AM Tom Lane

Re: speed up unicode decomposition and recomposition

2020-10-22 Thread Michael Paquier
On Thu, Oct 22, 2020 at 05:50:52AM -0400, John Naylor wrote: > Looks good to me. Thanks. Committed, then. Great work! -- Michael signature.asc Description: PGP signature

Re: Would it be helpful for share the patch merge result from cfbot

2020-10-22 Thread Michael Paquier
On Fri, Oct 23, 2020 at 11:05:34AM +0900, Michael Paquier wrote: > It seems to me that this problem is not completely related to the CF > bot, no? Automated testing and fetching from a mirror repository > that's automated to fetch patches from the mailing list and apply them > on some custom branc

Re: proposal: function pg_setting_value_split() to parse shared_preload_libraries etc.

2020-10-22 Thread Ian Lawrence Barwick
2020年10月23日(金) 9:53 Ian Lawrence Barwick : > > Hi > > From time to time I find myself in a situation where it would be very useful > to > be able to programatically determine whether a particular library is included > in > "shared_preload_libraries", which accepts a comma-separated list of values

Re: Would it be helpful for share the patch merge result from cfbot

2020-10-22 Thread Michael Paquier
On Fri, Oct 23, 2020 at 09:31:57AM +0800, Andy Fan wrote: > Currently when people want to review a patch, they have to download / apply > / maintain the branch manually. Would it be helpful that the reviewer can > just git fetch a remote branch where all the things have been done already. I > know

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
Mark Dilger writes: > Ahh, crud. It's because > syswrite($fh, '\x77\x77\x77\x77', 500) > is wrong twice. The 500 was wrong, but the string there isn't the bit > pattern we want -- it's just a string literal with backslashes and such. It > should have been double-quoted. Argh. So we re

Re: Would it be helpful for share the patch merge result from cfbot

2020-10-22 Thread Andy Fan
On Fri, Oct 23, 2020 at 9:58 AM Thomas Munro wrote: > On Fri, Oct 23, 2020 at 2:51 PM Thomas Munro > wrote: > > On Fri, Oct 23, 2020 at 2:32 PM Andy Fan > wrote: > > > Currently when people want to review a patch, they have to download / > apply / > > > maintain the branch manually. Would it b

Re: Would it be helpful for share the patch merge result from cfbot

2020-10-22 Thread Thomas Munro
On Fri, Oct 23, 2020 at 2:51 PM Thomas Munro wrote: > On Fri, Oct 23, 2020 at 2:32 PM Andy Fan wrote: > > Currently when people want to review a patch, they have to download / apply > > / > > maintain the branch manually. Would it be helpful that the reviewer can > > just > > git fetch a remot

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 6:50 PM, Mark Dilger wrote: > > > >> On Oct 22, 2020, at 6:46 PM, Tom Lane wrote: >> >> I wrote: >>> I get >>> off = , flags = 2, len = 3bbb >>> on a little-endian machine, and >>> off = 3bbb, flags = 2, len = >>> on big-endian. It'd be less symmetric if the

Re: Would it be helpful for share the patch merge result from cfbot

2020-10-22 Thread Thomas Munro
On Fri, Oct 23, 2020 at 2:32 PM Andy Fan wrote: > Currently when people want to review a patch, they have to download / apply / > maintain the branch manually. Would it be helpful that the reviewer can just > git fetch a remote branch where all the things have been done already. I know > that suc

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 6:46 PM, Tom Lane wrote: > > I wrote: >> I get >> off = , flags = 2, len = 3bbb >> on a little-endian machine, and >> off = 3bbb, flags = 2, len = >> on big-endian. It'd be less symmetric if the bytes weren't >> all the same ... > > ... but given that this is t

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 6:41 PM, Tom Lane wrote: > > I wrote: >> So now I think this is a REDIRECT on either architecture, but the >> offset and length fields have different values, causing the redirect >> pointer to point to different places. Maybe it happens to point >> at a DEAD tuple in the

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
I wrote: > I get > off = , flags = 2, len = 3bbb > on a little-endian machine, and > off = 3bbb, flags = 2, len = > on big-endian. It'd be less symmetric if the bytes weren't > all the same ... ... but given that this is the test value we are using, why don't both endiannesses whine abou

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
I wrote: > So now I think this is a REDIRECT on either architecture, but the > offset and length fields have different values, causing the redirect > pointer to point to different places. Maybe it happens to point > at a DEAD tuple in the big-endian case. Just to make sure, I tried this test prog

Re: Enumize logical replication message actions

2020-10-22 Thread Alvaro Herrera
On 2020-Oct-22, Ashutosh Bapat wrote: > On Thu, 22 Oct 2020 at 14:46, Kyotaro Horiguchi > wrote: > > pg_send_logicalrep_msg_type() looks somewhat too-much. If we need > > something like that we shouldn't do this refactoring, I think. > > Enum is an integer, and we want to send byte. The functi

Would it be helpful for share the patch merge result from cfbot

2020-10-22 Thread Andy Fan
Currently when people want to review a patch, they have to download / apply / maintain the branch manually. Would it be helpful that the reviewer can just git fetch a remote branch where all the things have been done already. I know that such cost saving is small, but it is a startup cost, so pers

Re: Enumize logical replication message actions

2020-10-22 Thread Kyotaro Horiguchi
At Fri, 23 Oct 2020 10:08:44 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 22 Oct 2020 16:37:18 +0530, Ashutosh Bapat > wrote in > > On Thu, 22 Oct 2020 at 14:46, Kyotaro Horiguchi > > wrote: > > pg_get_logicalrep_msg_type() seems doing the same check (that the > > > value is compared ag

Re: Enumize logical replication message actions

2020-10-22 Thread Kyotaro Horiguchi
At Thu, 22 Oct 2020 16:37:18 +0530, Ashutosh Bapat wrote in > On Thu, 22 Oct 2020 at 14:46, Kyotaro Horiguchi > wrote: > > > > > > > We shouldn't have the default: in the switch() block in > > apply_dispatch(). That prevents compilers from checking > > completeness. The content of the default:

heapam and bottom-up garbage collection, keeping version chains short (Was: Deleting older versions in unique indexes to avoid page splits)

2020-10-22 Thread Peter Geoghegan
On Thu, Oct 22, 2020 at 6:18 AM Robert Haas wrote: > But that being said, I'm not trying to derail this patch. It isn't, > and shouldn't be, the job of this patch to solve that problem. It's > just better if it doesn't regress things, or maybe even (as you say) > makes them a little better. I thin

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-10-22 Thread tsunakawa.ta...@fujitsu.com
From: Thomas Munro > > I'm probably being silly, but can't we avoid the problem by using fstat() > instead of lseek(SEEK_END)? Would they return the same value from the > i-node? > > Amazingly, st_size can disagree with SEEK_END when using the Linux NFS > client, but its behaviour is worse. Her

proposal: function pg_setting_value_split() to parse shared_preload_libraries etc.

2020-10-22 Thread Ian Lawrence Barwick
Hi >From time to time I find myself in a situation where it would be very useful to be able to programatically determine whether a particular library is included in "shared_preload_libraries", which accepts a comma-separated list of values. Unfortunately it's not as simple as splitting the list o

Re: [patch] Fix checksum verification in base backups for zero page headers

2020-10-22 Thread Michael Paquier
On Thu, Oct 22, 2020 at 03:11:45PM +0300, Anastasia Lubennikova wrote: > Most of such pages are valid and already in memory, because they were > changed just recently, so no need for pg_prewarm here. If such LSN appeared > because of a data corruption, page verification from inside ReadBuffer() > w

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
Mark Dilger writes: >> On Oct 22, 2020, at 2:06 PM, Tom Lane wrote: >> Oh, wait a second. ItemIdData has the flag bits in the middle: >> meaning that for that particular bit pattern, one endianness >> is going to see the flags as 01 (LP_NORMAL) and the other as 10 >> (LP_REDIRECT). > Well, the

Re: new heapcheck contrib module

2020-10-22 Thread Robert Haas
On Thu, Oct 22, 2020 at 4:04 PM Mark Dilger wrote: > I think the compiler warning was about fxid not being set. The callers pass > NULL for status if they don't want status checked, so writing *status > unconditionally would be an error. Also, if the xid being checked is out of > bounds, we c

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-10-22 Thread Thomas Munro
On Thu, Oct 22, 2020 at 8:32 PM tsunakawa.ta...@fujitsu.com wrote: > I'm probably being silly, but can't we avoid the problem by using fstat() > instead of lseek(SEEK_END)? Would they return the same value from the i-node? Amazingly, st_size can disagree with SEEK_END when using the Linux NFS c

Re: new heapcheck contrib module

2020-10-22 Thread Peter Geoghegan
On Thu, Oct 22, 2020 at 2:39 PM Mark Dilger wrote: > > This is great work. Thanks Mark and Robert. > > That's the first time I've laughed today. Having turned the build-farm red, > this is quite ironic feedback! Thanks all the same for the sentiment. Breaking the buildfarm is not a capital off

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 2:26 PM, Peter Geoghegan wrote: > > On Thu, Oct 22, 2020 at 5:51 AM Robert Haas wrote: >> Committed. Let's see what the buildfarm thinks. > > This is great work. Thanks Mark and Robert. That's the first time I've laughed today. Having turned the build-farm red, this

Re: new heapcheck contrib module

2020-10-22 Thread Peter Geoghegan
On Thu, Oct 22, 2020 at 5:51 AM Robert Haas wrote: > Committed. Let's see what the buildfarm thinks. This is great work. Thanks Mark and Robert. -- Peter Geoghegan

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
Mark Dilger writes: > Yeah, I'm already looking at that. The logic in verify_heapam skips over > line pointers that are unused or dead, and the test is reporting zero > corruption (and complaining about that), so it's probably not going to help > to overwrite all the line pointers with this pa

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 2:06 PM, Tom Lane wrote: > > I wrote: >> Mark Dilger writes: >>> It is seeking to position 32 and writing '\x77\x77\x77\x77'. x86_64 is >>> little-endian, and ppc32 and sparc64 are both big-endian, right? > >> They are, but that should not meaningfully affect the resul

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 2:06 PM, Tom Lane wrote: > > I wrote: >> Mark Dilger writes: >>> It is seeking to position 32 and writing '\x77\x77\x77\x77'. x86_64 is >>> little-endian, and ppc32 and sparc64 are both big-endian, right? > >> They are, but that should not meaningfully affect the resul

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
I wrote: > Mark Dilger writes: >> It is seeking to position 32 and writing '\x77\x77\x77\x77'. x86_64 is >> little-endian, and ppc32 and sparc64 are both big-endian, right? > They are, but that should not meaningfully affect the results of > that corruption step. You zapped only one line pointe

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
Mark Dilger writes: >> On Oct 22, 2020, at 1:31 PM, Tom Lane wrote: >> Hm, but why are we seeing the failure only on specific machine >> architectures? sparc64 and ppc32 is a weird pairing, too. > It is seeking to position 32 and writing '\x77\x77\x77\x77'. x86_64 is > little-endian, and ppc32

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 1:31 PM, Tom Lane wrote: > > Mark Dilger writes: >>> On Oct 22, 2020, at 1:09 PM, Tom Lane wrote: >>> ooh, looks like prairiedog sees the problem too. That means I should be >>> able to reproduce it under a debugger, if you're not certain yet where >>> the problem lies

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
Mark Dilger writes: >> On Oct 22, 2020, at 1:09 PM, Tom Lane wrote: >> ooh, looks like prairiedog sees the problem too. That means I should be >> able to reproduce it under a debugger, if you're not certain yet where >> the problem lies. > Thanks, Tom, but I question whether the regression test

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 1:23 PM, Tom Lane wrote: > > ... btw, having now looked more closely at get_xid_status(), I wonder > how come there aren't more compilers bitching about it, because it > is very very obviously broken. In particular, the case of > requesting status for an xid that is Boot

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 1:09 PM, Tom Lane wrote: > > ooh, looks like prairiedog sees the problem too. That means I should be > able to reproduce it under a debugger, if you're not certain yet where > the problem lies. Thanks, Tom, but I question whether the regression test failures are from a

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
... btw, having now looked more closely at get_xid_status(), I wonder how come there aren't more compilers bitching about it, because it is very very obviously broken. In particular, the case of requesting status for an xid that is BootstrapTransactionId or FrozenTransactionId *will* fall through

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
ooh, looks like prairiedog sees the problem too. That means I should be able to reproduce it under a debugger, if you're not certain yet where the problem lies. regards, tom lane

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 1:00 PM, Robert Haas wrote: > > On Thu, Oct 22, 2020 at 3:15 PM Mark Dilger > wrote: >> The 0001 attached patch addresses the -Werror=maybe-uninitialized problem. > > I am skeptical. Why so much code churn to fix a compiler warning? And > even in the revised code, *stat

Re: new heapcheck contrib module

2020-10-22 Thread Robert Haas
On Thu, Oct 22, 2020 at 3:15 PM Mark Dilger wrote: > The 0001 attached patch addresses the -Werror=maybe-uninitialized problem. I am skeptical. Why so much code churn to fix a compiler warning? And even in the revised code, *status isn't set in all cases, so I don't see why this would satisfy the

Re: Mop-up around psql's \connect behavior

2020-10-22 Thread Tom Lane
I wrote: > I did actually look into saving the active connection's PQconninfo > immediately at connection establishment and then referring to it in any > subsequent \connect. Then things could work the same even if the original > connection had failed meanwhile. But there are technical details th

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 9:01 AM, Mark Dilger wrote: > > > >> On Oct 22, 2020, at 7:06 AM, Robert Haas wrote: >> >> On Thu, Oct 22, 2020 at 8:51 AM Robert Haas wrote: >>> Committed. Let's see what the buildfarm thinks. >> >> It is mostly happy, but thorntail is not: >> >> https://buildfarm.

Re: Deleting older versions in unique indexes to avoid page splits

2020-10-22 Thread Peter Geoghegan
On Thu, Oct 22, 2020 at 10:12 AM Simon Riggs wrote: > > 18,988.762398 TPS for the patch > > 11,123.551707 TPS for the master branch. > > Very good. I'm happy with this result, but as I said it's not really the point. I can probably get up to a 5x or more improvement in TPS if I simply add enough

Re: ECPG gets embedded quotes wrong

2020-10-22 Thread Tom Lane
I wrote: > Poking at this further, I noticed that there's a semi-related bug > that this patch changes the behavior for, without fixing it exactly. > That has to do with use of a string literal as "execstring" in ECPG's > PREPARE ... FROM and EXECUTE IMMEDIATE commands. Right now, it > appears tha

Re: Deleting older versions in unique indexes to avoid page splits

2020-10-22 Thread Simon Riggs
On Fri, 16 Oct 2020 at 20:12, Peter Geoghegan wrote: > The TPS/throughput is about what you'd expect for the two hour run: > > 18,988.762398 TPS for the patch > 11,123.551707 TPS for the master branch. Very good. > Patch: > > statement latencies in milliseconds: > 0.294 UPDATE pgbench

Re: new heapcheck contrib module

2020-10-22 Thread Mark Dilger
> On Oct 22, 2020, at 7:06 AM, Robert Haas wrote: > > On Thu, Oct 22, 2020 at 8:51 AM Robert Haas wrote: >> Committed. Let's see what the buildfarm thinks. > > It is mostly happy, but thorntail is not: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=thorntail&dt=2020-10-22%2012%

Re: Is Recovery actually paused?

2020-10-22 Thread Dilip Kumar
On Thu, Oct 22, 2020 at 7:50 PM Dilip Kumar wrote: > > On Thu, Oct 22, 2020 at 6:59 AM Kyotaro Horiguchi > wrote: > > > > At Wed, 21 Oct 2020 11:14:24 -0400, Robert Haas > > wrote in > > > On Wed, Oct 21, 2020 at 7:16 AM Dilip Kumar wrote: > > > > One idea could be, if the recovery process is

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
lapwing just spit up a possibly relevant issue: ccache gcc -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -O2 -Werror -fPIC -I

Re: new heapcheck contrib module

2020-10-22 Thread Robert Haas
On Thu, Oct 22, 2020 at 10:28 AM Tom Lane wrote: > Considering this is a TAP test, why in the world is it designed to hide > all details of any unexpected amcheck messages? Surely being able to > see what amcheck is saying would be helpful here. > > IOW, don't have the tests abbreviate the module

Re: new heapcheck contrib module

2020-10-22 Thread Tom Lane
Robert Haas writes: > The messages in the log aren't very > illuminating, unfortunately. :-( Considering this is a TAP test, why in the world is it designed to hide all details of any unexpected amcheck messages? Surely being able to see what amcheck is saying would be helpful here. IOW, don't

Re: partition routing layering in nodeModifyTable.c

2020-10-22 Thread Alvaro Herrera
On 2020-Oct-22, Amit Langote wrote: > 0001 fixes a thinko of the recent commit 1375422c782 that I discovered > when debugging a problem with 0003. Hmm, how hard is it to produce a test case that fails because of this problem?

Re: Is Recovery actually paused?

2020-10-22 Thread Dilip Kumar
On Thu, Oct 22, 2020 at 6:59 AM Kyotaro Horiguchi wrote: > > At Wed, 21 Oct 2020 11:14:24 -0400, Robert Haas wrote > in > > On Wed, Oct 21, 2020 at 7:16 AM Dilip Kumar wrote: > > > One idea could be, if the recovery process is waiting for WAL and a > > > recovery pause is requested then we can

Re: Implementing Incremental View Maintenance

2020-10-22 Thread Adam Brusselback
Hey there Yugo, I've asked a coworker to prepare a self contained example that encapsulates our multiple use cases. The immediate/eager approach is exactly what we need, as within the same transaction we have statements that can cause one of those "materialized tables" to be updated, and then some

Re: new heapcheck contrib module

2020-10-22 Thread Robert Haas
On Thu, Oct 22, 2020 at 8:51 AM Robert Haas wrote: > Committed. Let's see what the buildfarm thinks. It is mostly happy, but thorntail is not: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=thorntail&dt=2020-10-22%2012%3A58%3A11 I thought that the problem might be related to the fact t

Re: partition routing layering in nodeModifyTable.c

2020-10-22 Thread Amit Langote
On Tue, Oct 20, 2020 at 9:57 PM Amit Langote wrote: > On Mon, Oct 19, 2020 at 8:55 PM Heikki Linnakangas wrote: > > It's probably true that there's no performance gain from initializing > > them more lazily. But the reasoning and logic around the initialization > > is complicated. After tracing t

Re: Deleting older versions in unique indexes to avoid page splits

2020-10-22 Thread Robert Haas
On Wed, Oct 21, 2020 at 3:36 PM Peter Geoghegan wrote: > Bear in mind that we actually do practically the same thing all the > time with the current LP_DEAD setting stuff, where we need to call > compute_xid_horizon_for_tuples/heap_compute_xid_horizon_for_tuples > with a leaf buffer lock held in a

Re: Collation versioning

2020-10-22 Thread Julien Rouhaud
On Thu, Oct 22, 2020 at 8:00 PM Thomas Munro wrote: > > On Thu, Sep 24, 2020 at 9:49 PM Julien Rouhaud wrote: > > On Sun, Sep 20, 2020 at 10:24:26AM +0800, Julien Rouhaud wrote: > > > On the other hand the *_pattern_ops are entirely hardcoded, and I > > > don't think that we'll ever have an exten

Re: new heapcheck contrib module

2020-10-22 Thread Robert Haas
On Wed, Oct 21, 2020 at 11:45 PM Mark Dilger wrote: > Done that way in the attached, which also include Robert's changes from v19 > he posted earlier today. Committed. Let's see what the buildfarm thinks. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Compan

Re: Get memory contexts of an arbitrary backend process

2020-10-22 Thread torikoshia
On Thu, Oct 1, 2020 at 4:06 PM Kasahara Tatsuhito wrote: Hi, On Fri, Sep 25, 2020 at 4:28 PM torikoshia wrote: > Thanks for all your comments, I updated the patch. Thanks for updating the patch. I did a brief test and code review. Thanks for your tests and review! > I added a shared hash

Re: should INSERT SELECT use a BulkInsertState?

2020-10-22 Thread Simon Riggs
On Fri, 16 Oct 2020 at 22:05, Justin Pryzby wrote: > > > I made this conditional on BEGIN BULK/SET bulk, so I'll solicit comments > > > on that. I think it would be better if this was self-tuning. So that we don't allocate a bulkinsert state until we've done say 100 (?) rows inserted. If there

Re: [patch] Fix checksum verification in base backups for zero page headers

2020-10-22 Thread Anastasia Lubennikova
On 22.10.2020 04:25, Michael Paquier wrote: On Thu, Oct 22, 2020 at 12:47:03AM +0300, Anastasia Lubennikova wrote: We can also read such pages via shared buffers to be 100% sure. Yeah, but this has its limits as well. One can use ignore_checksum_failure, but this can actually be very dangerous

Re: [HACKERS] Custom compression methods

2020-10-22 Thread Dilip Kumar
On Wed, Oct 21, 2020 at 8:51 PM Robert Haas wrote: > > On Thu, Oct 8, 2020 at 5:54 PM Tomas Vondra > wrote: > > And is the oidvector actually needed? If we have the extra catalog, > > can't we track this simply using the regular dependencies? So we'd have > > the attcompression OID of the current

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-22 Thread Amit Kapila
On Thu, Oct 22, 2020 at 4:09 PM Masahiko Sawada wrote: > > On Wed, 21 Oct 2020 at 12:56, Amit Kapila wrote: > > > > On Tue, Oct 13, 2020 at 10:33 AM Amit Kapila > > wrote: > > > > > > On Tue, Oct 13, 2020 at 10:21 AM Tom Lane wrote: > > > > > > > > > > > > I know I can go read the source code,

Re: Enumize logical replication message actions

2020-10-22 Thread Ashutosh Bapat
On Thu, 22 Oct 2020 at 14:46, Kyotaro Horiguchi wrote: > > > We shouldn't have the default: in the switch() block in > apply_dispatch(). That prevents compilers from checking > completeness. The content of the default: should be moved out to after > the switch() block. > > apply_dispatch() > { >

Re: Improper use about DatumGetInt32

2020-10-22 Thread Ashutosh Bapat
On Fri, 16 Oct 2020 at 19:26, Alvaro Herrera wrote: > On 2020-Sep-23, Ashutosh Bapat wrote: > > > > You're ignoring the xid use-case, for which DatumGetUInt32 actually is > > > the right thing. > > > > There is DatumGetTransactionId() which should be used instead. > > That made me search if there

Re: should INSERT SELECT use a BulkInsertState?

2020-10-22 Thread Simon Riggs
On Thu, 4 Jun 2020 at 18:31, Andres Freund wrote: > On 2020-05-08 02:25:45 -0500, Justin Pryzby wrote: > > Seems to me it should, at least conditionally. At least if there's a > > function > > scan or a relation or .. > > Well, the problem is that this can cause very very significant > regressi

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-22 Thread Masahiko Sawada
On Wed, 21 Oct 2020 at 12:56, Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 10:33 AM Amit Kapila wrote: > > > > On Tue, Oct 13, 2020 at 10:21 AM Tom Lane wrote: > > > > > > > > > I know I can go read the source code, but most users will not want to. > > > Is the documentation in monitoring.sgml

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-10-22 Thread Amit Kapila
On Thu, Oct 22, 2020 at 2:20 PM Kyotaro Horiguchi wrote: > > At Thu, 22 Oct 2020 07:31:55 +, "tsunakawa.ta...@fujitsu.com" > wrote in > > From: Thomas Munro > > > On Thu, Oct 22, 2020 at 7:33 PM Kyotaro Horiguchi > > > wrote: > > > > Mmm. Not exact. The requirement here is that we must be

Re: speed up unicode decomposition and recomposition

2020-10-22 Thread John Naylor
On Thu, Oct 22, 2020 at 12:34 AM Michael Paquier wrote: > Thanks for the updated version, that was fast. I have found a couple > of places that needed to be adjusted, like the comment at the top of > generate-unicode_norm_table.pl or some comments, an incorrect include > in the new headers and t

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-10-22 Thread Thomas Munro
On Thu, Oct 22, 2020 at 9:50 PM Kyotaro Horiguchi wrote: > By the way, heap scan finds the size of target relation using > smgrnblocks(). I'm not sure why we don't miss recently-extended pages > on a heap-scan? It seems to be possible that concurrent checkpoint > fsyncs relation files inbetween

Re: Enumize logical replication message actions

2020-10-22 Thread Kyotaro Horiguchi
At Thu, 22 Oct 2020 12:13:40 +0530, Ashutosh Bapat wrote in > Thanks Andres for your review. Thanks Li, Horiguchi-san and Amit for your > comments. > > On Tue, 20 Oct 2020 at 04:57, Andres Freund wrote: > > > Hi, > > > > On 2020-10-16 12:55:26 +0530, Ashutosh Bapat wrote: > > > Here's a patch

  1   2   >