Re: Support for NSS as a libpq TLS backend

2020-10-27 Thread Andres Freund
Hi, On 2020-10-27 21:07:01 +0100, Daniel Gustafsson wrote: > > On 2020-10-20 14:24:24 +0200, Daniel Gustafsson wrote: > >> From 0cb0e6a0ce9adb18bc9d212bd03e4e09fa452972 Mon Sep 17 00:00:00 2001 > >> From: Daniel Gustafsson > >> Date: Thu, 8 Oct 2020 18:44:28 +0200 > >> Subject: [PATCH] Support fo

Re: Get memory contexts of an arbitrary backend process

2020-10-27 Thread torikoshia
On 2020-10-23 13:46, Kyotaro Horiguchi wrote: Wait... Attachments: 0003-Enabled-pg_get_backend_memory_contexts-to-collect.patch For a moment I thought that the number is patch number but the predecessors are 0002-Enabled..collect.patch and 0001-(same name). It's not mandatory but we usually d

Re: list of extended statistics on psql

2020-10-27 Thread Tatsuro Yamada
Hi Michael-san and Hackers, On 2020/09/30 15:19, Michael Paquier wrote: On Thu, Sep 17, 2020 at 02:55:31PM +0900, Michael Paquier wrote: Could you provide at least a rebased version of the patch? The CF bot is complaning here. Not seeing this answered after two weeks, I have marked the patch

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2020-10-27 Thread Tatsuo Ishii
> Status update for a commitfest entry. > > This patch is ReadyForCommitter. It applies and passes the CI. There are no > unanswered questions in the discussion. > > The discussion started in 2015 with a patch by Jeff Janes. Later it was > revived by Pavan Deolasee. After it was picked up by

Re: [spam] Re: list of extended statistics on psql

2020-10-27 Thread Tatsuro Yamada
Hi Michael-san and Hackers, On 2020/09/30 15:19, Michael Paquier wrote: On Thu, Sep 17, 2020 at 02:55:31PM +0900, Michael Paquier wrote: Could you provide at least a rebased version of the patch? The CF bot is complaning here. Not seeing this answered after two weeks, I have marked the patch

Re: duplicate function oid symbols

2020-10-27 Thread Michael Paquier
On Tue, Oct 27, 2020 at 05:40:09PM -0400, John Naylor wrote: > Ok, here is a patch to fix that, and also throw an error if pg_proc.dat has > an explicitly defined symbol. I am not seeing any uses of HEAP_TABLE_AM_HANDLER_OID in the wild, so +1 for just removing it, and not back-patch. -- Michael

Re: [HACKERS] logical decoding of two-phase transactions

2020-10-27 Thread Peter Smith
Hi Ajin. I have re-checked the v13 patches for how my remaining review comments have been addressed. On Tue, Oct 27, 2020 at 8:55 PM Ajin Cherian wrote: > > > > > v12-0002. File: src/backend/replication/logical/reorderbuffer.c > > > > > > COMMENT > > Lin

Re: Add important info about ANALYZE after create Functional Index

2020-10-27 Thread Michael Paquier
On Tue, Oct 27, 2020 at 11:06:22AM -0300, Fabrízio de Royes Mello wrote: > When we create a new table or index they will not have statistics until an > ANALYZE happens. This is the default behaviour and I think is not a big > problem here, but we need to add some note on docs about the need of > st

Re: Online checksums verification in the backend

2020-10-27 Thread Michael Paquier
On Tue, Oct 27, 2020 at 07:47:19PM +0800, Julien Rouhaud wrote: > I think it's also worth noting that the IOLock is now acquired just > before getting the buffer state, and released after the read (or after > finding that the buffer is dirty). This is consistent with how it's > done elsewhere, so

Re: Implementing Incremental View Maintenance

2020-10-27 Thread Yugo NAGATA
Hi Anastasia Lubennikova, I am writing this to you because I would like to ask the commitfest manager something. The status of the patch was changed to "Waiting on Author" from "Ready for Committer" at the beginning of this montfor the reason that rebase was necessary. Now I updated the patch, s

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-27 Thread Bharath Rupireddy
On Wed, Oct 28, 2020 at 8:29 AM vignesh C wrote: > > Log message for GSS connection is missing once connection > authorization is successful. We have similar log messages for SSL > connections once the connection authorization is successful. This > message will help the user to identify the connec

RE: Disable WAL logging to speed up data loading

2020-10-27 Thread osumi.takami...@fujitsu.com
Hi, I wrote and attached the first patch to disable WAL logging. This patch passes the regression test of check-world already and is formatted by pgindent. Also, I conducted 3 types of performance tests to clarify the rough benefits of the patch. I compared two wal_levels both 'minimal' and 'no

Re: Internal key management system

2020-10-27 Thread Craig Ringer
On Wed, Oct 28, 2020 at 9:43 AM Bruce Momjian wrote: > > I don't know much about how to hook into that stuff so if you have an > idea, I am all ears. Yeah, I have a reasonable idea. The main thing will be to re-read the patch and put it into more concrete terms, which I'll try to find time for

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-10-27 Thread Andres Freund
Hi, On 2020-10-15 01:37:35 -0700, Andres Freund wrote: > Attached is a *prototype* implemention of this concept, which clearly is > lacking some comment work (and is intentionally lacking some > re-indentation). > > I described my thoughts about how to limit the horizons for temp tables in > http

Re: Track statistics for streaming of in-progress transactions

2020-10-27 Thread Amit Kapila
On Fri, Oct 23, 2020 at 10:24 AM Amit Kapila wrote: > > On Thu, Oct 22, 2020 at 2:09 PM Amit Kapila wrote: > > > > I have fixed the above comment and rebased the patch. I have changed > the docs a bit to add more explanation about the counters. Let me know > if you have any more comments. Thanks

Re: partition routing layering in nodeModifyTable.c

2020-10-27 Thread Amit Langote
On Tue, Oct 27, 2020 at 10:23 PM Heikki Linnakangas wrote: > On 23/10/2020 12:37, Amit Langote wrote: > > To explain these numbers a bit, "overheaul update/delete processing" > > patch improves the performance of that benchmark by allowing the > > updates to use run-time pruning when executing gen

Log message for GSS connection is missing once connection authorization is successful.

2020-10-27 Thread vignesh C
Hi, Log message for GSS connection is missing once connection authorization is successful. We have similar log messages for SSL connections once the connection authorization is successful. This message will help the user to identify the connection that was selected from the logfile. I'm not sure i

Re: Re: parallel distinct union and aggregate support patch

2020-10-27 Thread bu...@sohu.com
> On Tue, Oct 27, 2020 at 3:27 PM Dilip Kumar wrote: > > > > On Fri, Oct 23, 2020 at 11:58 AM bu...@sohu.com wrote: > > > > > > > Interesting idea. So IIUC, whenever a worker is scanning the tuple it > > > > will directly put it into the respective batch(shared tuple store), > > > > based on the

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-27 Thread Amit Kapila
On Wed, Oct 28, 2020 at 5:26 AM Tomas Vondra wrote: > > On Tue, Oct 27, 2020 at 06:19:42PM -0500, Justin Pryzby wrote: > >On Tue, Oct 27, 2020 at 09:17:43AM +0530, Amit Kapila wrote: > >> On Tue, Oct 27, 2020 at 8:51 AM Justin Pryzby wrote: > >> > > >> > On Fri, Oct 23, 2020 at 10:45:34AM +0530,

Re: Patch to fix FK-related selectivity estimates with constants

2020-10-27 Thread Tomas Vondra
On Tue, Oct 27, 2020 at 09:27:06PM -0400, Tom Lane wrote: Tomas Vondra writes: On Tue, Oct 27, 2020 at 01:58:56PM -0400, Tom Lane wrote: Attached is a patch series that attacks it that way. The patch sems fine to me, thanks for investigating and fixing this. Thanks for looking at it! I

Re: Internal key management system

2020-10-27 Thread Bruce Momjian
On Tue, Oct 27, 2020 at 10:20:35AM -0400, Bruce Momjian wrote: > I don't know much about how to hook into that stuff so if you have an > idea, I am all ears. I have used OpenSSL with Yubikey via pksc11. You > can see the use of it on slide 57 and following: > > https://momjian.us/main/writ

Re: MultiXact\SLRU buffers configuration

2020-10-27 Thread Tomas Vondra
On Tue, Oct 27, 2020 at 08:23:26PM +0300, Alexander Korotkov wrote: On Tue, Oct 27, 2020 at 8:02 PM Alexander Korotkov wrote: On Mon, Oct 26, 2020 at 6:45 PM Andrey Borodin wrote: > Thanks for your review, Alexander! > +1 for avoiding double locking in SimpleLruReadPage_ReadOnly(). > Other cha

Re: [patch] ENUM errdetail should mention bytes, not chars

2020-10-27 Thread Ian Lawrence Barwick
2020年10月27日(火) 20:00 Peter Eisentraut : > > On 2020-10-19 06:34, Julien Rouhaud wrote: > >> ERROR: invalid enum label "ああ" > >> DETAIL: Labels must be 63 characters or less. > >> > >> Attached trivial patch changes the message to: > >> > >> DETAIL: Labels must

Re: Patch to fix FK-related selectivity estimates with constants

2020-10-27 Thread Tom Lane
Tomas Vondra writes: > On Tue, Oct 27, 2020 at 01:58:56PM -0400, Tom Lane wrote: >>> Attached is a patch series that attacks it that way. > The patch sems fine to me, thanks for investigating and fixing this. Thanks for looking at it! > I find it a bit strange that generate_base_implied_equalit

DROP INDEX CONCURRENTLY on partitioned index

2020-10-27 Thread Justin Pryzby
Forking this thread, since the existing CFs have been closed. https://www.postgresql.org/message-id/flat/20200914143102.GX18552%40telsasoft.com#58b1056488451f8594b0f0ba40996afd On Mon, Sep 14, 2020 at 09:31:03AM -0500, Justin Pryzby wrote: > On Sat, Sep 12, 2020 at 10:35:34AM +0900, Michael Paquie

CLUSTER on partitioned index

2020-10-27 Thread Justin Pryzby
Forking this thread, since the existing CFs have been closed. https://www.postgresql.org/message-id/flat/20200914143102.GX18552%40telsasoft.com#58b1056488451f8594b0f0ba40996afd On Tue, Oct 06, 2020 at 01:38:23PM +0900, Michael Paquier wrote: > On Mon, Oct 05, 2020 at 10:07:33PM -0500, Justin Pryzb

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-27 Thread Tomas Vondra
On Tue, Oct 27, 2020 at 06:19:42PM -0500, Justin Pryzby wrote: On Tue, Oct 27, 2020 at 09:17:43AM +0530, Amit Kapila wrote: On Tue, Oct 27, 2020 at 8:51 AM Justin Pryzby wrote: > > On Fri, Oct 23, 2020 at 10:45:34AM +0530, Amit Kapila wrote: > > On Fri, Oct 23, 2020 at 8:59 AM Amit Kapila wrot

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-27 Thread Justin Pryzby
On Tue, Oct 27, 2020 at 09:17:43AM +0530, Amit Kapila wrote: > On Tue, Oct 27, 2020 at 8:51 AM Justin Pryzby wrote: > > > > On Fri, Oct 23, 2020 at 10:45:34AM +0530, Amit Kapila wrote: > > > On Fri, Oct 23, 2020 at 8:59 AM Amit Kapila > > > wrote: > > > > > > While updating the streaming stats p

Re: Patch to fix FK-related selectivity estimates with constants

2020-10-27 Thread Tomas Vondra
On Tue, Oct 27, 2020 at 01:58:56PM -0400, Tom Lane wrote: I wrote: Over in the thread at [1] it's discussed how our code for making selectivity estimates using knowledge about FOREIGN KEY constraints is busted in the face of EquivalenceClasses including constants. ... Attached is a patch series

Re: Question about make coverage-html

2020-10-27 Thread Peter Smith
> > Creating a coverage report is a two-step process. First, you run the > > test you're interested in, with "make check" or similar. Then you create > > a report for the source files you're interested in, with "make > > coverage-html". You can run these commands in different subdirectories. > > >

Re: cutting down the TODO list thread

2020-10-27 Thread Bruce Momjian
On Tue, Oct 27, 2020 at 04:54:24PM -0400, John Naylor wrote: > > > On Tue, Oct 27, 2020 at 3:52 PM Bruce Momjian wrote: > > > Do any of these limitations need to be documented before removing them > from the TODO list? > > > I see two areas that might use a mention: > > - pg_setting

Re: duplicate function oid symbols

2020-10-27 Thread John Naylor
On Tue, Oct 27, 2020 at 9:51 AM Tom Lane wrote: > John Naylor writes: > > I noticed that the table AM abstraction introduced the symbol > > HEAP_TABLE_AM_HANDLER_OID, although we already have a convention for > > defining symbols automatically for builtin functions, which in this case > is > > (

Re: cutting down the TODO list thread

2020-10-27 Thread John Naylor
On Tue, Oct 27, 2020 at 3:52 PM Bruce Momjian wrote: > > Do any of these limitations need to be documented before removing them > from the TODO list? > I see two areas that might use a mention: - pg_settings not displaying custom variables - SQL standard difference with REVOKE ROLE (I haven't l

Re: POC: GROUP BY optimization

2020-10-27 Thread Tomas Vondra
On Mon, Oct 26, 2020 at 11:40:40AM +0100, Dmitry Dolgov wrote: On Mon, Oct 26, 2020 at 01:28:59PM +0400, Pavel Borisov wrote: > Thanks for your interest! FYI there is a new thread about this topic [1] > with the next version of the patch and more commentaries (I've created > it for visibility pur

Re: cutting down the TODO list thread

2020-10-27 Thread John Naylor
On Tue, Oct 27, 2020 at 4:00 PM Thomas Munro wrote: > On Wed, Oct 28, 2020 at 8:36 AM Andres Freund wrote: > > On 2020-10-27 15:24:35 -0400, John Naylor wrote: > > > - Allow WAL replay of CREATE TABLESPACE to work when the directory > > > structure on the recovery computer is different from the

Re: Support for NSS as a libpq TLS backend

2020-10-27 Thread Heikki Linnakangas
On 27/10/2020 22:07, Daniel Gustafsson wrote: /* * Track whether the NSS database has a password set or not. There is no API * function for retrieving password status, so we simply flip this to true in * case NSS invoked the password callback - as that will only happen in case * there is a pa

Re: cutting down the TODO list thread

2020-10-27 Thread Thomas Munro
On Wed, Oct 28, 2020 at 8:36 AM Andres Freund wrote: > On 2020-10-27 15:24:35 -0400, John Naylor wrote: > > - Allow WAL replay of CREATE TABLESPACE to work when the directory > > structure on the recovery computer is different from the original > > Thread quote: "part of the difficult, perhaps-n

More aggressive vacuuming of unlogged relations?

2020-10-27 Thread Andres Freund
Hi, The patch in [1] makes the horizon logic in procarray.c aware of temp tables not needing to care about other session's snapshots (also discussed in[2]). Extending a7212be8b9e, which did that for VACUUM, but not HOT pruning etc. While polishing that patch I was wondering whether there are othe

Re: [patch] Fix checksum verification in base backups for zero page headers

2020-10-27 Thread Anastasia Lubennikova
On 26.10.2020 04:13, Michael Paquier wrote: On Fri, Oct 23, 2020 at 08:00:08AM +0900, Michael Paquier wrote: Yeah, we could try to make the logic a bit more complicated like that. However, for any code path relying on a page read without any locking insurance, we cannot really have a lot of tru

Re: cutting down the TODO list thread

2020-10-27 Thread Bruce Momjian
On Tue, Oct 27, 2020 at 03:46:14PM -0400, Bruce Momjian wrote: > On Tue, Oct 27, 2020 at 03:24:35PM -0400, John Naylor wrote: > > As I mentioned in [1], I've volunteered to clear out the TODO list of items > > that appear to be too difficult, controversial, or otherwise not worth > > doing to > >

Re: cutting down the TODO list thread

2020-10-27 Thread Bruce Momjian
On Tue, Oct 27, 2020 at 03:24:35PM -0400, John Naylor wrote: > As I mentioned in [1], I've volunteered to clear out the TODO list of items > that appear to be too difficult, controversial, or otherwise not worth doing > to > warrant being listed there. I'll be working a few sections at a time, and

Re: cutting down the TODO list thread

2020-10-27 Thread Andres Freund
Hi, On 2020-10-27 15:24:35 -0400, John Naylor wrote: > As I mentioned in [1], I've volunteered to clear out the TODO list of items > that appear to be too difficult, controversial, or otherwise not worth > doing to warrant being listed there. I'll be working a few sections at a > time, and every s

Re: SEARCH and CYCLE clauses

2020-10-27 Thread Anastasia Lubennikova
On 10.10.2020 08:25, Pavel Stehule wrote: Hi pá 9. 10. 2020 v 12:17 odesílatel Pavel Stehule mailto:pavel.steh...@gmail.com>> napsal: pá 9. 10. 2020 v 11:40 odesílatel Peter Eisentraut mailto:peter.eisentr...@2ndquadrant.com>> napsal: On 2020-09-22 20:29, Pavel Stehule wrot

cutting down the TODO list thread

2020-10-27 Thread John Naylor
As I mentioned in [1], I've volunteered to clear out the TODO list of items that appear to be too difficult, controversial, or otherwise not worth doing to warrant being listed there. I'll be working a few sections at a time, and every so often I'll have a list of proposed items for removal. If I d

Re: Use-after-free in 12- EventTriggerAlterTableEnd

2020-10-27 Thread Tom Lane
Arseny Sher writes: > Valgrind on our internal buildfarm complained about use-after-free > during currentEventTriggerState->commandList manipulations, e.g. lappend > in EventTriggerCollectSimpleCommand. I've discovered that the source of > problem is EventTriggerAlterTableEnd not bothering to swit

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2020-10-27 Thread Anastasia Lubennikova
Status update for a commitfest entry. This patch is ReadyForCommitter. It applies and passes the CI. There are no unanswered questions in the discussion. The discussion started in 2015 with a patch by Jeff Janes. Later it was revived by Pavan Deolasee. After it was picked up by Ibrar Ahmed an

Re: Deleting older versions in unique indexes to avoid page splits

2020-10-27 Thread Peter Geoghegan
On Tue, Oct 27, 2020 at 2:44 AM Simon Riggs wrote: > While it is important we investigate the worst cases, I don't see this > is necessarily bad. I looked at "perf top" a few times when the test from yesterday ran. I saw that the proposed delete mechanism was the top consumer of CPU cycles. It se

Re: Patch to fix FK-related selectivity estimates with constants

2020-10-27 Thread Tom Lane
I wrote: > Over in the thread at [1] it's discussed how our code for making > selectivity estimates using knowledge about FOREIGN KEY constraints > is busted in the face of EquivalenceClasses including constants. > ... > Attached is a patch series that attacks it that way. I'd failed to generate a

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2020-10-27 Thread Justin Pryzby
On Wed, Oct 21, 2020 at 02:02:37PM -0300, Alvaro Herrera wrote: > On 2020-Oct-21, Justin Pryzby wrote: > > > I came up with this, which probably needs more than a little finesse. > > Hmm, there are two important changes needed on this: 1) it must not emit > CREATE lines for the child triggers; on

Re: MultiXact\SLRU buffers configuration

2020-10-27 Thread Alexander Korotkov
On Tue, Oct 27, 2020 at 8:02 PM Alexander Korotkov wrote: > On Mon, Oct 26, 2020 at 6:45 PM Andrey Borodin wrote: > > Thanks for your review, Alexander! > > +1 for avoiding double locking in SimpleLruReadPage_ReadOnly(). > > Other changes seem correct to me too. > > > > > > I've tried to find opt

Re: select_common_typmod

2020-10-27 Thread Peter Eisentraut
On 2020-10-26 10:05, Heikki Linnakangas wrote: There might have been a tiny bug in transformValuesClause() because while consolidating the typmods it does not take into account whether the types are actually the same (as more correctly done in transformSetOperationTree() and buildMergedJoinVar())

Re: MultiXact\SLRU buffers configuration

2020-10-27 Thread Alexander Korotkov
On Mon, Oct 26, 2020 at 6:45 PM Andrey Borodin wrote: > Thanks for your review, Alexander! > +1 for avoiding double locking in SimpleLruReadPage_ReadOnly(). > Other changes seem correct to me too. > > > I've tried to find optimal value for cache size and it seems to me that it > affects multixact

Re: SQL:2011 PERIODS vs Postgres Ranges?

2020-10-27 Thread Paul Jungwirth
On 10/27/20 7:11 AM, Ibrar Ahmed wrote: I have spent some more time on the patch and did a lot of cleanup along with some fixes, compilation errors, and warnings. Thank you for taking a look at this! I've been swamped with ordinary work and haven't had a chance to focus on it for a while, but

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-27 Thread Tom Lane
Robert Haas writes: > If we change this, is it going to be a compatibility break for the > contents of postgresql.conf files? I think not, at least not for the sorts of values you'd ordinarily find in that variable, say '/tmp, /var/run/postgresql'. Possibly the behavior would change for pathname

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-27 Thread Robert Haas
On Tue, Oct 27, 2020 at 9:45 AM Tom Lane wrote: > Peter Eisentraut writes: > > If we're going to change it I think we need an updated patch that covers > > pg_dump. (Even if we argue that pg_dump would not normally dump this > > variable, keeping it up to date with respect to GUC_LIST_QUOTE seem

Re: Implementing Incremental View Maintenance

2020-10-27 Thread Adam Brusselback
That was a good bit more work to get ready than I expected. It's broken into two scripts, one to create the schema, the other to load data and containing a couple check queries to ensure things are working properly (checking the materialized tables against a regular view for accuracy). The first t

Re: Parallel copy

2020-10-27 Thread vignesh C
On Fri, Oct 23, 2020 at 6:58 PM Ashutosh Sharma wrote: > > > > > I think, if possible, all these if-else checks in CopyFrom() can be > > moved to a single function which can probably be named as > > IdentifyCopyInsertMethod() and this function can be called in > > IsParallelCopyAllowed(). This wil

Re: automatic analyze: readahead - add "IO read time" log message

2020-10-27 Thread Stephen Frost
Greetings Jakub, * Jakub Wartak (jakub.war...@tomtom.com) wrote: > > The analyze is doing more-or-less random i/o since it's skipping through > > the table picking out select blocks, not doing regular sequential i/o. > VS > >> Breakpoint 1, heapam_scan_analyze_next_block (scan=0x10c8098, > >> blo

Re: Parallel copy

2020-10-27 Thread vignesh C
Thanks Ashutosh for reviewing and providing your comments. On Fri, Oct 23, 2020 at 5:43 PM Ashutosh Sharma wrote: > > Hi Vignesh, > > Thanks for the updated patches. Here are some more comments that I can > find after reviewing your latest patches: > > +/* > + * This structure helps in storing th

Re: Parallel copy

2020-10-27 Thread vignesh C
On Wed, Oct 21, 2020 at 3:50 PM Amit Kapila wrote: > > On Wed, Oct 21, 2020 at 3:19 PM Bharath Rupireddy > wrote: > > > > > > 9. Instead of calling CopyStringToSharedMemory() for each string > > variable, can't we just create a linked list of all the strings that > > need to be copied into shm an

Re: Parallel copy

2020-10-27 Thread vignesh C
Thanks Heikki for reviewing and providing your comments. Please find my thoughts below. On Fri, Oct 23, 2020 at 2:01 PM Heikki Linnakangas wrote: > > I had a brief look at at this patch. Important work! A couple of first > impressions: > > 1. The split between patches > 0002-Framework-for-leader-

Re: Parallel copy

2020-10-27 Thread vignesh C
On Wed, Oct 21, 2020 at 4:20 PM Bharath Rupireddy wrote: > > On Wed, Oct 21, 2020 at 3:18 PM Bharath Rupireddy > wrote: > > > > 17. Remove extra lines after #define IsHeaderLine() > > (cstate->header_line && cstate->cur_lineno == 1) in copy.h > > > > I missed one comment: > > 18. I think we nee

Re: automatic analyze: readahead - add "IO read time" log message

2020-10-27 Thread Jakub Wartak
Hi Stephen, hackers, > The analyze is doing more-or-less random i/o since it's skipping through > the table picking out select blocks, not doing regular sequential i/o. VS >> Breakpoint 1, heapam_scan_analyze_next_block (scan=0x10c8098, >> blockno=19890910, bstrategy=0x1102278) at heapam_handler.

Re: parallel distinct union and aggregate support patch

2020-10-27 Thread Dilip Kumar
On Tue, Oct 27, 2020 at 5:43 PM Robert Haas wrote: > > On Thu, Oct 22, 2020 at 5:08 AM Dilip Kumar wrote: > > Interesting idea. So IIUC, whenever a worker is scanning the tuple it > > will directly put it into the respective batch(shared tuple store), > > based on the hash on grouping column and

Re: Re: parallel distinct union and aggregate support patch

2020-10-27 Thread Dilip Kumar
On Tue, Oct 27, 2020 at 3:27 PM Dilip Kumar wrote: > > On Fri, Oct 23, 2020 at 11:58 AM bu...@sohu.com wrote: > > > > > Interesting idea. So IIUC, whenever a worker is scanning the tuple it > > > will directly put it into the respective batch(shared tuple store), > > > based on the hash on group

Re: Internal key management system

2020-10-27 Thread Bruce Momjian
On Tue, Oct 27, 2020 at 10:02:53PM +0800, Craig Ringer wrote: > On Tue, 27 Oct 2020, 19:15 Bruce Momjian, wrote: > We don't need to do anything except provide a way to tell OpenSSL where to get > the KEK from, for situations where having Pg generate it internally > undesirable.  > > I proposed a

Re: Question about make coverage-html

2020-10-27 Thread Tom Lane
Heikki Linnakangas writes: > On 27/10/2020 10:09, Peter Smith wrote: >> The documentation [1] also says "The make commands also work in >> subdirectories." so I tried running them all in that folder. >> However, when I run "make coverage-html" in that subdirectory >> src/test/subscription it does

Re: Add important info about ANALYZE after create Functional Index

2020-10-27 Thread Fabrízio de Royes Mello
On Tue, Oct 27, 2020 at 4:12 AM Nikolay Samokhvalov wrote: > > On Mon, Oct 26, 2020 at 3:08 PM Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: >> >> Would be nice if add some information about it into our docs but not sure where. I'm thinking about: >> - doc/src/sgml/ref/create_index.sgm

Re: Add important info about ANALYZE after create Functional Index

2020-10-27 Thread Fabrízio de Royes Mello
On Mon, Oct 26, 2020 at 7:46 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > > It would seem preferable to call the lack of auto-analyzing after these operations a bug and back-patch a fix that injects an analyze side-effect just before their completion. It doesn't have to be smart eit

Re: Internal key management system

2020-10-27 Thread Craig Ringer
On Tue, 27 Oct 2020, 19:15 Bruce Momjian, wrote: > We could implement a 'command:' prefix now, and maybe > a 'pass:' one, and allow other methods like 'pkcs11' later. > We don't need to do anything except provide a way to tell OpenSSL where to get the KEK from, for situations where having Pg gen

Re: duplicate function oid symbols

2020-10-27 Thread Tom Lane
John Naylor writes: > I noticed that the table AM abstraction introduced the symbol > HEAP_TABLE_AM_HANDLER_OID, although we already have a convention for > defining symbols automatically for builtin functions, which in this case is > (currently unused) F_HEAP_TABLEAM_HANDLER. Yeah, that seems wr

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-27 Thread Tom Lane
Peter Eisentraut writes: > If we're going to change it I think we need an updated patch that covers > pg_dump. (Even if we argue that pg_dump would not normally dump this > variable, keeping it up to date with respect to GUC_LIST_QUOTE seems > proper.) Right, I was definitely assuming that th

Re: SQL-standard function body

2020-10-27 Thread Peter Eisentraut
Here is another updated patch. I did some merging and some small fixes and introduced the pg_proc column prosqlbody to store the parsed function body separately from probin. Aside from one TODO left it seems feature-complete to me for now. -- Peter Eisentraut http://www.2ndQuadr

Re: partition routing layering in nodeModifyTable.c

2020-10-27 Thread Heikki Linnakangas
On 23/10/2020 12:37, Amit Langote wrote: To explain these numbers a bit, "overheaul update/delete processing" patch improves the performance of that benchmark by allowing the updates to use run-time pruning when executing generic plans, which they can't today. However without "lazy-ResultRelInfo

duplicate function oid symbols

2020-10-27 Thread John Naylor
Hi, I noticed that the table AM abstraction introduced the symbol HEAP_TABLE_AM_HANDLER_OID, although we already have a convention for defining symbols automatically for builtin functions, which in this case is (currently unused) F_HEAP_TABLEAM_HANDLER. It seems a wart to have two symbols for the

Re: parallel distinct union and aggregate support patch

2020-10-27 Thread Robert Haas
On Thu, Oct 22, 2020 at 5:08 AM Dilip Kumar wrote: > Interesting idea. So IIUC, whenever a worker is scanning the tuple it > will directly put it into the respective batch(shared tuple store), > based on the hash on grouping column and once all the workers are > doing preparing the batch then eac

Re: Move catalog toast table and index declarations

2020-10-27 Thread John Naylor
On Tue, Oct 27, 2020 at 7:43 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2020-10-24 15:23, John Naylor wrote: > > Style: In genbki.h, "extern int no_such_variable" is now out of place. > > Also, the old comments like "The macro definitions are just to keep the > > C compile

Re: Online checksums verification in the backend

2020-10-27 Thread Julien Rouhaud
On Tue, Oct 27, 2020 at 3:07 PM Michael Paquier wrote: > > On Fri, Oct 23, 2020 at 06:06:30PM +0900, Michael Paquier wrote: > > On Fri, Oct 23, 2020 at 04:31:56PM +0800, Julien Rouhaud wrote: > >> Mmm, is it really an improvement to report warnings during this > >> function execution? Note also t

Re: Yet another fast GiST build

2020-10-27 Thread Heikki Linnakangas
On 21/10/2020 20:13, Andrey Borodin wrote: 7 окт. 2020 г., в 17:38, Heikki Linnakangas написал(а): On 07/10/2020 15:27, Andrey Borodin wrote: Here's draft patch with implementation of sortsupport for ints and floats. +static int +gbt_int4_cmp(Datum a, Datum b, SortSupport ssup) +{ + i

Re: Move catalog toast table and index declarations

2020-10-27 Thread Peter Eisentraut
On 2020-10-24 15:23, John Naylor wrote: This part created a syntax error: --- a/src/include/catalog/unused_oids +++ b/src/include/catalog/unused_oids @@ -28,7 +28,7 @@ chdir $FindBin::RealBin or die "could not cd to $FindBin::RealBin: $!\n";  use lib "$FindBin::RealBin/../../backend/catalog/"

Re: Prevent printing "next step instructions" in initdb and pg_upgrade

2020-10-27 Thread Bruce Momjian
On Tue, Oct 27, 2020 at 12:35:56PM +0100, Peter Eisentraut wrote: > On 2020-10-27 11:53, Bruce Momjian wrote: > > On Tue, Oct 27, 2020 at 11:35:25AM +0100, Peter Eisentraut wrote: > > > On 2020-10-06 12:26, Magnus Hagander wrote: > > > > I went with the name --no-instructions to have the same name

Re: Prevent printing "next step instructions" in initdb and pg_upgrade

2020-10-27 Thread Peter Eisentraut
On 2020-10-27 11:53, Bruce Momjian wrote: On Tue, Oct 27, 2020 at 11:35:25AM +0100, Peter Eisentraut wrote: On 2020-10-06 12:26, Magnus Hagander wrote: I went with the name --no-instructions to have the same name for both initdb and pg_upgrade. The downside is that "no-instructions" also causes

Re: Internal key management system

2020-10-27 Thread Bruce Momjian
On Tue, Oct 27, 2020 at 03:07:22PM +0800, Craig Ringer wrote: > On Mon, Oct 26, 2020 at 11:02 PM Stephen Frost wrote: > > > TL;DR: > > * Important to check that key rotation is possible on a replica, i.e. > primary and standby can have different cluster passphrase and KEK > encrypting the same

Re: Internal key management system

2020-10-27 Thread Bruce Momjian
On Mon, Oct 26, 2020 at 10:05:10PM +0800, Craig Ringer wrote: > For example if I want to lock my database with a YubiHSM I would configure > something like: > >     cluster_encryption_key = 'pkcs11:token=YubiHSM;id=0:0001;type=private' Well, openssl uses a prefix before the password string, e.g.:

Re: Prevent printing "next step instructions" in initdb and pg_upgrade

2020-10-27 Thread Magnus Hagander
On Tue, Oct 27, 2020 at 11:53 AM Bruce Momjian wrote: > On Tue, Oct 27, 2020 at 11:35:25AM +0100, Peter Eisentraut wrote: > > On 2020-10-06 12:26, Magnus Hagander wrote: > > > I went with the name --no-instructions to have the same name for both > > > initdb and pg_upgrade. The downside is that "

Re: [patch] ENUM errdetail should mention bytes, not chars

2020-10-27 Thread Peter Eisentraut
On 2020-10-19 06:34, Julien Rouhaud wrote: ERROR: invalid enum label "ああ" DETAIL: Labels must be 63 characters or less. Attached trivial patch changes the message to: DETAIL: Labels must be 63 bytes or less. This matches the documentation, which states:

Re: Prevent printing "next step instructions" in initdb and pg_upgrade

2020-10-27 Thread Bruce Momjian
On Tue, Oct 27, 2020 at 11:35:25AM +0100, Peter Eisentraut wrote: > On 2020-10-06 12:26, Magnus Hagander wrote: > > I went with the name --no-instructions to have the same name for both > > initdb and pg_upgrade. The downside is that "no-instructions" also > > causes the scripts not to be written i

Re: deferred primary key and logical replication

2020-10-27 Thread Amit Kapila
On Sun, Oct 25, 2020 at 9:39 PM Euler Taveira wrote: > > On Mon, 5 Oct 2020 at 08:34, Amit Kapila wrote: >> >> On Mon, May 11, 2020 at 2:41 AM Euler Taveira >> wrote: >> > >> > Hi, >> > >> > While looking at an old wal2json issue, I stumbled on a scenario that a >> > table >> > with a deferred

Re: "unix_socket_directories" should be GUC_LIST_INPUT?

2020-10-27 Thread Peter Eisentraut
On 2020-10-23 10:02, Michael Paquier wrote: So maybe we could get away with just changing it. It'd be good to verify though that this doesn't break existing string values for the variable, assuming they contain no unlikely characters that'd need quoting. Yeah, that's the kind of things I wante

Re: Prevent printing "next step instructions" in initdb and pg_upgrade

2020-10-27 Thread Peter Eisentraut
On 2020-10-06 12:26, Magnus Hagander wrote: I went with the name --no-instructions to have the same name for both initdb and pg_upgrade. The downside is that "no-instructions" also causes the scripts not to be written in pg_upgrade, which arguably is a different thing. We could go with "--no-in

Re: pg_upgrade analyze script

2020-10-27 Thread Peter Eisentraut
On 2020-10-06 11:43, Magnus Hagander wrote: For a long time now, pg_upgrade drops a script (analyze_new_cluster.sh) that just calls vacuumdb to run the analyze in stages. This script made a lot of sense back when it actually implemented the stages, but these days since it's just calling a singl

Use-after-free in 12- EventTriggerAlterTableEnd

2020-10-27 Thread Arseny Sher
Hi, Valgrind on our internal buildfarm complained about use-after-free during currentEventTriggerState->commandList manipulations, e.g. lappend in EventTriggerCollectSimpleCommand. I've discovered that the source of problem is EventTriggerAlterTableEnd not bothering to switch into its own context

Re: Re: parallel distinct union and aggregate support patch

2020-10-27 Thread Dilip Kumar
On Fri, Oct 23, 2020 at 11:58 AM bu...@sohu.com wrote: > > > Interesting idea. So IIUC, whenever a worker is scanning the tuple it > > will directly put it into the respective batch(shared tuple store), > > based on the hash on grouping column and once all the workers are > > doing preparing the

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-27 Thread Amit Kapila
On Thu, Oct 22, 2020 at 9:47 AM Greg Nancarrow wrote: > > On Fri, Oct 16, 2020 at 9:26 PM Amit Kapila wrote: > > > > Posting an update to the smaller patch (Parallel SELECT for INSERT > INTO...SELECT...). > > Most of this patch feeds into the larger Parallel INSERT patch, for > which I'll also be

Re: Deleting older versions in unique indexes to avoid page splits

2020-10-27 Thread Simon Riggs
On Mon, 26 Oct 2020 at 21:15, Peter Geoghegan wrote: > Now for the not-so-good news. The TPS numbers looked like this > (results in original chronological order of the runs, which I've > interleaved): While it is important we investigate the worst cases, I don't see this is necessarily bad. HOT

RE: Multiple hosts in connection string failed to failover in non-hot standby mode

2020-10-27 Thread tsunakawa.ta...@fujitsu.com
Please send emails in text format. Your email was in HTML, and I changed this reply to text format. From: Hubert Zhang > Libpq has supported to specify multiple hosts in connection string and enable > auto failover when the previous PostgreSQL instance cannot be accessed. > But when I tried

Re: Question about make coverage-html

2020-10-27 Thread Heikki Linnakangas
On 27/10/2020 10:09, Peter Smith wrote: Hi hackers. The example of test coverage in the documentation [1] works as advertised. But I wanted to generate test coverage results only of some TAP tests in src/test/subscription. The documentation [1] also says "The make commands also work in subdire

Re: Commitfest manager 2020-11

2020-10-27 Thread gkokolatos
‐‐‐ Original Message ‐‐‐ On Sunday, October 25, 2020 8:01 PM, Anastasia Lubennikova wrote: > On 20.10.2020 10:30, gkokola...@pm.me wrote: > > > ‐‐‐ Original Message ‐‐‐ > > > > > [snip] > > > Wow, that was well in advance) I am willing to assist if you need any > > > help.

Re: [PATCH] remove deprecated v8.2 containment operators

2020-10-27 Thread Magnus Hagander
On Tue, Oct 27, 2020 at 9:38 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2020-10-27 04:25, Justin Pryzby wrote: > > Forking this thread: > > > https://www.postgresql.org/message-id/fd93f1c5-7818-a02c-01e5-1075ac0d4...@iki.fi > > > > They have been deprecated for a Long Time

Multiple hosts in connection string failed to failover in non-hot standby mode

2020-10-27 Thread Hubert Zhang
Hi hackers, Libpq has supported to specify multiple hosts in connection string and enable auto failover when the previous PostgreSQL instance cannot be accessed. But when I tried to enable this feature for a non-hot standby, it cannot do the failover with the following messages. psql: error: co

  1   2   >