Re: Global snapshots

2020-10-28 Thread Fujii Masao
On 2020/10/23 11:58, Masahiko Sawada wrote: On Thu, 15 Oct 2020 at 01:41, Fujii Masao wrote: On 2020/09/17 15:56, Amit Kapila wrote: On Thu, Sep 10, 2020 at 4:20 PM Fujii Masao wrote: One alternative is to add only hooks into PostgreSQL core so that we can implement the global transa

Re: [patch] Fix checksum verification in base backups for zero page headers

2020-10-28 Thread Michael Paquier
On Tue, Oct 27, 2020 at 10:56:23PM +0300, Anastasia Lubennikova wrote: > In case you need a second opinion on the remaining patch, it still looks > good to me. Thanks. The patch for v13 cannot use a macro, but one of the versions of upthread would do just fine. I have been wondering about using

Re: Disable WAL logging to speed up data loading

2020-10-28 Thread Laurenz Albe
On Wed, 2020-10-28 at 04:11 +, osumi.takami...@fujitsu.com wrote: > I wrote and attached the first patch to disable WAL logging. > This patch passes the regression test of check-world already > and is formatted by pgindent. Without reading the code, I have my doubts about that feature. While

RE: Global snapshots

2020-10-28 Thread tsunakawa.ta...@fujitsu.com
Fujii-san, Sawada-san, all, From: Fujii Masao > Yeah, so if we need to guarantee the transaction linearizability even > in distributed env (probably this is yes. Right?), using only Clock-SI > is not enough. We would need to implement something more > in addition to Clock-SI or adopt the differen

Re: list of extended statistics on psql

2020-10-28 Thread Tatsuro Yamada
Hi, Results of \dX and \dX+: postgres=# \dX     List of extended statistics     Schema    |   Name    |   Definition    | N_distinct | Dependencies |   Mcv -+---+-++--+-   publ

Re: MultiXact\SLRU buffers configuration

2020-10-28 Thread Andrey Borodin
Tomas, thanks for looking into this! > 28 окт. 2020 г., в 06:36, Tomas Vondra > написал(а): > > > This thread started with a discussion about making the SLRU sizes > configurable, but this patch version only adds a local cache. Does this > achieve the same goal, or would we still gain somethin

Re: [patch] Fix checksum verification in base backups for zero page headers

2020-10-28 Thread Michael Paquier
On Wed, Oct 28, 2020 at 04:11:56PM +0900, Michael Paquier wrote: > Thanks. The patch for v13 cannot use a macro, but one of the versions > of upthread would do just fine. I have been wondering about using the > new CheckBuffer() for the purpose of the retry to make it > concurrent-safe, but by lo

Re: partition routing layering in nodeModifyTable.c

2020-10-28 Thread Amit Langote
On Tue, Oct 27, 2020 at 10:23 PM Heikki Linnakangas wrote: > This patch looks reasonable to me at a quick glance. I'm a bit worried > or unhappy about the impact on FDWs, though. It doesn't seem nice that > the ResultRelInfo is not available in the BeginDirectModify call. It's > not too bad, the F

Re: Implementing Incremental View Maintenance

2020-10-28 Thread Yugo NAGATA
On Tue, 27 Oct 2020 12:14:52 -0400 Adam Brusselback wrote: > That was a good bit more work to get ready than I expected. It's broken > into two scripts, one to create the schema, the other to load data and > containing a couple check queries to ensure things are working properly > (checking the m

Re: [PATCH] Add features to pg_stat_statements

2020-10-28 Thread seinoyu
2020-10-22 01:31 に Fujii Masao さんは書きました: On 2020/10/12 21:18, Yuki Seino wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:

RE: Disable WAL logging to speed up data loading

2020-10-28 Thread tsunakawa.ta...@fujitsu.com
From: Osumi, Takamichi/大墨 昂道 > I wrote and attached the first patch to disable WAL logging. > This patch passes the regression test of check-world already and is formatted I think make check-world uses the default setting for wal_level. You need to set wal_level = none and do make installcheck-

Re: Implementing Incremental View Maintenance

2020-10-28 Thread Anastasia Lubennikova
ср, 28 окт. 2020 г. в 08:02, Yugo NAGATA : > Hi Anastasia Lubennikova, > > I am writing this to you because I would like to ask the commitfest > manager something. > > The status of the patch was changed to "Waiting on Author" from > "Ready for Committer" at the beginning of this montfor the reaso

Re: Implementing Incremental View Maintenance

2020-10-28 Thread Yugo NAGATA
On Wed, 28 Oct 2020 12:01:58 +0300 Anastasia Lubennikova wrote: > ср, 28 окт. 2020 г. в 08:02, Yugo NAGATA : > > > Hi Anastasia Lubennikova, > > > > I am writing this to you because I would like to ask the commitfest > > manager something. > > > > The status of the patch was changed to "Waiting

Re: Wrong example in the bloom documentation

2020-10-28 Thread Daniel Westermann (DWE)
>I figured it out --- you have to use the larger generate_series value to >get the parallel output.  I have adjusted all the docs back to 9.6 to >show accurate output for that version, and simplified the query >ordering --- patch to master attached.  The other releases are similar. >Daniel, please

Re: Internal key management system

2020-10-28 Thread Craig Ringer
On Wed, Oct 28, 2020 at 12:02 PM Craig Ringer wrote: > On Wed, Oct 28, 2020 at 9:43 AM Bruce Momjian wrote: > > > >> I don't know much about how to hook into that stuff so if you have an >> idea, I am all ears. > > > Yeah, I have a reasonable idea. The main thing will be to re-read the > patch a

RE: pgbench: option delaying queries till connections establishment?

2020-10-28 Thread kuroda.hay...@fujitsu.com
Dear Fabien; > The current implementation is too simple. If nthreads >= 2 and connection > fails in the one thread, > the other one will wait forever. I attached the very preliminary patch for solving the problem. Even if threads fail to connect, the others can go through the barrier. But I thin

Re: parallel distinct union and aggregate support patch

2020-10-28 Thread bu...@sohu.com
Hi Here is patch for parallel distinct union aggregate and grouping sets support using batch hash agg. Please review. how to use: set enable_batch_hashagg = on how to work: like batch sort, but not sort each batch, just save hash value in each rows unfinished work: not support rescan yet. welco

Re: cutting down the TODO list thread

2020-10-28 Thread Oleksandr Shulgin
On Tue, Oct 27, 2020 at 8:25 PM John Naylor wrote: > As I mentioned in [1], I've volunteered to clear out the TODO list of > items that appear to be too difficult, controversial, or otherwise not > worth doing to warrant being listed there. I'll be working a few sections > at a time, and every so

RE: Disable WAL logging to speed up data loading

2020-10-28 Thread osumi.takami...@fujitsu.com
Hi, Laurenz > > I wrote and attached the first patch to disable WAL logging. > > This patch passes the regression test of check-world already and is > > formatted by pgindent. > > Without reading the code, I have my doubts about that feature. > > While it clearly will improve performance, it op

Re: shared tempfile was not removed on statement_timeout

2020-10-28 Thread Masahiko Sawada
On Wed, 29 Jul 2020 at 10:37, Justin Pryzby wrote: > > On Mon, Jul 27, 2020 at 05:39:02AM -0500, Justin Pryzby wrote: > > On Mon, Jul 27, 2020 at 08:00:46PM +1200, Thomas Munro wrote: > > > Why can't tuplesort_end do it? > > > > Because then I think the parallel workers remove their own files, wit

Re: cutting down the TODO list thread

2020-10-28 Thread Julien Rouhaud
On Wed, 28 Oct 2020, 17:55 Oleksandr Shulgin On Tue, Oct 27, 2020 at 8:25 PM John Naylor > wrote: > >> As I mentioned in [1], I've volunteered to clear out the TODO list of >> items that appear to be too difficult, controversial, or otherwise not >> worth doing to warrant being listed there. I'll

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2020-10-28 Thread Noah Misch
On Tue, Oct 27, 2020 at 07:14:14AM +, Hubert Zhang wrote: > Libpq has supported to specify multiple hosts in connection string and enable > auto failover when the previous PostgreSQL instance cannot be accessed. > But when I tried to enable this feature for a non-hot standby, it cannot do > t

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2020-10-28 Thread Hubert Zhang
Was the primary running and accepting connections when you encountered this error? That is, if you specified host="host1 host2", host1 was the non-hot standby and host2 was a running primary? Or only the non-hot standby was running? If a primary was running, I'd say it's a bug... Perhaps the

Re: cutting down the TODO list thread

2020-10-28 Thread Magnus Hagander
On Wed, Oct 28, 2020 at 11:15 AM Julien Rouhaud wrote: > > On Wed, 28 Oct 2020, 17:55 Oleksandr Shulgin wrote: >> >> On Tue, Oct 27, 2020 at 8:25 PM John Naylor >> wrote: >>> >>> As I mentioned in [1], I've volunteered to clear out the TODO list of items >>> that appear to be too difficult, co

RE: Multiple hosts in connection string failed to failover in non-hot standby mode

2020-10-28 Thread tsunakawa.ta...@fujitsu.com
From: Hubert Zhang > Hao Wu and I wrote a patch to fix this problem. Client side libpq should try > another hosts in connection string when it is rejected by a non-hot standby, > or the first host encounter some n/w problems during the libpq handshake. Thank you. Please add it to the November

Valgrind run error with Postgres on OSX

2020-10-28 Thread Pavel Borisov
Hi, hackers! I tried to run PG in a Valgrind on OSX 10.13 with Clang 9/10 and get an assertion error inside Valgrind valgrind --leak-check=no --gen-suppressions=all --time-stamp=yes \ --error-markers=VALGRINDERROR-BEGIN,VALGRINDERROR-END \ --log-file=%p.log --trace-children=yes postgres -D ./p

Re: A new function to wait for the backend exit after termination

2020-10-28 Thread Bharath Rupireddy
On Wed, Oct 21, 2020 at 6:43 PM Magnus Hagander wrote: > > I think it would be nicer to have a pg_terminate_backend(pid, wait=false), so > a function with a second parameter which defaults to the current behaviour of > not waiting. And it might be a good idea to also give it a timeout parameter?

Re: Allow some recovery parameters to be changed with reload

2020-10-28 Thread Sergei Kornilov
Hello Sorry for late response. >>  > ... but what's the corresponding hazard here, exactly? It doesn't seem >>  > that there's any way in which the decision one process makes affects >>  > the decision the other process makes. There's still a race condition: >>  > it's possible for a walsender >>

RE: Parallel copy

2020-10-28 Thread Hou, Zhijie
Hi I found some issue in v9-0002 1. + + elog(DEBUG1, "[Worker] Processing - line position:%d, block:%d, unprocessed lines:%d, offset:%d, line size:%d", +write_pos, lineInfo->first_block, +pg_atomic_read_u32(&data_blk_ptr->unprocessed_line_parts), +

Re: [PATCH] Add features to pg_stat_statements

2020-10-28 Thread Fujii Masao
On 2020/10/28 17:31, seinoyu wrote: 2020-10-22 01:31 に Fujii Masao さんは書きました: On 2020/10/12 21:18, Yuki Seino wrote: The following review has been posted through the commitfest application: make installcheck-world:  tested, passed Implements feature:   tested, passed Spec compliant:  

Re: Internal key management system

2020-10-28 Thread Masahiko Sawada
On Tue, 27 Oct 2020 at 20:34, Bruce Momjian wrote: > > On Tue, Oct 27, 2020 at 03:07:22PM +0800, Craig Ringer wrote: > > On Mon, Oct 26, 2020 at 11:02 PM Stephen Frost wrote: > > > > > > TL;DR: > > > > * Important to check that key rotation is possible on a replica, i.e. > > primary and standby c

Re: parallel distinct union and aggregate support patch

2020-10-28 Thread Tomas Vondra
Hi, On Wed, Oct 28, 2020 at 05:37:40PM +0800, bu...@sohu.com wrote: Hi Here is patch for parallel distinct union aggregate and grouping sets support using batch hash agg. Please review. how to use: set enable_batch_hashagg = on how to work: like batch sort, but not sort each batch, just save

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Fabrízio de Royes Mello
On Wed, Oct 28, 2020 at 2:15 AM Michael Paquier wrote: > > On Tue, Oct 27, 2020 at 11:06:22AM -0300, Fabrízio de Royes Mello wrote: > > When we create a new table or index they will not have statistics until an > > ANALYZE happens. This is the default behaviour and I think is not a big > > problem

Re: Allow some recovery parameters to be changed with reload

2020-10-28 Thread Fujii Masao
On 2020/10/28 21:02, Sergei Kornilov wrote: Hello Sorry for late response.  > ... but what's the corresponding hazard here, exactly? It doesn't seem  > that there's any way in which the decision one process makes affects  > the decision the other process makes. There's still a race condi

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-10-28 Thread k.jami...@fujitsu.com
Hi everyone, Attached are the updated set of patches (V28). 0004 - Truncate optimization is a new patch, while the rest are similar to V27. This passes the build, regression and TAP tests. Apologies for the delay. I'll post the benchmark test results on SSD soon, considering the suggested bench

Re: Disable WAL logging to speed up data loading

2020-10-28 Thread Laurenz Albe
On Wed, 2020-10-28 at 09:55 +, osumi.takami...@fujitsu.com wrote: > > > I wrote and attached the first patch to disable WAL logging. > > > This patch passes the regression test of check-world already and is > > > formatted by pgindent. > > > > Without reading the code, I have my doubts about th

Re: A new function to wait for the backend exit after termination

2020-10-28 Thread Fujii Masao
On 2020/10/28 20:50, Bharath Rupireddy wrote: On Wed, Oct 21, 2020 at 6:43 PM Magnus Hagander wrote: I think it would be nicer to have a pg_terminate_backend(pid, wait=false), so a function with a second parameter which defaults to the current behaviour of not waiting. And it might be a g

Re: cutting down the TODO list thread

2020-10-28 Thread John Naylor
On Tue, Oct 27, 2020 at 6:05 PM Bruce Momjian wrote: > On Tue, Oct 27, 2020 at 04:54:24PM -0400, John Naylor wrote: > > > > > > On Tue, Oct 27, 2020 at 3:52 PM Bruce Momjian wrote: > > > > > > Do any of these limitations need to be documented before removing > them > > from the TODO list

Re: cutting down the TODO list thread

2020-10-28 Thread John Naylor
On Wed, Oct 28, 2020 at 6:52 AM Magnus Hagander wrote: > On Wed, Oct 28, 2020 at 11:15 AM Julien Rouhaud > wrote: > > > > On Wed, 28 Oct 2020, 17:55 Oleksandr Shulgin < > oleksandr.shul...@zalando.de wrote: > >> I'm totally on board with cleaning the list up, but how about marking > as "won't fi

cannot freeze committed xmax

2020-10-28 Thread Konstantin Knizhnik
Hi hackers, The following error was encountered by our customers: Them have  very huge catalog (size of pg_class relation is more than 30Gb)  blowned by temporary relations. When them try to vacuum it, the following error is reported: vacuum full analyze pg_catalog.pg_class; ERROR:  cannot fre

Re: A new function to wait for the backend exit after termination

2020-10-28 Thread Bharath Rupireddy
Thanks for the comments. On Wed, Oct 28, 2020 at 6:41 PM Fujii Masao wrote: > > I prefer that false is returned when the timeout happens, > like pg_promote() does. > Earlier it was suggested to error out on timeout. Since users can not guess on time it takes to terminate or become idle, throwing

cleanup temporary files after crash

2020-10-28 Thread Euler Taveira
Hi, Bug #16427 mentioned that temporary files are not removed after a crash. I heard similar complaints from some customers. In the bug report [1], I proposed a new GUC to control whether temporary files are removed after a crash recovery. The current behavior is only useful for debugging purposes

Re: A new function to wait for the backend exit after termination

2020-10-28 Thread David G. Johnston
On Wed, Oct 28, 2020 at 6:50 AM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > Thanks for the comments. > > On Wed, Oct 28, 2020 at 6:41 PM Fujii Masao > wrote: > > > > I prefer that false is returned when the timeout happens, > > like pg_promote() does. > > > > Earlier it w

Re: cutting down the TODO list thread

2020-10-28 Thread Julien Rouhaud
On Wed, Oct 28, 2020 at 9:27 PM John Naylor wrote: > > On Wed, Oct 28, 2020 at 6:52 AM Magnus Hagander wrote: >> >> On Wed, Oct 28, 2020 at 11:15 AM Julien Rouhaud wrote: >> > >> > On Wed, 28 Oct 2020, 17:55 Oleksandr Shulgin > > wrote: >> >> I'm totally on board with cleaning the list up, but h

Re: Add Information during standby recovery conflicts

2020-10-28 Thread Fujii Masao
On 2020/10/27 9:41, Masahiko Sawada wrote: On Tue, 20 Oct 2020 at 22:02, Drouvot, Bertrand wrote: Hi, On 10/15/20 9:15 AM, Masahiko Sawada wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and

Re: cutting down the TODO list thread

2020-10-28 Thread Magnus Hagander
On Wed, Oct 28, 2020 at 3:35 PM Julien Rouhaud wrote: > > On Wed, Oct 28, 2020 at 9:27 PM John Naylor > wrote: > > > > On Wed, Oct 28, 2020 at 6:52 AM Magnus Hagander wrote: > >> > >> On Wed, Oct 28, 2020 at 11:15 AM Julien Rouhaud wrote: > >> > > >> > On Wed, 28 Oct 2020, 17:55 Oleksandr Shulg

Re: libpq compression

2020-10-28 Thread Daniil Zakhlystov
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested Hi, thanks for the patch! I’ve made a quick review and found one issue. I

Re: cannot freeze committed xmax

2020-10-28 Thread Mark Dilger
> On Oct 28, 2020, at 6:44 AM, Konstantin Knizhnik > wrote: > > Looks like there is no assumption that xmax should be set to > InvalidTransactionId when HEAP_XMAX_INVALID bit is set. > And I didn't find any check preventing cutoff_xid to be greater than XID of > some transaction which was

Re: Patch to fix FK-related selectivity estimates with constants

2020-10-28 Thread Alvaro Herrera
On 2020-Oct-27, Tom Lane wrote: > I had two concerns about possible extension breakage from a back-patch: > > * Changing the set of fields in ForeignKeyOptInfo is an ABI break. > We could minimize the risk by adding the new fields at the end in > the back branches, but it still wouldn't be zero-r

Re: duplicate function oid symbols

2020-10-28 Thread John Naylor
I wrote: > Ok, here is a patch to fix that, and also throw an error if pg_proc.dat > has an explicitly defined symbol. > It occurred to me I neglected to explain the error with a comment, which I've added in v2. -- John Naylor EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL

Re: cannot freeze committed xmax

2020-10-28 Thread Konstantin Knizhnik
On 28.10.2020 18:25, Mark Dilger wrote: On Oct 28, 2020, at 6:44 AM, Konstantin Knizhnik wrote: Looks like there is no assumption that xmax should be set to InvalidTransactionId when HEAP_XMAX_INVALID bit is set. And I didn't find any check preventing cutoff_xid to be greater than XID o

Re: Patch to fix FK-related selectivity estimates with constants

2020-10-28 Thread Tom Lane
Alvaro Herrera writes: > On 2020-Oct-27, Tom Lane wrote: >> * Changing the set of fields in ForeignKeyOptInfo is an ABI break. >> We could minimize the risk by adding the new fields at the end in >> the back branches, but it still wouldn't be zero-risk. > It'd be useful to be able to qualify this

Re: Patch to fix FK-related selectivity estimates with constants

2020-10-28 Thread Alvaro Herrera
On 2020-Oct-28, Tom Lane wrote: > Alvaro Herrera writes: > > I think if a struct is used as a function argument somewhere or arrays > > of the struct are formed, then it's certain that changing that struct's > > size is going to cause problems. > > I grasp the point about arrays, but not sure h

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-28 Thread Stephen Frost
Greetings, * vignesh C (vignes...@gmail.com) wrote: > Log message for GSS connection is missing once connection > authorization is successful. We have similar log messages for SSL > connections once the connection authorization is successful. This > message will help the user to identify the conne

Re: cannot freeze committed xmax

2020-10-28 Thread Mark Dilger
> On Oct 28, 2020, at 8:56 AM, Konstantin Knizhnik > wrote: > > > > On 28.10.2020 18:25, Mark Dilger wrote: >> >>> On Oct 28, 2020, at 6:44 AM, Konstantin Knizhnik >>> wrote: >>> >>> Looks like there is no assumption that xmax should be set to >>> InvalidTransactionId when HEAP_XMAX_IN

Re: Internal key management system

2020-10-28 Thread Bruce Momjian
On Wed, Oct 28, 2020 at 09:24:35PM +0900, Masahiko Sawada wrote: > On Tue, 27 Oct 2020 at 20:34, Bruce Momjian wrote: > > You need to use separate keys for heap/index and WAL so you can > > replicate to another server that uses a different heap/index key, but > > the same WAL. You can then fail-o

Re: libpq compression

2020-10-28 Thread Konstantin Knizhnik
On 28.10.2020 17:57, Даниил Захлыстов wrote: Hi, thanks for the patch! I’ve made a quick review and found one issue. If the backend sends a CompressionAck message followed by some already compressed message (for example, AuthenticationOk), then there is a chance that pqReadData() will read bo

Re: duplicate function oid symbols

2020-10-28 Thread Tom Lane
John Naylor writes: >> Ok, here is a patch to fix that, and also throw an error if pg_proc.dat >> has an explicitly defined symbol. > It occurred to me I neglected to explain the error with a comment, which > I've added in v2. Pushed with a bit of tweaking of the error message. I wondered about

Re: Internal key management system

2020-10-28 Thread Stephen Frost
Greetings, * Craig Ringer (craig.rin...@enterprisedb.com) wrote: > On Mon, Oct 26, 2020 at 11:02 PM Stephen Frost wrote: > > TL;DR: > > * Important to check that key rotation is possible on a replica, i.e. > primary and standby can have different cluster passphrase and KEK > encrypting the same

Re: cutting down the TODO list thread

2020-10-28 Thread Andres Freund
On 2020-10-28 16:20:03 +0100, Magnus Hagander wrote: > I would personally prefer a completely seprate page Same.

Re: [PATCH] Add `truncate` option to subscription commands

2020-10-28 Thread Rahila Syed
Hi David, The feature seems useful to me. The code will need to be refactored due to changes in commit : b05fe7b442 Please see the following comments. 1. Is there a specific reason behind having new relstate for truncate? The current state flow is INIT->DATATSYNC->SYNCWAIT->CATCHUP->SYNCDONE->RE

Re: duplicate function oid symbols

2020-10-28 Thread John Naylor
On Wed, Oct 28, 2020 at 12:25 PM Tom Lane wrote: > I wondered about introducing a similar prohibition for pg_type. > That might be worth doing, since some of the grandfathered macros are clustered together, which could lead to more cases creeping in as people match new types to examples nearby.

document pg_settings view doesn't display custom options

2020-10-28 Thread John Naylor
Starting separate threads to keep from cluttering the TODO list thread. Here's a patch for the subject, as mentioned in https://www.postgresql.org/message-id/20201027220555.GS4951%40momjian.us -- John Naylor EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company doc-pg-set

Re: duplicate function oid symbols

2020-10-28 Thread Andres Freund
Hi, Thanks for fixing the HEAP_TABLE_AM_HANDLER_OID one. On 2020-10-28 14:08:28 -0400, John Naylor wrote: > > The only existing oid_symbol in pg_type that I think has enough > > grandfather status to be tough to change is CASHOID for "money". > > But we could imagine special-casing that with a ha

Re: Internal key management system

2020-10-28 Thread Bruce Momjian
On Wed, Oct 28, 2020 at 12:02:46PM +0800, Craig Ringer wrote: > On Wed, Oct 28, 2020 at 9:43 AM Bruce Momjian wrote: >  I have used OpenSSL with Yubikey via pksc11.  You > can see the use of it on slide 57 and following: > >         https://momjian.us/main/writings/crypto_hw_config.pd

Re: cleanup temporary files after crash

2020-10-28 Thread Tomas Vondra
Hi Euler, On Wed, Oct 28, 2020 at 11:16:26AM -0300, Euler Taveira wrote: Hi, Bug #16427 mentioned that temporary files are not removed after a crash. I heard similar complaints from some customers. In the bug report [1], I proposed a new GUC to control whether temporary files are removed after

Autovacuum worker doesn't immediately exit on postmaster death

2020-10-28 Thread Alexander Kukushkin
Hello, I know, nobody in their mind should do that, but, if the postmaster process is killed with SIGKILL signal, most backend processes correctly notice the fact of the postmaster process absence and exit. There is one exception though, when there are autovacuum worker processes they are continui

Re: document pg_settings view doesn't display custom options

2020-10-28 Thread John Naylor
On Wed, Oct 28, 2020 at 2:15 PM John Naylor wrote: > Starting separate threads to keep from cluttering the TODO list thread. > > Here's a patch for the subject, as mentioned in > https://www.postgresql.org/message-id/20201027220555.GS4951%40momjian.us > I just realized I introduced a typo, so he

Re: duplicate function oid symbols

2020-10-28 Thread Tom Lane
Andres Freund writes: > I think changing type oid macro names is somewhat problematic - in > contrast to function oid macros the type macros are much more likely to > be used by client applications, e.g. for deciding whether to use binary > or text format for a type. > A quick code search shows a

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Tue, Oct 27, 2020 at 11:06:22AM -0300, Fabrízio de Royes Mello wrote: On Mon, Oct 26, 2020 at 7:46 PM David G. Johnston < david.g.johns...@gmail.com> wrote: It would seem preferable to call the lack of auto-analyzing after these operations a bug and back-patch a fix that injects an analyze

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Mon, Oct 26, 2020 at 03:46:10PM -0700, David G. Johnston wrote: On Mon, Oct 26, 2020 at 3:08 PM Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: Hi all, As you all already know Postgres supports functions in index expressions (marked as immutable ofc) and for this special index the

Re: [HACKERS] Custom compression methods

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 01:16:31PM +0530, Dilip Kumar wrote: ... I have added the next patch to support the compression options. I am storing the compression options only for the latest compression method. Basically, based on this design we would be able to support the options which are used

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread David G. Johnston
On Wed, Oct 28, 2020 at 11:55 AM Tomas Vondra wrote: > I agree the lack of stats may be quite annoying and cause issues, but my > guess is the chances of backpatching such change are about 0.01%. We > have a usable 'workaround' for this - manual analyze. > My guess is that it wouldn't be too

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tom Lane
Tomas Vondra writes: > On Mon, Oct 26, 2020 at 03:46:10PM -0700, David G. Johnston wrote: >> It would seem preferable to call the lack of auto-analyzing after these >> operations a bug and back-patch a fix that injects an analyze side-effect >> just before their completion. It doesn't have to be

Re: list of extended statistics on psql

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 03:07:56PM +0900, Tatsuro Yamada wrote: Hi Michael-san and Hackers, On 2020/09/30 15:19, Michael Paquier wrote: On Thu, Sep 17, 2020 at 02:55:31PM +0900, Michael Paquier wrote: Could you provide at least a rebased version of the patch? The CF bot is complaning here.

Re: list of extended statistics on psql

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 04:20:25PM +0900, Tatsuro Yamada wrote: Hi, Results of \dX and \dX+: postgres=# \dX     List of extended statistics     Schema    |   Name    |   Definition    | N_distinct | Dependencies |   Mcv -+---+

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread David G. Johnston
On Mon, Oct 26, 2020 at 9:44 PM Nikolay Samokhvalov wrote: > On Mon, Oct 26, 2020 at 7:03 PM David G. Johnston < > david.g.johns...@gmail.com> wrote: > >> On Monday, October 26, 2020, Nikolay Samokhvalov >> wrote: >>> >>> Although, this triggers a question – should ANALYZE be automated in, >>> s

Deduplicate aggregates and transition functions in planner

2020-10-28 Thread Heikki Linnakangas
Hi, Currently, ExecInitAgg() performs quite a lot of work, to deduplicate identical Aggrefs, as well as Aggrefs that can share the same transition state. That doesn't really belong in the executor, we should perform that work in the planner. It doesn't change from one invocation of the plan t

Re: Internal key management system

2020-10-28 Thread Bruce Momjian
On Wed, Oct 28, 2020 at 02:29:16PM -0400, Bruce Momjian wrote: > On Wed, Oct 28, 2020 at 12:02:46PM +0800, Craig Ringer wrote: > > Yes, that's possible. But in that case the passphrase will be asked for by > > openssl only when required, and we'll need to supply an openssl askpass > > hook. > > W

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 12:00:54PM -0700, David G. Johnston wrote: On Wed, Oct 28, 2020 at 11:55 AM Tomas Vondra wrote: I agree the lack of stats may be quite annoying and cause issues, but my guess is the chances of backpatching such change are about 0.01%. We have a usable 'workaround' f

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 03:05:39PM -0400, Tom Lane wrote: Tomas Vondra writes: On Mon, Oct 26, 2020 at 03:46:10PM -0700, David G. Johnston wrote: It would seem preferable to call the lack of auto-analyzing after these operations a bug and back-patch a fix that injects an analyze side-effect ju

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread David G. Johnston
On Wed, Oct 28, 2020 at 12:05 PM Tom Lane wrote: > This doesn't seem clearly different from any other situation where > auto-analyze doesn't react fast enough to suit you. > I would not > call it a bug, at least not without a wholesale redefinition of > how auto-analyze is supposed to work.

Re: duplicate function oid symbols

2020-10-28 Thread Andres Freund
Hi, On 2020-10-28 14:49:06 -0400, Tom Lane wrote: > The other extant oid_symbol entries are > > PGNODETREEOID > PGNDISTINCTOID > PGDEPENDENCIESOID > PGMCVLISTOID > PGDDLCOMMANDOID > EVTTRIGGEROID > The only one of these that client code would plausibly be using is LSNOID, > and even that is a bi

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tom Lane
Tomas Vondra writes: > On Wed, Oct 28, 2020 at 12:00:54PM -0700, David G. Johnston wrote: >> Given how simple the manual workaround is not having it be manual seems >> like it would be safe and straight-forward to implement. > Maybe, but I wouldn't be surprised if it was actually a bit trickier i

Re: duplicate function oid symbols

2020-10-28 Thread Tom Lane
Andres Freund writes: > On 2020-10-28 14:49:06 -0400, Tom Lane wrote: >> Moreover, this clearly shows the >> effect John mentioned that people have been copying the style of adjacent >> entries rather than making use of the standard oid_symbol convention like >> they should --- some of these don't

Re: libpq compression

2020-10-28 Thread Andres Freund
Hi, On 2020-10-26 19:20:46 +0300, Konstantin Knizhnik wrote: > diff --git a/configure b/configure > index ace4ed5..deba608 100755 > --- a/configure > +++ b/configure > @@ -700,6 +700,7 @@ LD > LDFLAGS_SL > LDFLAGS_EX > with_zlib > +with_zstd > with_system_tzdata > with_libxslt > XML2_LIBS

Re: duplicate function oid symbols

2020-10-28 Thread Andres Freund
On 2020-10-28 15:24:20 -0400, Tom Lane wrote: > Nah. What I'm imagining is just that pg_type.h contains > > #ifdef EXPOSE_TO_CLIENT_CODE > > /* > * Backwards compatibility for ancient random spellings of OID macros. > * Don't use these macros in new code. > */ > #define CASHOID MONEYOID > #d

Re: duplicate function oid symbols

2020-10-28 Thread John Naylor
On Wed, Oct 28, 2020 at 3:24 PM Tom Lane wrote: > and then the negotiation here is only about whether to make this list > longer. We don't need to complicate genbki.pl with a new facility. > Agreed, and reformat_dat_files.pl must also know about these special attributes. -- John Naylor Enterp

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-10-28 Thread Justin Pryzby
On Tue, Sep 08, 2020 at 02:51:26PM -0500, Justin Pryzby wrote: > On Sat, Jul 18, 2020 at 03:15:32PM -0500, Justin Pryzby wrote: > > Still waiting for feedback from a committer. > > This patch has been waiting for input from a committer on the approach I've > taken with the patches since March 10,

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 03:18:52PM -0400, Tom Lane wrote: Tomas Vondra writes: On Wed, Oct 28, 2020 at 12:00:54PM -0700, David G. Johnston wrote: Given how simple the manual workaround is not having it be manual seems like it would be safe and straight-forward to implement. Maybe, but I wou

Re: duplicate function oid symbols

2020-10-28 Thread Tom Lane
Andres Freund writes: > I assume you plan to error out if oid_symbol is defined for pg_type > going forward? Right, just like we just did for pg_proc. regards, tom lane

Re: MultiXact\SLRU buffers configuration

2020-10-28 Thread Alexander Korotkov
Hi, Tomas! Thank you for your review. On Wed, Oct 28, 2020 at 4:36 AM Tomas Vondra wrote: > I did a quick review on this patch series. A couple comments: > > > 0001 > > > This looks quite suspicious to me - SimpleLruReadPage_ReadOnly is > changed to return information about what lock was us

pg_prewarm bgworker could break fast shutdown

2020-10-28 Thread Alexander Kukushkin
Hello, I the fast shutdown was initiated before pg_prewarm managed to load buffers from the dump (and start the main loop), the pg_prewarm bgworker process never exits on SIGTERM and effectively preventing the clean shutdown of the cluster. This problem bite me a few times, but yesterday I manage

Re: Deduplicate aggregates and transition functions in planner

2020-10-28 Thread Andres Freund
Hi, On 2020-10-28 21:10:41 +0200, Heikki Linnakangas wrote: > Currently, ExecInitAgg() performs quite a lot of work, to deduplicate > identical Aggrefs, as well as Aggrefs that can share the same transition > state. That doesn't really belong in the executor, we should perform that > work in the p

Re: duplicate function oid symbols

2020-10-28 Thread John Naylor
On Wed, Oct 28, 2020 at 3:24 PM Tom Lane wrote: > > Nah. What I'm imagining is just that pg_type.h contains > > #ifdef EXPOSE_TO_CLIENT_CODE > > /* > * Backwards compatibility for ancient random spellings of OID macros. > * Don't use these macros in new code. > */ > #define CASHOID MONEYOID

Re: libpq compression

2020-10-28 Thread Alvaro Herrera
On 2020-Oct-26, Konstantin Knizhnik wrote: > + while (bufptr < bufend || zpq_buffered(PqStream) != 0) /* has more data > to flush or unsent data in internal compression buffer */ > { > - int r; > - > - r = secure_write(MyProcPort, bufptr, bufe

Re: Internal key management system

2020-10-28 Thread Bruce Momjian
On Wed, Oct 28, 2020 at 05:16:32PM +0800, Craig Ringer wrote: > On Wed, Oct 28, 2020 at 12:02 PM Craig Ringer > wrote: > > On Wed, Oct 28, 2020 at 9:43 AM Bruce Momjian wrote: > > > > I don't know much about how to hook into that stuff so if you have an > idea, I am all

Re: Disable WAL logging to speed up data loading

2020-10-28 Thread Laurenz Albe
On Wed, 2020-10-28 at 14:05 +0100, I wrote: > But what if someone sets wal_level=none, performs some data modifications, > sets wal_level=archive and after dome more processing decides to restore from > a backup that was taken before the cluster was set to wal_level=none? > Then they would end up w

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Fabrízio de Royes Mello
On Wed, Oct 28, 2020 at 4:35 PM Tomas Vondra wrote: > > I don't think anyone proposed to do this through autovacuum. There was a > reference to auto-analyze but I think that was meant as 'run analyze > automatically.' Which would work in transactions just fine, I think. > Maybe I was not very cle

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 05:43:08PM -0300, Fabrízio de Royes Mello wrote: On Wed, Oct 28, 2020 at 4:35 PM Tomas Vondra wrote: I don't think anyone proposed to do this through autovacuum. There was a reference to auto-analyze but I think that was meant as 'run analyze automatically.' Which would

  1   2   >