Re: Improve logging when using Huge Pages

2021-08-30 Thread Julien Rouhaud
On Tue, Aug 31, 2021 at 1:37 PM Shinoda, Noriyoshi (PN Japan FSIP) wrote: > > In the current version, when GUC huge_pages=try, which is the default > setting, no log is output regardless of the success or failure of the > HugePages acquisition. If you want to output logs, you need to set > log_

Re: .ready and .done files considered harmful

2021-08-30 Thread Bossart, Nathan
On 8/25/21, 4:11 AM, "Dipesh Pandit" wrote: > Please find attached patch v11. Apologies for the delay. I still intend to review this. Nathan

RE: suboverflowed subtransactions concurrency performance optimize

2021-08-30 Thread Pengchengliu
Hi Andrey, Thanks a lot for your replay and reference information. The default NUM_SUBTRANS_BUFFERS is 32. My implementation is local_cache_subtrans_pages can be adjusted dynamically. If we configure local_cache_subtrans_pages as 64, every backend use only extra 64*8192=512KB memory.

Re: Fix around conn_duration in pgbench

2021-08-30 Thread Tatsuo Ishii
>> >> > My 0.02€: From a benchmarking perspective, ISTM that it makes sense to >> >> > include disconnection times, which are clearly linked to connections, >> >> > especially with -C. So I'd rather have the more meaningful figure even >> >> > at the price of a small change in an undocumented featu

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread hubert depesz lubaczewski
On Mon, Aug 30, 2021 at 08:15:24PM -0400, Joe Conway wrote: > It would be interesting to step through a few times to see if it is really > stuck in that loop. That would be consistent with 100% CPU and not checking > for interrupts I think. If the problem will happen again, will do my best to get

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread hubert depesz lubaczewski
On Mon, Aug 30, 2021 at 09:09:20PM +0200, Laurenz Albe wrote: > On Mon, 2021-08-30 at 17:18 +0200, hubert depesz lubaczewski wrote: > > The thing is - I can't close it with pg_terminate_backend(), and I'd > > rather not kill -9, as it will, I think, close all other connections, > > and this is prod

Re: Can we get rid of repeated queries from pg_dump?

2021-08-30 Thread hubert depesz lubaczewski
On Mon, Aug 30, 2021 at 08:11:00PM -0400, Tom Lane wrote: > [ redirecting to -hackers ] > > I wrote: > > I experimented with the attached, very quick-n-dirty patch to collect > > format_type results during the initial scan of pg_type, instead. On the > > regression database in HEAD, it reduces th

Re: Fix around conn_duration in pgbench

2021-08-30 Thread Yugo NAGATA
On Tue, 31 Aug 2021 14:46:42 +0900 (JST) Tatsuo Ishii wrote: > >> > My 0.02€: From a benchmarking perspective, ISTM that it makes sense to > >> > include disconnection times, which are clearly linked to connections, > >> > especially with -C. So I'd rather have the more meaningful figure even > >

Re: Returning to Postgres community work

2021-08-30 Thread Gurjeet Singh
On Mon, Aug 30, 2021 at 10:53 PM Gurjeet Singh wrote: > PS: Hacker News announcement is at https://news.ycombinator.com/item?id= https://news.ycombinator.com/item?id=28364406 Best regards, -- Gurjeet Singh http://gurjeet.singh.im/

Returning to Postgres community work

2021-08-30 Thread Gurjeet Singh
Hi All, I'm very happy to announce that I now work for Supabase [1]. They have hired me so that I can participate in, and contribute to the Postgres community. I'm announcing it here in the hopes that more companies feel encouraged to contribute to Postgres. For those who don't know my p

Re: Separate out FileSet from SharedFileSet (was Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o)

2021-08-30 Thread Dilip Kumar
On Tue, Aug 31, 2021 at 7:39 AM houzj.f...@fujitsu.com < houzj.f...@fujitsu.com> wrote: > From Mon, Aug 30, 2021 2:15 PM Masahiko Sawada > wrote: > > On Fri, Aug 27, 2021 at 3:34 PM Dilip Kumar > wrote: > > > > > > On Fri, Aug 27, 2021 at 10:56 AM houzj.f...@fujitsu.com < > houzj.f...@fujitsu.co

Re: Failure of subscription tests with topminnow

2021-08-30 Thread Masahiko Sawada
On Tue, Aug 31, 2021 at 12:11 PM Amit Kapila wrote: > > On Mon, Aug 30, 2021 at 5:48 PM Ajin Cherian wrote: > > > > On Mon, Aug 30, 2021 at 7:52 PM Amit Kapila wrote: > > > > I have made the above changes on HEAD. > > > > Thanks, this looks mostly good to me. I'll push and backpatch this > tomor

Re: Fix around conn_duration in pgbench

2021-08-30 Thread Tatsuo Ishii
>> > My 0.02€: From a benchmarking perspective, ISTM that it makes sense to >> > include disconnection times, which are clearly linked to connections, >> > especially with -C. So I'd rather have the more meaningful figure even >> > at the price of a small change in an undocumented feature. >> >> +

Improve logging when using Huge Pages

2021-08-30 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Hi Hackers, In the current version, when GUC huge_pages=try, which is the default setting, no log is output regardless of the success or failure of the HugePages acquisition. If you want to output logs, you need to set log_min_messages=DEBUG3, but it will output a huge amount of extra logs. Wit

Re: Replication slot drop message is sent after pgstats shutdown.

2021-08-30 Thread Andres Freund
On 2021-08-31 14:22:39 +0900, Masahiko Sawada wrote: > You mean to move only the part of sending the message to its own > before_shmem_exit() callback? or move ReplicationSlotRelease() and > ReplicationSlotCleanup() from ProcKill() to it? The latter.

Re: Fix around conn_duration in pgbench

2021-08-30 Thread Yugo NAGATA
Hello Fabien, Ishii-san, On Tue, 31 Aug 2021 14:18:48 +0900 (JST) Tatsuo Ishii wrote: > >>> Ok. That makes sense. The output reports "including connections > >>> establishing" and "excluding connections establishing" regardless with > >>> -C, so we should measure delays in the same way. > >> > >

Re: Replication slot drop message is sent after pgstats shutdown.

2021-08-30 Thread Masahiko Sawada
On Tue, Aug 31, 2021 at 12:45 PM Andres Freund wrote: > > Hi, > > On 2021-08-31 11:37:08 +0900, Masahiko Sawada wrote: > > At step #2, wal sender waits for another transaction started at step > > #1 to complete after creating the replication slot. When the server is > > stopping, wal sender proces

Re: Added schema level support for publication.

2021-08-30 Thread Greg Nancarrow
On Tue, Aug 31, 2021 at 1:41 PM vignesh C wrote: > > > > > I notice that "CREATE PUBLICATION pub1 FOR ALL TABLES IN SCHEMA sc1, > > TABLE sc1.test;" maintains the table separately and results in the > > following in the \dRp+ output: > > > > Tables: > > "sc1.test" > > Schemas: > > "sc1" >

Re: Fix around conn_duration in pgbench

2021-08-30 Thread Tatsuo Ishii
>>> Ok. That makes sense. The output reports "including connections >>> establishing" and "excluding connections establishing" regardless with >>> -C, so we should measure delays in the same way. >> >> On second thought, it's more reasonable and less confusing not to >> measure the disconnection de

Re: Fix around conn_duration in pgbench

2021-08-30 Thread Yugo NAGATA
Hello Fujii-san, On Mon, 30 Aug 2021 23:36:30 +0900 Fujii Masao wrote: > > > On 2021/08/26 12:13, Yugo NAGATA wrote: > > Ok. That makes sense. The output reports "including connections > > establishing" > > and "excluding connections establishing" regardless with -C, so we should > > measure

Re: Fix around conn_duration in pgbench

2021-08-30 Thread Fabien COELHO
Ok. That makes sense. The output reports "including connections establishing" and "excluding connections establishing" regardless with -C, so we should measure delays in the same way. On second thought, it's more reasonable and less confusing not to measure the disconnection delays at all? Si

Re: prevent immature WAL streaming

2021-08-30 Thread Andres Freund
Hi, On 2021-08-23 18:52:17 -0400, Alvaro Herrera wrote: > Included 蔡梦娟 and Jakub Wartak because they've expressed interest on > this topic -- notably [2] ("Bug on update timing of walrcv->flushedUpto > variable"). > > As mentioned in the course of thread [1], we're missing a fix for > streaming re

Re: perlcritic: prohibit map and grep in void conext

2021-08-30 Thread Julien Rouhaud
On Tue, Aug 31, 2021 at 9:23 AM Michael Paquier wrote: > > On Mon, Aug 30, 2021 at 02:27:09PM -0400, Andrew Dunstan wrote: > > There doesn't seem to have been much pushback, so let's try it and see. > > Okay, fine by me. +1

Re: unpack_sql_state not called?

2021-08-30 Thread Michael Paquier
On Tue, Aug 31, 2021 at 02:01:02PM +1000, Peter Smith wrote: > Do you want me to re-post it as a patch attachment? No need. Thanks. -- Michael signature.asc Description: PGP signature

Re: Multi-Column List Partitioning

2021-08-30 Thread Amit Langote
On Mon, Aug 30, 2021 at 4:51 PM Rajkumar Raghuwanshi wrote: > > Hi Nitin. > > I have been testing these patches. Patches applied cleanly on the head. While > testing I found below a case where update row movement is not working > properly. > Please find the test case below. > > postgres=# create

Re: unpack_sql_state not called?

2021-08-30 Thread Peter Smith
On Tue, Aug 31, 2021 at 11:31 AM Michael Paquier wrote: > > On Tue, Aug 31, 2021 at 09:32:41AM +1000, Peter Smith wrote: > > Hi. I noticed some code that seems the same as the nearby function > > unpack_sql_state, and I wondered why it is not just calling it? > > This looks like a piece that could

Re: Added schema level support for publication.

2021-08-30 Thread vignesh C
On Mon, Aug 30, 2021 at 12:12 PM Amit Kapila wrote: > > > Okay, I got it but let's add few comments in the code related to it. > Also, I noticed that the code in InvalidatePublicationRels() already > exists in AlterPublicationOptions(). You can try to refactor the > existing code as a separate ini

Re: Replication slot drop message is sent after pgstats shutdown.

2021-08-30 Thread Andres Freund
Hi, On 2021-08-31 11:37:08 +0900, Masahiko Sawada wrote: > At step #2, wal sender waits for another transaction started at step > #1 to complete after creating the replication slot. When the server is > stopping, wal sender process drops the slot on releasing the slot > since it's still RS_EPHEMER

Re: Added schema level support for publication.

2021-08-30 Thread vignesh C
On Mon, Aug 30, 2021 at 2:14 PM Greg Nancarrow wrote: > > On Fri, Aug 27, 2021 at 4:13 PM vignesh C wrote: > > > > I have implemented this in the 0003 patch, I have kept it separate to > > reduce the testing effort and also it will be easier if someone > > disagrees with the syntax. I will merge

Re: AIX: Symbols are missing in libpq.a

2021-08-30 Thread Noah Misch
On Mon, Aug 30, 2021 at 03:35:23PM +, REIX, Tony wrote: > Yes, trying to use the create lib$(NAME).exp from $(SHLIB_EXPORTS) when it > exists was my first idea, too. > However, I do not master (or I forgot) this kind of "if" in a Makefile > and I was unable to find a solution by reading M

Re: [PATCH] Tab completion for ALTER TABLE … ADD …

2021-08-30 Thread Michael Paquier
On Mon, Aug 30, 2021 at 02:38:19PM +0100, Dagfinn Ilmari Mannsåker wrote: > That was because I forgot to exclude all the other object types that can > come after ADD. Attached is a patch that does that. I also moved it > right next to the ALTER TABLE … ADD completion, and added a comment to > kee

Re: Failure of subscription tests with topminnow

2021-08-30 Thread Amit Kapila
On Mon, Aug 30, 2021 at 5:48 PM Ajin Cherian wrote: > > On Mon, Aug 30, 2021 at 7:52 PM Amit Kapila wrote: > > I have made the above changes on HEAD. > Thanks, this looks mostly good to me. I'll push and backpatch this tomorrow unless you or someone else thinks otherwise. Minor comments ===

Re: Added missing invalidations for all tables publication

2021-08-30 Thread vignesh C
On Tue, Aug 31, 2021 at 7:40 AM houzj.f...@fujitsu.com wrote: > > From Tuesday, August 31, 2021 1:10 AM vignesh C wrote: > > Hi, > > > > Relation invalidation was missing in case of create publication and drop > > publication of "FOR ALL TABLES" publication, added so that the publication > > info

Re: pgstat_send_connstats() introduces unnecessary timestamp and UDP overhead

2021-08-30 Thread Laurenz Albe
On Fri, 2021-08-27 at 13:57 +0900, Michael Paquier wrote: > > I think in that case we'd have to do the bigger redesign and move "live" > > connection stats to backend_status.c... > > Hmm.  A redesign is not really an option for 14 at this stage.  And I > am not really comfortable with the latest p

Re: archive status ".ready" files may be created too early

2021-08-30 Thread Andres Freund
Hi, On 2021-08-30 22:39:04 +, Bossart, Nathan wrote: > On 8/30/21, 2:06 PM, "Andres Freund" wrote: > > When were you thinking of doing the latch sets? Adding a latch set for every > > XLogWrite() wouldn't be cheap either. Both because syscalls under a lock > > aren't free and because waking u

Replication slot drop message is sent after pgstats shutdown.

2021-08-30 Thread Masahiko Sawada
Hi all, I found another pass where we report stats after the stats collector shutdown. The reproducer and the backtrace I got are here: 1. psql -c "begin; create table a (a int); select pg_sleep(30); commit;" & 2. pg_recvlogical --create-slot -S slot -d postgres & 3. stop the server TRAP: Failed

RE: Added missing invalidations for all tables publication

2021-08-30 Thread houzj.f...@fujitsu.com
From Tuesday, August 31, 2021 1:10 AM vignesh C wrote: > Hi, > > Relation invalidation was missing in case of create publication and drop > publication of "FOR ALL TABLES" publication, added so that the publication > information can be rebuilt. Without these invalidation update/delete > operation

RE: Separate out FileSet from SharedFileSet (was Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o)

2021-08-30 Thread houzj.f...@fujitsu.com
From Mon, Aug 30, 2021 2:15 PM Masahiko Sawada wrote: > On Fri, Aug 27, 2021 at 3:34 PM Dilip Kumar wrote: > > > > On Fri, Aug 27, 2021 at 10:56 AM houzj.f...@fujitsu.com > > wrote: > > > > > > On Thu, Aug 26, 2021 2:18 PM Dilip Kumar wrote: > > > > > > > The patch looks good to me, I have reb

Re: unpack_sql_state not called?

2021-08-30 Thread Michael Paquier
On Tue, Aug 31, 2021 at 09:32:41AM +1000, Peter Smith wrote: > Hi. I noticed some code that seems the same as the nearby function > unpack_sql_state, and I wondered why it is not just calling it? This looks like a piece that could have been done in d46bc44, and would not matter performance-wise.

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread Michael Paquier
On Mon, Aug 30, 2021 at 09:16:51PM -0400, Joe Conway wrote: > On 8/30/21 8:22 PM, Tom Lane wrote: >> Yeah, this single data point is not enough justification to blame >> dynahash.c (which is *extremely* battle-tested code, you'll recall). >> I'm inclined to guess that the looping is happening a few

Re: perlcritic: prohibit map and grep in void conext

2021-08-30 Thread Michael Paquier
On Mon, Aug 30, 2021 at 02:27:09PM -0400, Andrew Dunstan wrote: > There doesn't seem to have been much pushback, so let's try it and see. Okay, fine by me. -- Michael signature.asc Description: PGP signature

Re: replace IDENTIFY_SYSTEM code in receivelog.c with RunIdentifySystem()

2021-08-30 Thread Michael Paquier
On Mon, Aug 30, 2021 at 01:01:16PM +0530, Bharath Rupireddy wrote: > Shound't we fix it in master branch to keep the code in sync with > other places where we usually follow that kind of type-casting? IMO, > we should just make that change, because it isn't a major change or we > aren't going to ba

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread Joe Conway
On 8/30/21 8:22 PM, Tom Lane wrote: Joe Conway writes: It would be interesting to step through a few times to see if it is really stuck in that loop. Yeah, this single data point is not enough justification to blame dynahash.c (which is *extremely* battle-tested code, you'll recall). I'm incl

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread Tom Lane
Joe Conway writes: > It would be interesting to step through a few times to see if it is > really stuck in that loop. Yeah, this single data point is not enough justification to blame dynahash.c (which is *extremely* battle-tested code, you'll recall). I'm inclined to guess that the looping is h

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread Joe Conway
On 8/30/21 3:34 PM, Justin Pryzby wrote: On Mon, Aug 30, 2021 at 09:09:20PM +0200, Laurenz Albe wrote: On Mon, 2021-08-30 at 17:18 +0200, hubert depesz lubaczewski wrote: > The thing is - I can't close it with pg_terminate_backend(), and I'd > rather not kill -9, as it will, I think, close all o

Re: Can we get rid of repeated queries from pg_dump?

2021-08-30 Thread Tom Lane
[ redirecting to -hackers ] I wrote: > I experimented with the attached, very quick-n-dirty patch to collect > format_type results during the initial scan of pg_type, instead. On the > regression database in HEAD, it reduces the number of queries pg_dump > issues from 3260 to 2905; but I'm having

unpack_sql_state not called?

2021-08-30 Thread Peter Smith
Hi. I noticed some code that seems the same as the nearby function unpack_sql_state, and I wondered why it is not just calling it? For example, diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c index a3e1c59..d91ed98 100644 --- a/src/backend/utils/error/elog.c +++ b/sr

Re: Avoid choose invalid number of partitions (src/backend/executor/nodeAgg.c)

2021-08-30 Thread Ranier Vilela
Em seg., 30 de ago. de 2021 às 07:44, David Rowley escreveu: > On Wed, 30 Jun 2021 at 02:33, Ranier Vilela wrote: > > hash_choose_num_partitions function has issues. > > There are at least two path calls made with used_bits = 0. > > See at hashagg_spill_init. > > > On Windows 64 bits (HEAD) fail

Re: archive status ".ready" files may be created too early

2021-08-30 Thread Bossart, Nathan
On 8/30/21, 3:40 PM, "Bossart, Nathan" wrote: > On 8/30/21, 2:06 PM, "Andres Freund" wrote: >> Although, the more I think about, the more I am confused about the trailing >> if (XLogArchivingActive()) >> NotifySegmentsReadyForArchive(LogwrtResult.Flush); >> >> in XLogWrite

Re: archive status ".ready" files may be created too early

2021-08-30 Thread Bossart, Nathan
On 8/30/21, 2:06 PM, "Andres Freund" wrote: > When were you thinking of doing the latch sets? Adding a latch set for every > XLogWrite() wouldn't be cheap either. Both because syscalls under a lock > aren't free and because waking up walsender even more often isn't free (we > already have a few th

Re: Use extended statistics to estimate (Var op Var) clauses

2021-08-30 Thread Zhihong Yu
On Mon, Aug 30, 2021 at 9:00 AM Tomas Vondra wrote: > On 8/28/21 6:30 PM, Mark Dilger wrote: > > > > > >> On Aug 28, 2021, at 6:52 AM, Tomas Vondra > >> wrote: > >> > >> Part 0003 fixes handling of those clauses so that we don't treat > >> them as simple, but it does that by tweaking > >> statex

Re: dynamic result sets support in extended query protocol

2021-08-30 Thread Zhihong Yu
On Mon, Aug 30, 2021 at 1:23 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > rebased patch set > > On 22.07.21 08:06, vignesh C wrote: > > On Tue, Jun 29, 2021 at 7:10 PM Peter Eisentraut > > wrote: > >> > >> Here is an updated patch with some merge conflicts resolved, to keep i

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread Justin Pryzby
On Mon, Aug 30, 2021 at 09:09:20PM +0200, Laurenz Albe wrote: > On Mon, 2021-08-30 at 17:18 +0200, hubert depesz lubaczewski wrote: > > The thing is - I can't close it with pg_terminate_backend(), and I'd > > rather not kill -9, as it will, I think, close all other connections, > > and this is prod

Re: archive status ".ready" files may be created too early

2021-08-30 Thread Andres Freund
Hi, On 2021-08-30 15:51:54 -0400, alvhe...@alvh.no-ip.org wrote: > On 2021-Aug-28, Andres Freund wrote: > > > While rebasing the aio patchset ontop of HEAD I noticed that this commit > > added > > another atomic operation to XLogWrite() with archiving enabled. The WAL > > write > > path is reall

Re: [PATCH] Error out if SKIP LOCKED and WITH TIES are both specified

2021-08-30 Thread David Christensen
On Mon, Aug 30, 2021 at 3:51 PM Alvaro Herrera wrote: > > On 2021-Aug-13, David Christensen wrote: > > > Both bugs #16676[1] and #17141[2] illustrate that the combination of > > SKIP LOCKED and FETCH FIRST WITH TIES break expectations when it comes > > to rows returned to other sessions accessing

Re: [PATCH] Error out if SKIP LOCKED and WITH TIES are both specified

2021-08-30 Thread Alvaro Herrera
On 2021-Aug-13, David Christensen wrote: > Both bugs #16676[1] and #17141[2] illustrate that the combination of > SKIP LOCKED and FETCH FIRST WITH TIES break expectations when it comes > to rows returned to other sessions accessing the same row. Since this > situation is detectable from the synta

Re: reporting TID/table with corruption error

2021-08-30 Thread Alvaro Herrera
Pushed to all branches, thanks. -- Álvaro Herrera Valdivia, Chile — https://www.EnterpriseDB.com/

Re: dynamic result sets support in extended query protocol

2021-08-30 Thread Peter Eisentraut
rebased patch set On 22.07.21 08:06, vignesh C wrote: On Tue, Jun 29, 2021 at 7:10 PM Peter Eisentraut wrote: Here is an updated patch with some merge conflicts resolved, to keep it fresh. It's still pending in the commit fest from last time. My focus right now is to work on the "psql - add

Re: archive status ".ready" files may be created too early

2021-08-30 Thread Bossart, Nathan
On 8/30/21, 12:52 PM, "alvhe...@alvh.no-ip.org" wrote: > On 2021-Aug-28, Andres Freund wrote: > >> While rebasing the aio patchset ontop of HEAD I noticed that this commit >> added >> another atomic operation to XLogWrite() with archiving enabled. The WAL write >> path is really quite hot, and at

Re: archive status ".ready" files may be created too early

2021-08-30 Thread alvhe...@alvh.no-ip.org
On 2021-Aug-28, Andres Freund wrote: > While rebasing the aio patchset ontop of HEAD I noticed that this commit added > another atomic operation to XLogWrite() with archiving enabled. The WAL write > path is really quite hot, and at least some of the > NotifySegmentsReadyForArchive() calls are don

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread Laurenz Albe
On Mon, 2021-08-30 at 17:18 +0200, hubert depesz lubaczewski wrote: > The thing is - I can't close it with pg_terminate_backend(), and I'd > rather not kill -9, as it will, I think, close all other connections, > and this is prod server. Of course the cause should be fixed, but to serve your immed

Re: verify_heapam for sequences?

2021-08-30 Thread Mark Dilger
> On Aug 30, 2021, at 1:22 AM, Peter Eisentraut > wrote: > > On 26.08.21 21:02, Mark Dilger wrote: >> I understand that sequences are really just heap tables, and since we >> already test corrupted heap tables, we could assume that we already have >> sufficient coverage. I'm not entirely co

Re: perlcritic: prohibit map and grep in void conext

2021-08-30 Thread Andrew Dunstan
On 8/27/21 6:32 AM, Dagfinn Ilmari Mannsåker wrote: > Michael Paquier writes: > >> On Wed, Jul 28, 2021 at 01:26:23PM +0200, Daniel Gustafsson wrote: >>> I'm fine with increasing this policy, but I don't have strong feelings. If >>> we >>> feel the perlcritic policy change is too much, I would

Re: extended stats objects are the only thing written like "%s"."%s"

2021-08-30 Thread Alvaro Herrera
On 2021-Aug-28, Tom Lane wrote: > I think what we are doing there is following the message style > guideline that says to put double quotes around inserted strings. > In this case schema.object (as a whole) is the inserted string. > People often confuse this with SQL double-quoted identifiers, but

Added missing invalidations for all tables publication

2021-08-30 Thread vignesh C
Hi, Relation invalidation was missing in case of create publication and drop publication of "FOR ALL TABLES" publication, added so that the publication information can be rebuilt. Without these invalidation update/delete operations on the relation will be successful in the publisher which will lat

Re: Patch: shouldn't timezone(text, timestamp[tz]) be STABLE?

2021-08-30 Thread Tom Lane
Aleksander Alekseev writes: > Got it. But in this case, what's your opinion on the differences between > date_trunc() and timezone()? Shouldn't date_trunc() be always IMMUTABLE as > well? No, because date_trunc depends on the current timezone setting, or at least its stable variants do.

Re: Possible missing segments in archiving on standby

2021-08-30 Thread Fujii Masao
On 2020/06/30 16:55, Kyotaro Horiguchi wrote: Hello. While looking a patch, I found that a standby with archive_mode=always fails to archive segments under certain conditions. I encountered this issue, too. 1. v1-0001-Make-sure-standby-archives-all-segments.patch: Fix for A and B. 2.

Re: Patch: shouldn't timezone(text, timestamp[tz]) be STABLE?

2021-08-30 Thread Aleksander Alekseev
Hi Tom, Thanks for the quick reply. > I don't see a lot of argument for treating tzdata changes differently > from OS locale changes. Got it. But in this case, what's your opinion on the differences between date_trunc() and timezone()? Shouldn't date_trunc() be always IMMUTABLE as well? I can s

Re: Use extended statistics to estimate (Var op Var) clauses

2021-08-30 Thread Tomas Vondra
On 8/28/21 6:30 PM, Mark Dilger wrote: On Aug 28, 2021, at 6:52 AM, Tomas Vondra wrote: Part 0003 fixes handling of those clauses so that we don't treat them as simple, but it does that by tweaking statext_is_compatible_clause(), as suggested by Dean. Function examine_opclause_args() doesn

Re: pg_dump, ATTACH, and independently restorable child partitions

2021-08-30 Thread Justin Pryzby
On Wed, Nov 25, 2020 at 06:35:19PM -0500, Tom Lane wrote: > What we'd need to do if we want this to work with direct-to-DB restore > > > is to split off the ATTACH P

RE: AIX: Symbols are missing in libpq.a

2021-08-30 Thread REIX, Tony
Hi Noah, Yes, trying to use the create lib$(NAME).exp from $(SHLIB_EXPORTS) when it exists was my first idea, too. However, I do not master (or I forgot) this kind of "if" in a Makefile and I was unable to find a solution by reading Makefile manuals or by searching for a similar example. So

Re: Added schema level support for publication.

2021-08-30 Thread vignesh C
On Mon, Aug 30, 2021 at 1:31 PM tanghy.f...@fujitsu.com wrote: > > On Friday, August 27, 2021 2:13 PM vignesh C wrote: > > > > I have implemented this in the 0003 patch, I have kept it separate to > > reduce the testing effort and also it will be easier if someone > > disagrees with the syntax. I

Re: Added schema level support for publication.

2021-08-30 Thread vignesh C
On Mon, Aug 30, 2021 at 9:10 AM houzj.f...@fujitsu.com wrote: > > On Friday, August 27, 2021 2:13 PM vignesh C wrote: > > > > I have implemented this in the 0003 patch, I have kept it separate to > > reduce the > > testing effort and also it will be easier if someone disagrees with the > > synt

Re: Added schema level support for publication.

2021-08-30 Thread vignesh C
On Mon, Aug 30, 2021 at 8:23 AM Greg Nancarrow wrote: > > On Fri, Aug 27, 2021 at 4:13 PM vignesh C wrote: > > > > I have implemented this in the 0003 patch, I have kept it separate to > > reduce the testing effort and also it will be easier if someone > > disagrees with the syntax. I will merge

Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread hubert depesz lubaczewski
Hi, Originally I posted it on -general, but Joe Conway suggested I repost in here for greater visibility... We hit a problem with Pg 12.6 (I know, we should upgrade, but that will take long time to prepare). Anyway - it's 12.6 on aarm64. Couple of days there was replication slot started, and now

Re: Patch: shouldn't timezone(text, timestamp[tz]) be STABLE?

2021-08-30 Thread Tom Lane
I wrote: > Aleksander Alekseev writes: >> [ why is timetz_zone volatile? ] Ah ... after a bit of digging in the git history, I found this [1]: Author: Tom Lane Branch: master Release: REL8_1_BR [35979e6c3] 2005-09-09 06:51:12 + Given its current definition that depends on time(NULL), t

Re: AIX: Symbols are missing in libpq.a

2021-08-30 Thread Noah Misch
On Mon, Aug 30, 2021 at 02:33:32PM +, REIX, Tony wrote: > It appears that the Makefile in src/interfaces/libpq has been modified > between v12 and v13, removing encnames.o (and wchar.o) from the object files > being used for creating the static libpq.a file which is used for creating > the l

Re: Patch: shouldn't timezone(text, timestamp[tz]) be STABLE?

2021-08-30 Thread Tom Lane
Aleksander Alekseev writes: > There are several overloaded versions of timezone() function. One > version accepts timezone name and timestamptz and returns timestamp: > This function is marked as IMMUTABLE and it's possible to use it in > functional indexes. I believe it's a bug. That's a deliber

Re: Fix around conn_duration in pgbench

2021-08-30 Thread Fujii Masao
On 2021/08/26 12:13, Yugo NAGATA wrote: Ok. That makes sense. The output reports "including connections establishing" and "excluding connections establishing" regardless with -C, so we should measure delays in the same way. On second thought, it's more reasonable and less confusing not to me

RE: AIX: Symbols are missing in libpq.a

2021-08-30 Thread REIX, Tony
Hi Noah, It appears that the Makefile in src/interfaces/libpq has been modified between v12 and v13, removing encnames.o (and wchar.o) from the object files being used for creating the static libpq.a file which is used for creating the libpq.so and libpq.a AIX shared-library files. And, since p

Re: Async-unsafe functions in signal handlers

2021-08-30 Thread Denis Smirnov
Honestly, I don’t know what to do with bgworker_die(). At the moment it produces ereport(FATAL) with async-unsafe proc_exit_prepare() and exit() underhood. I can see three solutions: 1. Leave the code as is. Then SIGTERM can produce deadlocks in bgworker's signal handler. The locked process can

Re: pgsql: Avoid using ambiguous word "positive" in error message.

2021-08-30 Thread Robert Haas
On Mon, Aug 30, 2021 at 10:16 AM Tom Lane wrote: > Robert Haas writes: > > The new style seems good, but I don't really agree that "positive" and > > "non-negative" are ambiguous. "positive" means >0 and "non-negative" > > means >= 0, because 0 is neither positive nor negative. > > Well, the poin

Patch: shouldn't timezone(text, timestamp[tz]) be STABLE?

2021-08-30 Thread Aleksander Alekseev
Hi hackers, There are several overloaded versions of timezone() function. One version accepts timezone name and timestamptz and returns timestamp: =# show time zone; TimeZone --- Europe/Moscow =# select timezone('MSK', '2021-08-30 12:34:56 MSK' :: timestamptz); timezone ---

Re: pgsql: Avoid using ambiguous word "positive" in error message.

2021-08-30 Thread Tom Lane
Robert Haas writes: > The new style seems good, but I don't really agree that "positive" and > "non-negative" are ambiguous. "positive" means >0 and "non-negative" > means >= 0, because 0 is neither positive nor negative. Well, the point is precisely that not everyone makes that distinction. I ag

Re: pgsql: Avoid using ambiguous word "positive" in error message.

2021-08-30 Thread Robert Haas
On Tue, Aug 24, 2021 at 10:49 PM Fujii Masao wrote: > Avoid using ambiguous word "positive" in error message. The new style seems good, but I don't really agree that "positive" and "non-negative" are ambiguous. "positive" means >0 and "non-negative" means >= 0, because 0 is neither positive nor n

Re: [PATCH] Tab completion for ALTER TABLE … ADD …

2021-08-30 Thread Dagfinn Ilmari Mannsåker
Michael Paquier writes: > On Fri, Aug 27, 2021 at 11:52:33AM +0100, Dagfinn Ilmari Mannsåker wrote: >> That was easy enough to add (just a bit of extra fiddling to handle >> COLUMN being optional), done in the attached v2 patch. > > This part was a bit misleading, as it would recommend a list of

Re: Failure of subscription tests with topminnow

2021-08-30 Thread Ajin Cherian
On Mon, Aug 30, 2021 at 7:52 PM Amit Kapila wrote: > Isn't it better to check the streaming state when we are fetching > oldpid? If we don't add, then I suspect that the next time someone > adding tests on similar lines might get confused about where to check > the state and where not. Also, if y

Re: [PATCH] Allow multiple recursive self-references

2021-08-30 Thread Denis Hirn
> This well-formedness check apparently needs to be enhanced to allow for more > than two branches in the union. The UNION operators' left associativity causes this problem. Currently, the recursive term must be enclosed in parentheses to make this example work: > WITH RECURSIVE nodes(x) AS (

Re: Avoid choose invalid number of partitions (src/backend/executor/nodeAgg.c)

2021-08-30 Thread David Rowley
On Wed, 30 Jun 2021 at 02:33, Ranier Vilela wrote: > hash_choose_num_partitions function has issues. > There are at least two path calls made with used_bits = 0. > See at hashagg_spill_init. > On Windows 64 bits (HEAD) fails with partition_prune: > parallel group (11 tests): reloptions hash_part

Re: Trap errors from streaming child in pg_basebackup to exit early

2021-08-30 Thread Bharath Rupireddy
On Thu, Aug 26, 2021 at 2:55 PM Daniel Gustafsson wrote: > > When using pg_basebackup with WAL streaming (-X stream), we have observed on a > number of times in production that the streaming child exited prematurely (to > no fault of the code it seems, most likely due to network middleboxes), whic

Re: suboverflowed subtransactions concurrency performance optimize

2021-08-30 Thread Andrey Borodin
Hi Pengcheng! You are solving important problem, thank you! > 30 авг. 2021 г., в 13:43, Pengchengliu написал(а): > > To resolve this performance problem, we think about a solution which cache > SubtransSLRU to local cache. > First we can query parent transaction id from SubtransSLRU, and copy

Re: Error code for checksum failure in origin.c

2021-08-30 Thread Kasahara Tatsuhito
On Mon, Aug 30, 2021 at 5:35 PM Amit Kapila wrote: > > On Fri, Aug 27, 2021 at 12:47 PM Daniel Gustafsson wrote: > > > > > On 27 Aug 2021, at 06:32, Amit Kapila wrote: > > > > > I think we need to backpatch this till 9.6 as this is introduced by > > > commit 5aa2350426. Any objections to that? >

Re: pg_receivewal starting position

2021-08-30 Thread Ronan Dunklau
Le samedi 28 août 2021, 14:10:25 CEST Bharath Rupireddy a écrit : > On Thu, Aug 26, 2021 at 5:45 PM Ronan Dunklau wrote: > > order to fail early if the replication slot doesn't exist, so please find > > attached v2 for that. > > Thanks for the patches. Here are some comments: > Thank you for t

Re: pg_receivewal starting position

2021-08-30 Thread Ronan Dunklau
Le vendredi 27 août 2021, 05:44:32 CEST Michael Paquier a écrit : > On Thu, Aug 26, 2021 at 02:14:27PM +0200, Ronan Dunklau wrote: > > Following the discussion at [1], I refactored the implementation into > > streamutil and added a third patch making use of it in pg_basebackup > > itself in order t

Re: Failure of subscription tests with topminnow

2021-08-30 Thread Amit Kapila
On Fri, Aug 27, 2021 at 4:33 PM Ajin Cherian wrote: > > On Fri, Aug 27, 2021 at 3:29 PM Amit Kapila wrote: > > > > > > I think the fix is correct but similar changes are required in > > 022_twophase_cascade.pl as well (search for $oldpid in tests). I am > > not completely sure but I think it is b

Re: Defer selection of asynchronous subplans until the executor initialization stage

2021-08-30 Thread Etsuro Fujita
On Mon, Aug 30, 2021 at 5:36 PM Andrey V. Lepikhov wrote: > On 8/23/21 2:18 PM, Etsuro Fujita wrote: > > To just execute what was planned at execution time, I think we should > > return to the patch in [1]. The patch was created for Horiguchi-san’s > > async-execution patch, so I modified it to w

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-08-30 Thread Andy Fan
On Tue, Jul 6, 2021 at 9:14 PM Tom Lane wrote: > > David Rowley writes: > > Tom, I'm wondering if you might get a chance to draw up a design for > > what you've got in mind with this? I assume adding a new field in > > Var, but I'm drawing a few blanks on how things might work for equal() > > wh

Re: Added schema level support for publication.

2021-08-30 Thread Greg Nancarrow
On Fri, Aug 27, 2021 at 4:13 PM vignesh C wrote: > > I have implemented this in the 0003 patch, I have kept it separate to > reduce the testing effort and also it will be easier if someone > disagrees with the syntax. I will merge it to the main patch later > based on the feedback. Attached v22 pa

suboverflowed subtransactions concurrency performance optimize

2021-08-30 Thread Pengchengliu
Hi hackers, I wrote a patch to resolve the subtransactions concurrency performance problems when suboverflowed. When we use more than PGPROC_MAX_CACHED_SUBXIDS(64) subtransactions per transaction concurrency, it will lead to subtransactions performance problems. All backend will be stuck at acqui

Re: Defer selection of asynchronous subplans until the executor initialization stage

2021-08-30 Thread Andrey V. Lepikhov
On 8/23/21 2:18 PM, Etsuro Fujita wrote: To just execute what was planned at execution time, I think we should return to the patch in [1]. The patch was created for Horiguchi-san’s async-execution patch, so I modified it to work with HEAD, and added a simplified version of your test cases. Plea

  1   2   >