RE: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-05-06 Thread tsunakawa.ta...@fujitsu.com
From: Robert Haas > On Wed, May 5, 2021 at 10:54 PM tsunakawa.ta...@fujitsu.com > wrote: > > As proposed in this thread and/or "Parallel INSERT SELECT take 2", we > thought of detecting parallel unsafe function execution during SQL statement > execution, instead of imposing much overhead to check

Re: Identify missing publications from publisher while create/alter subscription.

2021-05-06 Thread Dilip Kumar
On Thu, May 6, 2021 at 7:22 PM vignesh C wrote: > Some comments: 1. I don't see any change in pg_dump.c, don't we need to dump this option? 2. + /* Try to connect to the publisher. */ + wrconn = walrcv_connect(sub->conninfo, true, sub->name, &err); + if (!wrconn) + ereport(ERROR, + (errmsg("coul

Re: Identify missing publications from publisher while create/alter subscription.

2021-05-06 Thread Japin Li
On Thu, 06 May 2021 at 21:52, vignesh C wrote: > On Thu, May 6, 2021 at 9:08 AM Japin Li wrote: >> 3) Should we free the memory when finish the check_publications()? >> + publicationsCopy = list_copy(publications); > > I felt this list entries will be deleted in the success case, in error

Re: Race condition in recovery?

2021-05-06 Thread Dilip Kumar
On Fri, May 7, 2021 at 8:23 AM Kyotaro Horiguchi wrote: > > At Tue, 4 May 2021 17:41:06 +0530, Dilip Kumar wrote > in > Could you please fix the test script so that it causes your issue > correctly? And/or elaborate a bit more? > > The attached first file is the debugging aid logging. The second

doc issue missing type name "multirange" in chapter title

2021-05-06 Thread Pavel Stehule
Hi I searched operators for multirange type, and the current doc is little bit messy, because chapter "Range Functions and Operators" contains operators and functions for multirange type too. I think so this chapter should be renamed to Range, Multirange Functions and Operators" diff --git a/doc

Re: Implementing Incremental View Maintenance

2021-05-06 Thread Yugo NAGATA
On Mon, 26 Apr 2021 16:03:48 +0900 Yugo NAGATA wrote: > On Mon, 26 Apr 2021 15:46:21 +0900 > Yugo NAGATA wrote: > > > On Tue, 20 Apr 2021 09:51:34 +0900 > > Yugo NAGATA wrote: > > > > > On Mon, 19 Apr 2021 17:40:31 -0400 > > > Tom Lane wrote: > > > > > > > Andrew Dunstan writes: > > > > >

Re: Anti-critical-section assertion failure in mcxt.c reached by walsender

2021-05-06 Thread Tom Lane
Thomas Munro writes: > On Fri, May 7, 2021 at 1:43 PM Tom Lane wrote: >> The interesting part of this is frame 6, which points here: > Oh, and I see that 13 has 9989d37d "Remove XLogFileNameP() from the > tree" to fix this exact problem. Hah, so that maybe explains why thorntail has only shown

Re: Anti-critical-section assertion failure in mcxt.c reached by walsender

2021-05-06 Thread Thomas Munro
On Fri, May 7, 2021 at 1:43 PM Tom Lane wrote: > The interesting part of this is frame 6, which points here: > > case SYNC_METHOD_FDATASYNC: > if (pg_fdatasync(fd) != 0) > ereport(PANIC, > (errcode_for_file_access(), >

Re: Anti-critical-section assertion failure in mcxt.c reached by walsender

2021-05-06 Thread Andres Freund
Hi, On 2021-05-07 00:30:11 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2021-05-06 21:43:32 -0400, Tom Lane wrote: > >> That I'm not sure about. gdb is certainly installed, and thorntail is > >> visibly running the current buildfarm client and is configured with the > >> correct core_fi

Re: Anti-critical-section assertion failure in mcxt.c reached by walsender

2021-05-06 Thread Tom Lane
Andres Freund writes: > On 2021-05-06 21:43:32 -0400, Tom Lane wrote: >> That I'm not sure about. gdb is certainly installed, and thorntail is >> visibly running the current buildfarm client and is configured with the >> correct core_file_glob, and I can report that the crash did leave a 'core' >

Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-05-06 Thread Andres Freund
Hi, On 2021-05-07 11:32:57 +0800, Pengchengliu wrote: > Hi Hackers, > > Last email, format error, missing some information, so I resend this email. > > With PG 13.2(3fb4c75e857adee3da4386e947ba58a75f3e74b7), I tested > subtransaction with parallel scan, I got a subtransaction coredump as bel

Re: Anti-critical-section assertion failure in mcxt.c reached by walsender

2021-05-06 Thread Andres Freund
Hi, On 2021-05-06 21:43:32 -0400, Tom Lane wrote: > 2. We evidently need to put a bit more effort into this error > reporting logic. More generally, I wonder how we could audit > the code for similar hazards elsewhere, because I bet there are > some. (Or ... could it be sane to run functions inc

Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-05-06 Thread Pengchengliu
Hi Hackers, Last email, format error, missing some information, so I resend this email. With PG 13.2(3fb4c75e857adee3da4386e947ba58a75f3e74b7), I tested subtransaction with parallel scan, I got a subtransaction coredump as below: ``` (gdb) bt #0 0x1517ce61f7ff in raise () from /lib

Re: [BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-05-06 Thread Andres Freund
Hi, On 2021-05-06 13:35:56 -0700, Peter Geoghegan wrote: > On Thu, May 6, 2021 at 12:32 PM Andres Freund wrote: > > I think it'd be a good idea to audit the other uses of > > all_visible_according_to_vm to make sure there's no issues there. Can't > > this e.g. make us miss setting all-visible in

Re: Race condition in recovery?

2021-05-06 Thread Kyotaro Horiguchi
At Tue, 4 May 2021 17:41:06 +0530, Dilip Kumar wrote in > I think the fix for the problem will be that, after reading/validating > the checkpoint record, we can free the current value of expectedTLEs > and reinitialize it based on the recoveryTargetTLI as shown in the > attached patch? I'm not s

Re: Replication slot stats misgivings

2021-05-06 Thread Amit Kapila
On Fri, May 7, 2021 at 6:10 AM Masahiko Sawada wrote: > > On Sat, Mar 20, 2021 at 3:52 AM Andres Freund wrote: > > > > Hi, > > > > I started to write this as a reply to > > https://postgr.es/m/20210318015105.dcfa4ceybdjubf2i%40alap3.anarazel.de > > but I think it doesn't really fit under that hea

Re: Anti-critical-section assertion failure in mcxt.c reached by walsender

2021-05-06 Thread Noah Misch
On Thu, May 06, 2021 at 09:43:32PM -0400, Tom Lane wrote: > 2. We evidently need to put a bit more effort into this error > reporting logic. More generally, I wonder how we could audit > the code for similar hazards elsewhere, because I bet there are > some. (Or ... could it be sane to run functi

Re: Anti-critical-section assertion failure in mcxt.c reached by walsender

2021-05-06 Thread Tom Lane
Thomas Munro writes: > While looking for something else, I noticed thorntail has failed twice > like this, on REL_12_STABLE: > TRAP: FailedAssertion("!(CritSectionCount == 0 || > (context)->allowInCritSection)", File: > "/home/nm/farm/sparc64_deb10_gcc_64_ubsan/REL_12_STABLE/pgsql.build/../pgsql/s

Re: Replication slot stats misgivings

2021-05-06 Thread Masahiko Sawada
On Sat, Mar 20, 2021 at 3:52 AM Andres Freund wrote: > > Hi, > > I started to write this as a reply to > https://postgr.es/m/20210318015105.dcfa4ceybdjubf2i%40alap3.anarazel.de > but I think it doesn't really fit under that header anymore. > > On 2021-03-17 18:51:05 -0700, Andres Freund wrote: > >

Re: alter table set TABLE ACCESS METHOD

2021-05-06 Thread Jeff Davis
On Thu, 2021-05-06 at 17:19 -0500, Justin Pryzby wrote: > If ATRewriteTable didn't > handle this, > data would become inaccessible if an AM failed to de-toast tuples. If the AM fails to detoast tuples, it's got bigger problems than ALTER TABLE. What about INSERT INTO ... SELECT? It's the table AM

Re: pg_upgrade not preserving comments on predefined roles

2021-05-06 Thread Chapman Flack
On 05/02/21 19:25, Tom Lane wrote: > Chapman Flack writes: >> I've appended the comments we use for them at $work, anyway. > > IMO these would have to be shortened quite a bit to be friendly for > "\du+" displays. I'm not against the concept though. I'm certainly not wedded to the exact wording

Re: alter table set TABLE ACCESS METHOD

2021-05-06 Thread Justin Pryzby
On Thu, May 06, 2021 at 02:11:31PM -0700, Jeff Davis wrote: > On Thu, 2021-05-06 at 15:23 -0500, Justin Pryzby wrote: > > I think ALTER TABLE SET ACCESS METHOD should move all data off the > > old AM, > > including its toast table. > > Can you explain what you mean, and why? I'm still confused. >

Re: Remove redundant variable from transformCreateStmt

2021-05-06 Thread Alvaro Herrera
On 2021-Apr-29, Justin Pryzby wrote: > Getting rid of a redundant, boolean variable is good not because it's more > efficient but because it's one fewer LOC to read and maintain (and an > opportunity for inconsistency, I suppose). Makes sense. Pushed. Thanks everyone. > Also, this is a roundabo

Re: Docs for lock level of ALTER TABLE .. VALIDATE

2021-05-06 Thread Alvaro Herrera
On 2021-Apr-22, Simon Riggs wrote: > The docs don't explicitly mention the reduced lock level for this subcommand. Hmm, true. Pushed to all branches, thanks. -- Álvaro Herrera Valdivia, Chile

Re: alter table set TABLE ACCESS METHOD

2021-05-06 Thread Jeff Davis
On Thu, 2021-05-06 at 15:23 -0500, Justin Pryzby wrote: > I think ALTER TABLE SET ACCESS METHOD should move all data off the > old AM, > including its toast table. Can you explain what you mean, and why? I'm still confused. Let's say there are 4 table AMs: A, AT, B, and BT. A's relation_toast_am(

Re: AlterSubscription_refresh "wrconn" wrong variable?

2021-05-06 Thread Tom Lane
Alvaro Herrera writes: > On 2021-May-06, Peter Smith wrote: >> PSA v3 of the patch. Same as before, but now also renames the global >> variable from "wrconn" to "lrep_worker_wrconn". > I think there are two patches here -- the changes to > AlterSubscription_refresh are a backpatchable bugfix, and

Re: AlterSubscription_refresh "wrconn" wrong variable?

2021-05-06 Thread Alvaro Herrera
On 2021-May-06, Peter Smith wrote: > PSA v3 of the patch. Same as before, but now also renames the global > variable from "wrconn" to "lrep_worker_wrconn". I think there are two patches here -- the changes to AlterSubscription_refresh are a backpatchable bugfix, and the rest of it can just be app

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-05-06 Thread Álvaro Herrera
On 2021-May-06, Amit Langote wrote: > That makes sense, thanks for noticing. > > How about the attached? I tweaked the linkage; as submitted, the text in the link contained what is in the tag, so literally it had: ... see DETACH PARTITION partition_name [ CONCURRENTLY | FINALIZE ] for deta

Re: use `proc->pgxactoff` as the value of `index` in `ProcArrayRemove()`

2021-05-06 Thread 盏一
> Sounds like a plan! Do you want to write a patch? Add the patch. 0001-use-pgxactoff-as-the-value-of-index-in-ProcArrayRemo.patch Description: Binary data

Re: [BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-05-06 Thread Peter Geoghegan
On Thu, May 6, 2021 at 12:32 PM Andres Freund wrote: > I think it'd be a good idea to audit the other uses of > all_visible_according_to_vm to make sure there's no issues there. Can't > this e.g. make us miss setting all-visible in > > /* > * Handle setting visibil

Re: cache lookup failed for statistics object 123

2021-05-06 Thread Tomas Vondra
Hi, On 5/5/21 11:09 PM, Justin Pryzby wrote: Per sqlsmith. postgres=# SELECT pg_get_statisticsobjdef_expressions(123); ERROR: cache lookup failed for statistics object 123 postgres=# \errverbose ERROR: XX000: cache lookup failed for statistics object 123 LOCATION: pg_get_statisticsobjdef_exp

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-06 Thread Andreas Seltenreich
> On Thu, May 06, 2021 at 01:22:27PM -0400, Tom Lane wrote: >> I'm curious though why it took this long for anyone to complain. >> I'd supposed that people were running sqlsmith against HEAD on >> a pretty regular basis. Last time I ran it was November 27. I'm neglecting it on my spare time and t

Re: WIP: WAL prefetch (another approach)

2021-05-06 Thread Andres Freund
Hi, On 2021-05-04 18:08:35 -0700, Andres Freund wrote: > But the issue that 70b4f82a4b is trying to address seems bigger to > me. The reason it's so easy to hit the issue is that walreceiver does < > 8KB writes into recycled WAL segments *without* zero-filling the tail > end of the page - which wi

Re: alter table set TABLE ACCESS METHOD

2021-05-06 Thread Justin Pryzby
On Thu, May 06, 2021 at 01:10:53PM -0700, Jeff Davis wrote: > On Mon, 2021-03-08 at 16:30 +0900, Michael Paquier wrote: > > This toast issue is a kind of interesting one, and it seems rather > > right to rely on toast_build_flattened_tuple() to decompress things > > if > > both table AMs support to

Re: [BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-05-06 Thread Peter Geoghegan
On Thu, May 6, 2021 at 5:40 AM Masahiko Sawada wrote: > I've attached a patch removing the assertion. Pushed that just now -- thanks! -- Peter Geoghegan

Re: use `proc->pgxactoff` as the value of `index` in `ProcArrayRemove()`

2021-05-06 Thread Andrew Dunstan
On 5/6/21 3:27 PM, Tom Lane wrote: > Andres Freund writes: >> On 2021-05-07 00:30:13 +0800, 盏一 wrote: >>> Since we have introduced `pgxactoff` in >>> [941697c3c1ae5d6ee153065adb96e1e63ee11224](https://github.com/postgres/postgres/commit/941697c3c1ae5d6ee153065adb96e1e63ee11224), >>> and `pgxac

Re: alter table set TABLE ACCESS METHOD

2021-05-06 Thread Jeff Davis
On Mon, 2021-03-08 at 16:30 +0900, Michael Paquier wrote: > This toast issue is a kind of interesting one, and it seems rather > right to rely on toast_build_flattened_tuple() to decompress things > if > both table AMs support toast with the internals of toast knowing what > kind of compression has

Re: COPY table_name (single_column) FROM 'unknown.txt' DELIMITER E'\n'

2021-05-06 Thread Andrew Dunstan
On 5/6/21 7:41 AM, Isaac Morland wrote: > On Thu, 6 May 2021 at 02:21, Darafei "Komяpa" Praliaskouski > mailto:m...@komzpa.net>> wrote: >   > > What I would prefer is some new COPY mode like RAW that will just > push whatever it gets on the stdin/input into the cell on the > server si

Re: .ready and .done files considered harmful

2021-05-06 Thread Andres Freund
Hi, On 2021-05-06 21:23:36 +0200, Hannu Krosing wrote: > How are you envisioning the shared-memory signaling should work in the > original sample case, where the archiver had been failing for half a > year ? If we leave history files and gaps in the .ready sequence aside for a second, we really o

Re: Printing backtrace of postgres processes

2021-05-06 Thread Andres Freund
Hi, On 2021-05-06 15:22:02 -0400, Tom Lane wrote: > Andres Freund writes: > > we allow generating backtraces in all kind of places, including > > e.g. some inside critical sections via backtrace_functions. > > If there's an elog call inside a critical section, that seems > like a problem already

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-06 Thread Robert Haas
On Mon, Apr 12, 2021 at 10:04 AM Amul Sul wrote: > Rebased again. I started to look at this today, and didn't get very far, but I have a few comments. The main one is that I don't think this patch implements the design proposed in https://www.postgresql.org/message-id/CA+TgmoZ=cctbaxxmtyzogxegqzo

Re: Printing backtrace of postgres processes

2021-05-06 Thread Andres Freund
Hi, On 2021-05-06 15:31:02 -0400, Tom Lane wrote: > I'd probably vote for pg_read_all_data, considering that much of > the concern about this has to do with the possibility of exposure > of sensitive data. I'm not quite sure what the security expectations > are for pg_monitor. I was wondering th

Re: use `proc->pgxactoff` as the value of `index` in `ProcArrayRemove()`

2021-05-06 Thread Andres Freund
Hi, On 2021-05-06 15:27:29 -0400, Tom Lane wrote: > Andres Freund writes: > > If you do, I think it might be worthwhile to add an only-with-assertions > > loop checking that there's no other entry with the same pgprocno in the > > dense arrays. > > Hmm, I can definitely see keeping a check that t

Re: [BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-05-06 Thread Andres Freund
Hi, On 2021-05-06 21:40:18 +0900, Masahiko Sawada wrote: > Since we set all_visible_according_to_vm before acquiring the buffer > lock it's likely to happen that the page gets modified and all-visible > bit is cleared after setting true to all_visible_according_to_vm. This > assertion can easily b

Re: Printing backtrace of postgres processes

2021-05-06 Thread Tom Lane
Andres Freund writes: > On 2021-05-06 14:56:09 -0400, Tom Lane wrote: >> If we think it's worth having a predefined role for, OK. However, >> I don't like the future I see us heading towards where there are >> hundreds of random predefined roles. Is there an existing role >> that it'd be reasona

Re: use `proc->pgxactoff` as the value of `index` in `ProcArrayRemove()`

2021-05-06 Thread Tom Lane
Andres Freund writes: > On 2021-05-07 00:30:13 +0800, 盏一 wrote: >> Since we have introduced `pgxactoff` in >> [941697c3c1ae5d6ee153065adb96e1e63ee11224](https://github.com/postgres/postgres/commit/941697c3c1ae5d6ee153065adb96e1e63ee11224), >> and `pgxactoff` is always the index of `proc->pgprocn

Re: .ready and .done files considered harmful

2021-05-06 Thread Hannu Krosing
How are you envisioning the shared-memory signaling should work in the original sample case, where the archiver had been failing for half a year ? Or should we perhaps have a system table for ready-to-archive WAL files to get around limitation sof file system to return just the needed files with O

Re: Printing backtrace of postgres processes

2021-05-06 Thread Tom Lane
Andres Freund writes: > On 2021-05-06 14:38:51 -0400, Robert Haas wrote: >> On Wed, Feb 3, 2021 at 2:30 AM Tom Lane wrote: >>> This point is entirely separate from the question of whether >>> triggering stack traces at inopportune moments could cause system >>> malfunctions, but that question is

Re: Printing backtrace of postgres processes

2021-05-06 Thread Andres Freund
Hi, On 2021-05-06 14:56:09 -0400, Tom Lane wrote: > Robert Haas writes: > > On Wed, Feb 3, 2021 at 2:30 AM Tom Lane wrote: > >> TBH, I'm leaning to the position that this should be superuser > >> only. > > > I agree that ordinary users shouldn't be able to trigger it, but I > > think it should

Re: Printing backtrace of postgres processes

2021-05-06 Thread Andres Freund
Hi, On 2021-05-06 14:38:51 -0400, Robert Haas wrote: > On Wed, Feb 3, 2021 at 2:30 AM Tom Lane wrote: > > This point is entirely separate from the question of whether > > triggering stack traces at inopportune moments could cause system > > malfunctions, but that question is also not to be ignore

Re: use `proc->pgxactoff` as the value of `index` in `ProcArrayRemove()`

2021-05-06 Thread Andres Freund
Hi, On 2021-05-07 00:30:13 +0800, 盏一 wrote: > Since we have introduced `pgxactoff` in > [941697c3c1ae5d6ee153065adb96e1e63ee11224](https://github.com/postgres/postgres/commit/941697c3c1ae5d6ee153065adb96e1e63ee11224), > and `pgxactoff` is always the index of `proc->pgprocno` in > `procArray->pg

Re: Printing backtrace of postgres processes

2021-05-06 Thread Tom Lane
Robert Haas writes: > On Wed, Feb 3, 2021 at 2:30 AM Tom Lane wrote: >> TBH, I'm leaning to the position that this should be superuser >> only. > I agree that ordinary users shouldn't be able to trigger it, but I > think it should be restricted to some predefined role, new or > existing, rather

Re: Printing backtrace of postgres processes

2021-05-06 Thread Robert Haas
On Wed, Feb 3, 2021 at 2:30 AM Tom Lane wrote: > A backtrace normally exposes the text of the current query, for > instance, which could contain very sensitive data (passwords in ALTER > USER, customer credit card numbers in ordinary data, etc etc). We > don't allow the postmaster log to be seen

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-06 Thread Justin Pryzby
On Thu, May 06, 2021 at 01:22:27PM -0400, Tom Lane wrote: > I'm curious though why it took this long for anyone to complain. > I'd supposed that people were running sqlsmith against HEAD on > a pretty regular basis. I think it's also becase sqlsmith would need to run against the v14 *client* libra

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-05-06 Thread Álvaro Herrera
On 2021-May-05, Pavel Luzanov wrote: > Hello, > > I found this in the documentation, section '5.11.3. Partitioning Using > Inheritance'[1]: > "Some operations require a stronger lock when using declarative partitioning > than when using table inheritance. For example, removing a partition from a

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-06 Thread Tom Lane
Justin Pryzby writes: > 52a10224 broke sqlsmith, of all things. > It was using errmsg as a test of success, instead of checking if the > connection > result wasn't null: > conn = PQconnectdb(conninfo.c_str()); > char *errmsg = PQerrorMessage(conn); > if (strlen(errmsg)) >

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-05-06 Thread Álvaro Herrera
On 2021-Apr-30, Amit Langote wrote: > The case I was looking at is when a partition detach appears as > in-progress to a serializable transaction. Yeah, I was exceedingly sloppy on my reasoning about this, and you're right that that's what actually happens rather than what I said. > If the calle

Re: Asynchronous Append on postgres_fdw nodes.

2021-05-06 Thread Stephen Frost
Greetings, * Etsuro Fujita (etsuro.fuj...@gmail.com) wrote: > On Thu, Mar 4, 2021 at 1:00 PM Etsuro Fujita wrote: > > Another thing I'm concerned about in the postgres_fdw part is the case > > where all/many postgres_fdw ForeignScans of an Append use the same > > connection, because in that case

Re: COPY table_name (single_column) FROM 'unknown.txt' DELIMITER E'\n'

2021-05-06 Thread Isaac Morland
On Thu, 6 May 2021 at 12:02, Joel Jacobson wrote: > On Thu, May 6, 2021, at 13:41, Isaac Morland wrote: > > Yes! A significant missing feature is “take this arbitrary bucket of bits > and move it to/from the database from/to this file without modification of > any kind”. There are all sorts of tu

use `proc->pgxactoff` as the value of `index` in `ProcArrayRemove()`

2021-05-06 Thread 盏一
Hi, Since we have introduced `pgxactoff` in [941697c3c1ae5d6ee153065adb96e1e63ee11224](https://github.com/postgres/postgres/commit/941697c3c1ae5d6ee153065adb96e1e63ee11224), and `pgxactoff` is always the index of `proc->pgprocno` in `procArray->pgprocnos`. So it seems that we could directly use

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-06 Thread Justin Pryzby
On Sun, Jan 10, 2021 at 05:38:50PM -0500, Tom Lane wrote: > I wrote: > > The problems that I see in this area are first that there's no > > real standardization in libpq as to whether to append error messages > > together or just flush preceding messages; and second that no effort > > is made in mu

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-06 Thread Dilip Kumar
On Thu, May 6, 2021 at 5:42 PM Michael Paquier wrote: > > On Thu, May 06, 2021 at 10:45:53AM +0530, Dilip Kumar wrote: > > Thanks, Robert and Michael for your input. I will try to understand > > how it is done in the example shared by you and come up with the test > > once I get time. I assume t

Re: COPY table_name (single_column) FROM 'unknown.txt' DELIMITER E'\n'

2021-05-06 Thread Joel Jacobson
On Thu, May 6, 2021, at 13:41, Isaac Morland wrote: > On Thu, 6 May 2021 at 02:21, Darafei "Komяpa" Praliaskouski > wrote: > >> What I would prefer is some new COPY mode like RAW that will just push >> whatever it gets on the stdin/input into the cell on the server side. This >> way it can be

Re: Printing backtrace of postgres processes

2021-05-06 Thread vignesh C
On Thu, May 6, 2021 at 7:43 AM Justin Pryzby wrote: > > Here's a cleaned-up copy of the doc text. > > Send a request to the backend with the specified process ID to log its > backtrace. > The backtrace will be logged at message level LOG. > It will appear in the server log based on the log config

Multi-Column List Partitioning

2021-05-06 Thread Nitin Jadhav
Hi, While reviewing one of the 'Table partitioning' related patches, I found that Postgres does not support multiple column based LIST partitioning. Based on this understanding, I have started working on this feature. I also feel that 'Multi-Column List Partitioning' can be benefited to the Postgr

Re: Why do we have perl and sed versions of Gen_dummy_probes?

2021-05-06 Thread Andrew Dunstan
On 5/6/21 12:59 AM, Andres Freund wrote: > Hi, > > On 2021-05-06 00:18:12 -0400, Tom Lane wrote: >> Andres Freund writes: >>> I understand why we don't want to rely on sed because of windows - but >>> it's far from obvious why we can't just use the .pl variant all the >>> time? >> Perl is not co

Re: Toast compression method options

2021-05-06 Thread Dilip Kumar
On Mon, May 3, 2021 at 6:27 PM Dilip Kumar wrote: > > We have already pushed the configurable lz4 toast compression code[1]. > In the custom compression thread, we were already having the patch to > support the compression method options[2]. But the design for the > base patches was heavily modif

Re: Identify missing publications from publisher while create/alter subscription.

2021-05-06 Thread vignesh C
On Thu, May 6, 2021 at 9:08 AM Japin Li wrote: > > > On Tue, 04 May 2021 at 21:20, vignesh C wrote: > > On Tue, May 4, 2021 at 2:37 PM Bharath Rupireddy > > wrote: > >> > >> On Mon, May 3, 2021 at 7:59 PM vignesh C wrote: > >> > Thanks for the comments, these comments are handle in the v7 patch

Re: [BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-05-06 Thread Dilip Kumar
On Thu, May 6, 2021 at 6:11 PM Masahiko Sawada wrote: > Since we set all_visible_according_to_vm before acquiring the buffer > lock it's likely to happen that the page gets modified and all-visible > bit is cleared after setting true to all_visible_according_to_vm. This > assertion can easily be r

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-05-06 Thread Amit Kapila
On Thu, May 6, 2021 at 4:35 PM Robert Haas wrote: > > On Thu, May 6, 2021 at 3:00 AM Amit Kapila wrote: > > The idea here is to check for parallel safety of functions at > > someplace in the code during function invocation so that if we execute > > any parallel unsafe/restricted function via para

Re: [BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-05-06 Thread Masahiko Sawada
On Thu, Apr 29, 2021 at 1:22 PM Peter Geoghegan wrote: > > On Wed, Apr 28, 2021 at 7:34 PM tanghy.f...@fujitsu.com > wrote: > > >TRAP: FailedAssertion("!all_visible_according_to_vm || > > >prunestate.all_visible", File: "vacuumlazy.c", Line: 1347, PID: 1274675) > > > > BTW, in asynchronous mode,

Re: decoupling table and index vacuum

2021-05-06 Thread Dilip Kumar
On Thu, 6 May 2021 at 4:12 PM, Masahiko Sawada wrote: > On Thu, May 6, 2021 at 7:19 PM Robert Haas wrote: > > > > On Thu, May 6, 2021 at 5:02 AM Masahiko Sawada > wrote: > > > Not sure we will need to hold buffer locks for both the TID fork and > > > the heap at the same time but I agree that w

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-06 Thread Michael Paquier
On Thu, May 06, 2021 at 10:45:53AM +0530, Dilip Kumar wrote: > Thanks, Robert and Michael for your input. I will try to understand > how it is done in the example shared by you and come up with the test > once I get time. I assume this is not something urgent. Thanks. FWIW, I'd rather see this

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-06 Thread Michael Paquier
On Thu, May 06, 2021 at 05:01:23PM +0530, Dilip Kumar wrote: > I noticed that the error code for invalid compression method is not > perfect, basically when we pass the invalid compression method during > CREATE/ALTER table that time we give > ERRCODE_FEATURE_NOT_SUPPORTED. I think the correct err

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-05-06 Thread Greg Nancarrow
On Thu, May 6, 2021 at 5:26 PM tsunakawa.ta...@fujitsu.com wrote: > > Can anyone think of the need to check the parallel safety of built-in > functions in the context of parallel INSERT SELECT? The planner already > checks (or can check) the parallel safety of the SELECT part with > max_parall

Re: COPY table_name (single_column) FROM 'unknown.txt' DELIMITER E'\n'

2021-05-06 Thread Isaac Morland
On Thu, 6 May 2021 at 02:21, Darafei "Komяpa" Praliaskouski wrote: > What I would prefer is some new COPY mode like RAW that will just push > whatever it gets on the stdin/input into the cell on the server side. This > way it can be proxied by psql, utilize existing infra for passing streams > a

Re: RFC: Detailed reorder buffer stats dumps

2021-05-06 Thread Amit Kapila
On Thu, May 6, 2021 at 9:54 AM Craig Ringer wrote: > > On Thu, 6 May 2021 at 02:28, Andres Freund wrote: >> >> Hi, >> >> On 2021-05-05 18:33:27 +0800, Craig Ringer wrote: >> > I'm thinking of piggy-backing on the approach used in the "Get memory >> > contexts of an arbitrary backend process" patc

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-06 Thread Dilip Kumar
On Thu, May 6, 2021 at 10:45 AM Dilip Kumar wrote: I noticed that the error code for invalid compression method is not perfect, basically when we pass the invalid compression method during CREATE/ALTER table that time we give ERRCODE_FEATURE_NOT_SUPPORTED. I think the correct error code is ERRCO

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-05-06 Thread Pavel Luzanov
On 06.05.2021 08:35, Amit Langote wrote: On Wed, May 5, 2021 at 7:59 PM Pavel Luzanov wrote: I found this in the documentation, section '5.11.3. Partitioning Using Inheritance'[1]: "Some operations require a stronger lock when using declarative partitioning than when using table inheritance. F

Re: MaxOffsetNumber for Table AMs

2021-05-06 Thread Matthias van de Meent
On Thu, 6 May 2021 at 01:22, Peter Geoghegan wrote: > > On Wed, May 5, 2021 at 3:18 PM Matthias van de Meent > wrote: > > I believe that the TID is the unique identifier of that tuple, within > > context. > > > > For normal indexes, the TID as supplied directly by the TableAM is > > sufficient,

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-05-06 Thread Robert Haas
On Thu, May 6, 2021 at 3:00 AM Amit Kapila wrote: > The idea here is to check for parallel safety of functions at > someplace in the code during function invocation so that if we execute > any parallel unsafe/restricted function via parallel worker then we > error out. I think that is a good safet

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-05-06 Thread Robert Haas
On Wed, May 5, 2021 at 10:54 PM tsunakawa.ta...@fujitsu.com wrote: > (1) Is it better to get hardcoded function properties out of fmgr_builtins[]? > It's little worth doing so or thinking about that. It's no business for > users to change system objects, in this case system functions. I don't e

Re: decoupling table and index vacuum

2021-05-06 Thread Masahiko Sawada
On Thu, May 6, 2021 at 7:19 PM Robert Haas wrote: > > On Thu, May 6, 2021 at 5:02 AM Masahiko Sawada wrote: > > Not sure we will need to hold buffer locks for both the TID fork and > > the heap at the same time but I agree that we could need to lock on > > multiple TID fork buffers. We could need

Anti-critical-section assertion failure in mcxt.c reached by walsender

2021-05-06 Thread Thomas Munro
Hi, While looking for something else, I noticed thorntail has failed twice like this, on REL_12_STABLE: TRAP: FailedAssertion("!(CritSectionCount == 0 || (context)->allowInCritSection)", File: "/home/nm/farm/sparc64_deb10_gcc_64_ubsan/REL_12_STABLE/pgsql.build/../pgsql/src/backend/utils/mmgr/mcxt

Re: v14 mechanical code beautification patches

2021-05-06 Thread Bruce Momjian
On Wed, May 5, 2021 at 10:45:01PM -0400, Tom Lane wrote: > I located the "current" versions of those files in libbind 6.0. > (I put "current" in quotes because the file dates seem to be > 2005-2008, so indeed development came to a stop a long time ago.) > > They are *very* different from what we

Re: .ready and .done files considered harmful

2021-05-06 Thread Robert Haas
On Thu, May 6, 2021 at 3:23 AM Kyotaro Horiguchi wrote: > FWIW It's already done for v14 individually. > > Author: Fujii Masao > Date: Mon Mar 15 13:13:14 2021 +0900 > > Make archiver process an auxiliary process. Oh, I hadn't noticed. Thanks. -- Robert Haas EDB: http://www.enterprisedb.

Re: decoupling table and index vacuum

2021-05-06 Thread Robert Haas
On Thu, May 6, 2021 at 5:02 AM Masahiko Sawada wrote: > Not sure we will need to hold buffer locks for both the TID fork and > the heap at the same time but I agree that we could need to lock on > multiple TID fork buffers. We could need to add dead TIDs to up to two > pages for the TID fork durin

Re: Why do we have perl and sed versions of Gen_dummy_probes?

2021-05-06 Thread Dagfinn Ilmari Mannsåker
Andres Freund writes: > I tried to regenerate Gen_dummy_probes.pl using s2p - which doesn't seem > to exist for modern versions of perl anymore :( It still exists, it's just not part of the core Perl distribution any more (since 5.22, released in 2015): https://metacpan.org/pod/perl5220delta#

Re: AlterSubscription_refresh "wrconn" wrong variable?

2021-05-06 Thread Bharath Rupireddy
On Thu, May 6, 2021 at 3:00 PM Peter Smith wrote: > > On Thu, May 6, 2021 at 7:18 PM Japin Li wrote: > > > > > > On Thu, 06 May 2021 at 17:08, Peter Smith wrote: > > > On Wed, May 5, 2021 at 12:35 PM Tom Lane wrote: > > >> > > >> Peter Smith writes: > > >> > This patch replaces the global "wrc

Re: WIP: WAL prefetch (another approach)

2021-05-06 Thread Andres Freund
Hi, On 2021-05-04 09:46:12 -0400, Tom Lane wrote: > Yeah, I have also spent a fair amount of time trying to reproduce it > elsewhere, without success so far. Notably, I've been trying on a > PPC Mac laptop that has a fairly similar CPU to what's in the G4, > though a far slower disk drive. So th

Re: AlterSubscription_refresh "wrconn" wrong variable?

2021-05-06 Thread Japin Li
On Thu, 06 May 2021 at 17:30, Peter Smith wrote: > On Thu, May 6, 2021 at 7:18 PM Japin Li wrote: >> >> >> On Thu, 06 May 2021 at 17:08, Peter Smith wrote: >> > On Wed, May 5, 2021 at 12:35 PM Tom Lane wrote: >> >> >> >> Peter Smith writes: >> >> > This patch replaces the global "wrconn" in

PG 13.2 Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-05-06 Thread Pengchengliu
Hi hackers, With PG 13.2(3fb4c75e857adee3da4386e947ba58a75f3e74b7), I tested subtransaction with parallel scan, I got a subtransaction coredump as below: (gdb) bt #0 0x1517ce61f7ff in raise () from /lib64/libc.so.6 #1 0x1517ce609c35 in abort () from /lib64/libc.so.6 #2 0x00aa

Re: AlterSubscription_refresh "wrconn" wrong variable?

2021-05-06 Thread Peter Smith
On Thu, May 6, 2021 at 7:18 PM Japin Li wrote: > > > On Thu, 06 May 2021 at 17:08, Peter Smith wrote: > > On Wed, May 5, 2021 at 12:35 PM Tom Lane wrote: > >> > >> Peter Smith writes: > >> > This patch replaces the global "wrconn" in AlterSubscription_refresh > >> > with a local variable of th

Re: Replication slot stats misgivings

2021-05-06 Thread Amit Kapila
On Thu, May 6, 2021 at 2:05 PM Masahiko Sawada wrote: > > On Thu, May 6, 2021 at 5:28 PM Amit Kapila wrote: > > > > On Thu, May 6, 2021 at 1:30 PM Masahiko Sawada > > wrote: > > > > > > All issues pointed out in this thread are resolved and we can remove > > > this item from the open items? > >

Re: AlterSubscription_refresh "wrconn" wrong variable?

2021-05-06 Thread Japin Li
On Thu, 06 May 2021 at 17:08, Peter Smith wrote: > On Wed, May 5, 2021 at 12:35 PM Tom Lane wrote: >> >> Peter Smith writes: >> > This patch replaces the global "wrconn" in AlterSubscription_refresh with >> > a local variable of the same name, making it consistent with other >> > functions i

Re: Asynchronous Append on postgres_fdw nodes.

2021-05-06 Thread Etsuro Fujita
On Tue, Apr 27, 2021 at 3:57 PM Andrey V. Lepikhov wrote: > One more question. Append choose async plans at the stage of the Append > plan creation. > Later, the planner performs some optimizations, such as eliminating > trivial Subquery nodes. So, AsyncAppend is impossible in some > situations, f

Re: AlterSubscription_refresh "wrconn" wrong variable?

2021-05-06 Thread Peter Smith
On Wed, May 5, 2021 at 12:35 PM Tom Lane wrote: > > Peter Smith writes: > > This patch replaces the global "wrconn" in AlterSubscription_refresh with a > > local variable of the same name, making it consistent with other functions > > in subscriptioncmds.c (e.g. DropSubscription). > > The globa

Re: decoupling table and index vacuum

2021-05-06 Thread Masahiko Sawada
On Thu, May 6, 2021 at 3:38 PM Dilip Kumar wrote: > > On Thu, May 6, 2021 at 8:27 AM Masahiko Sawada wrote: > > > > > I'm doubtful about skipping WAL logging entirely - I'd have to think > > > harder about it, but I think that'd mean we'd restart from scratch after > > > crashes / immediate resta

Re: Replication slot stats misgivings

2021-05-06 Thread Masahiko Sawada
On Thu, May 6, 2021 at 5:28 PM Amit Kapila wrote: > > On Thu, May 6, 2021 at 1:30 PM Masahiko Sawada wrote: > > > > All issues pointed out in this thread are resolved and we can remove > > this item from the open items? > > > > I think so. Do you think we should reply to Andres's original email >

Re: Unresolved repliaction hang and stop problem.

2021-05-06 Thread Lukasz Biegaj
On 04.05.2021 16:35, Alvaro Herrera wrote: I would suggest to do that by running the problematic workload in the test system under "perf record -g" > [..] > you could ensure that in pg10 the same workload > does not cause the problem. We'll go with both propositions. I expect to come back to y

  1   2   >