Re: Invalid Unicode escape value at or near "\u0000"

2021-11-15 Thread Peter Eisentraut
On 13.11.21 06:40, Thomas Munro wrote: Yes, it is a valid codepoint, but unfortunately PostgreSQL can't support it because it sometimes deals in null terminated string, even though internally it does track string data and length separately. We have to do that to use libc facilities like strcoll_

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-15 Thread Peter Eisentraut
On 14.11.21 13:18, Bharath Rupireddy wrote: PSA v11 patch with 2 APIs with much simpler parameters and small function names: int errdetail_rel(Form_pg_class rd_rel); int errdetail_rel_v2(Oid relid, char relkind, char relpersistence); Please review it. I think this is not an improvement. It

pg_get_publication_tables() output duplicate relid

2021-11-15 Thread houzj.f...@fujitsu.com
Hi hackers, In another thread[1], we found the pg_get_publication_tables function will output duplicate partition relid when adding both child and parent table to the publication(pubviaroot = false). Example: create table tbl1 (a int) partition by range (a); create table tbl1_part1 partition of

Re: parallel vacuum comments

2021-11-15 Thread Masahiko Sawada
On Thu, Nov 11, 2021 at 6:38 PM Amit Kapila wrote: > > On Thu, Nov 11, 2021 at 8:11 AM Masahiko Sawada wrote: > > > > I've attached a draft patch that refactors parallel vacuum and > > separates parallel-vacuum-related code to new file vacuumparallel.c. > > After discussion, I'll divide the patch

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-15 Thread Daniel Gustafsson
> On 15 Nov 2021, at 09:15, Peter Eisentraut > wrote: > I think this is not an improvement. It loses the ability of the caller the > specify exactly why a relation is not acceptable. Agreed. > I think a separate errdetail_relpersistence_not_supported() would be a better > solution (assuming

Re: row filtering for logical replication

2021-11-15 Thread Dilip Kumar
On Fri, Nov 12, 2021 at 3:49 PM Ajin Cherian wrote: > > Attaching version 39- > Some comments on 0006 -- /* + * Write UPDATE to the output stream using cached virtual slots. + * Cached updates will have both old tuple and new tuple. + */ +void +logicalrep_write_update_cached(StringInfo out, Tra

Re: Skipping logical replication transactions on subscriber side

2021-11-15 Thread Masahiko Sawada
On Mon, Nov 15, 2021 at 4:49 PM Greg Nancarrow wrote: > > On Mon, Nov 15, 2021 at 1:49 PM Masahiko Sawada wrote: > > > > I've attached an updated patch that incorporates all comments I got so > > far. Please review it. > > > > Thanks for the updated patch. > A few minor comments: > > doc/src/sgml

Re: Add connection active, idle time to pg_stat_activity

2021-11-15 Thread Dilip Kumar
On Wed, Nov 10, 2021 at 1:47 PM Rafia Sabih wrote: > > > It seems that in beentry->st_idle_time, you want to compute the > > STATE_IDLE, but that state is not handled in the outer "if", that > > means whenever it comes out of the > > STATE_IDLE, it will not enter inside this if check. You can run

Re: Write visibility map during CLUSTER/VACUUM FULL

2021-11-15 Thread Daniel Gustafsson
> On 30 Aug 2021, at 02:26, Justin Pryzby wrote: > Rebased on f10f0ae420ee62400876ab34dca2c09c20dcd030. This patch no longer applies, please submit a rebased version. -- Daniel Gustafsson https://vmware.com/

RE: Failed transaction statistics to measure the logical replication progress

2021-11-15 Thread osumi.takami...@fujitsu.com
On Thursday, November 11, 2021 9:47 PM vignesh C wrote: > Few more comments: > 1) Here the tuple length is not considered in the calculation, else it will > always > show the fixed size for any size tuple. Ex varchar insert with 1 byte or > varchar > insert with 100's of bytes. So I feel we shou

RE: Failed transaction statistics to measure the logical replication progress

2021-11-15 Thread osumi.takami...@fujitsu.com
On Wednesday, November 10, 2021 7:13 PM vignesh C wrote: > On Tue, Nov 9, 2021 at 5:05 PM osumi.takami...@fujitsu.com > wrote: > > Yes. I've rebased and updated the patch, paying attention to this point. > > Attached the updated version. > > Thanks for the updated patch, few comments: > 1) you c

RE: Failed transaction statistics to measure the logical replication progress

2021-11-15 Thread osumi.takami...@fujitsu.com
On Wednesday, November 10, 2021 6:13 PM I wrote: > On Wednesday, November 10, 2021 3:43 PM Dilip Kumar > wrote: > > On Tue, Nov 9, 2021 at 5:05 PM osumi.takami...@fujitsu.com > > wrote: > > > On Tuesday, November 9, 2021 12:08 PM Greg Nancarrow > > wrote: > > > > On Fri, Nov 5, 2021 at 7:11 PM o

Re: row filtering for logical replication

2021-11-15 Thread Amit Kapila
On Mon, Nov 15, 2021 at 2:44 PM Dilip Kumar wrote: > > On Fri, Nov 12, 2021 at 3:49 PM Ajin Cherian wrote: > > > > This function definition header is too long to fit in one line, so > better to break it. I think running will be a good idea. > It seems in the last line you are suggesting to run

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-15 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 2:14 PM Daniel Gustafsson wrote: > > > On 15 Nov 2021, at 09:15, Peter Eisentraut > > wrote: > > > I think this is not an improvement. It loses the ability of the caller the > > specify exactly why a relation is not acceptable. > > > Agreed. +1. > > I think a separate

Re: row filtering for logical replication

2021-11-15 Thread Dilip Kumar
On Mon, 15 Nov 2021 at 3:07 PM, Amit Kapila wrote: > On Mon, Nov 15, 2021 at 2:44 PM Dilip Kumar wrote: > > > > On Fri, Nov 12, 2021 at 3:49 PM Ajin Cherian wrote: > > > > > > > This function definition header is too long to fit in one line, so > > better to break it. I think running will be a

Re: WIP: System Versioned Temporal Table

2021-11-15 Thread Daniel Gustafsson
> On 19 Sep 2021, at 20:32, Simon Riggs wrote: > My preferred approach would be to do this "for free" in the table > access method, but we're a long way from this in terms of actual > implementation. When Corey suggested earlier that we just put the > syntax in there, this was the direction I wa

Re: WIP: System Versioned Temporal Table

2021-11-15 Thread Vik Fearing
On 11/15/21 10:47 AM, Daniel Gustafsson wrote: >> On 19 Sep 2021, at 20:32, Simon Riggs wrote: > >> My preferred approach would be to do this "for free" in the table >> access method, but we're a long way from this in terms of actual >> implementation. When Corey suggested earlier that we just p

Re: row filtering for logical replication

2021-11-15 Thread Greg Nancarrow
On Mon, Nov 15, 2021 at 5:09 PM tanghy.f...@fujitsu.com wrote: > > I met a problem when using "ALTER PUBLICATION ... SET TABLE ... WHERE ...", > the > publisher was crashed after executing this statement. > > > > Backtrace is attached. I think maybe the problem is related to the below > change i

Re: Unnecessary delay in streaming replication due to replay lag

2021-11-15 Thread Daniel Gustafsson
> On 10 Nov 2021, at 00:41, Soumyadeep Chakraborty > wrote: > Thanks for the detailed review! Attached is a rebased patch that addresses > most of the feedback. This patch no longer applies after e997a0c64 and associated follow-up commits, please submit a rebased version. -- Daniel Gustafsson

Re: Split xlog.c

2021-11-15 Thread Daniel Gustafsson
> On 5 Oct 2021, at 03:09, Jaime Casanova wrote: > Are we waiting a rebased version? Currently this does not apply to head. > I'll mark this as WoA and move it to necxt CF. This patch still doesn't apply, exacerbated by the recent ThisTimelineID changes in xlog.c. I'm marking this Returned with

Re: Partial aggregates pushdown

2021-11-15 Thread Daniel Gustafsson
> On 3 Nov 2021, at 15:50, Alexander Pyhalov wrote: > > Daniel Gustafsson писал 2021-11-03 16:45: >>> On 2 Nov 2021, at 10:12, Alexander Pyhalov wrote: >>> Updated and rebased patch. >> +state = (Int128AggState *) palloc0(sizeof(Int128AggState)); >> +state->calcSumX2 = false; >> + >> +

Re: pg_get_publication_tables() output duplicate relid

2021-11-15 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 1:48 PM houzj.f...@fujitsu.com wrote: > > Hi hackers, > > In another thread[1], we found the pg_get_publication_tables function will > output > duplicate partition relid when adding both child and parent table to the > publication(pubviaroot = false). > > Example: > create

Re: WIP: WAL prefetch (another approach)

2021-11-15 Thread Daniel Gustafsson
> On 10 May 2021, at 06:11, Thomas Munro wrote: > On Thu, Apr 22, 2021 at 11:22 AM Stephen Frost wrote: >> I tend to agree with the idea to revert it, perhaps a +0 on that, but if >> others argue it should be fixed in-place, I wouldn’t complain about it. > > Reverted. > > Note: eelpout may re

Re: row filtering for logical replication

2021-11-15 Thread Amit Kapila
On Wed, Nov 10, 2021 at 12:36 PM Peter Smith wrote: > > On Mon, Nov 8, 2021 at 5:53 PM houzj.f...@fujitsu.com > wrote: > > > > 3) v37-0005 > > > > - no parse nodes of any kind other than Var, OpExpr, Const, BoolExpr, > > FuncExpr > > > > I think there could be other node type which can also be c

Re: [Proposal] Global temporary tables

2021-11-15 Thread Andrew Bille
Thanks for the patches. The feature has become much more stable. However, there is another simple case that generates an error: Master with v61 patches CREATE GLOBAL TEMPORARY TABLE t AS SELECT 1 AS a; ERROR: could not open file "base/13560/t3_16384": No such file or directory Andrew On Thu, Nov

Re: WIP: System Versioned Temporal Table

2021-11-15 Thread Simon Riggs
On Mon, 15 Nov 2021 at 09:47, Daniel Gustafsson wrote: > > > On 19 Sep 2021, at 20:32, Simon Riggs wrote: > > > My preferred approach would be to do this "for free" in the table > > access method, but we're a long way from this in terms of actual > > implementation. When Corey suggested earlier

Re: parallel vacuum comments

2021-11-15 Thread Amit Kapila
On Mon, Nov 15, 2021 at 2:01 PM Masahiko Sawada wrote: > > On Thu, Nov 11, 2021 at 6:38 PM Amit Kapila wrote: > > > > On Thu, Nov 11, 2021 at 8:11 AM Masahiko Sawada > > wrote: > > > > > > I've attached a draft patch that refactors parallel vacuum and > > > separates parallel-vacuum-related cod

Re: Add connection active, idle time to pg_stat_activity

2021-11-15 Thread Rafia Sabih
On Mon, 15 Nov 2021 at 10:24, Dilip Kumar wrote: > > On Wed, Nov 10, 2021 at 1:47 PM Rafia Sabih wrote: > > > > > It seems that in beentry->st_idle_time, you want to compute the > > > STATE_IDLE, but that state is not handled in the outer "if", that > > > means whenever it comes out of the > > >

Re: Add connection active, idle time to pg_stat_activity

2021-11-15 Thread Dilip Kumar
On Mon, Nov 15, 2021 at 4:46 PM Rafia Sabih wrote: > > On Mon, 15 Nov 2021 at 10:24, Dilip Kumar wrote: > > > > On Wed, Nov 10, 2021 at 1:47 PM Rafia Sabih > > wrote: > > > > > > > It seems that in beentry->st_idle_time, you want to compute the > > > > STATE_IDLE, but that state is not handled

RE: Failed transaction statistics to measure the logical replication progress

2021-11-15 Thread osumi.takami...@fujitsu.com
On Monday, November 15, 2021 6:28 PM I wrote: > On Thursday, November 11, 2021 9:47 PM vignesh C > wrote: > > Few more comments: ... > Fixed. Also, its name was too long > when aligned with other PgStat_StatDBEntry memebers. > Thus I renamed it as subworkers_preparedsizes. > > This depends on v21

Re: parallel vacuum comments

2021-11-15 Thread Masahiko Sawada
On Mon, Nov 15, 2021 at 8:04 PM Amit Kapila wrote: > > On Mon, Nov 15, 2021 at 2:01 PM Masahiko Sawada wrote: > > > > On Thu, Nov 11, 2021 at 6:38 PM Amit Kapila wrote: > > > > > > On Thu, Nov 11, 2021 at 8:11 AM Masahiko Sawada > > > wrote: > > > > > > > > I've attached a draft patch that ref

Re: RFC: Logging plan of the running query

2021-11-15 Thread torikoshia
On 2021-11-13 22:29, Bharath Rupireddy wrote: Thanks for your review! On Wed, Oct 13, 2021 at 7:58 PM Ekaterina Sokolova wrote: Thank you for working on this issue. I would be glad to continue to follow the development of this issue. Thanks for the patch. I'm not sure if v11 is the latest pa

Re: Partial aggregates pushdown

2021-11-15 Thread Alexander Pyhalov
Daniel Gustafsson писал 2021-11-15 13:16: On 3 Nov 2021, at 15:50, Alexander Pyhalov wrote: Daniel Gustafsson писал 2021-11-03 16:45: On 2 Nov 2021, at 10:12, Alexander Pyhalov wrote: Updated and rebased patch. + state = (Int128AggState *) palloc0(sizeof(Int128AggState)); + sta

Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2021-11-15 Thread Daniel Gustafsson
> On 21 Jul 2021, at 03:49, Yugo NAGATA wrote: > I attached the updated patch v2, which includes a comment fix and a TAP test. This patch fails the TAP test for pgbench: # Tests were run but no plan was declared and done_testing() was not seen. # Looks like your test exited with 25 just aft

Re: pg_get_publication_tables() output duplicate relid

2021-11-15 Thread Alvaro Herrera
> On Mon, Nov 15, 2021 at 1:48 PM houzj.f...@fujitsu.com > wrote: > > create table tbl1 (a int) partition by range (a); > > create table tbl1_part1 partition of tbl1 for values from (1) to (10); > > create table tbl1_part2 partition of tbl1 for values from (10) to (20); > > create publication pub

Re: RFC: Logging plan of the running query

2021-11-15 Thread torikoshia
On 2021-11-13 03:37, Justin Pryzby wrote: I reviewed this version of the patch - I have some language fixes. Thanks for your review! Attached patch that reflects your comments. Regards, -- Atsushi Torikoshi NTT DATA CORPORATIONFrom b8367e22d7a9898e4b85627ba8c203be273fc22f Mon Sep 17 00:00:0

pg_waldump stucks with options --follow or -f and --stats or -z

2021-11-15 Thread Bharath Rupireddy
Hi, pg_waldump options, --follow or -f(to keep polling once per second for new WAL to appear) and --stats or -z don't work well together i.e. the command stucks [1]. I think the pg_waldump should emit an error. Note that the pg_basebakup does error out on incompatible options. Here's a small patc

Re: RFC: Logging plan of the running query

2021-11-15 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 6:29 PM torikoshia wrote: > > 3) I think SendProcSignalForLogInfo can be more generic, meaning, it > > can also send signal to auxiliary processes if asked to do this will > > simplify the things for pg_log_backend_memory_contexts and other > > patches like pg_print_backtra

Re: Commitfest 2021-11 Patch Triage - Part 2

2021-11-15 Thread Robert Haas
On Tue, Nov 9, 2021 at 9:12 AM Daniel Gustafsson wrote: > 2773: libpq compression > === > This patch intended to provide libpq connection compression to "replace SSL > compression" which was doomed when the patch was written, and have since been > removed altogether. The initi

Should rename "startup process" to something else?

2021-11-15 Thread Rushabh Lathia
Hi, Robert and I wondered whether we would like to rename the startup process. The reason for doing this is that the current name doesn't make any sense, as in the stand-by mode replay loop as the main loop, the startup process stays around. I think a better name for the process may be “reco

Re: Skipping logical replication transactions on subscriber side

2021-11-15 Thread vignesh C
On Mon, Nov 15, 2021 at 2:48 PM Masahiko Sawada wrote: > > On Mon, Nov 15, 2021 at 4:49 PM Greg Nancarrow wrote: > > > > On Mon, Nov 15, 2021 at 1:49 PM Masahiko Sawada > > wrote: > > > > > > I've attached an updated patch that incorporates all comments I got so > > > far. Please review it. > >

Re: Parallelize correlated subqueries that execute within each worker

2021-11-15 Thread Robert Haas
On Wed, Nov 3, 2021 at 1:34 PM James Coleman wrote: > As I understand the current code, parallel plans are largely chosen > based not on where it's safe to insert a Gather node but rather by > determining if a given path is parallel safe. Through that lens params > are a bit of an odd man out -- t

Re: Printing backtrace of postgres processes

2021-11-15 Thread vignesh C
On Mon, Nov 15, 2021 at 11:37 AM Dilip Kumar wrote: > > On Mon, Nov 15, 2021 at 10:34 AM vignesh C wrote: > > > > > Thanks for the comments, the attached v12 patch has the changes for the > > same. > > I have reviewed this patch and have some comments on v12-0001, > > 1. > +This feature

Re: refactoring basebackup.c

2021-11-15 Thread Dmitry Dolgov
> On Fri, Nov 05, 2021 at 11:50:01AM -0400, Robert Haas wrote: > On Tue, Nov 2, 2021 at 10:32 AM Robert Haas wrote: > > Meanwhile, I think it's probably OK for me to go ahead and commit > > 0001-0003 from my patches at this point, since it seems we have pretty > > good evidence that the abstractio

Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

2021-11-15 Thread vignesh C
On Mon, Nov 15, 2021 at 7:47 AM Bharath Rupireddy wrote: > > On Fri, Nov 5, 2021 at 11:12 AM Bharath Rupireddy > wrote: > > PSA v2 patch and review it. > > I've modified the docs part a bit, please consider v3 for review. Thanks for the update patch, Few comments: 1) Should we change "CHECK_FOR_

Re: enhance pg_log_backend_memory_contexts() to log memory contexts of auxiliary processes

2021-11-15 Thread Bharath Rupireddy
On Mon, Nov 15, 2021 at 10:04 PM vignesh C wrote: > > On Mon, Nov 15, 2021 at 7:47 AM Bharath Rupireddy > wrote: > > > > On Fri, Nov 5, 2021 at 11:12 AM Bharath Rupireddy > > wrote: > > > PSA v2 patch and review it. > > > > I've modified the docs part a bit, please consider v3 for review. > > Th

Re: Time to drop plpython2?

2021-11-15 Thread Andres Freund
Hi, On 2021-11-14 21:24:31 -0500, Tom Lane wrote: > ... btw, there's a fairly critical gating factor for any plan to drop > python2 support: the buildfarm. I just counted, and there are exactly > as many members running python 2.x as 3.x (49 apiece), not counting > Windows machines that aren't ru

Re: JIT doing duplicative optimization?

2021-11-15 Thread Andres Freund
Hi, On 2021-11-11 17:02:09 -0300, Alvaro Herrera wrote: > Recently I noticed extremely suspicious behavior from queries under JIT. > A query (details for which sadly I cannot disclose) had an EXPLAIN > looking like this: > > JIT: >Functions: 743 >Options: Inlining true, Optimization true

Re: JIT doing duplicative optimization?

2021-11-15 Thread Andres Freund
Hi, On 2021-11-14 18:46:34 -0500, Tom Lane wrote: > BTW, I realized while working on this that I have little idea what the > "Functions:" count is. Nor does our documentation explain that (or any > other of these numbers), at least not anywhere I could find. That seems > like a pretty serious do

Re: Time to drop plpython2?

2021-11-15 Thread Tom Lane
Andres Freund writes: > On 2021-11-14 21:24:31 -0500, Tom Lane wrote: >> (It's likely that some fraction of them do already have python3 installed, >> in which case the search order change Peter recommended would be enough to >> fix it. But I'm sure not all do.) > How about committing the order

Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2021-11-15 Thread Yugo NAGATA
Hello Daniel Gustafsson, On Mon, 15 Nov 2021 14:13:32 +0100 Daniel Gustafsson wrote: > > On 21 Jul 2021, at 03:49, Yugo NAGATA wrote: > > > I attached the updated patch v2, which includes a comment fix and a TAP > > test. > > This patch fails the TAP test for pgbench: Thank you for pointing

Re: Test::More version

2021-11-15 Thread Tom Lane
Andrew Dunstan writes: > Thanks. pushed a slightly more robust version. I think we have enough info now to draw conclusions. There are several buildfarm animals that are running TAP tests but haven't reported since this went in. However, I don't think any of them will yield surprises: frogfis

Re: Read access for pg_monitor to pg_replication_origin_status view

2021-11-15 Thread Martín Marqués
Hello, I wanted to resurface this thread. The original intention I had with this patch I sent over a year ago was to have the possibility for monitoring ROLEs like pg_monitor and pg_read_all_stats to have read access for the replication origin status. Seems the patch only got half way through (we

Re: Improving psql's \password command

2021-11-15 Thread Tom Lane
"Bossart, Nathan" writes: > Here is an attempt at adding control-C support for \password. With > this patch, we pass sigint_interrupt_jmp and sigint_interrupt_enabled > all the way down to pg_get_line_append(), which is admittedly a bit > more complicated than I think would be ideal. I see a cou

Re: Time to drop plpython2?

2021-11-15 Thread Andres Freund
Hi, On 2021-11-15 12:19:51 -0500, Tom Lane wrote: > I don't think that's warranted. The existing design is that we let > the user say which python is "python", and I do not think we should > change that in advance of actually dropping python2 support. Hm. I think it'd be ok, given that python 2

Re: Time to drop plpython2?

2021-11-15 Thread Tom Lane
Andres Freund writes: > On 2021-11-15 12:19:51 -0500, Tom Lane wrote: >> I was wondering about simply probing to see if python3 exists (and if >> so, what version it is exactly), as an additional configure test that >> doesn't hook into anything. That would give us some information without >> sud

Re: [RFC] building postgres with meson

2021-11-15 Thread Andres Freund
Hi, FWIW, I tried building postgres on a few other operating systems using meson, after I got access to the gcc compile farm. Here's the results: - openbsd: Compiled fine. Hit one issue running tests: openbsd has *completely* broken $ORIGIN support. It uses CWD as $ORIGIN rpaths, which obvi

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-15 Thread Peter Eisentraut
On 15.11.21 10:38, Bharath Rupireddy wrote: I still think that the v8 patch posted earlier is the better option, which increase granularity of error reporting with a small code footprint. Thanks. Attaching the v8 here again. I find the use of RelationUsesLocalBuffers() confusing in this patch.

Re: Test::More version

2021-11-15 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > Based on this, I'm inclined to think we should select 0.98 as the > minimum version. Anything later would inconvenience some people. > OTOH, it doesn't look like there's any value in promising compatibility > with 0.96 instead, especially since I see a couple of subplan-relate

Re: Time to drop plpython2?

2021-11-15 Thread Peter Eisentraut
On 15.11.21 19:18, Tom Lane wrote: Andres Freund writes: On 2021-11-15 12:19:51 -0500, Tom Lane wrote: I was wondering about simply probing to see if python3 exists (and if so, what version it is exactly), as an additional configure test that doesn't hook into anything. That would give us som

Re: Commitfest 2021-11 Patch Triage - Part 2

2021-11-15 Thread Daniel Gustafsson
> On 15 Nov 2021, at 15:32, Robert Haas wrote: > > On Tue, Nov 9, 2021 at 9:12 AM Daniel Gustafsson wrote: >> 2773: libpq compression >> === >> This patch intended to provide libpq connection compression to "replace SSL >> compression" which was doomed when the patch was writ

Re: Frontend error logging style

2021-11-15 Thread Peter Eisentraut
On 09.11.21 23:20, Tom Lane wrote: 1. The distinction between "error" and "fatal" levels seems squishy to the point of uselessness. I think we should either get rid of it entirely, or make an effort to use "fatal" exactly for the cases that are going to give up and exit right away. Of the ap

Re: Frontend error logging style

2021-11-15 Thread Peter Eisentraut
On 09.11.21 23:20, Tom Lane wrote: 2. What is the preferred style for adding extra lines to log messages? I see a lot of direct prints to stderr: pg_log_error("missing required argument: database name"); fprintf(stderr, _("Try \"%s --help\" for more information.\n

Re: BUFFERS enabled by default in EXPLAIN (ANALYZE)

2021-11-15 Thread Justin Pryzby
On Fri, Nov 12, 2021 at 02:58:07PM -0800, Nikolay Samokhvalov wrote: > Re-reading the thread [1] (cannot answer there – don't have those emails in > my box anymore), I see that there was strong support for enabling BUFFERS > in EXPLAIN ANALYZE by default. And there were patches. Commitfest entry [2

Re: [RFC] building postgres with meson

2021-11-15 Thread Tom Lane
Andres Freund writes: > One important thing to call out: Meson has support for the AIX linker, but > *not* the xlc compiler. I.e. one has to use gcc (or clang, but I didn't > try). I don't know if we'd require adding support for xlc to meson - xlc is > pretty buggy and it doesn't seem part

Re: Test::More version

2021-11-15 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > Tom Lane writes: >> Based on this, I'm inclined to think we should select 0.98 as the >> minimum version. Anything later would inconvenience some people. >> OTOH, it doesn't look like there's any value in promising compatibility >> with 0.96 in

Re: Frontend error logging style

2021-11-15 Thread Peter Eisentraut
On 10.11.21 16:28, Robert Haas wrote: What I think we ought to be driving towards is having pg_log_fatal() forcibly exit, and pg_log_error() do the same unless the error is somehow caught. This is specifically designed not to do any flow control. In the backend, we have many instances, where

Re: [RFC] building postgres with meson

2021-11-15 Thread Andres Freund
Hi, On 2021-11-15 14:11:25 -0500, Tom Lane wrote: > Andres Freund writes: > > One important thing to call out: Meson has support for the AIX linker, but > > *not* the xlc compiler. I.e. one has to use gcc (or clang, but I didn't > > try). I don't know if we'd require adding support for xlc

Re: refactoring basebackup.c

2021-11-15 Thread Robert Haas
On Mon, Nov 15, 2021 at 11:25 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > Walsender tries to send a backup manifest, but crashes on the trottling sink: > > #2 0x560857b551af in ExceptionalCondition > (conditionName=0x560857d15d27 "sink->bbs_next != NULL", > errorType=0x560857d15c23

Re: Time to drop plpython2?

2021-11-15 Thread Tom Lane
Peter Eisentraut writes: > If we add this test first, then all we're going to learn is probably > that 60% of those who are currently using python2 don't have python3 > installed, and then we're still going to have to send that above email. I don't know what fraction don't have python3 installe

Re: Frontend error logging style

2021-11-15 Thread Robert Haas
On Mon, Nov 15, 2021 at 2:15 PM Peter Eisentraut wrote: > This is specifically designed not to do any flow control. In the > backend, we have many instances, where log messages are issued with the > wrong log level because the stronger log level would have flow control > impact that is not approp

Re: Printing backtrace of postgres processes

2021-11-15 Thread Justin Pryzby
On Mon, Nov 15, 2021 at 09:12:49PM +0530, vignesh C wrote: > The idea here is to implement & expose pg_print_backtrace function, internally This patch is closely related to this one https://commitfest.postgresql.org/35/3142/ | Logging plan of the currently running query I suggest to review that p

Re: Frontend error logging style

2021-11-15 Thread Tom Lane
Peter Eisentraut writes: > Several programs wrap, say, pg_log_fatal() into a pg_fatal(), that does > logging, cleanup, and exit, as the case may be. I think that's a good > solution. I agree, and my draft patch formalized that by turning pg_log_fatal into exactly that. The question that I thi

Re: [RFC] building postgres with meson

2021-11-15 Thread Robert Haas
On Mon, Nov 15, 2021 at 2:23 PM Andres Freund wrote: > It's good to make sure we're not growing too reliant on some compiler(s), but > imo only really makes sense if the alternative compilers are meaningfully > available and maintained. That's a sensible position. I do worry that with this propos

Re: Support for NSS as a libpq TLS backend

2021-11-15 Thread Joshua Brindle
On Wed, Nov 10, 2021 at 8:49 AM Daniel Gustafsson wrote: > > > On 9 Nov 2021, at 22:22, Joshua Brindle > > wrote: > > On Tue, Nov 9, 2021 at 2:02 PM Joshua Brindle > > wrote: > >> > >> On Tue, Nov 9, 2021 at 1:59 PM Joshua Brindle > >> wrote: > > >>> Hello, I'm looking to help out with reviews

Re: Commitfest 2021-11 Patch Triage - Part 2

2021-11-15 Thread Stephen Frost
Greetings, * Daniel Gustafsson (dan...@yesql.se) wrote: > > On 15 Nov 2021, at 15:32, Robert Haas wrote: > > On Tue, Nov 9, 2021 at 9:12 AM Daniel Gustafsson wrote: > >> 2773: libpq compression > >> === > >> This patch intended to provide libpq connection compression to "repl

Re: Time to drop plpython2?

2021-11-15 Thread Andres Freund
Hi, Continuing the discussion from https://www.postgresql.org/message-id/2146739.1637004415%40sss.pgh.pa.us On 2021-11-15 14:26:55 -0500, Tom Lane wrote: > * Meson only promises support back to python 3.6, but if that's > accurate it's going to be a problem for us, because there are lots > of li

Re: Schema variables - new implementation for Postgres 15

2021-11-15 Thread Pavel Stehule
Hi Justin so 6. 11. 2021 v 2:39 odesílatel Justin Pryzby napsal: > On Wed, Nov 03, 2021 at 02:05:02PM +0100, Tomas Vondra wrote: > > 3) I've reworded / tweaked a couple places in the docs, but this really > > needs a native speaker - I don't have a very good "feeling" for this > > technical lang

Re: Schema variables - new implementation for Postgres 15

2021-11-15 Thread Pavel Stehule
> > my last patch has 276KB and your patch has 293KB? > Please, can you resend your version of patch 0001? Thank you Pavel

Re: Test::More version

2021-11-15 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > =?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: Your MUA seems a bit MIME-deficient… >> Tom Lane writes: >>> Based on this, I'm inclined to think we should select 0.98 as the >>> minimum version. Anything later would inconvenience some people. >>> OTOH, it doesn't look l

Re: Commitfest 2021-11 Patch Triage - Part 2

2021-11-15 Thread Robert Haas
On Mon, Nov 15, 2021 at 2:51 PM Stephen Frost wrote: > I get that just compressing the entire stream is simpler and easier and > such, but it's surely cheaper and more efficient to not decompress and > then recompress data that's already compressed. Finding a way to pass > through data that's alr

Re: Schema variables - new implementation for Postgres 15

2021-11-15 Thread Justin Pryzby
On Mon, Nov 15, 2021 at 09:00:13PM +0100, Pavel Stehule wrote: > Thank you for review and fixes, I try to complete some version for next > work, and looks so your patch 0001 is broken > > gedit reports to me broken unicode \A0\A0\A0\A0\A0 > > my last patch has 276KB and your patch has 293KB? On

Re: Time to drop plpython2?

2021-11-15 Thread Tom Lane
Andres Freund writes: > On 2021-11-15 14:26:55 -0500, Tom Lane wrote: >> * Meson only promises support back to python 3.6, but if that's >> accurate it's going to be a problem for us, because there are lots >> of live LTS distributions with older python3 (RHEL7, Solaris 11.3, >> AIX 7.2 for starte

Re: Time to drop plpython2?

2021-11-15 Thread Robert Haas
On Mon, Nov 15, 2021 at 3:30 PM Tom Lane wrote: > The info I checked said that RHEL7 originally shipped with 3.3. > I'm not sure that Red Hat would've outright replaced that, but they > do have a notion of add-on "software collections", and I'm certain > that they would have provided newer pythons

Re: Commitfest 2021-11 Patch Triage - Part 2

2021-11-15 Thread Justin Pryzby
On Mon, Nov 15, 2021 at 03:23:17PM -0500, Robert Haas wrote: > On Mon, Nov 15, 2021 at 2:51 PM Stephen Frost wrote: > > I get that just compressing the entire stream is simpler and easier and > > such, but it's surely cheaper and more efficient to not decompress and > > then recompress data that's

Re: Commitfest 2021-11 Patch Triage - Part 2

2021-11-15 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Nov 15, 2021 at 2:51 PM Stephen Frost wrote: > > I get that just compressing the entire stream is simpler and easier and > > such, but it's surely cheaper and more efficient to not decompress and > > then recompress data that's alr

Re: support for MERGE

2021-11-15 Thread Álvaro Herrera
On 2021-Nov-14, Amit Langote wrote: > The only problem caused by the code block that follows the buggy if > statement unconditionally executing is wasted cycles. Fortunately, > there's no correctness issue, because rootRelInfo is the same as the > input result relation in the cases where the latt

Re: Improving psql's \password command

2021-11-15 Thread Tom Lane
"Bossart, Nathan" writes: > On 11/15/21, 10:13 AM, "Tom Lane" wrote: >> * I don't believe that this bit is necessary, or if it is, >> the comment fails to justify it: >> >> - initStringInfo(&buf); >> + /* make sure buf is palloc'd so we don't lose changes after a >> longjmp */ >> +

Re: Time to drop plpython2?

2021-11-15 Thread Andres Freund
Hi, On 2021-11-15 15:30:02 -0500, Tom Lane wrote: > > Is RHEL7 really an issue? I only have Centos 7 around, but that has python > > 3.6. > > The info I checked said that RHEL7 originally shipped with 3.3. > I'm not sure that Red Hat would've outright replaced that, but they > do have a notion of

Re: Time to drop plpython2?

2021-11-15 Thread Andres Freund
Hi, On November 15, 2021 12:36:11 PM PST, Robert Haas wrote: >It'd only be an issue if they want to compile from source, right? >We're not speaking of changing the runtime prerequisites, IIUC. Correct. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Time to drop plpython2?

2021-11-15 Thread Tom Lane
Robert Haas writes: > On Mon, Nov 15, 2021 at 3:30 PM Tom Lane wrote: >> The info I checked said that RHEL7 originally shipped with 3.3. > It'd only be an issue if they want to compile from source, right? > We're not speaking of changing the runtime prerequisites, IIUC. I'm not sure. Does it m

Re: RecoveryInProgress() has critical side effects

2021-11-15 Thread Robert Haas
On Fri, Nov 12, 2021 at 10:23 AM Bossart, Nathan wrote: > 0001 looks reasonable to me. Cool. > Unless there's a demonstrable performance benefit from adding the > branch, my preference is to leave it out. I could be off-base, but it > seems possible that future changes might end up depending on

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-11-15 Thread Daniel Gustafsson
> On 15 Nov 2021, at 19:42, Peter Eisentraut > wrote: > > On 15.11.21 10:38, Bharath Rupireddy wrote: >>> I still think that the v8 patch posted earlier is the better option, which >>> increase granularity of error reporting with a small code footprint. >> Thanks. Attaching the v8 here again. >

Re: Support for NSS as a libpq TLS backend

2021-11-15 Thread Daniel Gustafsson
> On 15 Nov 2021, at 20:51, Joshua Brindle > wrote: > Apologies for the delay, this didn't go to my inbox and I missed it on list. > > The bitcode generation is still broken, this time for nspr.h: Interesting, I am unable to replicate that in my tree but I'll investigate further tomorrow using

Re: [RFC] building postgres with meson

2021-11-15 Thread Thomas Munro
On Tue, Nov 16, 2021 at 8:23 AM Andres Freund wrote: > On 2021-11-15 14:11:25 -0500, Tom Lane wrote: > > Having said that, I don't plan to be the one trying to get meson > > to add xlc support ... > > It'd probably not be too hard. But given that it's quite hard to get access to > AIX + xlc, I'm n

Re: RecoveryInProgress() has critical side effects

2021-11-15 Thread Bossart, Nathan
On 11/15/21, 1:30 PM, "Robert Haas" wrote: > Here's a new version that does it that way. Any other opinions? LGTM > The best thing I could come up with for a test case for this was to > try repeatedly making a new connection and running "SELECT > txid_current()", which will cause just one WAL re

Re: [PATCH v2] use has_privs_for_role for predefined roles

2021-11-15 Thread Bossart, Nathan
On 11/12/21, 12:34 PM, "Joshua Brindle" wrote: > All of these and also adminpack.sgml updated. I think that is all of > them but docs broken across lines and irregular wording makes it > difficult. LGTM. I've marked this as ready-for-committer. Nathan

CREATE PUBLICATION should "See Also" CREATE SUBSCRIPTION

2021-11-15 Thread Peter Smith
Hi. I noticed recently that the CREATE PUBLICATION docs page [1] does not have any "See Also" reference to the CREATE SUBSCRIPTION docs page [2]. Perhaps it is deliberate (??) but IMO it just seemed like a strange/annoying omission that there was no easy way to jump both ways between those two pa

Re: [RFC] building postgres with meson

2021-11-15 Thread Tom Lane
Thomas Munro writes: > ... The interesting thing about 16.1 is that you can invoke it > as xlclang to get the new clang frontend and, I think, possibly use > more clang/gcc-ish compiler switches[2]. > [2] > https://www.ibm.com/docs/en/xl-c-and-cpp-aix/16.1?topic=new-clang-based-front-end Ho, tha

Re: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display

2021-11-15 Thread Bossart, Nathan
On 11/14/21, 4:18 AM, "Bharath Rupireddy" wrote: > Thanks. Attaching the v2 to avoid that by directly using the message > in ereport instead of activitymsg. +ereport(DEBUG1, +(errmsg_internal("waiting for %s", xlogfname))); As a general comment, I think we should

  1   2   >