Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-08-15 Thread Peter Eisentraut
On 12.08.23 04:35, Jeff Davis wrote: The attached patch implements a new SEARCH clause for CREATE FUNCTION. The SEARCH clause controls the search_path used when executing functions that were created without a SET clause. I don't understand this. This adds a new option for cases where the exis

Re: Extract numeric filed in JSONB more effectively

2023-08-15 Thread Andy Fan
update with the correct patch.. v8-0001-optimize-casting-jsonb-to-a-given-type.patch Description: Binary data

Re: Remove distprep

2023-08-15 Thread Peter Eisentraut
On 14.07.23 11:48, Tom Lane wrote: Peter Eisentraut writes: Ah, there was a reason. The headerscheck and cpluspluscheck targets need jsonpath_gram.h to be built first. Which previously happened indirectly somehow? I have fixed this in the new patch version. I also fixed the issue that Álvar

Re: WIP: new system catalog pg_wait_event

2023-08-15 Thread Michael Paquier
On Wed, Aug 16, 2023 at 07:04:53AM +0200, Drouvot, Bertrand wrote: > I'd prefer the singular form. There is a lot of places where it's already used > (pg_database, pg_user, pg_namespace...to name a few) and it looks like that > using > the plural form are exceptions. Okay, fine by me. Also, I wo

Re: Extract numeric filed in JSONB more effectively

2023-08-15 Thread Andy Fan
On Tue, Aug 15, 2023 at 1:24 PM Pavel Stehule wrote: > Hi > > út 15. 8. 2023 v 5:24 odesílatel Andy Fan > napsal: > >> >>> jsonb_extract_xx_type just cares about the argtype, but >>> 'explain select xx' will still access the const->constvalue. >>> const->constvalue is 0 which is set by makeNul

some code cleanup in index.c and indexcmds.c

2023-08-15 Thread Peter Eisentraut
Here is a patch set with some straightforward code cleanup in index.c and indexcmds.c and some adjacent places. First, I have added const qualifiers to all the function prototypes as appropriate. This didn't require any additional casts or tricks. Then, I have renamed some function arguments

Re: Synchronizing slots from primary to standby

2023-08-15 Thread Ajin Cherian
On Mon, Aug 14, 2023 at 8:38 PM shveta malik wrote: > > On Mon, Aug 14, 2023 at 3:22 PM shveta malik wrote: > > > > On Tue, Aug 8, 2023 at 11:11 AM Drouvot, Bertrand > > wrote: > > > > > > Hi, > > > > > > On 8/8/23 7:01 AM, shveta malik wrote: > > > > On Mon, Aug 7, 2023 at 3:17 PM Drouvot, Bert

Re: Memory consumed by child SpecialJoinInfo in partitionwise join planning

2023-08-15 Thread Ashutosh Bapat
On Fri, Aug 4, 2023 at 2:11 PM Ashutosh Bapat wrote: > > Attached patchset fixing those. > 0001 - patch to report planning memory, with to explain.out regression output > fix. We may consider committing this as well. > 0002 - with your comment addressed above. 0003 - Added this patch for handlin

Re: WIP: new system catalog pg_wait_event

2023-08-15 Thread Drouvot, Bertrand
Hi, On 8/14/23 6:37 AM, Michael Paquier wrote: On Thu, Aug 10, 2023 at 08:09:34PM +0200, Drouvot, Bertrand wrote: Agree that's worth it given the fact that iterating one more time is not that costly here. I have reviewed v4, and finished by putting my hands on it to see what I could do. Tha

Re: remaining sql/json patches

2023-08-15 Thread Amit Langote
On Tue, Aug 15, 2023 at 5:58 PM jian he wrote: > Hi. > in v11, json_query: > +The returned data_type has the > same semantics > +as for constructor functions like > json_objectagg; > +the default returned type is text. > +The ON EMPTY clause specifies the behavior

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Andres Freund
Hi, On 2023-08-16 12:37:21 +0900, Michael Paquier wrote: > I won't fight much if there are objections to backpatching, but that's > not really wise (no idea how much EDB's close flavor of BDR relies on > that). To be clear: I don't just object to backpatching, I also object to making existing inv

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Andres Freund
Hi, On 2023-08-16 03:20:53 +0200, Tomas Vondra wrote: > On 8/16/23 02:33, Andres Freund wrote: > > Hi, > > > > On 2023-08-16 06:58:56 +0900, Michael Paquier wrote: > >> On Tue, Aug 15, 2023 at 11:37:32AM +0200, Tomas Vondra wrote: > >>> Shouldn't the flush be done only for non-transactional messag

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-08-15 Thread Peter Smith
On Fri, Aug 11, 2023 at 11:45 PM Melih Mutlu wrote: > > Again, I couldn't reproduce the cases where you saw significantly degraded > performance. I wonder if I'm missing something. Did you do anything not > included in the test scripts you shared? Do you think v26-0001 will perform > 84% worse

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Michael Paquier
On Wed, Aug 16, 2023 at 03:20:53AM +0200, Tomas Vondra wrote: > So are you objecting to adding the flush in general, or just to the > backpatching part? > > IMHO we either guarantee durability of non-transactional messages, in > which case this would be a clear bug - and I'd say a fairly serious o

Re: [PATCH] Add function to_oct

2023-08-15 Thread John Naylor
On Wed, Aug 16, 2023 at 12:17 AM Nathan Bossart wrote: > > On Tue, Aug 15, 2023 at 01:53:25PM +0700, John Naylor wrote: > > - *ptr = '\0'; > > + Assert(base == 2 || base == 8 || base == 16); > > > > + *ptr = '\0'; > > > > Spurious whitespace change? > > I think this might just be a weird artifact

RE: [PoC] pg_upgrade: allow to upgrade publisher node

2023-08-15 Thread Hayato Kuroda (Fujitsu)
Dear Hou, Thanks for posting the patch! I want to open a question to gather opinions from others. > > It was primarily for upgrade purposes only. So, as we can't see a good > > reason to > > go via pg_dump let's do it in upgrade unless someone thinks otherwise. > > Removed the new option in pg

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Tomas Vondra
On 8/16/23 02:33, Andres Freund wrote: > Hi, > > On 2023-08-16 06:58:56 +0900, Michael Paquier wrote: >> On Tue, Aug 15, 2023 at 11:37:32AM +0200, Tomas Vondra wrote: >>> Shouldn't the flush be done only for non-transactional messages? The >>> transactional case will be flushed by regular commit f

Re: Test case for parameterized remote path in postgres_fdw

2023-08-15 Thread Richard Guo
On Tue, Aug 15, 2023 at 7:50 PM Etsuro Fujita wrote: > So we should have modified the second one as well? Attached is a > small patch for that. Agreed, nice catch! +1 to the patch. Thanks Richard

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Andres Freund
Hi, On 2023-08-16 06:58:56 +0900, Michael Paquier wrote: > On Tue, Aug 15, 2023 at 11:37:32AM +0200, Tomas Vondra wrote: > > Shouldn't the flush be done only for non-transactional messages? The > > transactional case will be flushed by regular commit flush. > > Indeed, that would be better. I am

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-08-15 Thread Peter Smith
Here is another review comment about patch v26-0001. The tablesync worker processes include the 'relid' as part of their name. See launcher.c: snprintf(bgw.bgw_name, BGW_MAXLEN, "logical replication tablesync worker for subscription %u sync %u", subid, relid); ~~ And if that worker

Re: should frontend tools use syncfs() ?

2023-08-15 Thread Michael Paquier
On Tue, Aug 08, 2023 at 01:06:06PM -0700, Nathan Bossart wrote: > I ran a couple of tests for pg_upgrade with 100k tables (created using the > script here [0]) in order to demonstrate the potential benefits of this > patch. That shows some nice numbers with many files, indeed. How does the size o

Re: Avoid a potential unstable test case: xmlmap.sql

2023-08-15 Thread Andy Fan
> > > Please look at the bug #18014: > > https://www.postgresql.org/message-id/flat/18014-28c81cb79d44295d%40postgresql.org > There were other aspects of the xmlmap test failure discussed in that > thread as well. > Thank you Alexander for the information, I will go through there for discussion.

Re: Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

2023-08-15 Thread Michael Paquier
On Tue, Aug 15, 2023 at 03:39:10PM -0700, Jacob Champion wrote: > I'm not super comfortable with saying "connection authenticated" when > it explicitly hasn't been (nor with switching the meaning of a > non-NULL SYSTEM_USER from "definitely authenticated somehow" to "who > knows; parse it apart to

Re: Would it be possible to backpatch Close support in libpq (28b5726) to PG16?

2023-08-15 Thread Alvaro Herrera
On 2023-Aug-16, Michael Paquier wrote: > > Personally I think backpatching 28b5726 has a really low risk of > > breaking anything. > > I agree about the low-risk argument, though. This is just new code. Here's a way to think about it. If 16.1 was already out, would we add libpq support for Clo

Re: Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

2023-08-15 Thread Jacob Champion
On Tue, Aug 15, 2023 at 3:24 PM Michael Paquier wrote: > The first message from Jacob outlines the idea behind the handling of > trust. We could perhaps add one extra set_authn_id() for the uaTrust > case (not uaCert!) if that's more helpful. I'm not super comfortable with saying "connection aut

Re: Would it be possible to backpatch Close support in libpq (28b5726) to PG16?

2023-08-15 Thread Michael Paquier
On Wed, Aug 16, 2023 at 12:14:21AM +0200, Jelte Fennema wrote: > 28b5726 allows sending Close messages from libpq, as opposed to > sending DEALLOCATE queries to deallocate prepared statements. Without > support for Close messages, libpq based clients won't be able to > deallocate prepared statement

Re: Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

2023-08-15 Thread Michael Paquier
On Tue, Aug 15, 2023 at 04:49:47PM -0500, Shaun Thomas wrote: > The switch statement that decodes port->hba->auth_method ends by > simply setting status = STATUS_OK; with no supplementary output since > it never calls set_authn_id. So in theory, a malicious user could add > a trust line to pg_hba.c

Would it be possible to backpatch Close support in libpq (28b5726) to PG16?

2023-08-15 Thread Jelte Fennema
Hi, I know that this will probably get a staunch "No" as an answer, but... I'm still going to ask: Would it be possible to backport 28b5726 to the PG16 branch? Even though it's clearly a new feature? I'm working on named prepared statement support in PgBouncer: https://github.com/pgbouncer/pgboun

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Michael Paquier
On Tue, Aug 15, 2023 at 11:37:32AM +0200, Tomas Vondra wrote: > Shouldn't the flush be done only for non-transactional messages? The > transactional case will be flushed by regular commit flush. Indeed, that would be better. I am sending an updated patch. I'd like to backpatch that, would there

Logging of matching pg_hba.conf entry during auth skips trust auth, potential security issue

2023-08-15 Thread Shaun Thomas
Hi everyone, This started as a conversation on Discord. Someone asked if Postgres logs which line in pg_hba.conf matched against a certain login attempt, and I said no. That's not quite right, as enabling log_connections includes a line like this: 2023-08-15 13:26:03.159 PDT [692166] postgres@sni

Re: Using defines for protocol characters

2023-08-15 Thread Tatsuo Ishii
> On Wed, Aug 16, 2023 at 06:25:09AM +0900, Tatsuo Ishii wrote: >> Currently pqcomm.h needs c.h which is not problem for Pgpool-II. But >> what about other middleware? > > Why do you need to include directly c.h? There are definitions in > there that are not intended to be exposed. pqcomm.h has

Re: Using defines for protocol characters

2023-08-15 Thread Alvaro Herrera
On 2023-Aug-16, Michael Paquier wrote: > On Wed, Aug 16, 2023 at 06:25:09AM +0900, Tatsuo Ishii wrote: > > Currently pqcomm.h needs c.h which is not problem for Pgpool-II. But > > what about other middleware? > > Why do you need to include directly c.h? There are definitions in > there that are

Re: Using defines for protocol characters

2023-08-15 Thread Michael Paquier
On Wed, Aug 16, 2023 at 06:25:09AM +0900, Tatsuo Ishii wrote: > Currently pqcomm.h needs c.h which is not problem for Pgpool-II. But > what about other middleware? Why do you need to include directly c.h? There are definitions in there that are not intended to be exposed. -- Michael signature.a

Re: Using defines for protocol characters

2023-08-15 Thread Tatsuo Ishii
> On Tue, Aug 15, 2023 at 02:46:24PM +0900, Tatsuo Ishii wrote: >> Is it possible to put the new define staff into protocol.h then let >> pqcomm.h include protocol.h? This makes Pgpool-II and other middle >> ware/drivers written in C easier to use the defines so that they only >> include protocol.h

Re: run pgindent on a regular basis / scripted manner

2023-08-15 Thread Nathan Bossart
On Fri, Aug 11, 2023 at 01:59:40PM -0700, Peter Geoghegan wrote: > I'm starting to have doubts about this policy. There have now been > quite a few follow-up "fixes" to indentation issues that koel > complained about. None of these fixups have been included in > .git-blame-ignore-revs. If things co

Re: [PATCH] Add function to_oct

2023-08-15 Thread Nathan Bossart
On Tue, Aug 15, 2023 at 01:53:25PM +0700, John Naylor wrote: > If we're not going to have a general SQL conversion function, here are some > comments on the current patch. I appreciate the review. > +static char *convert_to_base(uint64 value, int base); > > Not needed if the definition is above

Re: initial pruning in parallel append

2023-08-15 Thread Robert Haas
On Wed, Aug 9, 2023 at 8:57 AM Amit Langote wrote: > I checked enough to be sure that IsParallelWorker() is reliable at the > time of ExecutorStart() / ExecInitNode() in ParallelQueryMain() in a > worker. However, ParallelWorkerContext is not available at that > point. Here's the relevant part

Re: Faster "SET search_path"

2023-08-15 Thread Robert Haas
On Mon, Aug 7, 2023 at 7:24 PM Jeff Davis wrote: > I might just avoid guc.c entirely and directly set > namespace_search_path and baseSearchPathValid=false. The main thing we > lose there is the GUC stack (AtEOXact_GUC()), but there's already a > PG_TRY/PG_FINALLY block in fmgr_security_definer, s

Re: Extending SMgrRelation lifetimes

2023-08-15 Thread Heikki Linnakangas
On 14/08/2023 05:41, Thomas Munro wrote: The new idea is to overload smgrrelease(it) so that it also clears the owner, which means that AtEOXact_SMgr() will eventually smgrclose(it), unless it is re-owned by a relation before then. That choice stems from the complete lack of information availabl

Re: Using defines for protocol characters

2023-08-15 Thread Nathan Bossart
On Tue, Aug 15, 2023 at 02:46:24PM +0900, Tatsuo Ishii wrote: > Is it possible to put the new define staff into protocol.h then let > pqcomm.h include protocol.h? This makes Pgpool-II and other middle > ware/drivers written in C easier to use the defines so that they only > include protocol.h to us

Re: [PATCH] Add function to_oct

2023-08-15 Thread Nathan Bossart
On Tue, Aug 15, 2023 at 07:58:17AM +0200, Vik Fearing wrote: > On 8/15/23 06:11, Nathan Bossart wrote: >> If there are no objections, I'd like to commit this patch soon. > > I just took a look at this (and the rest of the thread). I am a little bit > disappointed that we don't have a generic base

Re: pgbench - adding pl/pgsql versions of tests

2023-08-15 Thread Nathan Bossart
On Tue, Aug 15, 2023 at 09:46:59AM +0200, Fabien COELHO wrote: > I'm unclear about what variety of scripts that could be provided given the > tables made available with pgbench. ISTM that other scenari would involve > both an initialization and associated scripts, and any proposal would be > bared

Re: Replace known_assigned_xids_lck by memory barrier

2023-08-15 Thread Nathan Bossart
On Tue, Aug 15, 2023 at 12:29:24PM +0200, Michail Nikolaev wrote: >> What sort of benefits do you see from this patch? It might be worthwhile >> in itself to remove spinlocks when possible, but IME it's much easier to >> justify such changes when there is a tangible benefit we can point to. > > Oh

Converting sql anywhere to postgres

2023-08-15 Thread Russell Rose | Passfield Data Systems
Hi there I am trying to convert a SQL Anywhere database to postgres. Within SQL anywhere a field can have a default value of 'last user'. This means that when you perform an update on a table, if the field is not explicitly set then the current user is used. So for instance if I have a field ca

Re: Avoid a potential unstable test case: xmlmap.sql

2023-08-15 Thread Alexander Lakhin
Hi Andy, 15.08.2023 14:09, Andy Fan wrote: Hi: In the test case of xmlmap.sql, we have the query below under schema_to_xml. Please look at the bug #18014: https://www.postgresql.org/message-id/flat/18014-28c81cb79d44295d%40postgresql.org There were other aspects of the xmlmap test failure d

Re: BUG #17946: LC_MONETARY & DO LANGUAGE plperl - BUG

2023-08-15 Thread Heikki Linnakangas
On 01/08/2023 16:48, Joe Conway wrote: Any further comments on the posted patch[1]? I would like to apply/push this prior to the beta and minor releases next week. I'm not sure about the placement of the uselocale() calls. In plperl_spi_exec(), for example, I think we should switch to the glob

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-08-15 Thread Önder Kalacı
Hi Etsuro, all The commit[1] seems to break some queries in Citus[2], which is an extension which relies on set_join_pathlist_hook. Although the comment says */*Finally, give extensions a chance to manipulate the path list.*/ *we use it to extract lots of information about the joins and do the p

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-08-15 Thread John Naylor
On Tue, Aug 15, 2023 at 9:34 AM Masahiko Sawada wrote: > BTW cfbot reported that some regression tests failed due to OOM. I've > attached the patch to fix it. Seems worth doing now rather than later, so added this and squashed most of the rest together. I wonder if that test uses too much memory

Test case for parameterized remote path in postgres_fdw

2023-08-15 Thread Etsuro Fujita
Hi, While working on the join pushdown issue, I noticed this bit in commit e4106b252: --- parameterized remote path +-- parameterized remote path for foreign table EXPLAIN (VERBOSE, COSTS false) - SELECT * FROM ft2 a, ft2 b WHERE a.c1 = 47 AND b.c1 = a.c2; + SELECT * FROM "S 1"."T 1" a, ft2 b

Re: Avoid a potential unstable test case: xmlmap.sql

2023-08-15 Thread Andy Fan
I overlooked the fact even in the bitmap index scan loose mode, the recheck is still executed before the qual, so bitmap index scan is OK in this case. Sort Output: oid, relname Sort Key: pg_class.relname -> Bitmap Heap Scan on pg_catalog.pg_class Output: oid, relname

Avoid a potential unstable test case: xmlmap.sql

2023-08-15 Thread Andy Fan
Hi: In the test case of xmlmap.sql, we have the query below under schema_to_xml. explain (costs off, verbose) SELECT oid FROM pg_catalog.pg_class WHERE relnamespace = 28601 AND relkind IN ('r','m','v') AND pg_catalog.has_table_privilege (oid, 'SELECT') ORDER BY relname; If the query is using S

Re: Replace known_assigned_xids_lck by memory barrier

2023-08-15 Thread Michail Nikolaev
Hello, Nathan. > What sort of benefits do you see from this patch? It might be worthwhile > in itself to remove spinlocks when possible, but IME it's much easier to > justify such changes when there is a tangible benefit we can point to. Oh, it is not an easy question :) The answer, probably, lo

Re: cataloguing NOT NULL constraints

2023-08-15 Thread Alvaro Herrera
On 2023-Aug-15, Dean Rasheed wrote: > I think perhaps for ALTER TABLE INHERIT, it should check that the > child has a NOT NULL constraint, and error out if not. That's the > current behaviour, and also matches other constraints types (e.g., > CHECK constraints). Yeah, I reached the same conclusio

Re: cataloguing NOT NULL constraints

2023-08-15 Thread Dean Rasheed
On Fri, 11 Aug 2023 at 14:54, Alvaro Herrera wrote: > > Right, in the end I got around to that point of view. I abandoned the > idea of adding these dependency links, and I'm back at relying on the > coninhcount/conislocal markers. But there were a couple of bugs in the > accounting for that, so

Re: pgbench: allow to exit immediately when any client is aborted

2023-08-15 Thread Fabien COELHO
About pgbench exit on abort v4: Patch applies cleanly, compiles, local make check ok, doc looks ok. This looks ok to me. -- Fabien.

Re: pg_logical_emit_message() misses a XLogFlush()

2023-08-15 Thread Tomas Vondra
On 8/15/23 08:38, Michael Paquier wrote: > Hi all, > > While playing with pg_logical_emit_message() and WAL replay, I have > noticed that LogLogicalMessage() inserts a record but forgets to make > sure that the record has been flushed. So, for example, if the system > crashes the message inser

Re: LLVM 16 (opaque pointers)

2023-08-15 Thread Fabien COELHO
[...] Further changes are already needed for their "main" branch (LLVM 17-to-be), so this won't quite be enough to shut seawasp up. For information, the physical server which was hosting my 2 bf animals (seawasp and moonjelly) has given up rebooting after a power cut a few weeks/months ago,

Generating memory trace from Postgres backend process

2023-08-15 Thread Muneeb Anwar
Hi, Has anyone tried generating a dynamic memory trace of a backend Postgres process while it's running a query? I want to characterize the memory access pattern of the Postgres database engine when it's running any given query. The usual way to do this would be to attach a dynamic instrumentation

Re: remaining sql/json patches

2023-08-15 Thread jian he
Hi. in v11, json_query: +The returned data_type has the same semantics +as for constructor functions like json_objectagg; +the default returned type is text. +The ON EMPTY clause specifies the behavior if the +path_expression yields no value at all; the +

Re: New WAL record to detect the checkpoint redo location

2023-08-15 Thread Dilip Kumar
On Fri, Jul 14, 2023 at 8:46 PM Andres Freund wrote: > > Hi, > > As I think I mentioned before, I like this idea. However, I don't like the > implementation too much. Thanks for looking into it. > This might work right now, but doesn't really seem maintainable. Nor do I like > adding branches i

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-08-15 Thread Etsuro Fujita
On Tue, Aug 8, 2023 at 6:30 PM Richard Guo wrote: > On Tue, Aug 8, 2023 at 4:40 PM Etsuro Fujita wrote: >> I modified the code a bit further to use an if-test to avoid a useless >> function call, and added/tweaked comments and docs further. Attached >> is a new version of the patch. I am planni

Re: pgbench - adding pl/pgsql versions of tests

2023-08-15 Thread Fabien COELHO
Hello Nathan, 1. so I don't have to create the script and function manually each time I want to test mainly the database (instead of the client-database system) 2. so that new users of PostgreSQL can easily see how much better OLTP workloads perform when packaged up as a server-side function

Re: Extract numeric filed in JSONB more effectively

2023-08-15 Thread Pavel Stehule
út 15. 8. 2023 v 9:05 odesílatel Andy Fan napsal: > > >> a) effectiveness. The ending performance should be similar like your >> current patch, but without necessity to use planner support API. >> > > So the cost is we need to create a new & different framework. > yes, it can be less work, code

Re: Extract numeric filed in JSONB more effectively

2023-08-15 Thread Andy Fan
> > a) effectiveness. The ending performance should be similar like your > current patch, but without necessity to use planner support API. > So the cost is we need to create a new & different framework. > > b) because you can write only var := j->'f', and plpgsql forces cast > function execution