Re: IPC/MultixactCreation on the Standby server

2025-06-25 Thread Dmitry
On 25.06.2025 12:34, Andrey Borodin wrote: On 25 Jun 2025, at 11:11, Dmitry wrote: #6 GetMultiXactIdMembers (multi=45559845, members=0x7ffdaedc84b0, from_pgupgrade=, isLockOnly=) at /usr/src/postgresql-17-17.5-1.pgdg24.04+1/build/../src/backend/access/transam/multixact.c

IPC/MultixactCreation on the Standby server

2025-06-24 Thread Dmitry
  #24 0x57c4402bcd2d in main (argc=17, argv=0x57c4747fb140) at /usr/src/postgresql-17-17.5-1.pgdg24.04+1/build/../src/backend/main/main.c:199 Could you please help me to fix the problem of stuck 'client backend' processes? I kindly ask you for any ideas and recommendations! Best regards, Dmitry

Re: Changing shared_buffers without restart

2025-06-20 Thread Dmitry Dolgov
> On Fri, Jun 20, 2025 at 12:19:31PM +0200, Dmitry Dolgov wrote: > Thanks! I've reworked the series to implement approach suggested by > Thomas, and applied your patches to support buffers shrinking on top. I > had to restructure the patch set, here is how it looks like right now:

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2025-06-11 Thread Dmitry Koval
ion as partition of permanent relation "t" -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.com

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2025-06-05 Thread Dmitry Koval
Hi, jian he! Thank you very much for your emails! Unfortunately, due to urgent tasks at my work, I do not have time to look through your notes today and tomorrow. But I will definitely do it at the beginning of next week. -- With best regards, Dmitry Koval Postgres Professional: http

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2025-06-03 Thread Dmitry Koval
Some notes to the patch. ... Thanks for the notes! I'll try to make edits in the next few days. -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.com

Re: Automatically sizing the IO worker pool

2025-05-27 Thread Dmitry Dolgov
On Mon, May 26, 2025, 8:01 AM Thomas Munro wrote: > But ... I'm not even sure if we can say that our > I/O arrivals have a Poisson distribution, since they are not all > independent. > Yeah, a good point, one have to be careful with assumptions about distribution -- from what I've read many proc

Re: Automatically sizing the IO worker pool

2025-05-24 Thread Dmitry Dolgov
> On Sun, Apr 13, 2025 at 04:59:54AM GMT, Thomas Munro wrote: > It's hard to know how to set io_workers=3. If it's too small, > io_method=worker's small submission queue overflows and it silently > falls back to synchronous IO. If it's too high, it generates a lot of > pointless wakeups and sched

Re: I/O worker and ConfigReload

2025-05-24 Thread Dmitry Dolgov
> On Sun, May 25, 2025 at 02:15:12AM GMT, Thomas Munro wrote: > On Sun, May 25, 2025 at 1:56 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > I've been rebasing the patch for online resizing of shared memory, and > > noticed something strange about IoWorkerMain: alt

I/O worker and ConfigReload

2025-05-24 Thread Dmitry Dolgov
Hi, I've been rebasing the patch for online resizing of shared memory, and noticed something strange about IoWorkerMain: although it sets the handler SignalHandlerForConfigReload, it doesn't look like it acts upon ConfigReloadPending. From what I see it happens because it only does CHECK_FOR_INTER

Re: queryId constant squashing does not support prepared statements

2025-05-23 Thread Dmitry Dolgov
ctually. This is the behavior on HEAD, and I don't > > > get why > > > the first list with the casts does not get squashed, while the second one > > > does. > > > I will check IsSquashableConst tomorrow unless Dmitry gets to it first. > > > >

Re: queryId constant squashing does not support prepared statements

2025-05-23 Thread Dmitry Dolgov
gt; > $3 in ($4, cast($5 as int), $6, (cast($7 as int)), $8, $9, (cast($10 as > > text))::int) > > This is interesting actually. This is the behavior on HEAD, and I don't get > why > the first list with the casts does not get squashed, while the second one > does. &

Re: queryId constant squashing does not support prepared statements

2025-05-22 Thread Dmitry Dolgov
> On Wed, May 21, 2025 at 08:22:19PM GMT, Sami Imseih wrote: > > > > At the same time AFAICT there isn't much more code paths > > > > to worry about in case of a LocationExpr as a node > > > > > > I can imagine there are others like value expressions, > > > row expressions, json array expressions,

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2025-05-22 Thread Dmitry Koval
superuser can bypass all permission checks. superuser can attach any table to the partitioned table as he wants. That's right. Using SUPERUSER may be a rare situation, but it needs to be processed. -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.com

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2025-05-21 Thread Dmitry Koval
for example: ... If in both examples you replace create role bob; with create role bob SUPERUSER; and in the second example add "set role bob;" before "alter table ..." query, then no error will be occur. -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.com

Re: queryId constant squashing does not support prepared statements

2025-05-21 Thread Dmitry Dolgov
> On Tue, May 20, 2025 at 04:50:12PM GMT, Sami Imseih wrote: > > At the same time AFAICT there isn't much more code paths > > to worry about in case of a LocationExpr as a node > > I can imagine there are others like value expressions, > row expressions, json array expressions, etc. that we may > w

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2025-05-21 Thread Dmitry Koval
wner of the split partition.) -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.com

Re: queryId constant squashing does not support prepared statements

2025-05-20 Thread Dmitry Dolgov
> On Tue, May 20, 2025 at 06:30:25AM GMT, Michael Paquier wrote: > On Mon, May 12, 2025 at 06:40:43PM -0400, Sami Imseih wrote: > > Also, LocationExpr is not really an expression node, but a wrapper to > > an expression node, so I think it's wrong to define it as a Node and be > > required to add t

Re: queryId constant squashing does not support prepared statements

2025-05-20 Thread Dmitry Dolgov
> On Tue, May 20, 2025 at 11:03:52AM GMT, Dmitry Dolgov wrote: > > By the way, the new test cases for ARRAY lists are sent in the last > > patch of the series posted on this thread: > > https://www.postgresql.org/message-id/7zbzwk4btnxoo4o3xbtzefoqvht54cszjj4bol22fm

Re: queryId constant squashing does not support prepared statements

2025-05-13 Thread Dmitry Dolgov
> On Mon, May 12, 2025 at 06:40:43PM GMT, Sami Imseih wrote: > > The static variables was only part of the concern, another part was > > using A_Expr to carry this information, which will have impact on lots > > of unrelated code. > > What would be the problem if A_Expr carries an extra pointer to

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2025-05-12 Thread Dmitry Koval
merged have different owners? a) current user (since he is the one who creates new table); b) merging of partitions should be forbidden if they have different owners. Please, advise what seems to be the best solution for points 1 and 2. With best regards, Dmitry Koval Postgres Professional: http

Re: queryId constant squashing does not support prepared statements

2025-05-12 Thread Dmitry Dolgov
> On Fri, May 09, 2025 at 12:47:19PM GMT, Sami Imseih wrote: > So, I think we can create a new parse node ( parsenode.h ) that will only be > used in parsing (and gram.c only ) to track the start/end locations > and List and > based on this node we can create A_ArrayExpr and A_Expr with the List >

Re: queryId constant squashing does not support prepared statements

2025-05-12 Thread Dmitry Dolgov
> On Fri, May 09, 2025 at 10:12:24AM GMT, Dmitry Dolgov wrote: > Agree, I'll try to extend number of test cases here as a separate patch. Here is the extended version, where start/end is replaced by location/length, array_expr is handled as well, and more ARRAY cases are a

Re: queryId constant squashing does not support prepared statements

2025-05-09 Thread Dmitry Dolgov
> On Fri, May 09, 2025 at 08:47:58AM GMT, Michael Paquier wrote: > SELECT query, calls FROM pg_stat_statements ORDER BY query COLLATE "C"; > - query| calls > -+--- > - SELECT ARRAY[$1 /*, ... */]

Re: queryId constant squashing does not support prepared statements

2025-05-09 Thread Dmitry Dolgov
> On Fri, May 09, 2025 at 02:35:33PM GMT, Michael Paquier wrote: > On Fri, May 09, 2025 at 11:05:43AM +0800, Junwang Zhao wrote: > > Why not a location and a length, it should be more natural, it > > seems we use this convention in some existing nodes, like > > RawStmt, InsertStmt etc. > > These ar

Re: queryId constant squashing does not support prepared statements

2025-05-08 Thread Dmitry Dolgov
> On Thu, May 08, 2025 at 02:22:00PM GMT, Michael Paquier wrote: > On Wed, May 07, 2025 at 10:41:22AM +0200, Dmitry Dolgov wrote: > > Ah, I see what you mean. I think the idea is fine, it will simplify > > certain things as well as address the issue. But I'm afraid adding &g

Re: queryId constant squashing does not support prepared statements

2025-05-07 Thread Dmitry Dolgov
> On Tue, May 06, 2025 at 03:01:32PM GMT, Sami Imseih wrote: > > > Without properly accounting for the boundaries of the list of > > > expressions, i.e., > > > the start and end positions of '(' and ')' or '[' and ']' and normalizing > > > the > > > expressions in between, it will be very difficu

Re: queryId constant squashing does not support prepared statements

2025-05-06 Thread Dmitry Dolgov
> On Tue, May 06, 2025 at 01:32:48PM GMT, Sami Imseih wrote: > > I also agree with Alvaro that this discussion doesn't justify a > > revert. If the pre-v18 behavior wasn't chiseled on stone tablets, > > the new behavior isn't either. We can improve it some more later. > > As I was looking further

Re: queryId constant squashing does not support prepared statements

2025-05-06 Thread Dmitry Dolgov
> On Tue, May 06, 2025 at 11:50:07PM GMT, Junwang Zhao wrote: > Would it make sense to rename `RecordConstLocation` to something like > `RecordExpressionLocation` instead? Yeah, naming is hard. RecordExpressionLocation is somehow more vague, but I see what you mean, maybe something along these lin

Re: Changing shared_buffers without restart

2025-05-06 Thread Dmitry Dolgov
> On Tue, May 06, 2025 at 04:23:07AM GMT, Jack Ng wrote: > Thanks Dmitry. Right, the coordination mechanism in v4-0006 works as expected > in various tests (sorry, I misunderstood some details initially). Great, thanks for checking. > I also want to report a couple of minor issues

Re: queryId constant squashing does not support prepared statements

2025-05-04 Thread Dmitry Dolgov
> On Thu, May 01, 2025 at 09:55:47PM GMT, Dmitry Dolgov wrote: > > On Thu, May 01, 2025 at 09:29:13AM GMT, Michael Paquier wrote: > > > > I agree that the current solution we have in the tree feels incomplete > > because we are not taking into account the most common

Re: queryId constant squashing does not support prepared statements

2025-05-02 Thread Dmitry Dolgov
> On Fri, May 02, 2025 at 04:18:37PM GMT, Michael Paquier wrote: > On Fri, May 02, 2025 at 09:13:39AM +0200, Dmitry Dolgov wrote: > > Squashing constants was ment to be a first step towards doing the same > > for other types of queries (params, rte_values), reverting it to > &

Re: queryId constant squashing does not support prepared statements

2025-05-02 Thread Dmitry Dolgov
> On Fri, May 02, 2025 at 07:10:19AM GMT, Michael Paquier wrote: > > I am really leaning towards that we should revert this feature as the > > limitation we have now with parameters is a rather large one and I think > > we need to go back and address this issue. > > I am wondering if this would not

Re: queryId constant squashing does not support prepared statements

2025-05-01 Thread Dmitry Dolgov
> On Thu, May 01, 2025 at 09:29:13AM GMT, Michael Paquier wrote: > > I agree that the current solution we have in the tree feels incomplete > because we are not taking into account the most common cases that > users would care about. Now, allowing PARAM_EXTERN means that we > allow any expression

Re: Changing shared_buffers without restart

2025-04-21 Thread Dmitry Dolgov
> On Thu, Apr 17, 2025 at 07:05:36PM GMT, Ni Ku wrote: > I also have a related question about how ftruncate() is used in the patch. > In my testing I also see that when using ftruncate to shrink a shared > segment, the memory is freed immediately after the call, even if other > processes still have

Re: Changing shared_buffers without restart

2025-04-21 Thread Dmitry Dolgov
> On Fri, Apr 18, 2025 at 10:06:23AM GMT, Konstantin Knizhnik wrote: > The only drawback is that we are loosing content of shared buffers in case > of resize. It may be sadly, but not looks like there is no better > alternative. No, why would we loose the content? If we do mremap, it will leave th

Re: Changing shared_buffers without restart

2025-04-21 Thread Dmitry Dolgov
> On Fri, Apr 18, 2025 at 09:17:21PM GMT, Thomas Munro wrote: > I was imagining that you might map some maximum possible size at the > beginning to reserve the address space permanently, and then adjust > the virtual memory object's size with ftruncate as required to provide > backing. Doesn't tha

Re: Changing shared_buffers without restart

2025-04-18 Thread Dmitry Dolgov
> On Thu, Apr 17, 2025 at 02:21:07PM GMT, Konstantin Knizhnik wrote: > > 1. Performance of Postgres CLOCK page eviction algorithm depends on number > of shared buffers. My first native attempt just to mark unused buffers as > invalid cause significant degrade of performance Thanks for sharing! Ri

Re: Changing shared_buffers without restart

2025-04-18 Thread Dmitry Dolgov
> On Thu, Apr 17, 2025 at 03:22:28PM GMT, Ashutosh Bapat wrote: > > In an offlist chat Thomas Munro mentioned that just ftruncate() would > be enough to resize the shared memory without touching address maps > using mmap and munmap(). > > ftruncate man page seems to concur with him > >If th

Re: Changing shared_buffers without restart

2025-04-14 Thread Dmitry Dolgov
> On Mon, Apr 14, 2025 at 10:40:28AM GMT, Ashutosh Bapat wrote: > > However, when we put back the patches to shrink buffers, we will evict > the extra buffers, and shrink - if all the processes haven't > participated in the barrier by then, some of them may try to access > those buffers - re-instal

Re: Proposal: Adding compression of temporary files

2025-04-13 Thread Dmitry Dolgov
> On Fri, Mar 28, 2025 at 09:23:13AM GMT, Filip Janus wrote: > > +else > > +if (nbytesOriginal - file->pos != 0) > > +/* curOffset must be corrected also if compression is > > + * enabled, nbytes was changed by compression but we > > + * have to use t

Re: Changing shared_buffers without restart

2025-04-11 Thread Dmitry Dolgov
> On Fri, Apr 11, 2025 at 08:04:39PM GMT, Ashutosh Bapat wrote: > On Mon, Apr 7, 2025 at 2:13 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > Yes, you're right, plain dynamic Barrier does not ensure all available > > processes will be synchronized.

Re: Changing shared_buffers without restart

2025-04-09 Thread Dmitry Dolgov
> On Wed, Apr 09, 2025 at 01:20:16PM GMT, Ashutosh Bapat wrote: > ../../coderoot/pg/src/include/storage/s_lock.h:93:2: error: #error > "s_lock.h may not be included from frontend code" > > How about this? Why is that happening? The same -- as you can see it comes from compiling pg_numa.c, which as

Re: Changing shared_buffers without restart

2025-04-09 Thread Dmitry Dolgov
> On Wed, Apr 09, 2025 at 11:12:18AM GMT, Ashutosh Bapat wrote: > On Mon, Apr 7, 2025 at 2:13 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > In the new v4 version > > of the patch the first option is implemented. > > > > The patches don't a

Re: Changing shared_buffers without restart

2025-04-05 Thread Dmitry Dolgov
> On Thu, Mar 20, 2025 at 04:55:47PM GMT, Ni Ku wrote: > > I ran some simple tests (outside of PG) on linux kernel v6.1, which has > this commit that added some hugepage support to mremap ( > https://patchwork.kernel.org/project/linux-mm/patch/20211013195825.3058275-1-almasrym...@google.com/ > ). >

Re: System views for versions reporting

2025-04-01 Thread Dmitry Dolgov
> On Sun, Mar 23, 2025 at 06:21:33PM GMT, Tom Lane wrote: > > FWIW, I think the 0004 patch is about to be mostly obsoleted by > Andrei's proposal at [1]. To the extent that it's not obsoleted, > I question whether it's something we want at all, given the ground > rule that unprivileged users are n

Re: Squash constant lists in query jumbling by default

2025-03-25 Thread Dmitry Dolgov
On Tue, Mar 25, 2025, 7:40 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > >>> On Tue, Mar 25, 2025, 6:28 PM Álvaro Herrera > wrote: > > On 2025-Mar-25, Tom Lane wrote: > > > If this GUC sticks around, it should be at least PGC_SUSET (on > > > the a

Re: Squash constant lists in query jumbling by default

2025-03-25 Thread Dmitry Dolgov
> > >> On Tue, Mar 25, 2025, 6:28 PM Álvaro Herrera wrote: On 2025-Mar-25, Tom Lane wrote: > If this GUC sticks around, it should be at least PGC_SUSET (on > the analogy of compute_query_id) to make it harder to break > pg_stat_statements that way. I have no problem making it superuser-only,

Re: Changing shared_buffers without restart

2025-03-21 Thread Dmitry Dolgov
> On Fri, Mar 21, 2025 at 04:48:30PM GMT, Ni Ku wrote: > Thanks for your insights and confirmation, Dmitry. > Right, I think the anonymous fd approach would work to keep the memory > contents intact in between munmap and mmap with the new size, so bufferpool > expansion would work.

Re: pg_stat_statements and "IN" conditions

2025-03-19 Thread Dmitry Dolgov
> On Tue, Mar 18, 2025 at 02:54:18PM GMT, Sami Imseih wrote: > I want to mention that the current patch does not deal > with external parameters ( prepared statements ) [0] [1]. This could be a > follow-up, as it may need some further discussion. It is important to > address this case, IMO. Sure,

Re: pg_stat_statements and "IN" conditions

2025-03-19 Thread Dmitry Dolgov
> On Tue, Mar 18, 2025 at 07:33:25PM GMT, Álvaro Herrera wrote: > > By the way, I'm still open to adding the 'powers' mechanism that was > discussed earlier and other simple additions on top of what's there now, > if you have some spare energy to spend on this. (For full disclosure: > the "powers"

Re: pg_stat_statements and "IN" conditions

2025-03-18 Thread Dmitry Dolgov
> On Mon, Mar 17, 2025 at 08:14:16PM GMT, Álvaro Herrera wrote: > On 2025-Mar-17, Álvaro Herrera wrote: > > > You can see my patch on top of yours here: > > https://github.com/alvherre/postgres/commits/query_id_squash_values/ > > and the CI run here: > > https://cirrus-ci.com/build/5660053472018432

Re: pg_stat_statements and "IN" conditions

2025-03-17 Thread Dmitry Dolgov
> On Mon, Mar 17, 2025 at 12:07:44PM GMT, vignesh C wrote: > > I noticed that the feedback from Sami at [1] has not yet been > addressed, I have changed the status to Waiting on Author, kindly > address them and update the status to Needs review. > [1] - > https://www.postgresql.org/message-id/CAA

Re: pg_stat_statements and "IN" conditions

2025-03-03 Thread Dmitry Dolgov
> On Mon, Mar 03, 2025 at 12:56:24PM GMT, Álvaro Herrera wrote: > In the meantime, here's v28 which is Dmitry's v27 plus pgindent. No > other changes. Dmitry, were you planning to submit a new version? Nope, there was nothing I wanted to add so far.

Re: Changing shared_buffers without restart

2025-02-27 Thread Dmitry Dolgov
> On Tue, Feb 25, 2025 at 10:52:05AM GMT, Dmitry Dolgov wrote: > > On Fri, Oct 18, 2024 at 09:21:19PM GMT, Dmitry Dolgov wrote: > > TL;DR A PoC for changing shared_buffers without PostgreSQL restart, via > > changing shared memory mapping layout. Any feedback is appreciated.

Re: Changing shared_buffers without restart

2025-02-25 Thread Dmitry Dolgov
> On Fri, Oct 18, 2024 at 09:21:19PM GMT, Dmitry Dolgov wrote: > TL;DR A PoC for changing shared_buffers without PostgreSQL restart, via > changing shared memory mapping layout. Any feedback is appreciated. Hi, Here is a new version of the patch, which contains a proposal about how to c

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2025-02-21 Thread Dmitry Dolgov
> On Mon, Nov 25, 2024 at 11:20:05AM GMT, Ashutosh Bapat wrote: > > I've tried to reproduce some subset of those results, in case if I would > > be able to notice anything useful. Strangely enough, I wasn't able to > > get much boost in planning time e.g. with 4 first patches, 100 > > partitions an

Assert when executing query on partitioned table

2025-02-20 Thread Dmitry Koval
ng. --- I suggest replace Assert with an error message, see [v1-0001-draft-of-fix.patch]. This is not a final fix as I am confused by the comment for Assert: "we don't support an UPDATE of INSERT ON CONFLICT for a partitioned table". (Why "don't support an UPDATE"?

Re: pg_stat_statements and "IN" conditions

2025-02-18 Thread Dmitry Dolgov
> On Mon, Feb 17, 2025 at 01:50:00PM GMT, Sami Imseih wrote: > > This test was to catch a crash that was happening in older version of > > the patch, so it doesn't have to verify the actual pgss entry. > > It seems odd to keep this test because of crash behavior experienced > in a previous version

Re: pg_stat_statements and "IN" conditions

2025-02-17 Thread Dmitry Dolgov
> On Mon, Feb 17, 2025 at 09:51:32AM GMT, Sami Imseih wrote: > > This should do it. The last patch for today, > > I looked at v27 today and have a few comments. > > 1/ It looks like the CTE test is missing a check for results. This test was to catch a crash that was happening in older version of t

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 09:06:24AM GMT, Sami Imseih wrote: > I think it will be sad to not include this very common case from > the start, because it is going to be one of the most common > cases. > > Wouldn't doing something like this inside IsMergeableConst > """ > if (!IsA(arg, Const) && !IsA(

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 11:12:25PM GMT, Julien Rouhaud wrote: > > > There seems to be an off-by-1 error in parameter numbering when merging > > > them. > > > > There are indeed three constants, but the second is not visible in the > > query text. Maybe makes sense to adjust the number in this ca

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 10:39:45PM GMT, Julien Rouhaud wrote: > There seems to be an off-by-1 error in parameter numbering when merging them. There are indeed three constants, but the second is not visible in the query text. Maybe makes sense to adjust the number in this case, let me try. > Not

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 05:26:19AM GMT, Sami Imseih wrote: > > > On Thu, Feb 13, 2025 at 05:08:45PM GMT, Sami Imseih wrote: > > > Constants passed as parameters to a prepared statement will not be > > > handled as expected. I did not not test explicit PREPARE/EXECUTE > > > statement, > > > but I

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 05:57:01PM GMT, Julien Rouhaud wrote: > On Fri, Feb 14, 2025 at 10:36:48AM +0100, Álvaro Herrera wrote: > > On 2025-Feb-14, Julien Rouhaud wrote: > > > > > Since the merging is a yes/no option (I think there used to be some > > > discussions > > > about having a threshold

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Thu, Feb 13, 2025 at 05:08:45PM GMT, Sami Imseih wrote: > Constants passed as parameters to a prepared statement will not be > handled as expected. I did not not test explicit PREPARE/EXECUTE statement, > but I assume it will have the same issue. This is the same question of supporting variou

Assignment before assert

2025-02-13 Thread Dmitry Koval
Hi! Function EvalPlanQualFetchRowMark contains an assignment ExecRowMark *erm = earm->rowmark; before assert Assert(earm != NULL); Maybe these lines need to be swapped? -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.com diff --git a/src/back

Re: pg_stat_statements and "IN" conditions

2025-02-13 Thread Dmitry Dolgov
> On Thu, Feb 13, 2025 at 01:47:01PM GMT, Álvaro Herrera wrote: > Also, how wed are you to > "query_id_merge_values" as a name? It's not in any way obvious that > this is about values in arrays. How about query_id_squash_arrays? Or > are you thinking in having values in other types of structures

Re: pg_stat_statements and "IN" conditions

2025-02-13 Thread Dmitry Dolgov
> On Wed, Feb 12, 2025 at 08:48:03PM GMT, Dmitry Dolgov wrote: > > On Wed, Feb 12, 2025 at 07:39:39PM GMT, Álvaro Herrera wrote: > > The nastiness level of this seems quite low, compared to what happens to > > this other example if we didn't handle these easy cases: >

Re: pg_stat_statements and "IN" conditions

2025-02-12 Thread Dmitry Dolgov
> On Wed, Feb 12, 2025 at 07:39:39PM GMT, Álvaro Herrera wrote: > The nastiness level of this seems quite low, compared to what happens to > this other example if we didn't handle these easy cases: > > create table t (a float); > select i from t where i in (1, 2); > select i from t where i in (1, '

Re: pg_stat_statements and "IN" conditions

2025-02-12 Thread Dmitry Dolgov
> On Tue, Feb 11, 2025 at 08:00:27PM GMT, Dmitry Dolgov wrote: > > Hmm, what about doing something much simpler, such as testing whether > > there's just a CoerceViaIO/RelabelType around a Const or a one-parameter > > function call of an immutable boostrap-OID fu

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Dmitry Dolgov
> On Tue, Feb 11, 2025 at 07:51:43PM GMT, Álvaro Herrera wrote: > On 2025-Feb-11, Dmitry Dolgov wrote: > > > > On Tue, Feb 11, 2025 at 10:49:59AM GMT, Sami Imseih wrote: > > > I have only looked at 0001, but I am wondering why > > > query_id_const_merge is a

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Dmitry Dolgov
;2', '3', '4'); > and so on. There are no casts here because these literals are of type > unknown. > > I suppose this is telling us that detecting the case with consts wrapped > in casts is not really optional. (Dmitry said this was supported at > early stag

Re: Increased work_mem for "logical replication tablesync worker" only?

2025-02-04 Thread Dmitry Koterov
waits for COPY to finish, and only then moves that temp files to data area, because of the large transaction or something. Would be cool to understand the reason. On Mon, Feb 3, 2025 at 22:31 Amit Kapila wrote: > On Mon, Feb 3, 2025 at 5:51 PM Dmitry Koterov > wrote: > > > > H

Re: Increased work_mem for "logical replication tablesync worker" only?

2025-02-03 Thread Dmitry Koterov
y on the destination they are different, right after the copying. (There are not a lot of writes to this table happening on the source happening while copying BTW.) Maybe that could hint on why temp files are used? On Mon, Feb 3, 2025 at 4:21 AM Dmitry Koterov wrote: > Here is the list of tmp

Re: Increased work_mem for "logical replication tablesync worker" only?

2025-02-03 Thread Dmitry Koterov
/main/base/4 2.3M17/main/base/5 12G 17/main/base/pgsql_tmp 6.3G17/main/pg_wal So the question, why does it use temp files. Why not just writes directly to WAL+data. On Mon, Feb 3, 2025 at 3:04 AM Dmitry Koterov wrote: > Hi. > > 1. Those are temp files on the destination n

Re: Increased work_mem for "logical replication tablesync worker" only?

2025-02-03 Thread Dmitry Koterov
when copying) definitely helps with temp files. Thanks! On Sun, Feb 2, 2025 at 19:10 Amit Kapila wrote: > On Sun, Feb 2, 2025 at 5:13 PM Dmitry Koterov > wrote: > > > > Trying to monitor perf during the initial tablesync phase (COPY) right > after CREATE SUBSCRIPTION. I not

Increased work_mem for "logical replication tablesync worker" only?

2025-02-02 Thread Dmitry Koterov
Hi. Trying to monitor perf during the initial tablesync phase (COPY) right after CREATE SUBSCRIPTION. I noticed that the size of 17/main/base/pgsql_tmp on the destination node grows (tens of gigabytes) as the COPY command (running internally on the publisher) progresses. Then in the end (when its

hash_search_with_hash_value is high in "perf top" on a replica

2025-01-31 Thread Dmitry Koterov
Hi. Debugging some replication lag on a replica when the master node experiences heavy writes. PG "startup recovering" eats up a lot of CPU (like 65 %user and 30 %sys), which is a little surprising (what is it doing with all those CPU cycles? it looked like WAL replay should be more IO bound than

Re: System views for versions reporting

2025-01-25 Thread Dmitry Dolgov
> On Thu, Jan 02, 2025 at 10:36:48AM GMT, jian he wrote: > hi. > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/cf%2F5318 > shows lots of failures, but it doesn't seem to tell you about doc build > failure. Thanks for checking this out. Here is the updated version with applied changes.

Re: Invalid index on partitioned table - is this a bug or feature?

2025-01-22 Thread Dmitry Koval
22.01.2025 20:57, Álvaro Herrera пишет: Yes. You need to attach child indexes on all partitions so that this index becomes valid. Thanks! -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.com

Invalid index on partitioned table - is this a bug or feature?

2025-01-22 Thread Dmitry Koval
eyIndex() returns InvalidOid for INVALID PK. Is this a bug or a feature? Should this be corrected? Links. [1] https://github.com/postgres/postgres/blob/master/src/backend/commands/indexcmds.c#L1211 -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.com

Re: Fwd: Re: proposal: schema variables

2025-01-17 Thread Dmitry Dolgov
> On Fri, Jan 17, 2025 at 11:01:41AM GMT, Bruce Momjian wrote: > On Fri, Jan 17, 2025 at 04:55:07PM +0100, Pavel Stehule wrote: > > pá 17. 1. 2025 v 16:35 odesílatel Bruce Momjian napsal: > > > > So this feature would be like global GUC variables, with permission > > control? > > > > + typ

Re: pg_stat_statements and "IN" conditions

2024-12-03 Thread Dmitry Dolgov
> On Thu, Nov 28, 2024 at 08:36:47PM GMT, Kirill Reshke wrote: > > Hi! Can you please send a rebased version of this? Sure, here it is. >From 2de1af6489d46449b2884a9194515cd1090d5e8c Mon Sep 17 00:00:00 2001 From: Dmitrii Dolgov <9erthali...@gmail.com> Date: Tue, 3 Dec 2024 14:55:45 +0100 Subject:

Re: Changing shared_buffers without restart

2024-12-02 Thread Dmitry Dolgov
> On Fri, Nov 29, 2024 at 05:47:27PM GMT, Dmitry Dolgov wrote: > > On Fri, Nov 29, 2024 at 01:56:30AM GMT, Matthias van de Meent wrote: > > > > I mean, we can do the following to get a nice contiguous empty address > > space no other mmap(NULL)s will get put into: > &

Re: Consider the number of columns in the sort cost model

2024-12-02 Thread Dmitry Dolgov
Hi folks, Just wanted to mention, that looks like some CFBot test are failing, something around level_tracking in pgss.

Re: Partition-wise join with whole row vars

2024-12-02 Thread Dmitry Dolgov
> On Tue, Oct 08, 2024 at 09:24:15AM GMT, Alexander Pyhalov wrote: > > Attaching rebased patches. Just to let you know, looks like CFBot tests are red again, but this time there are some unexpected differences in some test query plan.

Re: CRC32C Parallel Computation Optimization on ARM

2024-12-01 Thread Dmitry Dolgov
> On Mon, Dec 04, 2023 at 10:18:09PM -0600, Nathan Bossart wrote: > > Thanks for the new patch. I am hoping to spend much more time on this in > the near future... Hi, The patch looks interesting, having around 8% improvement on that sounds attractive. Nathan, do you plan to come back to it and

Unclear code - please elaborate

2024-11-30 Thread Dmitry Nikitin
. Nothing wrong here? Either zero is simply not possible after that bms_next_member() because of some factors behind the code? -- Best regards, Dmitry mailto:pgsql-hack...@dima.nikitin.name

Re: Changing shared_buffers without restart

2024-11-29 Thread Dmitry Dolgov
> On Fri, Nov 29, 2024 at 01:56:30AM GMT, Matthias van de Meent wrote: > > I mean, we can do the following to get a nice contiguous empty address > space no other mmap(NULL)s will get put into: > > /* reserve size bytes of memory */ > base = mmap(NULL, size, PROT_NONE, ...flags, ...); >

Re: Changing shared_buffers without restart

2024-11-28 Thread Dmitry Dolgov
> On Thu, Nov 28, 2024 at 12:18:54PM GMT, Robert Haas wrote: > > All that having been said, what does concern me a bit is our ability > to predict what Linux will do well enough to keep what we're doing > safe; and also whether the Linux behavior might abruptly change in the > future. Users would b

Re: Changing shared_buffers without restart

2024-11-28 Thread Dmitry Dolgov
> On Wed, Nov 27, 2024 at 04:05:47PM GMT, Robert Haas wrote: > On Wed, Nov 27, 2024 at 3:48 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > My understanding is that clashing of mappings (either at creation time > > or when resizing) could happen only withing the proces

Re: Changing shared_buffers without restart

2024-11-27 Thread Dmitry Dolgov
> On Wed, Nov 27, 2024 at 10:20:27AM GMT, Robert Haas wrote: > > > > > > code, but I'm not sure exactly which points are safe. If we have no > > > code anywhere that assumes the address of an unpinned buffer can't > > > change before we pin it, then I guess the check for pins is the only > > > thin

Re: [PoC] Reducing planning time when tables have many partitions

2024-11-27 Thread Dmitry Dolgov
> On Tue, Oct 15, 2024 at 12:20:04PM GMT, Yuya Watari wrote: > > The previous patches do not apply to the current master, so I have > attached the rebased version. Thanks for keeping it up to date. > v25-0001 > This patch is one of the main parts of my optimization. Traditionally, > EquivalenceCl

Re: [PATCH] Missing Assert in the code

2024-11-27 Thread Dmitry Nikitin
r sometimes that "business-logic" changes later on... So this is a moment when debug asserts come to play and save a lot of time and sane. Although I don't mind of elog/whatever. I'm just not that familiar with the "business-logic" to make an appropriate choice. -- Best

Re: [PATCH] Missing Assert in the code

2024-11-27 Thread Dmitry Nikitin
gt; large bunch. Mostly it used in loops. So some code will be gracefully skipped and that's it. However the case discussed is different because -1 index of an array will be accessed. Which is much worse than a bare assertion at least. -- Best regards, Dmitrymailto:pgsql-hack...@dima.nikitin.name

Re: Changing shared_buffers without restart

2024-11-26 Thread Dmitry Dolgov
> On Mon, Nov 25, 2024 at 02:33:48PM GMT, Robert Haas wrote: > > I think the idea of having multiple shared memory segments is > interesting and makes sense, but I would prefer to see them called > "segments" rather than "slots" just as do we do for DSMs. The name > "slot" is somewhat overused, and

[PATCH] Missing Assert in the code

2024-11-25 Thread Dmitry Nikitin
Hello , The patch is pretty trivial. -- Best regards, Dmitry mailto:pgsql-hack...@dima.nikitin.nameFrom 0ff6a234740d7a7d2bb9572271a1dbdfd4f45f39 Mon Sep 17 00:00:00 2001 From: Dmitry Nikitin Date: Mon, 25 Nov 2024 10:31:53 +0300 Subject: [PATCH 1/2] Get rid off the

Re: Reducing memory consumed by RestrictInfo list translations in partitionwise join planning

2024-11-24 Thread Dmitry Dolgov
> On Thu, Oct 10, 2024 at 05:36:10PM GMT, Ashutosh Bapat wrote: > > 3. With just patch 0001 applied, planning time usually shows > degradation (column Q and X in planning time sheets) with or without > PWJ enabled. I first thought that it might be because of the increased > size of PlannerInfo. We

Re: Extract numeric filed in JSONB more effectively

2024-11-22 Thread Dmitry Dolgov
> On Mon, Nov 18, 2024 at 08:23:52AM GMT, Andy Fan wrote: > > >> > I imagined you'd the patch should create a SupportRequestSimplify > >> > support function for jsonb_numeric() that checks if the input > >> > expression is an OpExpr with funcid of jsonb_object_field(). All you > >> > do then is di

  1   2   3   4   5   6   7   8   9   >