Re: Introduce some randomness to autovacuum

2025-04-30 Thread Sami Imseih
oblems, this will not solve the "spinning" problem because if the most common reason for this is a long-running transaction, etc., all your tables will eventually end up with wasted vacuum cycles because the xmin horizon is not advancing. -- Sami Imseih

Improve explicit cursor handling in pg_stat_statements

2025-04-30 Thread Sami Imseih
also was not happy with the approach I took in [0]. Looking forward to feedback! Regards, -- Sami Imseih Amazon Web Services (AWS) [0] https://www.postgresql.org/message-id/flat/203CFCF7-176E-4AFC-A48E-B2CECFECD6AA%40amazon.com v1-0001-Improve-cursor-handling-in-pg_stat_statements.patch Descript

queryId constant squashing does not support prepared statements

2025-04-30 Thread Sami Imseih
ich will be very disappointing. Thoughts? Sami Imseih Amazon Web Services (AWS) v1-0001-Allow-query-jumble-to-squash-a-list-external-para.patch Description: Binary data

Re: stats.sql fails during installcheck on mac

2025-04-10 Thread Sami Imseih
o be tightened SELECT sum(writes) AS writes, sum(fsyncs) AS fsyncs FROM pg_stat_io WHERE context = 'normal' AND object = 'wal' \gset io_sum_wal_normal_after_ -- Sami Imseih Amazon Web Services (AWS)

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-10 Thread Sami Imseih
> On Fri, 11 Apr 2025 at 02:01, Sami Imseih wrote: > > I created an entry for the July CF > > https://commitfest.postgresql.org/patch/5691/ > > > > ... and I realized I forgot to include David's code comment patch yesterday, > > Reattaching both patches.

Re: Conflicting updates of command progress

2025-04-14 Thread Sami Imseih
herefore, only the top level command is updating progress. what do you think? [1] https://commitfest.postgresql.org/patch/5117/ -- Sami Imseih Amazon Web Services (AWS)

Re: POC: Parallel processing of indexes in autovacuum

2025-05-02 Thread Sami Imseih
in thread[1], it would > also likely be beneficial to incorporate it into autovacuum during > aggressive vacuum or failsafe mode. IIRC, index cleanup is disabled by failsafe. -- Sami Imseih Amazon Web Services (AWS)

Re: POC: Parallel processing of indexes in autovacuum

2025-05-02 Thread Sami Imseih
> On Fri, May 2, 2025 at 11:58 PM Sami Imseih wrote: > > > > I am generally -1 on the idea of autovacuum performing parallel > > index vacuum, because I always felt that the parallel option should > > be employed in a targeted manner for a specific table. if you have

Re: Improve explicit cursor handling in pg_stat_statements

2025-05-02 Thread Sami Imseih
> Hmm. What are the workloads that you have seen as problematic? Do > these involve cursor names generated randomly, where most of them are > similar with a random factor for the name? postgres_fdw, as an example, in which cursor name get reused for different queries. Notice below "c1" and "c2"

Re: POC: Parallel processing of indexes in autovacuum

2025-05-05 Thread Sami Imseih
> On Sat, May 3, 2025 at 1:10 AM Daniil Davydov <3daniss...@gmail.com> > wrote: > > > > On Sat, May 3, 2025 at 5:28 AM Masahiko Sawada > wrote: > > > > > > > In current implementation, the leader process sends a signal to the > > > > a/v launcher, and the launcher tries to launch all requested wor

Re: queryId constant squashing does not support prepared statements

2025-05-06 Thread Sami Imseih
| calls --+-+--- t| select where $1 in ($2 /*, ... */) and $3 in ($4 /*, ... */($5 as int)) | 1 (1 row) """ Without properly accounting for the boundaries of the list of expressions, i.e., the star

Re: Conflicting updates of command progress

2025-04-23 Thread Sami Imseih
d by a top-level command, such as the COPY case above. 2/ a top-level command triggered some other progress code implicitly, such as CLUSTER triggering CREATE INDEX code. I also like the shared memory approach because we can then not have to use a message like the one introduced in f1889729

Re: Conflicting updates of command progress

2025-04-24 Thread Sami Imseih
> pgstat_progress_start_command() is called twice: First with > cmdtype=PROGRESS_COMMAND_CLUSTER, second with > PROGRESS_COMMAND_CREATE_INDEX. The first happens in cluster_rel() the second > in cluster_rel() -> rebuild_relation() -> finish_heap_swap() -> > reindex_relation() -> reindex_index(). > >

Re: [BUG] temporary file usage report with extended protocol and unnamed portals

2025-04-25 Thread Sami Imseih
Thanks for testing. I also tested it a bit more today with other patterns like different fetch sizes, named portal, etc. and I can't find an issue with this, but I could be missing something. I will go ahead and attach this change in patch form. -- Sami Imseih Amazon Web Services (AWS)

stats.sql fails during installcheck on mac

2025-04-10 Thread Sami Imseih
emented when the wal_sync_method is either fdatasync, fsync or fsync_writethrough". Perhaps, the same clarification will be beneficial for the pg_stat_io.fsyncs* fields? [0] https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-WAL-VIEW [1] https://github

Re: track generic and custom plans in pg_stat_statements

2025-04-10 Thread Sami Imseih
e explain, and we may not even need it at all since generic plans are already displayed with $1, $2 parameters. Let me know if you have other comments for v5, the pg_stat_statements enhancements. — Sami Imseih Amazon Web Services (AWS)

Re: track generic and custom plans in pg_stat_statements

2025-04-10 Thread Sami Imseih
rebased in the attached v5. -- Sami Imseih Amazon Web Services (AWS) v5-0001-Add-plan_cache-counters-to-pg_stat_statements.patch Description: Binary data

Re: Disallow redundant indexes

2025-04-28 Thread Sami Imseih
tect these ( besides a query in the Wiki ), but I don't think we should prevent it. While a WARNING will be a good to have, it could easily go unnoticed, but it's still good to have. I also think we should either provide a psql shortcut to detect these indexes or to add annotation to an index in the \d command, or perhaps both of these things. -- Sami Imseih Amazon Web Services (AWS)

Re: Disallow redundant indexes

2025-04-29 Thread Sami Imseih
> On Mon, Apr 28, 2025 at 01:12:18PM -0500, Sami Imseih wrote: > > If I had a dollar for every time I asked a user to check for duplicate > > indexes :) So > > I think it's a good idea to provide some way to detect these ( besides > > a query in the > > Wiki

Re: POC: Parallel processing of indexes in autovacuum

2025-05-02 Thread Sami Imseih
> I think it would more make > sense to maintain the existing autovacuum_max_workers parameter while > introducing a new parameter that would either control the maximum > number of parallel vacuum workers per autovacuum worker or set a > system-wide cap on the total number of parallel vacuum worker

Re: queryId constant squashing does not support prepared statements

2025-05-01 Thread Sami Imseih
I spent a few hours looking into this today and to your points below: > > I agree that the current solution we have in the tree feels incomplete > > because we are not taking into account the most common cases that > > users would care about. Now, allowing PARAM_EXTERN means that we > > allow any

Re: [BUG] temporary file usage report with extended protocol and unnamed portals

2025-04-19 Thread Sami Imseih
CreatePortal call inside exec_bind_message; that seems incorrect. So, I believe we should temporarily switch the debug_query_string value only while running PortalDrop. Attached is what I think could be safer to do. What do you think? -- Sami Imseih Amazon Web Services (AWS) v2-0001-Fix-race

Re: [BUG] temporary file usage report with extended protocol and unnamed portals

2025-04-18 Thread Sami Imseih
care of the problem because it ensures that the drop portal, when it occurs, is referencing the correct sql. I am not yet sure if the patch the right solution yet, it maybe the best answer. I don't have a better answer, but wanted to share this research as well. -- Sami Imseih Amazon Web Services (AWS) Test.java Description: Binary data

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-18 Thread Sami Imseih
On Thu, Apr 17, 2025 at 11:13 PM David Rowley wrote: > > On Sat, 12 Apr 2025 at 07:33, Sami Imseih wrote: > > What do you think of the attached? > > I looked at the v3 patch and I'm having trouble getting excited about it. > > I'd say this part is

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-18 Thread Sami Imseih
> I can see it being confusing. What about this wording to make it more > clear when the field is > updated? here are both of the changes in v4. -- Sami Imseih Amazon Web Services (AWS) v4-0001-Clarify-when-aborted-rows-are-tracked-for-tuple-r.patch Description: Binary data

Re: pg_stat_statements and "IN" conditions

2025-02-17 Thread Sami Imseih
> This should do it. The last patch for today, I looked at v27 today and have a few comments. 1/ It looks like the CTE test is missing a check for results. """ -- Test constants evaluation in a CTE, which was causing issues in the past WITH cte AS ( SELECT 'const' as const FROM test_merge ) SELE

Re: pg_stat_statements and "IN" conditions

2025-02-18 Thread Sami Imseih
> > > This test was to catch a crash that was happening in older version of > > > the patch, so it doesn't have to verify the actual pgss entry. > > > > It seems odd to keep this test because of crash behavior experienced > > in a previous version of the patch. if the crash reason was understood >

Re: pg_stat_statements and "IN" conditions

2025-02-17 Thread Sami Imseih
> This test was to catch a crash that was happening in older version of > the patch, so it doesn't have to verify the actual pgss entry. It seems odd to keep this test because of crash behavior experienced in a previous version of the patch. if the crash reason was understood and resolved, why kee

Re: Proposal - Allow extensions to set a Plan Identifier

2025-02-17 Thread Sami Imseih
On Sun, Feb 16, 2025 at 5:34 PM Michael Paquier wrote: > > On Sat, Feb 15, 2025 at 10:29:41AM +0100, Andrei Lepikhov wrote: > > I have already implemented it twice in different ways as a core patch. > > In my projects, we need to track queryId and plan node ID for two reasons: > > Are these availa

Re: making EXPLAIN extensible

2025-02-28 Thread Sami Imseih
discussion [0] for showing FDW remote plans ( postgres_fdw specifically), and I think we will need to add some new options to EXPLAIN to make that possible. Have not looked at your patches, but I will do so now. Regards, Sami Imseih Amazon Web Services (AWS) [0] https://www.postgresql.

Re: Should work_mem be stable for a prepared statement?

2025-02-28 Thread Sami Imseih
> The argument for treating work_mem specially is that it has effects at > both plan time and run time, so that the planner's cost assumptions > are invalidated if the executor uses a different value than the > planner did. I see that now. Thanks! > Maybe that refactoring is one that would conven

Re: Should work_mem be stable for a prepared statement?

2025-02-28 Thread Sami Imseih
d to turn that off in a hurry when it impacts their oltp workload. -- Sami Imseih Amazon Web Services (AWS)

Re: Doc: clarify possibility of ephemeral discrepancies between state and wait_event in pg_stat_activity

2025-03-03 Thread Sami Imseih
Thanks for the update. This LGTM! and I will mark as RFC. -- Sami

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-06 Thread Sami Imseih
lse { JUMBLE_NODE(limitOffset); } if (!expr->limitCount) { JUMBLE_NULL(); } else { JUMBLE_NODE(limitCount); } """ What do you think? Maybe someone can suggest a better

Re: track generic and custom plans in pg_stat_statements

2025-03-05 Thread Sami Imseih
> > Please see v2 > oops, had to fix a typo in meson.build. Please see v3. -- Sami v3-0001-add-plan_cache-counters-to-pg_stat_statements.patch Description: Binary data

Re: track generic and custom plans in pg_stat_statements

2025-03-05 Thread Sami Imseih
> + Total number of non-utility statements executed using a generic plan > > I'm not sure we need to specify non-utility here. > fair, I did not have strong feeling about this either. Please see v2 Thanks! -- Sami Imseih Amazon Web Services (AWS) v2-0001-add-plan_cache-counters-to-pg_stat_statements.patch Description: Binary data

Re: track generic and custom plans in pg_stat_statements

2025-03-10 Thread Sami Imseih
> I don't quite understand why do we need to differentiate between > PLAN_CACHE_STATUS_GENERIC_PLAN_BUILD and > PLAN_CACHE_STATUS_GENERIC_PLAN_REUSE? > We could simply keep PLAN_CACHE_STATUS_GENERIC_PLAN_REUSE. > I don't think users would see much of a difference in either > pg_stat_statements or

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-11 Thread Sami Imseih
> It seems to me that if this fixes the issue, that the next similar one > is already lurking in the shadows waiting to jump out on us. Yes, this is true that there may be other cases, but I am not sure if it's worth carrying all the extra bytes in the jumble to deal with a few cases like this. Th

Re: Should work_mem be stable for a prepared statement?

2025-02-28 Thread Sami Imseih
> I'm also not convinced that the argument requires us to solve > the problem by re-planning. It would work just as well to stamp > each PlannedStmt with the value that the planner used and > refer to that in the executor instead of looking directly at the GUC. hmm, if work_mem influences the pla

Re: Doc: clarify possibility of ephemeral discrepancies between state and wait_event in pg_stat_activity

2025-02-27 Thread Sami Imseih
as the below: """ To keep the reporting overhead low, the system does not attempt to synchronize activity data for a backend. As a result, ephemeral discrepancies may exist between the view’s columns. """ -- Sami Imseih Amazon Web Services (AWS)

Re: making EXPLAIN extensible

2025-03-13 Thread Sami Imseih
E_PLANS and ANALYZE cannot be used together postgres=# explain (analyze, remote_plans) select * from t_r1; ERROR: EXPLAIN options REMOTE_PLANS and ANALYZE cannot be used together Regards, Sami From d4350df06cfeb0d9d5d7fe99b898c1a7ef237c97 Mon Sep 17 00:00:00 2001 From: Sami Imseih Date: Thu, 1

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread Sami Imseih
ery jumbling is weightless compared to > the overall query processing yeah, that is a good point. At least benchmarking the above on a SELECT only pgbench did not reveal any regression. -- Sami Imseih Amazon Web Services (AWS)

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread Sami Imseih
> Then we have the same problem with another Node using this Foo node > two times in a row, depending on how it's used by the query parsing > and transform: hmm, it's hard to imagine such a case being real-world and useful for query jumbling purposes. Also, I think if we introduce something like J

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread Sami Imseih
> FWIW, another idea I have on top of my mind is the addition of a > counter in JumbleState that we increment each time we enter > _jumbleNode(), then simply call JUMBLE_FIELD_SINGLE() after the > incrementation. And we can rely on this counter to be unique each > time we jumble a node.. With thi

should num_custom_plans be reset after plan invalidation?

2025-03-05 Thread Sami Imseih
a plan cache invalidation might be a better approach. I've attached an example for reference. The fix seems straightforward, but since generic plans may already not handle skewed data optimally, I want to see if others have thoughts on this being something to fix. -- Sami Imseih Amazo

track generic and custom plans in pg_stat_statements

2025-03-05 Thread Sami Imseih
information can be added to EXPLAIN output and perhaps pg_stat_database. Maybe that's a good idea also? This patch bumps the version pf pg_stat_statements. -- Sami Imseih Amazon Web Services (AWS) [0] https://www.postgresql.org/message-id/add1e591fbe8874107e75d04328859ec%40oss.nttdata.com

Re: explain plans for foreign servers

2025-03-05 Thread Sami Imseih
have any > mechanism for addressing that at al FWIW, I had the same thought [0] and planned on doing the investigation. [0] https://www.postgresql.org/message-id/CAA5RZ0tLrNOw-OgPkv49kbNmZS4nFn9vzpN5HXX_xvOaM9%3D5ww%40mail.gmail.com -- Sami Imseih

Re: should num_custom_plans be reset after plan invalidation?

2025-03-05 Thread Sami Imseih
> Yes, it is. There's little reason to expect that the invalidation > would change our decision, and re-planning five times to confirm that > is a high price to pay. Re-planning 5 times did not sound appealing to me either, but I was on the fence as to what the proper behavior should be. Thanks

Re: pg_stat_statements and "IN" conditions

2025-03-03 Thread Sami Imseih
> > > It's not a question about whether it's possible to implement this, > > > but about whether it makes sense. In case of plain constants it's > > > straightforward -- they will not change anything meaningfully and > > > hence could be squashed from the query. Now for a function, that > > > might

Re: making EXPLAIN extensible

2025-03-12 Thread Sami Imseih
ary : es->analyze; What do you think? [0] https://www.postgresql.org/message-id/flat/CAP%2BB4TD%3Diy-C2EnsrJgjpwSc7_4pd3Xh-gFzA0bwsw3q8u860g%40mail.gmail.com [1] https://github.com/postgres/postgres/blob/master/src/backend/commands/explain.c#L2013 Thanks -- Sami Imseih Amazon Web Services (AWS)

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-11 Thread Sami Imseih
> and simple. However, Sami seems to have concerns about the overhead of > doing this. Is that warranted at all? Potentially, there could be a > decent number of NULL fields. It'll probably be much cheaper than the > offsetof idea I came up with. I have not benchmarked the overhead, so maybe there

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-11 Thread Sami Imseih
> transformation. For these reasons, variant A where we put the > LimitOption between the two int8 expression nodes feels like the > "okay" approach here. But we must document this expectation in the > structure, and check for more grammar variants of LIMIT and OFFSET > clauses in pgss. Please s

Re: making EXPLAIN extensible

2025-03-12 Thread Sami Imseih
ext formats, which is not desirable behavior. I get that also. I have no strong feelings for this, but wanted to see what others think. thanks! -- Sami Imseih

Re: pg_stat_statements and "IN" conditions

2025-02-13 Thread Sami Imseih
Hi, Thanks for the updated patch! I spent some time looking at v24 today, and I have some findings/comments. 1/ Constants passed as parameters to a prepared statement will not be handled as expected. I did not not test explicit PREPARE/EXECUTE statement, but I assume it will have the same issue.

Re: Proposal - Allow extensions to set a Plan Identifier

2025-02-13 Thread Sami Imseih
Thanks for the feedback! > I think that makes sense and then the idea would be later on to move to > something > like 5fd9dfa5f50, but for the "planId": is my understanding correct? correct. This is adding infrastructure to eventually have an in-core planId; but in the meanwhile give extensions

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-02-13 Thread Sami Imseih
> I don't understand why we would change any naming here at all. I think > you should be looking at a much broader consensus and plus-ones that a > renaming is needed. -1 from me. The reason for the change is because "query jumble" will no longer make sense if the jumble code can now be used for

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
Yeah that works too and it's probably way easier to implement. +1 Just to throw out an alternate idea using comments. What about adding a comment at the start to the query "/* query_id_squash_values */" and keep the parameter symbols as-is. The comment at the start will indicate that th

Proposal - Allow extensions to set a Plan Identifier

2025-02-12 Thread Sami Imseih
used. However, this is already the case for queryId when compute_query_id is disabled and no extension sets it, so it may not be a concern. Looking forward to feedback on this approach. If there is agreement, I will work on preparing the patches. Regards, Sami Imseih Amazon Web Services (AWS

Re: pg_stat_statements and "IN" conditions

2025-02-12 Thread Sami Imseih
> The nastiness level of this seems quite low, compared to what happens to > this other example if we didn't handle these easy cases: > > create table t (a float); > select i from t where i in (1, 2); > select i from t where i in (1, '2'); > select i from t where i in ('1', 2); > select i from t wh

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > I do see the discussion here [1], sorry for not noticing it. > > > > I am not sure about this though. At minimum this needs to be documented, > > However, I think the prepared statement case is too common of a case to > > skip for the first release of tis feature, and users that will likely >

Re: Sample rate added to pg_stat_statements

2025-02-19 Thread Sami Imseih
> But instead of blindly reducing the frequency via PRNG, we can take a more > thoughtful approach with threshold by execute time: > Find the most frequent query by column 'calls' in pg_stat_statements; > In this query look at info about execution time: min_exec_time, > max_exec_time, etc; > Gra

Re: Sample rate added to pg_stat_statements

2025-02-19 Thread Sami Imseih
I don't see v18 attached. But, I think it's wrong that you changed the design of this patch (sample rate to duration based ) after it was ready for committer. This CF [0] should go back to "Needs Review" if this is the case. -- Sami [0] https://commitfest.postgresql.org/patch/5390/

Re: Sample rate added to pg_stat_statements

2025-02-19 Thread Sami Imseih
> As you can see, the query has not been normalized. Is it a bug, expected or > undefined behavior? No, this behavior is expected. The ability to normalize a statement is only available during post_parse_analyze (pgss_post_parse_analyze), as this is when we have access to JumbleState. In your ex

Re: Psql meta-command conninfo+

2025-02-21 Thread Sami Imseih
the protocol traffic is > going to be if such messages are emitted every time you run a security > invoker function or things like that With the latest version of the patch, 'role' is not needed as 'session authorization' is not shown either [1]. The latest version

Re: Doc fix of aggressive vacuum threshold for multixact members storage

2025-02-21 Thread Sami Imseih
> Maybe we could also add a comment in multixact.c to update the doc > accordingly if > the computation changes? (I think that will be easy to miss). Thanks for the comments! I rather we not touch the .c file for this update. It's unlikely the actual computation will change.

Re: Redact user password on pg_stat_statements

2025-02-21 Thread Sami Imseih
with that either. Regards, Sami Imseih Amazon Web Services (AWS)

Re: Proposal - Allow extensions to set a Plan Identifier

2025-02-20 Thread Sami Imseih
I put together patches to do as is being proposed. v1-0001: 1. Adds a planId field in PlannedStmt 2. Added an st_plan_id fields in PgBackendStatus 3. APIs to report and to retrieve a planId to PgBackendStatus An extension is able to set a planId in PlannedStmt directly, and while they can do the

Re: Psql meta-command conninfo+

2025-02-21 Thread Sami Imseih
would have handled this as the list of PQ parameters was dynamically generated and only those parameters reported by the specific version of the server showed up in \conninfo+. Of course that may have led to confusion in which some versions show role while others did not, but that could be dea

Re: explain plans for foreign servers

2025-02-26 Thread Sami Imseih
ill call the remote sql using this EXPLAIN option and at the end of execution, it will be responsible to fetch the plans from pg_last_explain. I Have not fully formulated this idea, but wanted to share it. Regards, Sami Imseih Amazon Web Services (AWS)

Re: Doc: clarify possibility of ephemeral discrepancies between state and wait_event in pg_stat_activity

2025-02-26 Thread Sami Imseih
we do need to document anything, which I am not convinced we should, it should be more generic. -- Sami Imseih Amazon Web Services (AWS)

Re: Redact user password on pg_stat_statements

2025-02-25 Thread Sami Imseih
> > Well sure, but best effort is better than no effort at all. Preventing > CREATE/ALTER will catch 99% of items, and as I advocated, there really is no > reason for them to be in pg_stat_statements in the first place. > >> >> The clients that set passwords could simply turn off track_utility on

Re: Redact user password on pg_stat_statements

2025-02-25 Thread Sami Imseih
> What about a more general solution, such as a flag to turn off logging of > ALTER ROLE statements completely? IMO, flags for a specific type of utility statement seems way too much for pg_stat_statements, and this will also not completely prevent leaking plain text passwords from all ways that

Re: [PATCH] Re: Proposal to Enable/Disable Index using ALTER INDEX

2025-04-02 Thread Sami Imseih
s patch will provide the ability for the user to create an index as initially invisible and a GUC, use_invisible_index if set to TRUE to experiment with the new index to see if it improves performance. So, I think providing this pattern to experiment with a new index will fit nicely as a new bul

Re: [PATCH] Re: Proposal to Enable/Disable Index using ALTER INDEX

2025-04-02 Thread Sami Imseih
isible indexes as one of the tools for experimentation can be added here. What do you think? [0] https://www.postgresql.org/docs/current/indexes-examine.html -- Sami Imseih Amazon Web Services (AWS)

Re: rename pg_log_standby_snapshot

2025-04-04 Thread Sami Imseih
t; similar to the existing function pg_create_restore_point(). I think we > > need to think about backward compatibility if we agree with moving in > > this direction. > > +1 This is slightly better. pg_create_restore_point also writes to wal and _create_ has a more generic meaning. -- Sami Imseih Amazon Web Services (AWS)

Re: making EXPLAIN extensible

2025-04-04 Thread Sami Imseih
> On Tue, Mar 18, 2025 at 11:21 PM Sami Imseih wrote: > > > > Do you want to propose a patch? > > > > > > yes, will attach a patch shortly. > > > > Attached is a patch to add a hook to allow extensions > > to add additional option validations. Th

Re: pg_stat_statements and "IN" conditions

2025-03-22 Thread Sami Imseih
lanner turns the Param into a Const during eval_const_expressions_mutator. If it's as simple as I think it is, I hope we can get this committed for 18. If not, and a longer discussion is needed, a new thread can be started for this. -- Sami Imseih Amazon Web Services (AWS) v1-0001-Allow-que

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-25 Thread Sami Imseih
ecutions of queries, two apparently-identical queries will be considered the same. However, if the alias for a table is different for semantically similar queries, these queries will be considered distinct" -- Sami Imseih Amazon Web Services (AWS)

Re: New criteria for autovacuum

2025-04-03 Thread Sami Imseih
e. index-only scan with heap fetches or a regular index scan. I think having a counter specifically for heap fetches due to index-only scans could be valuable. -- Sami Imseih Amazon Web Services (AWS)

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-25 Thread Sami Imseih
-cases that rely on the existing behavior. But I do agree that pg_s_s bloat is a big pain point, so this change should be positive overall. Let's see if there are enough complaints to force us to reconsider. -- Sami Imseih Amazon Web Services (AwS)

Re: Proposal - Allow extensions to set a Plan Identifier

2025-04-02 Thread Sami Imseih
> { > + Assert(jstate->clocations); > + Here is v3 with the Assert in place -- Sami Imseih Amazon Web Services (AWS) v3-0001-Allow-plugins-to-Jumble-an-expression.patch Description: Binary data

Re: New criteria for autovacuum

2025-04-04 Thread Sami Imseih
sn't that what anti-wraparound autovac does? or, I may have missed the point here completely [0] https://www.postgresql.org/message-id/CAA5RZ0t1U38qtVAmg3epjh5RBbpT4VRB_Myfp0oGm_73w-UNRA%40mail.gmail.com -- Sami Imseih Amazon Web Services (AWS)

Re: queryId constant squashing does not support prepared statements

2025-05-12 Thread Sami Imseih
> > On Fri, May 09, 2025 at 12:47:19PM GMT, Sami Imseih wrote: > > So, I think we can create a new parse node ( parsenode.h ) that will only be > > used in parsing (and gram.c only ) to track the start/end locations > > and List and > > based on this node we can crea

Re: Prolonged truncation phase during vacuum on toast table with repeated interruptions by lock waiters and a proposed POC patch

2025-05-08 Thread Sami Imseih
> actually determine you are in this situation, or how bad the situation > was, in order to know that setting truncate off would help? To that Adding counters for this area is not a bad idea in general, as this hits customers particularly hard on hot standbys when the truncate does actually occur

Re: queryId constant squashing does not support prepared statements

2025-05-09 Thread Sami Imseih
> > To clarify, I had in mind something like in the attached patch. The > > idea is to make start/end location capturing relatively independent from > > the constants squashing. The new parsing node conveys the location > > information, which is then getting transformed to be a part of an > > Array

Re: queryId constant squashing does not support prepared statements

2025-05-08 Thread Sami Imseih
On Thu, May 8, 2025 at 2:36 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Thu, May 08, 2025 at 02:22:00PM GMT, Michael Paquier wrote: > > On Wed, May 07, 2025 at 10:41:22AM +0200, Dmitry Dolgov wrote: > > > Ah, I see what you mean. I think the idea is fine, it will simplify > > > certain

Re: Prolonged truncation phase during vacuum on toast table with repeated interruptions by lock waiters and a proposed POC patch

2025-05-08 Thread Sami Imseih
even try at all? [0] https://postgr.es/m/Z2DE4lDX4tHqNGZt%40dev.null -- Sami Imseih Amazon Web Services (AWS)

Re: queryId constant squashing does not support prepared statements

2025-05-06 Thread Sami Imseih
query| calls --++--- t | select where $1 in ($2 /*, ... */) | 2 (1 row) ``` What do you think? -- Sami Imseih From 15f1313ef66e964e588b0bf19ede676437ea5a42 Mon Sep 17 00:00:00 2001 From: Sami Imseih Date: Tue, 6 May 2

Re: POC: Parallel processing of indexes in autovacuum

2025-05-06 Thread Sami Imseih
> On Mon, May 5, 2025 at 5:21 PM Sami Imseih wrote: > > > > > >> On Sat, May 3, 2025 at 1:10 AM Daniil Davydov <3daniss...@gmail.com> wrote: > >> > > >> > On Sat, May 3, 2025 at 5:28 AM Masahiko Sawada > >> > wrote: > &g

Re: Possible regression in PG18 beta1

2025-05-17 Thread Sami Imseih
both versions use >> > "Parallel Seq Scan", with PG18 being ~25% faster. (190ms vs. 150ms). I'll be curious about tests with a normal table as well with a sufficiently large shared_buffers. -- Sami Imseih Amazon Web Services (AWS)

Re: Possible regression in PG18 beta1

2025-05-17 Thread Sami Imseih
>> I'll be curious about tests with a normal table as well with a >> sufficiently large shared_buffers. > Here are results for a normal table with default shared_buffers (128 MB) and > large shared_buffers (4GB): thanks. I don't see regression for a normal table, at least for this test. In term

Re: Regression in statement locations

2025-05-20 Thread Sami Imseih
Tested the patch and it looks good to me. Not that I thought it would fail, but I also confirmed the pgaudit case works as expected. ``` LOG: AUDIT: SESSION,10,2,DDL,CREATE TABLE,,,"CREATE TABLE do_table (""weird name"" INT)", LOG: AUDIT: SESSION,10,3,DDL,DROP TABLE,,,DROP table do_table, DO ``

Re: Add comment explaining why queryid is int64 in pg_stat_statements

2025-05-20 Thread Sami Imseih
likely(jumble_len >= JUMBLE_SIZE)) { - uint64 start_hash; + int64 start_hash; - start_hash = DatumGetUInt64(hash_any_extended(jumble, - JUMBLE_SIZE, 0)); + start_hash = DatumGetInt64(hash_any_extended(jumble, + JUMBLE_SIZE, 0)); memcpy(jumble, &start_hash, sizeof(start_hash)); jumble_len = sizeof(start_hash); -- Sami Imseih Amazon Web Services (AWS)

Re: queryId constant squashing does not support prepared statements

2025-05-20 Thread Sami Imseih
ion ParseLoc in A_ArrayExpr and A_Expr. Doing it this will keep changes to the parse_expr.c code to a minimum, only the IN transformation will need to set the values of the A_Expr into the final A_ArrayExpr. -- Sami Imseih Amazon Web Services (AWS) diff --git a/src/backend/parser/gram.y b/src/backend

Re: Add comment explaining why queryid is int64 in pg_stat_statements

2025-05-19 Thread Sami Imseih
FWIW, all the hash function SQL interfaces, \df hash*, have this behavior in which the result is a signed (int/bigint), but the internal representation of the hash is an unsigned (int/bigint). I am not sure why a comment is needed specifically for pg_stat_statements -- Sami Imseih Amazon Web

Re: Regression in statement locations

2025-05-19 Thread Sami Imseih
ows) I am still not sure why this is the case, but wanted to share this for now. -- Sami Imseih Amazon Web Services (AWS)

Re: Possible regression in PG18 beta1

2025-05-18 Thread Sami Imseih
On Sat, May 17, 2025 at 2:00 PM Sadeq Dousti wrote: >> >> thanks. I don't see regression for a normal table, at least for this test. > > > No, there isn't. I just added them as per your request ;) > > >> In terms of your original test, I tried it out on my Ubuntu machine >> >> and with your test a

Re: Possible regression in PG18 beta1

2025-05-18 Thread Sami Imseih
> I'm now thinking maybe the compilation options for PG 17.5 have been > different. I'm using the default version that comes with Homebrew, but for > PG 18, I compiled it myself. Here are the results for `select version();` > on both: > > PostgreSQL 17.5 (Homebrew) on aarch64-apple-darwin24.4.0, >

Re: Relstats after VACUUM FULL and CLUSTER

2025-05-24 Thread Sami Imseih
> If you subtract recently dead from that number within the heap > implementation, then it will no longer > reflect non-removable tuples and the log message in the cluster > function "found %.0f removable, %.0f nonremovable row versions" will no > longer be correct. Yes, that's correct. I did no

Re: queryId constant squashing does not support prepared statements

2025-05-24 Thread Sami Imseih
> In v17, we are a bit smarter with the numbering, with a normalization > giving the following, starting at $1: > select where $5 in ($1, $2, $3) and $6 = $4 > > So your argument about the $n parameters is kind of true, but I think > the numbering logic in v17 to start at $1 is a less-confusing res

<    1   2   3   4   >