Re: Removing unneeded self joins

2025-04-26 Thread Alexander Korotkov
On Sat, Apr 26, 2025 at 11:04 PM Alexander Korotkov wrote: > On Sun, Apr 6, 2025 at 12:02 AM Alena Rybakina > wrote: > > Should we add more regression tests covering these cases? > > > > I experimented with some examples like this and noticed that it does affect &g

Re: Removing unneeded self joins

2025-04-26 Thread Alexander Korotkov
27;m going to push it if no objections. -- Regards, Alexander Korotkov Supabase v2-0001-Disallow-removing-placeholders-during-Self-Join-E.patch Description: Binary data

Re: Fortify float4 and float8 regression tests by ordering test results

2025-04-22 Thread Alexander Korotkov
On Tue, Apr 22, 2025 at 7:20 PM Tom Lane wrote: > Alexander Korotkov writes: > > I'd like to add that float4.out not only assumes that insert-ordering is > > preserved (this could be more-or-less portable between table AMs). It also > > assumes the way UPDATE moves

Re: Fortify float4 and float8 regression tests by ordering test results

2025-04-22 Thread Alexander Korotkov
abs_f1 FROM FLOAT4_TBL f; f1 |abs_f1 ---+--- 0 | 0 1004.3 |1004.3 -34.84 | 34.84 1.2345679e+20 | 1.2345679e+20 1.2345679e-20 | 1.2345679e-20 (5 rows) UPDATE FLOAT4_TBL SET f1 = FLOAT4_TBL.f1 * '-1' WHERE FLOAT4_TBL.f1 > '0.0'; SELECT * FROM FLOAT4_TBL; f1 0 -34.84 -1004.3 -1.2345679e+20 -1.2345679e-20 (5 rows) -- Regards, Alexander Korotkov Supabase

Re: type cache cleanup improvements

2025-04-20 Thread Alexander Korotkov
Hi, Noah! On Sat, Apr 12, 2025 at 12:43 AM Alexander Korotkov wrote: > > On Fri, Apr 11, 2025 at 11:32 PM Noah Misch wrote: > > > > On Tue, Oct 22, 2024 at 08:33:24PM +0300, Alexander Korotkov wrote: > > > On Tue, Oct 22, 2024 at 6:10 PM Pavel Borisov > > >

Re: type cache cleanup improvements

2025-04-11 Thread Alexander Korotkov
On Fri, Apr 11, 2025 at 11:32 PM Noah Misch wrote: > > On Tue, Oct 22, 2024 at 08:33:24PM +0300, Alexander Korotkov wrote: > > On Tue, Oct 22, 2024 at 6:10 PM Pavel Borisov > > wrote: > > > On Tue, 22 Oct 2024 at 11:34, Alexander Korotkov > > > wrote:

Re: MergeJoin beats HashJoin in the case of multiple hash clauses

2025-04-11 Thread Alexander Korotkov
On Fri, Apr 11, 2025 at 5:06 AM Andres Freund wrote: > On 2025-04-11 00:47:19 +0200, Matthias van de Meent wrote: > > On Fri, 11 Apr 2025 at 00:27, Andres Freund wrote: > > > > > > Hi, > > > > > > On 2025-03-09 14:13:52 +0200, Alexander Korotk

Re: Some problems regarding the self-join elimination code

2025-04-10 Thread Alexander Korotkov
move > ChangeVarNodes out of rewriteManip and make it multi-purpose routine, > allowing to transform expression that may happen after a Var node change? What about adding a callback to ChangeVarNodes_context that would called for each RestrictInfo after changing varnodes itself? SJE could use a callback that replaces OpExpr with NullTest when needed. -- Regards, Alexander Korotkov Supabase

Re: pgsql: Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate

2025-04-07 Thread Alexander Korotkov
{1,2}'::oid[])) > > + -> Sort > > + Sort Key: a.oid > > + -> Seq Scan on pg_am a > > +(7 rows) > > Are these failures from patches applied to master prior to 3ba2cdaa? Yes, these failures appears before 3ba2cdaa. Tom committed 3ba2cdaa to fix the problem before I get into it [1]. Links. 1. https://www.postgresql.org/message-id/srnuqlttuimzmvoulhsrbgvj4vnul6b65osswvua7sfkqsvmuy%40yg7apybpxp34 -- Regards, Alexander Korotkov Supabase

Re: Removing unneeded self joins

2025-04-07 Thread Alexander Korotkov
com Thank you for pointing. Looking into that! -- Regards, Alexander Korotkov Supabase

Re: pgsql: Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate

2025-04-07 Thread Alexander Korotkov
On Fri, Apr 4, 2025 at 6:47 PM Melanie Plageman wrote: > > On Fri, Apr 4, 2025 at 9:17 AM Alexander Korotkov > wrote: > > > > Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate > > > > This commit implements the automatic convers

Re: Removing unneeded self joins

2025-04-06 Thread Alexander Korotkov
On Sun, Apr 6, 2025 at 5:41 PM Tom Lane wrote: > Alexander Korotkov writes: > > Nevertheless, should we consider revisiting this flag? I see the only > > other GUC simultaneously QUERY_TUNING_METHOD and GUC_NOT_IN_SAMPLE is > > optimize_bounded_sort, which is not expose

Re: Removing unneeded self joins

2025-04-06 Thread Alexander Korotkov
GUC_NOT_IN_SAMPLE flag. Thanks for the reminder. > Please ignore the noise. Also, sorry for thoughtless commit of that. Nevertheless, should we consider revisiting this flag? I see the only other GUC simultaneously QUERY_TUNING_METHOD and GUC_NOT_IN_SAMPLE is optimize_bounded_sort, which is not exposed in a standard build. -- Regards, Alexander Korotkov Supabase

Re: Get rid of WALBufMappingLock

2025-04-05 Thread Alexander Korotkov
t; configurations with hundreds of clients. So, there're more bottlenecks ahead > )) > > > Yes, it is still not "real-world" benchmark. But it at least shows patch is > harmless. Thank you for your experiments. Your results shows up to 30% speedups on real hardware, not tmpfs. While this is still a corner case, I think this is quite a results for a pretty local optimization. On small connection number there are some cases above and below 1.0. I think this due to statistical error. If we would calculate average tps ratio across different experiments, for low number of clients it's still above 1.0. sqlite> select clients, avg(ratio) from (select walseg, walbuf, recsize, clients, (avg(tps) filter (where branch = 'nowalbuf'))/(avg(tps) filter (where branch = 'master')) as ratio from results where branch in ('master', 'nowalbuf') group by walseg, walbuf, recsize, clients) x group by clients; 1|1.00546614169766 4|1.00782085856889 16|1.02257892337757 64|1.04400167838906 128|1.04134006876033 256|1.04627949500578 I'm going to push the first patch ("nowalbuf") if no objections. I think the second one ("Several attempts") still needs more work, as there are regressions. -- Regards, Alexander Korotkov Supabase

Re: Remove an unnecessary check on semijoin_target_ok() on postgres_fdw.c

2025-04-05 Thread Alexander Korotkov
ns FALSE, >> > !bms_is_member(var->varno, outerrel->relids) will not execute due to >> > short circuit. >> > >> > So I think we can remove the "!bms_is_member(var->varno, >> > outerrel->relids)" from if. >> > Any thoughts? >> >> Hi. >> Seems good to me. >> -- >> Best regards, >> Alexander Pyhalov, >> Postgres Professional > > > Thanks for looking at that. Pushed. But I've decided to keep the redundant check as an assertion. -- Regards, Alexander Korotkov Supabase

Re: Removing unneeded self joins

2025-04-05 Thread Alexander Korotkov
On Fri, Apr 4, 2025 at 11:35 AM Andrei Lepikhov wrote: > > On 4/4/25 04:53, Richard Guo wrote: > > On Fri, Apr 4, 2025 at 1:02 AM Alexander Korotkov > > wrote: > >> I've got an off-list bug report from Alexander Lakhin involving a > >> placeholder variab

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-04 Thread Alexander Korotkov
not stable I think the next step is to nuke both > test queries, since I remain of the opinion that they're likely > a waste of cycles. Thank you for pushing this! I just start looking at the problem and have just seen your commit. -- Regards, Alexander Korotkov Supabase

Re: Changing the state of data checksums in a running cluster

2025-04-04 Thread Alexander Korotkov
uot;unable to enable data checksums in cluster"))); 2) ProcessAllDatabases() contains loop, which repeats scanning the new databases for checkums. It continues while there are new database on each iteration. Could we just limit the number of iterations to 2? Given at each step we're calling WaitForAllTransactionsToFinish(), everything that gets created after first WaitForAllTransactionsToFinish() call should have checksums enabled in the beginning. -- Regards, Alexander Korotkov Supabase

Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly

2025-04-03 Thread Alexander Korotkov
1. https://www.postgresql.org/message-id/e3ac0535-e7a2-4a96-9b36-9f765e9cfec5%40vondra.me -- Regards, Alexander Korotkov Supabase

Re: Removing unneeded self joins

2025-04-03 Thread Alexander Korotkov
s attached. I'm going to fix this if no objections. -- Regards, Alexander Korotkov Supabase v1-0001-Disallow-removing-placeholders-during-Self-Join-E.patch Description: Binary data

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-03 Thread Alexander Korotkov
s if no objections. -- Regards, Alexander Korotkov Supabase v9-0001-Extract-make_SAOP_expr-function-from-match_orclau.patch Description: Binary data v9-0002-Convert-x-IN-VALUES-.-to-x-ANY-.-then-appropriate.patch Description: Binary data

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-02 Thread Alexander Korotkov
g Time: 0.321 ms Execution Time: 210.409 ms (12 rows) Only when we have all consts in the array, we can have SAOP hashing which is competitive with hashing of VALUES node. So, I'm continuing with my version of patch. Links. 1. https://www.postgresql.org/message-id/CAPpHfds-7eJ3ZMgyeVrMfC5E1nTHD4Bp0ch5MZhrYSoiCfERXw%40mail.gmail.com -- Regards, Alexander Korotkov Supabase

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-04-01 Thread Alexander Korotkov
=1) Buckets: 1024 Batches: 1 Memory Usage: 9kB -> Values Scan on "*VALUES*" (cost=0.00..0.08 rows=6 width=4) (actual time=0.009..0.032 rows=6.00 loops=1) Planning Time: 0.627 ms Execution Time: 225.155 ms (12 rows) I think we should allow our transformation only

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-03-31 Thread Alexander Korotkov
ers: shared hit=1 -> Hash (cost=0.03..0.03 rows=2 width=32) (actual time=0.111..0.112 rows=2.00 loops=1) Buckets: 1024 Batches: 1 Memory Usage: 9kB -> Values Scan on "*VALUES*" (cost=0.00..0.03 rows=2 width=32) (actual time=0.004..0.065 rows=2.00 loops=1) Planning Time: 0.250 ms Execution Time: 0.267 ms (10 rows) -- Regards, Alexander Korotkov Supabase

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-03-30 Thread Alexander Korotkov
Hi, Alena! On Sat, Mar 29, 2025 at 9:03 PM Alena Rybakina wrote: > On 29.03.2025 14:03, Alexander Korotkov wrote: >> One thing I have to fix: we must do >> IncrementVarSublevelsUp() unconditionally for all expressions as Vars >> could be deeper inside. > > Yes, I

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-03-29 Thread Alexander Korotkov
On Wed, Mar 12, 2025 at 8:11 PM Alena Rybakina wrote: > On 06.03.2025 11:23, Alexander Korotkov wrote: > > Hi, Alena! > > On Sat, Mar 1, 2025 at 1:39 PM Alena Rybakina > wrote: > > On 09.02.2025 18:38, Alexander Korotkov wrote: > > Also, aren't

Re: POC, WIP: OR-clause support for indexes

2025-03-28 Thread Alexander Korotkov
On Fri, Mar 28, 2025 at 1:32 PM Andrei Lepikhov wrote: > On 3/28/25 00:18, Alexander Korotkov wrote: > > The attached patch changes the reordering algorithm of > > group_similar_or_args() in the following way. We reorder each group > > of similar clauses so that the first it

Re: POC, WIP: OR-clause support for indexes

2025-03-27 Thread Alexander Korotkov
he clauses remain in their places. -- Regards, Alexander Korotkov Supabase v1-0001-Make-group_similar_or_args-reorder-clause-list-as.patch Description: Binary data

Re: Add semi-join pushdown to postgres_fdw

2025-03-25 Thread Alexander Korotkov
On Mon, Mar 24, 2025 at 6:56 PM Alexander Pyhalov wrote: > Alexander Korotkov писал(а) 2025-03-24 11:49: > > On Mon, Mar 24, 2025 at 9:07 AM Alexander Pyhalov > > wrote: > >> Alexander Korotkov писал(а) 2025-03-24 04:21: > >> > Hi, Alexander! > >

Re: Add semi-join pushdown to postgres_fdw

2025-03-24 Thread Alexander Korotkov
On Mon, Mar 24, 2025 at 9:07 AM Alexander Pyhalov wrote: > Alexander Korotkov писал(а) 2025-03-24 04:21: > > Hi, Alexander! > > > > On Tue, Mar 18, 2025 at 6:04 PM Alexander Pyhalov > > wrote: > >> This shouldn't. When semi-join is found below left/r

Re: Add semi-join pushdown to postgres_fdw

2025-03-23 Thread Alexander Korotkov
from semi-joins, then their further pulls will be disabled automatically? See the attached patch. It also contains other (mostly cosmetic improvements). -- Regards, Alexander Korotkov Supabase v2-0001-Avoid-pulling-up-restrict-infos-from-subqueries.patch Description: Binary data

Re: Add semi-join pushdown to postgres_fdw

2025-03-18 Thread Alexander Korotkov
Hi, Alexander! On Tue, Mar 18, 2025 at 1:13 PM Alexander Pyhalov wrote: > Alexander Korotkov писал(а) 2025-03-18 03:27: > > Hi, Robins! > > > > On Tue, Mar 18, 2025 at 2:20 AM Robins Tharakan > > wrote: > >> On Mon, 4 Dec 2023 at 07:22, Alexander Korotkov

Re: Proposal: Adding compression of temporary files

2025-03-18 Thread Alexander Korotkov
On Tue, Mar 18, 2025 at 12:13 AM Tomas Vondra wrote: > On 3/15/25 11:40, Alexander Korotkov wrote: > > On Sun, Jan 5, 2025 at 1:43 AM Filip Janus wrote: > >> > >> I apologize for multiple messages, but I found a small bug in the previous > >> version. > &

Re: Add semi-join pushdown to postgres_fdw

2025-03-17 Thread Alexander Korotkov
Hi, Robins! On Tue, Mar 18, 2025 at 2:20 AM Robins Tharakan wrote: > On Mon, 4 Dec 2023 at 07:22, Alexander Korotkov wrote: > > > > > > Now, I think this looks good. I'm going to push this if no objections. > > After this commit, I began seeing an unexpe

Re: Proposal: Adding compression of temporary files

2025-03-15 Thread Alexander Korotkov
compress. Then, 0002 would apply this to temporary file compression. -- Regards, Alexander Korotkov Supabase

Re: Parallel CREATE INDEX for GIN indexes

2025-03-15 Thread Alexander Korotkov
ess items array lazily. But could we leave just items-related fields in GinBuffer, but have the rest always in GinBuffer.cached? So, if GinBuffer.items != NULL then we have items decompressed already, otherwise have to decompress them when needed. -- Regards, Alexander Korotkov Supabase

Re: Enhancing Memory Context Statistics Reporting

2025-03-15 Thread Alexander Korotkov
when there is not a huge amount of free memory. So, failure is probable on DSA allocation. Could we do simpler? For instance, allocate some amount of static shared memory and use it as a message queue between processes. As a heavy load is not supposed to be here, I think one queue would be enough. -- Regards, Alexander Korotkov Supabase

Re: Get rid of WALBufMappingLock

2025-03-13 Thread Alexander Korotkov
On Fri, Mar 7, 2025 at 5:08 PM Alexander Korotkov wrote: > On Sun, Mar 2, 2025 at 1:58 PM Alexander Korotkov > wrote: > > > > On Fri, Feb 28, 2025 at 3:13 PM Álvaro Herrera > > wrote: > > > On 2025-Feb-28, Michael Paquier wrote: > > > > > >

Re: pgsql: reindexdb: Add the index-level REINDEX with multiple jobs

2025-03-11 Thread Alexander Korotkov
On Mon, Mar 10, 2025 at 1:12 PM Álvaro Herrera wrote: > On 2025-Mar-09, Alexander Korotkov wrote: > > > After second thought it's not so hard to fix. The attached patch does > > it by putting REINDEX commands related to the same table into a single > > SQL statem

Re: Implement waiting for wal lsn replay: reloaded

2025-03-10 Thread Alexander Korotkov
On Fri, Feb 28, 2025 at 3:55 PM Yura Sokolov wrote: > 28.02.2025 16:03, Yura Sokolov пишет: > > 17.02.2025 00:27, Alexander Korotkov wrote: > >> On Thu, Feb 6, 2025 at 10:31 AM Yura Sokolov > >> wrote: > >>> I briefly looked into patch and have couple o

Re: pgsql: reindexdb: Add the index-level REINDEX with multiple jobs

2025-03-09 Thread Alexander Korotkov
On Sun, Mar 9, 2025 at 4:53 AM Alexander Korotkov wrote: > On Sat, Mar 8, 2025 at 12:49 PM Alexander Korotkov > wrote: > > On Fri, Mar 7, 2025 at 8:20 PM Álvaro Herrera > > wrote: > > > > > > On 2024-Mar-25, Alexander Korotkov wrote: > > > > &

Re: MergeJoin beats HashJoin in the case of multiple hash clauses

2025-03-09 Thread Alexander Korotkov
On Wed, Mar 5, 2025 at 4:43 AM Alexander Korotkov wrote: > > On Mon, Mar 3, 2025 at 10:24 AM Andrei Lepikhov wrote: > > On 17/2/2025 01:34, Alexander Korotkov wrote: > > > Hi, Andrei! > > > > > > On Tue, Oct 8, 2024 at 8:00 AM Andrei Lepikhov wrote: > &

Re: Considering fractional paths in Append node

2025-03-09 Thread Alexander Korotkov
On Wed, Mar 5, 2025 at 1:20 PM Alexander Korotkov wrote: > On Wed, Mar 5, 2025 at 8:32 AM Andrei Lepikhov wrote: > > On 5/3/2025 03:27, Alexander Korotkov wrote: > > > On Mon, Mar 3, 2025 at 1:04 PM Andrei Lepikhov wrote: > > >>> 2. As usage of root->t

Re: pgsql: reindexdb: Add the index-level REINDEX with multiple jobs

2025-03-08 Thread Alexander Korotkov
On Sat, Mar 8, 2025 at 12:49 PM Alexander Korotkov wrote: > On Fri, Mar 7, 2025 at 8:20 PM Álvaro Herrera wrote: > > > > On 2024-Mar-25, Alexander Korotkov wrote: > > > > > reindexdb: Add the index-level REINDEX with multiple jobs > > > > > > Strai

Re: pg_atomic_compare_exchange_*() and memory barriers

2025-03-08 Thread Alexander Korotkov
; barrier in case CAS fails. Thank you, I'm good with this solution. Can I leave this on you? I'm not feeling myself strong to word this correctly. -- Regards, Alexander Korotkov Supabase

Re: pg_atomic_compare_exchange_*() and memory barriers

2025-03-08 Thread Alexander Korotkov
Hi, Andres! On Fri, Mar 7, 2025 at 7:54 PM Alexander Korotkov wrote: > On Fri, Mar 7, 2025 at 7:46 PM Andres Freund wrote: > > On 2025-03-07 19:44:20 +0200, Alexander Korotkov wrote: > > > On Fri, Mar 7, 2025 at 7:38 PM Andres Freund wrote: > > > > On 2025-0

Re: pgsql: reindexdb: Add the index-level REINDEX with multiple jobs

2025-03-08 Thread Alexander Korotkov
On Fri, Mar 7, 2025 at 8:20 PM Álvaro Herrera wrote: > > On 2024-Mar-25, Alexander Korotkov wrote: > > > reindexdb: Add the index-level REINDEX with multiple jobs > > > > Straight-forward index-level REINDEX is not supported with multiple jobs as > > we cannot con

Re: pgsql: reindexdb: Add the index-level REINDEX with multiple jobs

2025-03-08 Thread Alexander Korotkov
ready used in some user scripts. Should we replace pg_fatal() with some notice and then run in a single job? So, user scripts wouldn't error out. -- Regards, Alexander Korotkov Supabase

Re: pg_atomic_compare_exchange_*() and memory barriers

2025-03-07 Thread Alexander Korotkov
On Fri, Mar 7, 2025 at 7:07 PM Andres Freund wrote: > On 2025-03-07 18:39:42 +0200, Alexander Korotkov wrote: > > On Fri, Mar 7, 2025 at 6:02 PM Andres Freund wrote: > > > > > > On 2025-03-07 17:47:08 +0200, Alexander Korotkov wrote: > > > > While investi

Re: pg_atomic_compare_exchange_*() and memory barriers

2025-03-07 Thread Alexander Korotkov
On Fri, Mar 7, 2025 at 7:46 PM Andres Freund wrote: > On 2025-03-07 19:44:20 +0200, Alexander Korotkov wrote: > > On Fri, Mar 7, 2025 at 7:38 PM Andres Freund wrote: > > > On 2025-03-07 19:15:23 +0200, Alexander Korotkov wrote: > > > > On Fri, Mar 7, 2025 a

Re: pg_atomic_compare_exchange_*() and memory barriers

2025-03-07 Thread Alexander Korotkov
On Fri, Mar 7, 2025 at 7:38 PM Andres Freund wrote: > On 2025-03-07 19:15:23 +0200, Alexander Korotkov wrote: > > On Fri, Mar 7, 2025 at 7:07 PM Andres Freund wrote: > > > What is the access pattern and the observed problems with it that made you > > > look at the disa

Re: pg_atomic_compare_exchange_*() and memory barriers

2025-03-07 Thread Alexander Korotkov
On Fri, Mar 7, 2025 at 7:15 PM Alexander Korotkov wrote: > > On Fri, Mar 7, 2025 at 7:07 PM Andres Freund wrote: > > On 2025-03-07 18:39:42 +0200, Alexander Korotkov wrote: > > > On Fri, Mar 7, 2025 at 6:02 PM Andres Freund wrote: > > > > > > > > On

Re: pg_atomic_compare_exchange_*() and memory barriers

2025-03-07 Thread Alexander Korotkov
On Fri, Mar 7, 2025 at 7:07 PM Andres Freund wrote: > On 2025-03-07 18:39:42 +0200, Alexander Korotkov wrote: > > On Fri, Mar 7, 2025 at 6:02 PM Andres Freund wrote: > > > > > > On 2025-03-07 17:47:08 +0200, Alexander Korotkov wrote: > > > > While investi

Re: pg_atomic_compare_exchange_*() and memory barriers

2025-03-07 Thread Alexander Korotkov
Hi, Andres. Thank you for reply. On Fri, Mar 7, 2025 at 6:02 PM Andres Freund wrote: > > On 2025-03-07 17:47:08 +0200, Alexander Korotkov wrote: > > While investigating a bug in the patch to get rid of WALBufMappingLock, I > > found that the surrounding pg_atomic_compare_ex

pg_atomic_compare_exchange_*() and memory barriers

2025-03-07 Thread Alexander Korotkov
need explicit memory barriers now. Any thoughts? Links 1. https://www.postgresql.org/message-id/CAPpHfdsWcQb-u-9K%3DipneBf8CMhoUuBWKYc%2BXWJEHVdtONOepQ%40mail.gmail.com 2. https://developer.arm.com/documentation/100941/0101/Barriers -- Regards, Alexander Korotkov Supabase

Re: Get rid of WALBufMappingLock

2025-03-07 Thread Alexander Korotkov
On Sun, Mar 2, 2025 at 1:58 PM Alexander Korotkov wrote: > > On Fri, Feb 28, 2025 at 3:13 PM Álvaro Herrera > wrote: > > On 2025-Feb-28, Michael Paquier wrote: > > > > > Saying that, I have also done similar tests with your v12 for a couple > > &g

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-03-06 Thread Alexander Korotkov
Hi, Alena! On Sat, Mar 1, 2025 at 1:39 PM Alena Rybakina wrote: > On 09.02.2025 18:38, Alexander Korotkov wrote: >> >> Also, aren't we too restrictive while requiring is_simple_values_sequence()? >> For instance, I believe cases like this (containing Var) co

Re: Considering fractional paths in Append node

2025-03-05 Thread Alexander Korotkov
On Wed, Mar 5, 2025 at 8:32 AM Andrei Lepikhov wrote: > On 5/3/2025 03:27, Alexander Korotkov wrote: > > On Mon, Mar 3, 2025 at 1:04 PM Andrei Lepikhov wrote: > >>> 2. As usage of root->tuple_fraction RelOptInfo it has been criticized, > >>> do you think we c

Re: MergeJoin beats HashJoin in the case of multiple hash clauses

2025-03-04 Thread Alexander Korotkov
On Mon, Mar 3, 2025 at 10:24 AM Andrei Lepikhov wrote: > On 17/2/2025 01:34, Alexander Korotkov wrote: > > Hi, Andrei! > > > > On Tue, Oct 8, 2024 at 8:00 AM Andrei Lepikhov wrote: > > Thank you for your work on this subject. I agree with the general > > di

Re: Considering fractional paths in Append node

2025-03-04 Thread Alexander Korotkov
On Mon, Mar 3, 2025 at 1:04 PM Andrei Lepikhov wrote: > > On 2/3/2025 09:53, Alexander Korotkov wrote: > > Hi, Andrei! > > > > On Fri, Dec 6, 2024 at 10:13 AM Andrei Lepikhov wrote: > >> > >> On 12/6/24 13:48, Andrei Lepikhov wrote: > >>>

Re: Get rid of WALBufMappingLock

2025-03-02 Thread Alexander Korotkov
only > tested the patch and it was stable while I've noticed a few diffs with > the previous version, but I did *not* check its internals at all, nor > do I mean that I endorse its logic. I hope that's clear now. Got it. Michael, thank you very much for your help. -- Regards, Alexander Korotkov Supabase

Re: Get rid of WALBufMappingLock

2025-03-02 Thread Alexander Korotkov
t tests looks much more stable now. I think he is absolutely correct with this. -- Regards, Alexander Korotkov Supabase

Re: Considering fractional paths in Append node

2025-03-02 Thread Alexander Korotkov
sure this wouldn't affect other callers of this function? 2. As usage of root->tuple_fraction RelOptInfo it has been criticized, do you think we could limit this to some simple cases? For instance, check that RelOptInfo is the final result relation for given root. Links. https://www.postgresql.org/message-id/871q0fvbje.fsf%40163.com -- Regards, Alexander Korotkov Supabase

Re: Considering fractional paths in Append node

2025-03-02 Thread Alexander Korotkov
ke the startup cost as > cost to retrieve cost for the first tuple, we can do the below quick hack, > > @@ -355,8 +355,8 @@ cost_seqscan(Path *path, PlannerInfo *root, > } > > path->disabled_nodes = enable_seqscan ? 0 : 1; > - path->startup_cost = startup_cost; > path->total_cost = startup_cost + cpu_run_cost + disk_run_cost; > + path->startup_cost = startup_cost + (cpu_run_cost + disk_run_cost) > * (1 - path->rows / baserel->tuples); > } You can check I've already proposed similar change years before. https://www.postgresql.org/message-id/CAPpHfdvfDAXPXhFQT3Ww%3DkZ4tpyAsD07_oz8-fh0%3Dp6eckEBKQ%40mail.gmail.com It appears that according to the current model startup_cost is not the cost of the first tuple. It should be the cost of preparation work, while extraction of tuples should be distributed uniformly through total_cost - startup_cost. -- Regards, Alexander Korotkov Supabase

Re: suspicious lockup on widowbird in AdvanceXLInsertBuffer (could it be due to 6a2275b8953?)

2025-02-27 Thread Alexander Korotkov
the same as posted by Michael from batta. My apologies for committing not well-tested patch. -- Regards, Alexander Korotkov Supabase

Re: Removing unneeded self joins

2025-02-26 Thread Alexander Korotkov
On Mon, Feb 24, 2025 at 2:22 PM Andrei Lepikhov wrote: > > On 24/2/2025 11:57, Alexander Korotkov wrote: > > Hi, Andrei! > > > > Thank you for your feedback. > > > > On Mon, Feb 24, 2025 at 12:12 AM Andrei Lepikhov wrote: > >> On 23/2/2025 22:15

Re: Removing unneeded self joins

2025-02-26 Thread Alexander Korotkov
On Mon, Feb 24, 2025 at 12:57 PM Alexander Korotkov wrote: > On Mon, Feb 24, 2025 at 12:12 AM Andrei Lepikhov wrote: > > On 23/2/2025 22:15, Alexander Korotkov wrote: > > > There is my attempt to implement this approach. Getting rid of local > > > variable (and co

Re: Get rid of WALBufMappingLock

2025-02-26 Thread Alexander Korotkov
Hi, Michael! On Wed, Feb 26, 2025 at 3:04 AM Michael Paquier wrote: > > On Tue, Feb 25, 2025 at 05:19:29PM +0200, Alexander Korotkov wrote: > > It seems that I managed to reproduce the issue on my Raspberry PI 4. > > After running our test suite in a loop for 2 days I found on

Re: Get rid of WALBufMappingLock

2025-02-25 Thread Alexander Korotkov
On Tue, Feb 18, 2025 at 2:29 AM Alexander Korotkov wrote: > > On Tue, Feb 18, 2025 at 2:21 AM Michael Paquier wrote: > > > > On Mon, Feb 17, 2025 at 11:25:05AM -0500, Tom Lane wrote: > > > This timeout failure on hachi looks suspicious as well: > > > > &

Re: Removing unneeded self joins

2025-02-24 Thread Alexander Korotkov
Hi, Andrei! Thank you for your feedback. On Mon, Feb 24, 2025 at 12:12 AM Andrei Lepikhov wrote: > On 23/2/2025 22:15, Alexander Korotkov wrote: > > There is my attempt to implement this approach. Getting rid of local > > variable (and computation of the same value other w

Re: Removing unneeded self joins

2025-02-23 Thread Alexander Korotkov
On Sun, Feb 23, 2025 at 7:02 PM Tom Lane wrote: > > Alexander Korotkov writes: > > I've corrected some spelling error reported by Alexander Lakhin > > privately to me. Also, I've revised comments around ChangeVarNodes() > > and ChangeVarNodesExtended(). I&#

Re: Improve statistics estimation considering GROUP-BY as a 'uniqueiser'

2025-02-19 Thread Alexander Korotkov
On Tue, Feb 18, 2025 at 2:52 PM Andrei Lepikhov wrote: > On 17/2/2025 02:06, Alexander Korotkov wrote: > > On Thu, Nov 28, 2024 at 4:39 AM Andrei Lepikhov wrote: > >> Here we also could count number of scanned NULLs separately in > >> vardata_extra and use it i

Re: Improve statistics estimation considering GROUP-BY as a 'uniqueiser'

2025-02-19 Thread Alexander Korotkov
x | y ---+--- 3 | 2 2 | 2 3 | 1 2 | 1 1 | 3 1 | 2 1 | 1 2 | 3 3 | 3 (9 rows) x and y are unique here as a pair. But individual x and y values have repeats. -- Regards, Alexander Korotkov Supabase

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Alexander Korotkov
patch. Sounds > plausible to me that some other buildfarm animals could be stuck > without their owners knowing about it. It's proving to be a good idea > to force a timeout value in the configuration file of these animals.. Tom, Michael, thank you for the information. This patch will be better tested before next attempt. -- Regards, Alexander Korotkov Supabase

Re: Get rid of WALBufMappingLock

2025-02-17 Thread Alexander Korotkov
t quickly guess the reason. I'm going to revert patch for now, then we investigate ------ Regards, Alexander Korotkov Supabase

Re: Add Postgres module info

2025-02-16 Thread Alexander Korotkov
on >= '1.0'; For this, and probably other purposes, it's desirable for version to be something comparable at SQL level. Should we add some builtin analogue of pg_text_semver? -- Regards, Alexander Korotkov Supabase

Re: Improve statistics estimation considering GROUP-BY as a 'uniqueiser'

2025-02-16 Thread Alexander Korotkov
ould be the type of vardata_extra? And what information could it store? Yet seems too sketchy for me to understand. But, I think for now we should go with the original patch. It seems to be quite straightforward extension to what 4767bc8ff2 does. I've revised commit message and applied pg_indent to sources. I'm going to push this if no objections. -- Regards, Alexander Korotkov Supabase v2-0001-Improve-statistics-estimation-for-single-column-G.patch Description: Binary data

Re: MergeJoin beats HashJoin in the case of multiple hash clauses

2025-02-16 Thread Alexander Korotkov
/saveList/origin_rinfos/g for better readability. Also, the patch badly needs regression test coverage. We can't include costs in expected outputs. But that could be some plans, which previously were reliably merge joins but now become reliable hash joins. -- Regards, Alexander Korotkov Supabase v2-0001-Use-extended-statistics-for-precise-estimation-of.patch Description: Binary data

Re: Implement waiting for wal lsn replay: reloaded

2025-02-16 Thread Alexander Korotkov
> Otherwise, the patch looks quite strong to me. Great, thank you! -- Regards, Alexander Korotkov Supabase v2-0001-Implement-WAIT-FOR-command.patch Description: Binary data

Re: Get rid of WALBufMappingLock

2025-02-15 Thread Alexander Korotkov
Hi! On Fri, Feb 14, 2025 at 4:11 PM Yura Sokolov wrote: > 14.02.2025 17:09, Yura Sokolov пишет: > > 14.02.2025 13:24, Alexander Korotkov пишет: > >> On Fri, Feb 14, 2025 at 11:45 AM Pavel Borisov > >> wrote: > >>> On Fri, 14 Feb 2025 at 00:59, Alexande

Re: Get rid of WALBufMappingLock

2025-02-14 Thread Alexander Korotkov
On Fri, Feb 14, 2025 at 11:45 AM Pavel Borisov wrote: > On Fri, 14 Feb 2025 at 00:59, Alexander Korotkov wrote: > > On Thu, Feb 13, 2025 at 6:39 PM Pavel Borisov > > wrote: > > > On Thu, 13 Feb 2025 at 14:08, Alexander Korotkov > > > wrote: > > >

Re: Get rid of WALBufMappingLock

2025-02-13 Thread Alexander Korotkov
Hi, Pavel! On Thu, Feb 13, 2025 at 6:39 PM Pavel Borisov wrote: > On Thu, 13 Feb 2025 at 14:08, Alexander Korotkov wrote: > > > > On Thu, Feb 13, 2025 at 11:45 AM Yura Sokolov > > wrote: > > > 13.02.2025 12:34, Alexander Korotkov пишет: > > > > On

Re: Get rid of WALBufMappingLock

2025-02-13 Thread Alexander Korotkov
On Thu, Feb 13, 2025 at 11:45 AM Yura Sokolov wrote: > 13.02.2025 12:34, Alexander Korotkov пишет: > > On Wed, Feb 12, 2025 at 8:16 PM Yura Sokolov > > wrote: > >> 08.02.2025 13:07, Alexander Korotkov пишет: > >>> On Fri, Feb 7, 2025 at 1:39 PM Alexander

Re: Get rid of WALBufMappingLock

2025-02-13 Thread Alexander Korotkov
On Wed, Feb 12, 2025 at 8:16 PM Yura Sokolov wrote: > 08.02.2025 13:07, Alexander Korotkov пишет: > > On Fri, Feb 7, 2025 at 1:39 PM Alexander Korotkov > > wrote: > >> Good, thank you. I think 0001 patch is generally good, but needs some > >> further polishi

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-02-09 Thread Alexander Korotkov
On Sun, Feb 9, 2025 at 1:58 PM Alexander Korotkov wrote: > > On Thu, Jan 9, 2025 at 3:11 PM Alena Rybakina > wrote: > > On 04.10.2024 12:05, Andrei Lepikhov wrote: > > > We also have an implementation of VALUES -> ARRAY transformation. > > > Because enterp

Re: Replace IN VALUES with ANY in WHERE clauses during optimization

2025-02-09 Thread Alexander Korotkov
quence()? For instance, I believe cases like this (containing Var) could be transformed too. select * from t t1, lateral (select * from t t2 where t2.i in (values (t1.i), (1))); -- Regards, Alexander Korotkov Supabase

Re: Get rid of WALBufMappingLock

2025-02-08 Thread Alexander Korotkov
On Fri, Feb 7, 2025 at 1:39 PM Alexander Korotkov wrote: > Good, thank you. I think 0001 patch is generally good, but needs some > further polishing, e.g. more comments explaining how does it work. Two things comes to my mind worth rechecking about 0001. 1) Are XLogCtl->Initializ

Re: Get rid of WALBufMappingLock

2025-02-07 Thread Alexander Korotkov
On Fri, Feb 7, 2025 at 1:24 PM Yura Sokolov wrote: > 07.02.2025 14:02, Yura Sokolov пишет: > > 07.02.2025 01:26, Alexander Korotkov пишет: > >> Hi! > >> > >> On Sun, Jan 19, 2025 at 2:11 AM Yura Sokolov > >> wrote: > >>> > >>

Re: Get rid of WALBufMappingLock

2025-02-06 Thread Alexander Korotkov
InitializedUpTo in a loop. So, multiple processes will try to advance InitializedUpTo, they could hijack initiative from each other, but there is always a leader which will finish the work. There is only one ConditionVariable to wait for InitializedUpTo being advanced. I didn't benchm

Re: Vacuum statistics

2025-02-04 Thread Alexander Korotkov
On Tue, Feb 4, 2025 at 5:22 PM Alena Rybakina wrote: > > Hi! Thank you for your review! > > On 02.02.2025 23:43, Alexander Korotkov wrote: > > On Mon, Jan 13, 2025 at 3:26 PM Alena Rybakina > > wrote: > >> I noticed that the cfbot is bad, the reason seems to

Re: SIGSEGV, FPE fix in pg_controldata

2025-02-04 Thread Alexander Korotkov
lse "as is" doesn't make sense to me). Now we have WalSegMaxSize equals to 1GB and signed int is enough to store value within this range. So, if we would badly need to increase WalSegMaxSize, that would give the motivation to change, but I don't see that. -- Regards, Alexander Korotkov Supabase

Re: SIGSEGV, FPE fix in pg_controldata

2025-02-03 Thread Alexander Korotkov
ment size to uint32 as it's already defined as int in awfully a lot of places. Additionally WalSegMaxSize is clearly within the 32-bit integer max value. So, I think we can just adjust the check before XLByteToSeg(). What do you think? -- Regards, Alexander Korotkov Supabase v2-000

Re: POC, WIP: OR-clause support for indexes

2025-02-03 Thread Alexander Korotkov
On Mon, Feb 3, 2025 at 12:22 PM Alena Rybakina wrote: > > Thank you for updated version! I agree for your version of the code. > > On 02.02.2025 21:00, Alexander Korotkov wrote: > > On Fri, Jan 31, 2025 at 4:31 PM Alena Rybakina > wrote: > > I started reviewing at th

Re: Vacuum statistics

2025-02-02 Thread Alexander Korotkov
le AMs as well. There are deletions of empty lines in src/include/utils/pgstat_internal.h and src/include/pgstat.h. Please, remote them as it's not purpose of this patchset. -- Regards, Alexander Korotkov Supabase

Re: POC, WIP: OR-clause support for indexes

2025-02-02 Thread Alexander Korotkov
ttps://www.postgresql.org/message-id/CAPpHfdsBZmNt9qUoJBqsQFiVDX1%3DyCKpuVAt1YnR7JCpP%3Dk8%2BA%40mail.gmail.com -- Regards, Alexander Korotkov Supabase

Re: POC, WIP: OR-clause support for indexes

2025-02-02 Thread Alexander Korotkov
On Tue, Jan 28, 2025 at 10:42 AM Andrei Lepikhov wrote: > On 1/28/25 11:36, Andrei Lepikhov wrote: > > On 1/27/25 16:50, Alexander Korotkov wrote: > > qsort(matches, n, sizeof(OrArgIndexMatch), or_arg_index_match_cmp); > > > > To fit an index, the order of elements

Re: POC, WIP: OR-clause support for indexes

2025-02-02 Thread Alexander Korotkov
On Thu, Jan 30, 2025 at 3:23 PM Pavel Borisov wrote: > On Tue, 28 Jan 2025 at 12:42, Andrei Lepikhov wrote: > > > > On 1/28/25 11:36, Andrei Lepikhov wrote: > > > On 1/27/25 16:50, Alexander Korotkov wrote: > > > qsort(matches, n, sizeof(OrArgIndexMatch), or_arg

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2025-01-31 Thread Alexander Korotkov
SmL%3DEEiph-8rfHg%40mail.gmail.com 2. https://www.postgresql.org/message-id/CAPpHfdvVMdUX0DGSK3oAbt9C5TPup%3DBEq8QkmvDrMbuD4BR9Fw%40mail.gmail.com -- Regards, Alexander Korotkov Supabase

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2025-01-31 Thread Alexander Korotkov
o be atomic and transactional. And that seems to be extremely hard to achieve if we move the data between existing relnodes. How can we rollback or recover after error? So, it least for initial implementation I would leave data movement as it is. -- Regards, Alexander Korotkov Supabase

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2025-01-31 Thread Alexander Korotkov
t; The patches are attached to the email. Thank you for the rebase. I don't think we need a separate 0003 patch with refactoring. It's probably good idea to keep this functionality as a separate patch, but let's make then it a 0001, which prepares functions used by 0002 and 0003. -- Regards, Alexander Korotkov Supabase

  1   2   3   4   5   6   7   8   9   10   >