Re: Poor row estimates from planner, stat `most_common_elems` sometimes missing for a text[] column

2025-06-06 Thread Frédéric Yhuel
On 6/5/25 23:52, Tom Lane wrote: The idea of treating lack of MCELEM differently from complete lack of stats still seems to have merit, though. Couldn't we count / estimate the number of distinct two-by-two elements, and use that instead of the default selectivity estimate?

Re: Poor row estimates from planner, stat `most_common_elems` sometimes missing for a text[] column

2025-06-05 Thread Frédéric Yhuel
On 6/5/25 17:42, Mark Frost wrote: Is there any good explanation for this behaviour? Preferably we’d like some way for proper `most_common_elems` statistics to be collected in our production database, in the hope that influences a good query plan to always be selected. most_common_elems h

Re: ALTER TABLE ADD FOREIGN KEY to partitioned table, is not parallelized

2025-06-05 Thread Frédéric Yhuel
On 6/5/25 16:13, Frédéric Yhuel wrote: On 6/4/25 16:12, Dimitrios Apostolou wrote: In general I have noticed most operations are slower after a succesful pg_restore until VACUUM is complete, which is unfortunate as the database is huge and it takes days to run. Something I have on my list

Re: ALTER TABLE ADD FOREIGN KEY to partitioned table, is not parallelized

2025-06-05 Thread Frédéric Yhuel
On 6/4/25 16:12, Dimitrios Apostolou wrote: In general I have noticed most operations are slower after a succesful pg_restore until VACUUM is complete, which is unfortunate as the database is huge and it takes days to run. Something I have on my list to try, is whether a COPY FREEZE would al

Re: ALTER TABLE ADD FOREIGN KEY to partitioned table, is not parallelized

2025-06-04 Thread Frédéric Yhuel
On 6/3/25 17:34, Dimitrios Apostolou wrote: The backend process for each of the above ALTER TABLE commands, does not   parallelize the foreign key checks for the different partitions. I   know, because in the logs I see gigabytes of temporary files being   written, with the CONTEXT showing

Re: Why a bitmap scan in this case?

2024-12-20 Thread Frédéric Yhuel
On 12/20/24 09:16, Frédéric Yhuel wrote: On 12/19/24 20:09, Jon Zeppieri wrote: The table is freshly vacuumed. If I disable bitmap scans, it will do an index only scan, which performs better. For the bitmap heap scan, it says "Heap Blocks: exact=27393," whereas for the index

Re: Why a bitmap scan in this case?

2024-12-20 Thread Frédéric Yhuel
On 12/19/24 20:09, Jon Zeppieri wrote: The table is freshly vacuumed. If I disable bitmap scans, it will do an index only scan, which performs better. For the bitmap heap scan, it says "Heap Blocks: exact=27393," whereas for the index only scan, it's "Heap Fetches: 27701." So you have 100% h

Re: Cardinality estimate of the inner relation

2024-11-25 Thread Frédéric Yhuel
On 11/23/24 03:07, Andrei Lepikhov wrote: Thanks for the case provided! Thanks for your answer! I wonder if data science has invented a statistic or selectivity estimation technique that could tackle your case in general. As I see, we should touch the table of products to realise which s

Cardinality estimate of the inner relation

2024-11-22 Thread Frédéric Yhuel
My colleague Christophe Courtois and I have been trying to fix a bad plan for one of Dalibo's clients. It is a (probably well-known) problem with skewed data and a parameterized Nested Loop with an underestimation of the cardinality of the inner relation. Here is a test case (the script to cre

Re: Has gen_random_uuid() gotten much slower in v17?

2024-11-20 Thread Frédéric Yhuel
On 9/11/24 12:47, David Mullineux wrote: Good idea. Thanks.    I did check. It's not enabled by default but just in case I did another build. This time explicitly defining --disable- debug and --disable-cassert. And I tested. Still slower than old versions. 4.5 seconds is very surprising.

Re: Which side of a Merge Join gets executed first? Do both sides always get executed?

2023-12-20 Thread Frédéric Yhuel
Le 20/12/2023 à 20:04, Jerry Brenner a écrit : Thanks.  Does this make sense? * There are 3 nodes under the Merge Join * The first node is an InitPlan, due to the ANY(ARRAY()) - that gets executed and finds 0 matching rows * The second node is the outer node in the Merge Join and th

Re: Which side of a Merge Join gets executed first? Do both sides always get executed?

2023-12-20 Thread Frédéric Yhuel
Le 20/12/2023 à 15:40, Jerry Brenner a écrit : Whichever side gets executed first, is the execution of the side that would be second get short circuited if 0 rows are returned by the first side? Indeed, if 0 rows are returned from the outer relation, the scan of the inner relation is never

Re: Which side of a Merge Join gets executed first? Do both sides always get executed?

2023-12-20 Thread Frédéric Yhuel
Le 20/12/2023 à 15:40, Jerry Brenner a écrit : The attached query plan is from 11. We are getting Merge Joins on both sides of the UNION.  In both cases, the first node under the Merge Join returns 0 rows but the other side of the Merge Join (the one being sorted) is executed and that's wher