Re: Difference in execution plans pg12 vs pg14

2021-12-11 Thread Дмитрий Иванов
Ok, I will check.
Now I have even more questions. This behavior appeared, it was not there in
the first place. jit-compilation down my long aggregation queries by about
half on average. How do I adjust this correctly?

#jit_provider = 'llvmjit' # JIT library to use
#jit_above_cost = 10 # perform JIT compilation if available
# and query more expensive than this;
# -1 disables
#jit_inline_above_cost = 50 # inline small functions if query is
# more expensive than this; -1 disables
#jit_optimize_above_cost = 50 # use expensive JIT optimizations if
# query is more expensive than this;
# -1 disables
jit = off # allow JIT compilation
--
Regards, Dmitry!


сб, 11 дек. 2021 г. в 09:12, Imre Samu :

> Hi Dmitry,
>
> pg12:
> > Execution Time: 44.123 ms
>
> pg14:
> > JIT:
> >   Functions: 167
> >   Options: Inlining true, Optimization true, Expressions true, Deforming
> true
> >   Timing: Generation 9.468 ms, Inlining 55.237 ms, Optimization 507.548
> ms, Emission 347.932 ms, Total 920.185 ms
> > Execution Time: 963.258 ms
>
> please check the JIT settings ;
> and test with:   jit = off;
>
> regards,
>  Imre
>
>
> Дмитрий Иванов  ezt írta (időpont: 2021. dec. 11.,
> Szo, 2:01):
>
>> Afternoon. I was able to make the necessary changes to my base needed to
>> migrate win_pg12 to debian pg14.
>> But there is a new problem, which was not there at the initial stage so I
>> checked:
>>
>> win_pg12:
>> ->  Index Scan using index_class_tree_full on class c  (cost=0.28..2.50
>> rows=1 width=235) (actual time=0.030..0.032 rows=1 loops=1)
>>  Index Cond: (id = 650)
>> debian_pg14:
>> ->  Index Scan using index_class_tree_full on class c  (cost=0.28..2.50
>> rows=1 width=235) (actual time=842.795..842.796 rows=1 loops=1)
>>  Index Cond: (id = 650)
>>
>> I'm wondering where to dig next...
>> --
>> Best regards, Dmitry!
>>
>
Uchet=# EXPLAIN ANALYZE SELECT * FROM bpd.vclass WHERE "id" = 650;

 QUERY PLAN
-
 Nested Loop Left Join  (cost=26599.87..856768.45 rows=4348 width=311) (actual 
time=38.159..38.182 rows=1 loops=1)
   Join Filter: (c_2.id = c.id)
   ->  Nested Loop Left Join  (cost=2.82..96.06 rows=1 width=244) (actual 
time=3.612..3.621 rows=1 loops=1)
 Join Filter: (con.id = c.id_con)
 Rows Removed by Join Filter: 1
 ->  Nested Loop Left Join  (cost=2.82..94.68 rows=1 width=236) (actual 
time=3.606..3.614 rows=1 loops=1)
   Join Filter: (c_1.id = c.id)
   ->  Index Scan using index_class_tree_full on class c  
(cost=0.28..2.50 rows=1 width=235) (actual time=0.014..0.015 rows=1 loops=1)
 Index Cond: (id = 650)
   ->  GroupAggregate  (cost=2.54..92.15 rows=1 width=9) (actual 
time=3.590..3.595 rows=1 loops=1)
 Group Key: c_1.id
 ->  Nested Loop Left Join  (cost=2.54..91.82 rows=1 
width=233) (actual time=0.270..3.576 rows=20 loops=1)
   ->  Nested Loop Left Join  (cost=2.26..91.52 rows=1 
width=233) (actual time=0.262..3.545 rows=20 loops=1)
 ->  Nested Loop Left Join  (cost=1.97..23.18 
rows=1 width=225) (actual time=0.057..0.252 rows=20 loops=1)
   ->  Nested Loop Left Join  
(cost=1.70..21.60 rows=1 width=213) (actual time=0.050..0.209 rows=20 loops=1)
 ->  Nested Loop Left Join  
(cost=1.42..21.30 rows=1 width=208) (actual time=0.045..0.185 rows=20 loops=1)
   ->  Nested Loop Left Join  
(cost=1.13..18.89 rows=1 width=192) (actual time=0.035..0.134 rows=20 loops=1)
 ->  Nested Loop Left 
Join  (cost=0.86..17.68 rows=1 width=131) (actual time=0.028..0.095 rows=20 
loops=1)
   ->  Nested Loop 
Left Join  (cost=0.57..15.18 rows=1 width=26) (actual time=0.019..0.037 rows=20 
loops=1)
 Join 
Filter: (c_1.id = cp.id_class)
 ->  Index 
Scan using index_class_tree_full on class c_1  (cost=0.28..2.50 rows=1 width=9) 
(actual time=0.008..0.008 rows=1 loops=1)
   
Index Cond: (id = 650)
 ->  Index 
Scan using index_class_prop_class on class_prop cp  (cost=0.29..12.53 rows=12 
width=25) (actual time=0.009..0.017 rows=20 loops=1)

Re: Difference in execution plans pg12 vs pg14

2021-12-11 Thread Peter J. Holzer
On 2021-12-11 06:00:40 +0500, Дмитрий Иванов wrote:
> Afternoon. I was able to make the necessary changes to my base needed to
> migrate win_pg12 to debian pg14.
> But there is a new problem, which was not there at the initial stage so I
> checked:
> 
> win_pg12:
> ->  Index Scan using index_class_tree_full on class c  (cost=0.28..2.50 rows=1
> width=235) (actual time=0.030..0.032 rows=1 loops=1)
>                      Index Cond: (id = 650)
> debian_pg14:  
> ->  Index Scan using index_class_tree_full on class c  (cost=0.28..2.50 rows=1
> width=235) (actual time=842.795..842.796 rows=1 loops=1)
>                      Index Cond: (id = 650)

Is this repeatable or did it just occur once?

32 µs to retrieve a single row via index probably means that it was
already cached in RAM
842796 µs to retrieve a single row via index doesn't even look realistic
for a completely cold database on a slow rotating hard disk. If this
happened only once I suspect that something else interfered (maybe
another I/O intensive process, if this is on a VM maybe even on another
guest). If it is repeatable, something very weird is going on.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature


Re: Difference in execution plans pg12 vs pg14

2021-12-11 Thread Дмитрий Иванов
Thanks to Imre Samu's help, I found out that this is an unwarranted
interference of the JIT compilation. When it is disabled, the short queries
work stably. Before the problem started, I purposely increased the amount
of surrogate data to evaluate performance. Perhaps the logic for enabling
JIT compilation is different in different versions of Postgres. It didn't
show up as clearly on long queries because they were rewritten without JOIN
VIEW and provide filtering before aggregation and linking. I want to make
rougher JIT compiler settings (I think disabling it is fundamentally wrong)
and rewrite all queries similar to long queries.  Thanks.
--
Regards, Dmitry!


сб, 11 дек. 2021 г. в 16:18, Peter J. Holzer :

> Is this repeatable or did it just occur once?
>
> 32 µs to retrieve a single row via index probably means that it was
> already cached in RAM
> 842796 µs to retrieve a single row via index doesn't even look realistic
> for a completely cold database on a slow rotating hard disk. If this
> happened only once I suspect that something else interfered (maybe
> another I/O intensive process, if this is on a VM maybe even on another
> guest). If it is repeatable, something very weird is going on.
>
> hp
>
> --
>_  | Peter J. Holzer| Story must make more sense than reality.
> |_|_) ||
> | |   | h...@hjp.at |-- Charles Stross, "Creative writing
> __/   | http://www.hjp.at/ |   challenge!"
>


Re: Postgresql + containerization possible use case

2021-12-11 Thread Michael Lewis
Interesting. I'm not sure that moving an image for a VM with Postgres is
compatible with the goal of minimizing time/data but if upgrades are yearly
or something, perhaps that is reasonable. It has been 9+ years since that
post as well. But the full data will need to be included in the image if
the goal is not requiring any (significant) tech support at each location,
right? Unless the data that needs to be stored at each site is small, that
seems a bit unlikely to be a feasible option.