Re: plan with result cache is very slow when work_mem is not enough

2021-05-09 Thread Andres Freund
Hi, On 2021-05-09 15:57:22 +0300, Yura Sokolov wrote: > Occasionally there is a need to run cassert builds in production to > catch an issue. It is usually ok if cassert build O(1) slower than > optimized biuld (ie it is slower in some constant factor C). But > if cassert build will be quadratical

Re: plan with result cache is very slow when work_mem is not enough

2021-05-09 Thread Yura Sokolov
David Rowley писал 2021-05-09 04:01: On Sun, 9 May 2021 at 03:29, Pavel Stehule wrote: Personally, I have not problem with too slow assertions, although it is not too practical. The main problem is some shock, and feeling so some is wrong. I spent 1 hour detecting if it is a bug or not. Than

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Pavel Stehule
ne 9. 5. 2021 v 3:01 odesílatel David Rowley napsal: > On Sun, 9 May 2021 at 03:29, Pavel Stehule > wrote: > > Personally, I have not problem with too slow assertions, although it is > not too practical. The main problem is some shock, and feeling so some is > wrong. I spent 1 hour detecting if

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread David Rowley
On Sun, 9 May 2021 at 03:29, Pavel Stehule wrote: > Personally, I have not problem with too slow assertions, although it is not > too practical. The main problem is some shock, and feeling so some is wrong. > I spent 1 hour detecting if it is a bug or not. Thanks for spending the time figuring

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread David Rowley
On Sun, 9 May 2021 at 00:07, Tomas Vondra wrote: > > > On 5/8/21 1:27 PM, David Rowley wrote: > > On Sat, 8 May 2021 at 22:33, Tomas Vondra > > wrote: > >> I don't know if there's a better way to do these tests, but if there's > >> not I'd not worry about it too much for now. > > > > So you're -

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Tom Lane
Pavel Stehule writes: > so 8. 5. 2021 v 9:39 odesílatel David Rowley napsal: >> So, I'm -1 for "heavy asserts". > Personally, I have not problem with too slow assertions, although it is not > too practical. I'm very much on David's side here. I'm currently trying to figure out why CLOBBER_CACH

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Pavel Stehule
so 8. 5. 2021 v 9:39 odesílatel David Rowley napsal: > On Sat, 8 May 2021 at 19:03, Yura Sokolov > wrote: > > Perhaps there is need for flag "heavy asserts". Or option > "--enable-cassert=heavy". Then USE_ASSERT_CHECKING could be defined to > integer 1 or 2 depending on "heaviness" of enabled ch

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Tomas Vondra
On 5/8/21 1:27 PM, David Rowley wrote: On Sat, 8 May 2021 at 22:33, Tomas Vondra wrote: I don't know if there's a better way to do these tests, but if there's not I'd not worry about it too much for now. So you're -1 on the proposed patch? Oh! I have not noticed there was a patch. No, I'

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread David Rowley
On Sat, 8 May 2021 at 22:33, Tomas Vondra wrote: > I don't know if there's a better way to do these tests, but if there's > not I'd not worry about it too much for now. So you're -1 on the proposed patch? David

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Tomas Vondra
On 5/8/21 9:39 AM, David Rowley wrote: On Sat, 8 May 2021 at 19:03, Yura Sokolov wrote: Perhaps there is need for flag "heavy asserts". Or option "--enable-cassert=heavy". Then USE_ASSERT_CHECKING could be defined to integer 1 or 2 depending on "heaviness" of enabled checks. I'd rather ex

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread David Rowley
On Sat, 8 May 2021 at 19:03, Yura Sokolov wrote: > Perhaps there is need for flag "heavy asserts". Or option > "--enable-cassert=heavy". Then USE_ASSERT_CHECKING could be defined to > integer 1 or 2 depending on "heaviness" of enabled checks. I'd rather explore all other options before we went

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Yura Sokolov
8 мая 2021 г. 00:16:54 GMT+03:00, Tomas Vondra пишет: >On 5/7/21 11:04 PM, David Rowley wrote: >> On Sat, 8 May 2021 at 08:18, Pavel Stehule >wrote: >>> >>> pá 7. 5. 2021 v 21:56 odesílatel David Rowley >napsal: With USE_ASSERT_CHECKING builds, I did add some code that verifies >the >>>

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 14:43, Justin Pryzby wrote: > You could put this into a separate function called by ExecEndResultCache(). > Then anyone that breaks the memory accounting can also call the function in > the > places they changed to help figure out what they broke. I almost did it that way a

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Justin Pryzby
On Sat, May 08, 2021 at 02:26:44PM +1200, David Rowley wrote: > On Sat, 8 May 2021 at 09:16, Tomas Vondra > wrote: > > On 5/7/21 11:04 PM, David Rowley wrote: > > > Another thought I have is that maybe it would be ok just to move > > > memory accounting debug code so it only runs once in > > > Ex

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 09:16, Tomas Vondra wrote: > > On 5/7/21 11:04 PM, David Rowley wrote: > > Another thought I have is that maybe it would be ok just to move > > memory accounting debug code so it only runs once in > > ExecEndResultCache. I struggling to imagine that if the memory > > trackin

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Tomas Vondra
On 5/7/21 11:04 PM, David Rowley wrote: On Sat, 8 May 2021 at 08:18, Pavel Stehule wrote: pá 7. 5. 2021 v 21:56 odesílatel David Rowley napsal: With USE_ASSERT_CHECKING builds, I did add some code that verifies the memory tracking is set correctly when evicting from the cache. This code is p

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 08:18, Pavel Stehule wrote: > > pá 7. 5. 2021 v 21:56 odesílatel David Rowley napsal: >> With USE_ASSERT_CHECKING builds, I did add some code that verifies the >> memory tracking is set correctly when evicting from the cache. This >> code is pretty expensive as it loops over

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Pavel Stehule
pá 7. 5. 2021 v 21:56 odesílatel David Rowley napsal: > On Sat, 8 May 2021 at 07:18, Pavel Stehule > wrote: > > yes, the slowdown is related to debug assertions > > With USE_ASSERT_CHECKING builds, I did add some code that verifies the > memory tracking is set correctly when evicting from the ca

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 07:18, Pavel Stehule wrote: > yes, the slowdown is related to debug assertions With USE_ASSERT_CHECKING builds, I did add some code that verifies the memory tracking is set correctly when evicting from the cache. This code is pretty expensive as it loops over the entire cach

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Pavel Stehule
pá 7. 5. 2021 v 21:06 odesílatel Yura Sokolov napsal: > Pavel Stehule писал 2021-05-07 21:45: > >> > >> Samples: 119K of event 'cycles', 4000 Hz, Event count (approx.): > >> Overhead Shared Object Symbol > >> 79.20% postgres [.] cache_reduce_memory > >> 1.94% [kern

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Tomas Vondra
On 5/7/21 9:06 PM, Yura Sokolov wrote: Pavel Stehule писал 2021-05-07 21:45: Samples: 119K of event 'cycles', 4000 Hz, Event count (approx.): Overhead  Shared Object Symbol 79.20%  postgres  [.] cache_reduce_memory 1.94%  [kernel]  [k] native_write

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Yura Sokolov
Pavel Stehule писал 2021-05-07 21:45: Samples: 119K of event 'cycles', 4000 Hz, Event count (approx.): Overhead Shared Object Symbol 79.20% postgres [.] cache_reduce_memory 1.94% [kernel] [k] native_write_msr_safe 1.63% [kernel]

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Pavel Stehule
pá 7. 5. 2021 v 20:24 odesílatel Pavel Stehule napsal: > Hi > > I am testing new features of Postgres 14, and now I am trying to check the > result cache. Unfortunately on my test data, the result is not too good. > the behaviour is very non linear. Is it expected? > > create table t1(a int, t2_i

plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Pavel Stehule
Hi I am testing new features of Postgres 14, and now I am trying to check the result cache. Unfortunately on my test data, the result is not too good. the behaviour is very non linear. Is it expected? create table t1(a int, t2_id int); insert into t1 select random() * 10, random() * 10 fr