Re: plan with result cache is very slow when work_mem is not enough

2021-05-09 Thread Andres Freund
Hi, On 2021-05-09 15:57:22 +0300, Yura Sokolov wrote: > Occasionally there is a need to run cassert builds in production to > catch an issue. It is usually ok if cassert build O(1) slower than > optimized biuld (ie it is slower in some constant factor C). But > if cassert build will be quadratical

Re: plan with result cache is very slow when work_mem is not enough

2021-05-09 Thread Yura Sokolov
David Rowley писал 2021-05-09 04:01: On Sun, 9 May 2021 at 03:29, Pavel Stehule wrote: Personally, I have not problem with too slow assertions, although it is not too practical. The main problem is some shock, and feeling so some is wrong. I spent 1 hour detecting if it is a bug or not. Than

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Pavel Stehule
ne 9. 5. 2021 v 3:01 odesílatel David Rowley napsal: > On Sun, 9 May 2021 at 03:29, Pavel Stehule > wrote: > > Personally, I have not problem with too slow assertions, although it is > not too practical. The main problem is some shock, and feeling so some is > wrong. I spent 1 hour detecting if

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread David Rowley
On Sun, 9 May 2021 at 03:29, Pavel Stehule wrote: > Personally, I have not problem with too slow assertions, although it is not > too practical. The main problem is some shock, and feeling so some is wrong. > I spent 1 hour detecting if it is a bug or not. Thanks for spending the time figuring

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread David Rowley
On Sun, 9 May 2021 at 00:07, Tomas Vondra wrote: > > > On 5/8/21 1:27 PM, David Rowley wrote: > > On Sat, 8 May 2021 at 22:33, Tomas Vondra > > wrote: > >> I don't know if there's a better way to do these tests, but if there's > >> not I'd not worry about it too much for now. > > > > So you're -

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Tom Lane
Pavel Stehule writes: > so 8. 5. 2021 v 9:39 odesílatel David Rowley napsal: >> So, I'm -1 for "heavy asserts". > Personally, I have not problem with too slow assertions, although it is not > too practical. I'm very much on David's side here. I'm currently trying to figure out why CLOBBER_CACH

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Pavel Stehule
so 8. 5. 2021 v 9:39 odesílatel David Rowley napsal: > On Sat, 8 May 2021 at 19:03, Yura Sokolov > wrote: > > Perhaps there is need for flag "heavy asserts". Or option > "--enable-cassert=heavy". Then USE_ASSERT_CHECKING could be defined to > integer 1 or 2 depending on "heaviness" of enabled ch

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Tomas Vondra
On 5/8/21 1:27 PM, David Rowley wrote: On Sat, 8 May 2021 at 22:33, Tomas Vondra wrote: I don't know if there's a better way to do these tests, but if there's not I'd not worry about it too much for now. So you're -1 on the proposed patch? Oh! I have not noticed there was a patch. No, I'

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread David Rowley
On Sat, 8 May 2021 at 22:33, Tomas Vondra wrote: > I don't know if there's a better way to do these tests, but if there's > not I'd not worry about it too much for now. So you're -1 on the proposed patch? David

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Tomas Vondra
On 5/8/21 9:39 AM, David Rowley wrote: On Sat, 8 May 2021 at 19:03, Yura Sokolov wrote: Perhaps there is need for flag "heavy asserts". Or option "--enable-cassert=heavy". Then USE_ASSERT_CHECKING could be defined to integer 1 or 2 depending on "heaviness" of enabled checks. I'd rather ex

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread David Rowley
On Sat, 8 May 2021 at 19:03, Yura Sokolov wrote: > Perhaps there is need for flag "heavy asserts". Or option > "--enable-cassert=heavy". Then USE_ASSERT_CHECKING could be defined to > integer 1 or 2 depending on "heaviness" of enabled checks. I'd rather explore all other options before we went

Re: plan with result cache is very slow when work_mem is not enough

2021-05-08 Thread Yura Sokolov
8 мая 2021 г. 00:16:54 GMT+03:00, Tomas Vondra пишет: >On 5/7/21 11:04 PM, David Rowley wrote: >> On Sat, 8 May 2021 at 08:18, Pavel Stehule >wrote: >>> >>> pá 7. 5. 2021 v 21:56 odesílatel David Rowley >napsal: With USE_ASSERT_CHECKING builds, I did add some code that verifies >the >>>

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 14:43, Justin Pryzby wrote: > You could put this into a separate function called by ExecEndResultCache(). > Then anyone that breaks the memory accounting can also call the function in > the > places they changed to help figure out what they broke. I almost did it that way a

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Justin Pryzby
On Sat, May 08, 2021 at 02:26:44PM +1200, David Rowley wrote: > On Sat, 8 May 2021 at 09:16, Tomas Vondra > wrote: > > On 5/7/21 11:04 PM, David Rowley wrote: > > > Another thought I have is that maybe it would be ok just to move > > > memory accounting debug code so it only runs once in > > > Ex

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 09:16, Tomas Vondra wrote: > > On 5/7/21 11:04 PM, David Rowley wrote: > > Another thought I have is that maybe it would be ok just to move > > memory accounting debug code so it only runs once in > > ExecEndResultCache. I struggling to imagine that if the memory > > trackin

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Tomas Vondra
On 5/7/21 11:04 PM, David Rowley wrote: On Sat, 8 May 2021 at 08:18, Pavel Stehule wrote: pá 7. 5. 2021 v 21:56 odesílatel David Rowley napsal: With USE_ASSERT_CHECKING builds, I did add some code that verifies the memory tracking is set correctly when evicting from the cache. This code is p

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 08:18, Pavel Stehule wrote: > > pá 7. 5. 2021 v 21:56 odesílatel David Rowley napsal: >> With USE_ASSERT_CHECKING builds, I did add some code that verifies the >> memory tracking is set correctly when evicting from the cache. This >> code is pretty expensive as it loops over

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Pavel Stehule
pá 7. 5. 2021 v 21:56 odesílatel David Rowley napsal: > On Sat, 8 May 2021 at 07:18, Pavel Stehule > wrote: > > yes, the slowdown is related to debug assertions > > With USE_ASSERT_CHECKING builds, I did add some code that verifies the > memory tracking is set correctly when evicting from the ca

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread David Rowley
On Sat, 8 May 2021 at 07:18, Pavel Stehule wrote: > yes, the slowdown is related to debug assertions With USE_ASSERT_CHECKING builds, I did add some code that verifies the memory tracking is set correctly when evicting from the cache. This code is pretty expensive as it loops over the entire cach

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Pavel Stehule
pá 7. 5. 2021 v 21:06 odesílatel Yura Sokolov napsal: > Pavel Stehule писал 2021-05-07 21:45: > >> > >> Samples: 119K of event 'cycles', 4000 Hz, Event count (approx.): > >> Overhead Shared Object Symbol > >> 79.20% postgres [.] cache_reduce_memory > >> 1.94% [kern

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Tomas Vondra
On 5/7/21 9:06 PM, Yura Sokolov wrote: Pavel Stehule писал 2021-05-07 21:45: Samples: 119K of event 'cycles', 4000 Hz, Event count (approx.): Overhead  Shared Object Symbol 79.20%  postgres  [.] cache_reduce_memory 1.94%  [kernel]  [k] native_write

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Yura Sokolov
Pavel Stehule писал 2021-05-07 21:45: Samples: 119K of event 'cycles', 4000 Hz, Event count (approx.): Overhead Shared Object Symbol 79.20% postgres [.] cache_reduce_memory 1.94% [kernel] [k] native_write_msr_safe 1.63% [kernel]

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Pavel Stehule
pá 7. 5. 2021 v 20:24 odesílatel Pavel Stehule napsal: > Hi > > I am testing new features of Postgres 14, and now I am trying to check the > result cache. Unfortunately on my test data, the result is not too good. > the behaviour is very non linear. Is it expected? > > create table t1(a int, t2_i