Re: Getting better results from valgrind leak tracking

2021-03-29 Thread David Rowley
On Tue, 30 Mar 2021 at 06:38, Andres Freund wrote: > On 2021-03-29 11:48:47 +1300, David Rowley wrote: > > I've done that in the attached. I added the > > ParallelBlockTableScanWorkerData as a pointer field in > > HeapScanDescData and change it so we only allocate memory for it for > > just paral

Re: Getting better results from valgrind leak tracking

2021-03-29 Thread Andres Freund
Hi, On 2021-03-29 11:48:47 +1300, David Rowley wrote: > I've done that in the attached. I added the > ParallelBlockTableScanWorkerData as a pointer field in > HeapScanDescData and change it so we only allocate memory for it for > just parallel scans. The field is left as NULL for non-parallel >

Re: Getting better results from valgrind leak tracking

2021-03-28 Thread David Rowley
On Wed, 17 Mar 2021 at 15:31, Andres Freund wrote: > I'm a bit confused about the precise design of rs_private / > ParallelBlockTableScanWorkerData, specifically why it's been added to > TableScanDesc, instead of just adding it to HeapScanDesc? And why is it > allocated unconditionally, instead of

Re: Getting better results from valgrind leak tracking

2021-03-18 Thread Tom Lane
I wrote: > Andres Freund writes: >> There's plenty other hits, but I think I should get back to working on >> making the shared memory stats patch committable. I really wouldn't want >> it to slip yet another year. > +1, so far there's little indication that we're finding any serious leaks > here

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Tom Lane
Andres Freund writes: > The most glaring case is the RelationInitTableAccessMethod() call in > RelationBuildLocalRelation(). Seems like the best fix is to just move > the MemoryContextSwitchTo() to just before the > RelationInitTableAccessMethod(). Although I wonder if we shouldn't go > further,

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Andres Freund
On 2021-03-17 22:33:59 -0400, Tom Lane wrote: > >> I've not yet tried to grok the comment that purports to justify it, > >> but I fail to see why it'd ever be useful to drop values inherited > >> from the postmaster. > > > I can't really make sense of it either. I think it may be trying to > > res

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Andres Freund
Hi, On 2021-03-17 00:01:55 -0400, Tom Lane wrote: > As for the particular point about ParallelBlockTableScanWorkerData, > I agree with your question to David about why that's in TableScanDesc > not HeapScanDesc, but I can't get excited about it not being freed in > heap_endscan. That's mainly beca

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Tom Lane
Andres Freund writes: > On 2021-03-17 21:30:48 -0400, Tom Lane wrote: >> I believe I've just tracked down the cause of that. Those errors >> are (as far as I've seen) only happening in parallel workers, and >> the reason is this gem in RestoreGUCState: ... > Ouch. At least it's a short lived iss

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Andres Freund
Hi, On 2021-03-17 21:30:48 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2021-03-17 18:07:36 -0400, Tom Lane wrote: > >> Huh, interesting. I wonder why that makes the ident problem go away? > > > I spent a bit of time looking at valgrind, and it looks to be explicit > > behaviour: > > >

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Tom Lane
Andres Freund writes: > On 2021-03-17 18:07:36 -0400, Tom Lane wrote: >> Huh, interesting. I wonder why that makes the ident problem go away? > I spent a bit of time looking at valgrind, and it looks to be explicit > behaviour: > >// Our goal is to construct a set of chunks that includes eve

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Andres Freund
Hi, On 2021-03-17 18:07:36 -0400, Tom Lane wrote: > Andres Freund writes: > >> I found a way around that late last night. Need to mark the context > >> itself as an allocation. But I made a mess on the way to that and need > >> to clean the patch up before sending it (and need to drop my > >> gir

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Tom Lane
Andres Freund writes: >> I found a way around that late last night. Need to mark the context >> itself as an allocation. But I made a mess on the way to that and need >> to clean the patch up before sending it (and need to drop my >> girlfriend off first). > Unfortunately I didn't immediately fin

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Andres Freund
Hi, (really need to fix my mobile phone mail program to keep the CC list...) On 2021-03-17 08:15:43 -0700, Andres Freund wrote: > On Wed, Mar 17, 2021, at 07:16, Tom Lane wrote: > > Andres Freund writes: > > > On 2021-03-16 20:50:17 -0700, Andres Freund wrote: > > Meanwhile, I'm still trying to

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Andres Freund
Hi, On Wed, Mar 17, 2021, at 07:16, Tom Lane wrote: > Andres Freund writes: > > On 2021-03-16 20:50:17 -0700, Andres Freund wrote: > Meanwhile, I'm still trying to understand why valgrind is whining > about the rd_indexcxt identifier strings. AFAICS it shouldn't. I found a way around that late

Re: Getting better results from valgrind leak tracking

2021-03-17 Thread Tom Lane
Andres Freund writes: > On 2021-03-16 20:50:17 -0700, Andres Freund wrote: >> What I meant was that I didn't understand how there's not a leak >> danger when compilation fails halfway through, given that the context >> in question is below TopMemoryContext and that I didn't see a relevant >> TRY b

Re: Getting better results from valgrind leak tracking

2021-03-16 Thread Andres Freund
Hi, On 2021-03-16 20:50:17 -0700, Andres Freund wrote: > What I meant was that I didn't understand how there's not a leak > danger when compilation fails halfway through, given that the context > in question is below TopMemoryContext and that I didn't see a relevant > TRY block. But that probably

Re: Getting better results from valgrind leak tracking

2021-03-16 Thread Tom Lane
"Andres Freund" writes: > On Tue, Mar 16, 2021, at 20:01, Tom Lane wrote: >> That seems both unnecessary and impractical. We have to consider that >> everything-still-reachable is an OK final state. > I don't consider "still reachable" a leak. Just definitely unreachable. OK, we're in violent a

Re: Getting better results from valgrind leak tracking

2021-03-16 Thread Andres Freund
Hi, On Tue, Mar 16, 2021, at 20:01, Tom Lane wrote: > Andres Freund writes: > > On 2021-03-16 19:36:10 -0400, Tom Lane wrote: > >> It doesn't appear to me that we leak much at all, at least not if you > >> are willing to take "still reachable" blocks as not-leaked. > > > Well, I think for any so

Re: Getting better results from valgrind leak tracking

2021-03-16 Thread Tom Lane
Andres Freund writes: > On 2021-03-16 19:36:10 -0400, Tom Lane wrote: >> It doesn't appear to me that we leak much at all, at least not if you >> are willing to take "still reachable" blocks as not-leaked. > Well, I think for any sort of automated testing - which I think would be > useful - we'd

Re: Getting better results from valgrind leak tracking

2021-03-16 Thread Zhihong Yu
Hi, For the second last trace involving SearchCatCacheList (catcache.c:1691), the ctlist's members are stored in cl->members array where cl is returned at the end of SearchCatCacheList. Maybe this was not accounted for by valgrind ? Cheers On Tue, Mar 16, 2021 at 7:31 PM Andres Freund wrote: >

Re: Getting better results from valgrind leak tracking

2021-03-16 Thread Andres Freund
Hi, David, there's a question about a commit of yours below, hence adding you. On 2021-03-16 19:36:10 -0400, Tom Lane wrote: > Out of curiosity I poked at this for a little while. Cool. > It doesn't appear to me that we leak much at all, at least not if you > are willing to take "still reachab

Getting better results from valgrind leak tracking

2021-03-16 Thread Tom Lane
[ starting a new thread for this ] Andres Freund writes: > I wonder if it'd be worth starting to explicitly annotate all the places > that do allocations and are fine with leaking them. E.g. by introducing > malloc_permanently() or such. Right now it's hard to use valgrind et al > to detect leaks