On Wed, Oct 7, 2020 at 12:51 AM Tom Lane <t...@sss.pgh.pa.us> wrote: > Ashutosh Bapat <ashutosh.bapat....@gmail.com> writes: > > On Mon, Oct 5, 2020 at 11:59 PM Tom Lane <t...@sss.pgh.pa.us> wrote: > >> ... we could avoid the growth in eclass members for large partition sets > >> if we simply didn't store child eclass members, instead translating > >> on-the-fly when we need to deal with child rels. I have a patch > >> about half done, but it won't be possible to determine the true > >> performance implications of that idea until it's all done. More > >> later.
+1 to this idea. We've seen mainly get_eclass_for_sort_expr() become a bottleneck with large partition sets and getting rid of it would be really great. > > If we translate more than ones, every time someone comes searching for > > an EC member, we will leak memory in the planner memory context, which > > anyway gets bloated because of the huge number of translations even > > when done only once. So that needs to be done rather carefully. > > I'm not terribly concerned about that. There's not a "huge number" > of such translations to be done --- it's more like one per index. > (Also, we could very possibly build the translations in a temp > context that gets reset every so often, if it becomes an issue.) Yeah, the memory bloat from this should be minimal. AFAICS, the only place that demands a child expression from a given matched EC thus needing a translation is createplan.c, which I would expect to be a place that doesn't spend a lot of memory because there's no exploratory processing. > I am a bit worried about whether we'll be spending a lot more cycles > to do the added translations; though some of that should be bought > back by having fewer EC members to compare to. In any event, testing > a working patch will be a lot more illuminating than speculation. Agreed. -- Amit Langote EDB: http://www.enterprisedb.com [1] https://www.postgresql.org/message-id/CAApHDvrEXcadNYAAdq6RO0eKZUG6rRHXJGAbpzj8y432gCD9bA%40mail.gmail.com
