Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-10-18 Thread Justin Pryzby
Hi, I just ran into this again in another context (see original dicussion, quoted below). Some time ago, while initially introducting non-default stats target, I set our non-filtering columns to "STATISTICS 10", lower than default, to minimize the size of pg_statistic, which (at least at one poin

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-21 Thread Greg Stark
On 20 July 2017 at 14:19, Tom Lane wrote: > Greg Stark writes: > >> Would it be useful to keep in one of the memory checking assertion builds? > > Why? Code that expects to continue accessing a relcache entry's tupdesc > after closing the relcache entry is broken, independently of whether it's >

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-20 Thread Amit Langote
On 2017/07/20 22:19, Tom Lane wrote: > Greg Stark writes: >> On 19 July 2017 at 00:26, Tom Lane wrote: >>> It's probably a bit late in the v10 cycle to be taking any risks in >>> this area, but I'd vote for ripping out RememberToFreeTupleDescAtEOX >>> as soon as the v11 cycle opens, unless someon

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-20 Thread Tom Lane
Greg Stark writes: > On 19 July 2017 at 00:26, Tom Lane wrote: >> It's probably a bit late in the v10 cycle to be taking any risks in >> this area, but I'd vote for ripping out RememberToFreeTupleDescAtEOX >> as soon as the v11 cycle opens, unless someone can show an example >> of non-broken codi

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-20 Thread Greg Stark
On 19 July 2017 at 00:26, Tom Lane wrote: > It's probably a bit late in the v10 cycle to be taking any risks in > this area, but I'd vote for ripping out RememberToFreeTupleDescAtEOX > as soon as the v11 cycle opens, unless someone can show an example > of non-broken coding that requires it. (An

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-18 Thread Craig Ringer
On 19 July 2017 at 07:26, Tom Lane wrote: > Justin Pryzby writes: > > I've seen this before while doing SET STATISTICS on a larger number of > columns > > using xargs, but just came up while doing ADD of a large number of > columns. > > Seems to be roughly linear in number of children but superl

Re: [HACKERS] [GENERAL] huge RAM use in multi-command ALTER of table heirarchy

2017-07-18 Thread Tom Lane
Justin Pryzby writes: > I've seen this before while doing SET STATISTICS on a larger number of columns > using xargs, but just came up while doing ADD of a large number of columns. > Seems to be roughly linear in number of children but superlinear WRT columns. > I think having to do with catalog u