Re: Temporarily very slow planning time after a big delete

2019-05-21 Thread Peter Geoghegan
On Tue, May 21, 2019 at 11:27 AM Walter Smith wrote: > Very low. Probably less than ten over all time. I suspect the only use of the > index is to rapidly find the processed=false rows, so the notifiable_type > value isn’t important, really. It would probably work just as well on any > other co

Re: Temporarily very slow planning time after a big delete

2019-05-21 Thread didier
On Tue, May 21, 2019 at 8:27 PM Walter Smith wrote: > On Tue, May 21, 2019 at 11:17 AM Peter Geoghegan wrote: > >> On Tue, May 21, 2019 at 11:16 AM Walter Smith >> wrote: >> > It occurs to me that is a somewhat unusual index -- it tracks >> unprocessed notifications so it gets an insert and del

Re: Temporarily very slow planning time after a big delete

2019-05-21 Thread Walter Smith
On Tue, May 21, 2019 at 11:17 AM Peter Geoghegan wrote: > On Tue, May 21, 2019 at 11:16 AM Walter Smith wrote: > > It occurs to me that is a somewhat unusual index -- it tracks > unprocessed notifications so it gets an insert and delete for every row, > and is normally almost empty. > > Is it a

Re: Temporarily very slow planning time after a big delete

2019-05-21 Thread Peter Geoghegan
On Tue, May 21, 2019 at 11:16 AM Walter Smith wrote: > It occurs to me that is a somewhat unusual index -- it tracks unprocessed > notifications so it gets an insert and delete for every row, and is normally > almost empty. Is it a very low cardinality index? In other words, is the total number

Re: Temporarily very slow planning time after a big delete

2019-05-21 Thread Walter Smith
On Tue, May 21, 2019 at 11:15 AM Peter Geoghegan wrote: > On Tue, May 21, 2019 at 11:12 AM Walter Smith wrote > > I did a VACUUM overnight and got the following. The thing that stands > out to me is that one index (index_unproc_notifications_on_notifiable_type) > took 100x longer to scan than th

Re: Temporarily very slow planning time after a big delete

2019-05-21 Thread Peter Geoghegan
On Tue, May 21, 2019 at 11:12 AM Walter Smith wrote > I did a VACUUM overnight and got the following. The thing that stands out to > me is that one index (index_unproc_notifications_on_notifiable_type) took > 100x longer to scan than the others. That's not the index used in the slow > query, th

Re: Temporarily very slow planning time after a big delete

2019-05-21 Thread Walter Smith
On Mon, May 20, 2019 at 7:15 PM David Rowley wrote: > It would be good if you could confirm the problem is resolved after a > vacuum. Maybe run VACUUM VERBOSE on the table and double check > there's not some large amount of tuples that are "nonremovable". > As I say, the problem resolved itself

Re: Temporarily very slow planning time after a big delete

2019-05-20 Thread David Rowley
On Tue, 21 May 2019 at 14:04, Walter Smith wrote: > I'm so sorry -- I meant to give the version, of course. It's 9.6.13. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=3ca930fc3 has been applied since then. It would be good if you could confirm the problem is resolved after a vac

Re: Temporarily very slow planning time after a big delete

2019-05-20 Thread Walter Smith
Tom Lane writes: >I'm assuming your problem >query involved a join on the indexed column --- whether or not the >final plan did a mergejoin, the planner would consider this There's no join -- the query is SELECT "notifications".* FROM "notifications" WHERE "notifications"."person_id" = ? AND

Re: Temporarily very slow planning time after a big delete

2019-05-20 Thread Walter Smith
I'm so sorry -- I meant to give the version, of course. It's 9.6.13. Thanks, Walter On Mon, May 20, 2019 at 6:05 PM Tom Lane wrote: > Walter Smith writes: > > Today we deleted about 15 million rows in one transaction from this > table. > > Immediately afterwards, a particular SELECT started r

Re: Temporarily very slow planning time after a big delete

2019-05-20 Thread Tom Lane
Walter Smith writes: > Today we deleted about 15 million rows in one transaction from this table. > Immediately afterwards, a particular SELECT started running very slowly -- > 500 to 3000 ms rather than the usual <1ms. > We did an EXPLAIN ANALYZE on this select and it was still doing an index >

Re: Temporarily very slow planning time after a big delete

2019-05-20 Thread David Rowley
On Tue, 21 May 2019 at 12:44, Walter Smith wrote: > > We had a mysterious (to us) slowdown today that I'm hoping someone can > explain just based on PG's principles of operation. It got better by itself > so it seems like it was "normal" behavior -- I just don't know what behavior > it was exhi