Re: Partial index creation always scans the entire table

2020-02-16 Thread Tom Lane
"Mike Sofen" writes: >> From: Tom LaneSent: Sunday, February 16, 2020 7:30 AM >>> I've seen people try to do this before. I wonder if the manual page about >>> partial indexes should explicitly say "don't do that". > Yes please (seriously). The utter beauty of Postgres is the flexibility a

Re: Partial index creation always scans the entire table

2020-02-16 Thread Justin Pryzby
On Sun, Feb 16, 2020 at 04:43:10PM -0800, Mike Sofen wrote: > >From: Tom LaneSent: Sunday, February 16, 2020 7:30 > AM > >I've seen people try to do this before. I wonder if the manual page about > partial indexes should explicitly say "don't do that". > > regards, tom lane > > Yes pleas

RE: Partial index creation always scans the entire table

2020-02-16 Thread Mike Sofen
>From: Tom LaneSent: Sunday, February 16, 2020 7:30 AM >I've seen people try to do this before. I wonder if the manual page about partial indexes should explicitly say "don't do that". > regards, tom lane Yes please (seriously). The utter beauty of Postgres is the flexibility and powe

SubtransControlLock and performance problems

2020-02-16 Thread Lars Aksel Opsahl
Hi On a server with 32 cores and 250 GB memory, with CentOS 7 and kernel 4.4.214-1.el7.elrepo.x86_64, I try to run 30 parallel threads using dblink. (https://github.com/larsop/postgres_execute_parallel) . I have tried to disconnect and reconnect in the dblink code and that did not help. If I

Re: Partial index creation always scans the entire table

2020-02-16 Thread Tom Lane
Justin Pryzby writes: > I was reminded of reading this, but I think it's a pretty different case. > https://heap.io/blog/engineering/running-10-million-postgresql-indexes-in-production Yeah, the critical paragraph in that is This isn’t as scary as it sounds for a two main reasons. First, we

Re: Partial index creation always scans the entire table

2020-02-16 Thread Justin Pryzby
On Sat, Feb 15, 2020 at 10:15:53PM +0100, Laurenz Albe wrote: > > My scenario is that I have a table with 50M rows that are categorized into > > 10K categories. > > I need to create a partial index for each category. I have created a index > > on the category > > column, hoping that postgres can

Re: Partial index creation always scans the entire table

2020-02-16 Thread Tom Lane
Laurenz Albe writes: > On Sat, 2020-02-15 at 19:04 +0800, MingJu Wu wrote: >> My scenario is that I have a table with 50M rows that are categorized into >> 10K categories. >> I need to create a partial index for each category. > You don't need a partial index per category, you need a single inde