Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-20 Thread Ashutosh Bapat
On Fri, Jul 17, 2020 at 10:00 PM Etsuro Fujita wrote: > > On Sat, Jul 18, 2020 at 12:44 AM Ashutosh Bapat > wrote: > > On Fri, Jul 17, 2020 at 8:24 PM Etsuro Fujita > > wrote: > > > On Fri, Jul 17, 2020 at 1:56 AM Alexey Kondratov > > > wrote: > > > > However, there is an issue with aggregates

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-17 Thread Etsuro Fujita
On Sat, Jul 18, 2020 at 12:44 AM Ashutosh Bapat wrote: > On Fri, Jul 17, 2020 at 8:24 PM Etsuro Fujita wrote: > > On Fri, Jul 17, 2020 at 1:56 AM Alexey Kondratov > > wrote: > > > However, there is an issue with aggregates as well. For a query like: > > > > > > SELECT > > > count(*) > > > F

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-17 Thread Ashutosh Bapat
On Fri, Jul 17, 2020 at 8:24 PM Etsuro Fujita wrote: > > On Fri, Jul 17, 2020 at 1:56 AM Alexey Kondratov > wrote: > > However, there is an issue with aggregates as well. For a query like: > > > > SELECT > > count(*) > > FROM > > documents > > WHERE > > company_id = 5; > > > > It w

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-17 Thread Etsuro Fujita
On Fri, Jul 17, 2020 at 1:56 AM Alexey Kondratov wrote: > However, there is an issue with aggregates as well. For a query like: > > SELECT > count(*) > FROM > documents > WHERE > company_id = 5; > > It would be great to teach planner to understand, that it's a > partition-wise aggre

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-16 Thread Andrey V. Lepikhov
On 7/16/20 9:35 PM, Etsuro Fujita wrote: On Thu, Jul 16, 2020 at 8:56 PM Andrey Lepikhov wrote: On 7/16/20 9:55 AM, Etsuro Fujita wrote: On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov wrote: Some real-life test queries show, that all single-node queries aren't pushed-down to the require

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-16 Thread Alexey Kondratov
On 2020-07-16 19:35, Etsuro Fujita wrote: On Thu, Jul 16, 2020 at 8:56 PM Andrey Lepikhov wrote: On 7/16/20 9:55 AM, Etsuro Fujita wrote: On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov wrote: > Some real-life test queries show, that all single-node queries aren't > push

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-16 Thread Alexey Kondratov
On 2020-07-16 14:56, Andrey Lepikhov wrote: On 7/16/20 9:55 AM, Etsuro Fujita wrote: On Wed, Jul 15, 2020 at 9:02 PM Etsuro Fujita wrote: On Wed, Jul 15, 2020 at 12:12 AM Alexey Kondratov wrote: On 2020-07-14 15:27, Ashutosh Bapat wrote: On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov wr

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-16 Thread Etsuro Fujita
On Thu, Jul 16, 2020 at 8:56 PM Andrey Lepikhov wrote: > On 7/16/20 9:55 AM, Etsuro Fujita wrote: > On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov > wrote: > > Some real-life test queries show, that all single-node queries aren't > > pushed-down to the required node. For exa

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-16 Thread Andrey Lepikhov
On 7/16/20 9:55 AM, Etsuro Fujita wrote: On Wed, Jul 15, 2020 at 9:02 PM Etsuro Fujita wrote: On Wed, Jul 15, 2020 at 12:12 AM Alexey Kondratov wrote: On 2020-07-14 15:27, Ashutosh Bapat wrote: On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov wrote: Some real-life test queries show, th

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-15 Thread Etsuro Fujita
On Wed, Jul 15, 2020 at 9:02 PM Etsuro Fujita wrote: > On Wed, Jul 15, 2020 at 12:12 AM Alexey Kondratov > wrote: > > On 2020-07-14 15:27, Ashutosh Bapat wrote: > > > On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov > > > wrote: > > >> Some real-life test queries show, that all single-node quer

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-15 Thread Etsuro Fujita
On Wed, Jul 15, 2020 at 12:12 AM Alexey Kondratov wrote: > On 2020-07-14 15:27, Ashutosh Bapat wrote: > > On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov > > wrote: > >> Some real-life test queries show, that all single-node queries aren't > >> pushed-down to the required node. For example: > >

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-14 Thread Alexey Kondratov
On 2020-07-14 15:27, Ashutosh Bapat wrote: On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov wrote: I built a simple two node multi-tenant schema for tests, which can be easily set up with attached scripts. It creates three tables (companies, users, documents) distributed over two nodes. Every

Re: Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-14 Thread Ashutosh Bapat
On Tue, Jul 14, 2020 at 12:48 AM Alexey Kondratov wrote: > > Hi Hackers, > > The idea of achieving Postgres scaling via sharding using postgres_fdw + > partitioning got a lot of attention last years. Many optimisations have > been done in this direction: partition pruning, partition-wise > aggrega

Partitioning and postgres_fdw optimisations for multi-tenancy

2020-07-13 Thread Alexey Kondratov
Hi Hackers, The idea of achieving Postgres scaling via sharding using postgres_fdw + partitioning got a lot of attention last years. Many optimisations have been done in this direction: partition pruning, partition-wise aggregates / joins, postgres_fdw push-down of LIMIT, GROUP BY, etc. In ma