Re: Partitioning a table by integer value (preferably in place)

2021-08-13 Thread Vijaykumar Jain
On Fri, 13 Aug 2021 at 21:07, Michael Lewis wrote: > On Fri, Aug 13, 2021 at 3:02 AM Vijaykumar Jain < > vijaykumarjain.git...@gmail.com> wrote: > >> ... use binary split for large partitions, to avoid large row movements. >> > > Would you expound on this? > if we have range partitions based on

Re: Partitioning a table by integer value (preferably in place)

2021-08-13 Thread Michael Lewis
On Fri, Aug 13, 2021 at 3:02 AM Vijaykumar Jain < vijaykumarjain.git...@gmail.com> wrote: > ... use binary split for large partitions, to avoid large row movements. > Would you expound on this?

Re: Partitioning a table by integer value (preferably in place)

2021-08-13 Thread Vijaykumar Jain
On Fri, 13 Aug 2021 at 14:13, Pól Ua Laoínecháin wrote: > Thanks again for your questions - they gave me pause for thought and I > will try to apply them in future partitioning scenarios. (Unfortunatly > :-) ) there is no magic number of partitions for, say, a given size of > table - otherwise it

Re: Partitioning a table by integer value (preferably in place)

2021-08-13 Thread Pól Ua Laoínecháin
Dear Ron, > > Just a quick follow-up - could I script the creation of 1000 > > partitions using bash > Sure. That's what scripting languages are for. Thank you so much for your helpful and expansive answer. No wonder everyone talks about how friendly and welcoming the PostgreSQL community is!

Re: Partitioning a table by integer value (preferably in place)

2021-08-13 Thread Pól Ua Laoínecháin
Hi Vijay, and thanks for replying, >> I have a 400GB joining table (one SMALLINT and the other INTEGER - >> Primary Keys on other tables) with 1000 fields on one side and 10M on >> the other, so 10,000M (or 10Bn) records all told. > My queries: >> results with the existing setup? Does it look

Re: Partitioning a table by integer value (preferably in place)

2021-08-12 Thread Pól Ua Laoínecháin
Hi all - I resolved my issue - and took a 25 minute query down to 5 seconds. > I have a 400GB joining table (one SMALLINT and the other INTEGER - > What I would like to do is to partition by the SMALLINT (1 - 1000) > value - which would give 1,000 tables of 400MB each. I found this site very help

Re: Partitioning a table by integer value (preferably in place)

2021-08-09 Thread Vijaykumar Jain
> > > > I have a 400GB joining table (one SMALLINT and the other INTEGER - > Primary Keys on other tables) with 1000 fields on one side and 10M on > the other, so 10,000M (or 10Bn) records all told. > My queries: > > Do you have any explain analyze,buffers results with the existing setup? Does

Re: Partitioning a table by integer value (preferably in place)

2021-08-09 Thread Ron
On 8/9/21 9:14 AM, Pól Ua Laoínecháin wrote: Hi again all, Just a quick follow-up - could I script the creation of 1000 partitions using bash Sure.  That's what scripting languages are for. or PL/pgSQL? I think (rightly/wrongly?) that this may be the best solution? I've found samples on t

Re: Partitioning a table by integer value (preferably in place)

2021-08-09 Thread Pól Ua Laoínecháin
Hi again all, Just a quick follow-up - could I script the creation of 1000 partitions using bash or PL/pgSQL? I think (rightly/wrongly?) that this may be the best solution? I've found samples on the web, but they are for partitioning by date - a quick sample by integer would be gratefully receive

Partitioning a table by integer value (preferably in place)

2021-08-09 Thread Pól Ua Laoínecháin
Hi all, Linux Fedora 34 1TB Samsung SSD 4 CPUs, 2 cores PostgreSQL 12.7 (can upgrade if a better solution is to be found in 13 or even 14 beta2 - currently testing a proposed solution, so by the time it's fully implemented, 14 should be on GA and hey, I might even find a bug or two to help with