Re: Plans for partitioning of inheriting tables

2024-11-02 Thread Thiemo Kellner
My bad. I was expecting primary and unique to be mentioned here, so I did not read on. **INCLUDING CONSTRAINTS*** **#[https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-PARMS-LIKE-OPT-CONSTRAINTS]* /*CHECK*// constraints will be copied. No distinction is made between

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread Adrian Klaver
On 11/1/24 16:10, thi...@gelassene-pferde.biz wrote: Adrian Klaver escribió: On 11/1/24 13:47, Thiemo Kellner wrote: It looks to me basically to be a "create table A as select * from B where false". No it more capable then that. Yes, I wrote basically, not exactly. CREATE TABLE LIKE h

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread thiemo
Adrian Klaver escribió: On 11/1/24 13:47, Thiemo Kellner wrote: It looks to me basically to be a "create table A as select * from B where false". No it more capable then that. Yes, I wrote basically, not exactly. CREATE TABLE LIKE has like_option which allows to transfer over more

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread Adrian Klaver
E_TECH table and expect to see all the rows from the other 2 tables in that one table combined, then you could use CREATE TABLE (LIKE ...) instead of inheritance. That way your "child" tables would become normal tables and you could use declarative partitioning on them. Even if you are q

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread Thiemo Kellner
PLATE_TECH table and expect to see all the rows >>> from the other 2 tables in that one table combined, then you could use >>> CREATE TABLE (LIKE ...) instead of inheritance. That way your "child" >>> tables would become normal tables and you could use declara

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread Adrian Klaver
re not actually querying the TEMPLATE_TECH table and expect to see all the rows from the other 2 tables in that one table combined, then you could use CREATE TABLE (LIKE ...) instead of inheritance. That way your "child" tables would become normal tables and you could use declarative

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread thiemo
es and you could use declarative partitioning on them. �� Even if you are querying the TEMPLATE_TECH table, you could still do that by turning the TEMPLATE_TECH table into a view which performs a UNION ALL over the other tables.

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread thiemo
Adrian Klaver escribió: Even if there where plans, any changes would happen in the future and would not be help the now problem. Yes and no. I can live without the partitioning, as I do not intend to load data from more than one source. Other might. But until others want to load data

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread Torsten Förtsch
CREATE TABLE (LIKE ...) instead of inheritance. That way your "child" tables would become normal tables and you could use declarative partitioning on them. Even if you are querying the TEMPLATE_TECH table, you could still do that by turning the TEMPLATE_TECH table into a view which performs a

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread Adrian Klaver
you can convince then early enough or at all. I was not trying to convince anyone to do anything about the implementation of declarative partitioning. I have been just curious if there were plans. If I have raised the impression of the former, I am sorry. Even if there where plans, any changes

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread thiemo
not trying to convince anyone to do anything about the implementation of declarative partitioning. I have been just curious if there were plans. If I have raised the impression of the former, I am sorry. What I getting at is that you need to start thinking of another way of doing this if

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread Adrian Klaver
On 11/1/24 01:41, thi...@gelassene-pferde.biz wrote: Adrian Klaver escribió: It is just not the way you want to do it, see: https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-USING-INHERITANCE Thanks for your patience. Maybe I am not clever enough to

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread Achilleas Mantzios - cloud
On 10/24/24 21:58, David G. Johnston wrote: On Thursday, October 24, 2024, wrote: Up to version 17, partitioning of tables inheriting from other tables is not possible. psql:../code_files/data_storage/PostgreSQL/tables/TOPO_FILES.pg_sql:68: ERROR:  no se puede crear

Re: Plans for partitioning of inheriting tables

2024-11-01 Thread thiemo
Adrian Klaver escribió: It is just not the way you want to do it, see: https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-USING-INHERITANCE Thanks for your patience. Maybe I am not clever enough to understand you. I shall try to explain what I try to do

Re: Plans for partitioning of inheriting tables

2024-10-31 Thread Adrian Klaver
On 10/25/24 11:47, Thiemo Kellner wrote: Am 25.10.2024 um 17:57 schrieb Adrian Klaver: I do not feel it applies to my case. I tried to create a partitioned table that inherits columns from a base table. The documentation you provided the URL seems to speak of realising partitioning by using

Plans for partitioning of inheriting tables

2024-10-25 Thread Thiemo Kellner
Am 25.10.2024 um 17:57 schrieb Adrian Klaver: I do not feel it applies to my case. I tried to create a partitioned table that inherits columns from a base table. The documentation you provided the URL seems to speak of realising partitioning by using inheritance. This needs a code example

Re: Plans for partitioning of inheriting tables

2024-10-25 Thread Thiemo Kellner
Am 25.10.2024 um 17:57 schrieb Adrian Klaver: I do not feel it applies to my case. I tried to create a partitioned table that inherits columns from a base table. The documentation you provided the URL seems to speak of realising partitioning by using inheritance. This needs a code example

Re: Plans for partitioning of inheriting tables

2024-10-25 Thread Adrian Klaver
On 10/24/24 22:33, Thiemo Kellner wrote: Thanks for taking this up. 24.10.2024 22:44:11 Adrian Klaver : 1) Have you looked at?: https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-USING-INHERITANCE I do not feel it applies to my case. I tried to create a

Re: Plans for partitioning of inheriting tables

2024-10-25 Thread Adrian Klaver
On 10/24/24 12:47 PM, thi...@gelassene-pferde.biz wrote: Hi Up to version 17, partitioning of tables inheriting from other tables is not possible. psql:../code_files/data_storage/PostgreSQL/tables/TOPO_FILES.pg_sql:68: ERROR:  no se puede crear una tabla particionada como hija de herencia

Re: Plans for partitioning of inheriting tables

2024-10-24 Thread Thiemo Kellner
Thanks for taking this up. 24.10.2024 22:44:11 Adrian Klaver : > > 1) Have you looked at?: > > https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-USING-INHERITANCE I do not feel it applies to my case. I tried to create a partitioned table that inhe

Re: Plans for partitioning of inheriting tables

2024-10-24 Thread Thiemo Kellner
24.10.2024 22:58:39 David G. Johnston : > > My impression of things is that directly using “inherit” for table creation > is considered deprecated at this point.  No one has interest in expanding on > the feature nor even recommends it be used in new development.  That > particular unique featu

Plans for partitioning of inheriting tables

2024-10-24 Thread thiemo
Hi Up to version 17, partitioning of tables inheriting from other tables is not possible. psql:../code_files/data_storage/PostgreSQL/tables/TOPO_FILES.pg_sql:68: ERROR: no se puede crear una tabla particionada como hija de herencia Are there plans to support this in the future? I could

Re: Plans for partitioning of inheriting tables

2024-10-24 Thread David G. Johnston
On Thursday, October 24, 2024, wrote: > > Up to version 17, partitioning of tables inheriting from other tables is > not possible. > >> psql:../code_files/data_storage/PostgreSQL/tables/TOPO_FILES.pg_sql:68: >> ERROR: no se puede crear una tabla particionada como hij

Re: Partitioning and unique key

2024-09-02 Thread Laurenz Albe
On Tue, 2024-09-03 at 10:39 +0530, veem v wrote: > As you rightly said "they will make it more difficult to detach a partition." > , > we are really seeing a longer time when detaching parent table partitions. > It runs forever sometimes. So do you mean it's because we have primary key > defined t

Re: Partitioning and unique key

2024-09-02 Thread veem v
On Tue, 3 Sept 2024 at 01:14, Laurenz Albe wrote: > > You can keep the primary key defined on both columns if it is good enough > for you. > But it will give you lower guarantees of uniqueness: with that primary > key, there could > be two rows with a different timestamp, but the same "txn_id", a

Re: Partitioning and unique key

2024-09-02 Thread Laurenz Albe
t; only on the transaction_id column, we have to include > > > transaction_timestamp with it as > > > a composite key. So I want to understand from experts if there is any > > > possible way to > > > satisfy both partitioning on transaction_timestamp column and

Re: Partitioning and unique key

2024-09-02 Thread veem v
tamp with it as > > a composite key. So I want to understand from experts if there is any > possible way to > > satisfy both partitioning on transaction_timestamp column and unique key > or pk just on > > transaction_id only? > > No, you cannot have both. > > Usu

Re: Partitioning and unique key

2024-09-02 Thread Laurenz Albe
xperts if there is any > possible way to > satisfy both partitioning on transaction_timestamp column and unique key or > pk just on > transaction_id only?  No, you cannot have both. Usually the solution is to *not* create a primary key on the partitioned table and instead create a pr

Re: Partitioning and unique key

2024-09-01 Thread veem v
On Sun, 1 Sept 2024 at 11:38, veem v wrote: > > On Sun, 1 Sept 2024 at 10:03, veem v wrote: > >> >> On Sun, 1 Sept 2024 at 09:13, David G. Johnston < >> david.g.johns...@gmail.com> wrote: >> >>> On Saturday, August 31, 2024, veem v wrote: >>> iii)And then alter the datatype of th

Re: Partitioning and unique key

2024-08-31 Thread veem v
On Sun, 1 Sept 2024 at 10:03, veem v wrote: > > On Sun, 1 Sept 2024 at 09:13, David G. Johnston < > david.g.johns...@gmail.com> wrote: > >> On Saturday, August 31, 2024, veem v wrote: >> >>> >>> >>> iii)And then alter the datatype of the partition key transaction_date to >>> DATE in one shot at

Re: Partitioning and unique key

2024-08-31 Thread veem v
On Sun, 1 Sept 2024 at 09:13, David G. Johnston wrote: > On Saturday, August 31, 2024, veem v wrote: > >> >> >> iii)And then alter the datatype of the partition key transaction_date to >> DATE in one shot at the table level(which should be fast as its having more >> granularity as compare to exi

Re: Partitioning and unique key

2024-08-31 Thread David G. Johnston
mestamp? > There is presently no such thing as a cross-partition unique constraint. If you define the constraint on the [partitioned] table the documentation is perfectly clear, as are I believe the error messages, that it will require all partitioning columns to be included - since that is what happ

Re: Partitioning and unique key

2024-08-31 Thread veem v
On Sun, 1 Sept 2024 at 03:58, Adrian Klaver wrote: > > The model is at odds with itself and untenable. If the tables hold > multiple rows for a given transaction_id then you cannot have a > PK/Unique constraint on that column. Seems there is a decided lack of > any planning. The only way I can s

Re: Partitioning and unique key

2024-08-31 Thread Adrian Klaver
in future and will be queried on the transaction_timestamp filter so we can't really avoid the partitioning option here considering future growth. But due to postgres limitations we are unable to have this unique constraint or primary key only on the transaction_id column, we h

Partitioning and unique key

2024-08-31 Thread veem v
ied on the transaction_timestamp filter so we can't really avoid the partitioning option here considering future growth. But due to postgres limitations we are unable to have this unique constraint or primary key only on the transaction_id column, we have to include transaction_timestamp with it as a composite

Re: Question related to partitioning with pg_partman

2024-03-10 Thread Adrian Klaver
nsideration of whether daily partitions are really what you want? 2) What you hope to get out of the partitioning? a) If it is confining queries to the partition boundaries then you have already stated that is not going to happen. b) If it is for data pruning purposes, then you have s

Re: Question related to partitioning with pg_partman

2024-03-10 Thread sud
On Sun, Mar 10, 2024 at 11:31 PM Adrian Klaver wrote: > 1) The partition will be across one day(24 hours) it is just the times > may confuse people. Per you example 2024-03-07 00:00:00+00 is the same > time as 2024-03-06 19:00:00-05 for EST. The issue is that the +00 and > -05 maybe ignored. Als

Re: Question related to partitioning with pg_partman

2024-03-10 Thread Adrian Klaver
On 3/10/24 10:51, sud wrote: On Sun, Mar 10, 2024 at 10:32 PM Adrian Klaver mailto:adrian.kla...@aklaver.com>> wrote: On 3/10/24 05:12, sud wrote: > > In my example in the first post, I see, if someone connected to a RDS > Postgres database and run the create partition

Re: Question related to partitioning with pg_partman

2024-03-10 Thread sud
On Sun, Mar 10, 2024 at 10:32 PM Adrian Klaver wrote: > On 3/10/24 05:12, sud wrote: > > > > In my example in the first post, I see, if someone connected to a RDS > > Postgres database and run the create partition command using pg_partman > > by setting the timezone as "UTC", the 7th march partit

Re: Question related to partitioning with pg_partman

2024-03-10 Thread Adrian Klaver
On 3/10/24 05:12, sud wrote: '2024-03-07 00:00:00+00' and '2024-03-06 19:00:00-05' are the same time as is '2024-03-07 00:00:00-05' and '2024-03-07 05:00:00+00'. Still I would think for sanity sake you would want to stick with UTC. Thank you so much Adrian. In my example in

Re: Question related to partitioning with pg_partman

2024-03-10 Thread sud
On Sat, Mar 9, 2024 at 3:41 AM Adrian Klaver wrote: > On 3/8/24 00:23, sud wrote: > > > > Starting a new thread... > > > > Something interesting and not sure if its expected behaviour as below. > > We are also confused a bit here. > > > > In the below example we created two partitioned tables on

Re: Question related to partitioning with pg_partman

2024-03-08 Thread Adrian Klaver
On 3/8/24 00:23, sud wrote: Starting a new thread... Something interesting and not sure if its expected behaviour as below. We are also confused a bit here. In the below example we created two partitioned tables on timestamptz type columns with different time zones and the child partitions

Re: Question related to partitioning with pg_partman

2024-03-08 Thread sud
Can somebody help me to understand the behaviour? >

Question related to partitioning with pg_partman

2024-03-08 Thread sud
Starting a new thread... Something interesting and not sure if its expected behaviour as below. We are also confused a bit here. In the below example we created two partitioned tables on timestamptz type columns with different time zones and the child partitions are created appropriately with bou

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-22 Thread Greg Sabino Mullane
> > I have a table that is capturing what is, basically, time series data Time series data usually is concerned with "recent" data, and has a subsequent drop off date. This is ideal for partitioning by timestamp - not only do your queries only need to hit a few of the total tab

Re: Partitioning options

2024-02-21 Thread Alec Lazarescu
l.org/message-id/CAE%2BE%3DSQacy6t_3XzCWnY1eiRcNWfz4pp02FER0N7mU_F%2Bo8G_Q%40mail.gmail.com Alec On Tue, Feb 20, 2024 at 11:59 AM Justin wrote: > > > On Sun, Feb 18, 2024 at 5:20 PM Alec Lazarescu wrote: >> >> "Would probably look at a nested partitioning" >> &g

Re: Partitioning options

2024-02-20 Thread Justin
On Sun, Feb 18, 2024 at 5:20 PM Alec Lazarescu wrote: > "Would probably look at a nested partitioning" > > I'm not the original poster, but I have a schema with nested > (composite) partitions and I do run into some significant > inefficiencies compared to fla

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-19 Thread Greg Sabino Mullane
> case is to query across all sources, asking for a set is unusual) a > reasonable approach? I am struggling to see the overall benefit of partitioning here. How many total rows and inserts per second, anyway? > If the app already knows the src (as evidenced by adding in "and sr

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-19 Thread David Rowley
On Mon, 19 Feb 2024 at 22:07, Darryl Green wrote: > > On Mon, 19 Feb 2024 at 14:23, David Rowley wrote: > > > > On Mon, 19 Feb 2024 at 16:32, Darryl Green wrote: > > > 2) It would be nice to be able to specify the id as pk on the table being > > > partitioned (as it was in the non-partitioned d

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-19 Thread David G. Johnston
On Monday, February 19, 2024, David G. Johnston wrote: > On Monday, February 19, 2024, Darryl Green wrote: > >> >> > It may be possible to still have it work by doing a speculative record >> > in the index for the target table then go and check all of the other >> > indexes before marking the sp

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-19 Thread David G. Johnston
On Monday, February 19, 2024, Darryl Green wrote: > > > It may be possible to still have it work by doing a speculative record > > in the index for the target table then go and check all of the other > > indexes before marking the speculative entry as valid. > > It is always valid - except in the

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-19 Thread Darryl Green
On Mon, 19 Feb 2024 at 17:31, Peter Eisentraut wrote: > > On 19.02.24 04:32, Darryl Green wrote: > > I note that in Postgresql 16 identity column handling in partitioned > > tables has been aligned to the view that the partitioned table as a > > whole is a single relation (and so a unique identity

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-19 Thread Darryl Green
that all partitions are using the same IDENTITY source of (unique) key values. Yes I realise that even with this it is possible to force a value OVERRIDING SYSTEM VALUE and of course by changing constraints over time. I can imagine this could make some changes expensive. > > The problem wi

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-19 Thread Ketan Popat
; would seem only to need the "partition by" validation to allow a column not > mentioned in partition by clause to be defined as pk or unique if and only > if the pk/unique column is an identity column. Not a big deal but is this > practical/valid? > > One of the potential sol

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-18 Thread Peter Eisentraut
On 19.02.24 04:32, Darryl Green wrote: I note that in Postgresql 16 identity column handling in partitioned tables has been aligned to the view that the partitioned table as a whole is a single relation (and so a unique identity across partitions). This makes sense. The change that I think yo

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-18 Thread David Rowley
eing inserted doesn't exist in some other partition? The problem with a single index is that it kinda defeats the purpose of partitioning, i.e., "my table is large and I want to split it up". Operations such as DETACH PARTITION would have to become more than just a metadata operation

Re: Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-18 Thread David G. Johnston
lude the identity in the partitioning key (if I have it as the pk) so > that the uniqueness can be checked. But - using a (shared across all > partition tables) identity sequence should be enough to ensure uniqueness > without this. > A sequences is a value generator - its stat

Partitioning, Identity and Uniqueness (given pg 16 changes)

2024-02-18 Thread Darryl Green
ity column. But I have to include the identity in the partitioning key (if I have it as the pk) so that the uniqueness can be checked. But - using a (shared across all partition tables) identity sequence should be enough to ensure uniqueness without this. Currently I need to define my parti

Re: Partitioning options

2024-02-18 Thread Alec Lazarescu
"Would probably look at a nested partitioning" I'm not the original poster, but I have a schema with nested (composite) partitions and I do run into some significant inefficiencies compared to flat partitions in various schema metadata operations (queries to get the list of t

Re: Partitioning options

2024-02-11 Thread Justin
Hi Marc, Nested partitioning still allows for simple data deletion by dropping the table that falls in that date range. Probably thinking of partitioning by multicolomn rules which is very complex to set up On Fri, Feb 9, 2024, 10:29 AM Marc Millas wrote: > > > > On Thu, Feb 8

Re: Partitioning options

2024-02-09 Thread Marc Millas
On Thu, Feb 8, 2024 at 10:25 PM Justin wrote: > Hi Sud, > > Would not look at HASH partitioning as it is very expensive to add or > subtract the number of partitions. > > Would probably look at a nested partitioning using customer ID using > range or list of IDs then by t

Re: Partitioning options

2024-02-08 Thread Justin
Hi Sud, Would not look at HASH partitioning as it is very expensive to add or subtract the number of partitions. Would probably look at a nested partitioning using customer ID using range or list of IDs then by transaction date, Its easy to add partitions and balance the partitions segments

Re: Partitioning options

2024-02-08 Thread Greg Sabino Mullane
really gain you much, given you would be hashing it, the customers are unevenly distributed, and OP talked about filtering on the customer_id column. A hash partition would just be a lot more work and complexity for us humans and for Postgres. Partitioning for the sake of partitioning is not a good th

Re: Partitioning options

2024-02-08 Thread Jim Nasby
On 2/8/24 1:43 PM, veem v wrote: On Thu, 8 Feb 2024 at 20:08, Greg Sabino Mullane <mailto:htamf...@gmail.com>> wrote: Should we go for simple daily range partitioning on the transaction_date column? This one gets my vote. That and some good indexes. H

Re: Partitioning options

2024-02-08 Thread veem v
; "transaction date" column. >> > ... > >> Should we go for simple daily range partitioning on the transaction_date >> column? >> > > This one gets my vote. That and some good indexes. > > Cheers, > Greg > > Hello Greg, Out of curiosity, As OP m

Re: Partitioning options

2024-02-08 Thread Greg Sabino Mullane
On Thu, Feb 8, 2024 at 12:42 AM sud wrote: ... > The key transaction table is going to have ~450 Million transactions per > day and the data querying/filtering will always happen based on the > "transaction date" column. > ... > Should we go for simple daily

Partitioning options

2024-02-07 Thread sud
ly distributed , they will be skewed in nature for e.g. Some of the big customers will hold majority of the transactions (say 20-30% of total transactions) and other are distributed among others, but again not equally. So my question was , in the above scenario should we go for a composite pa

Re: Question on partitioning

2024-02-06 Thread Ron Johnson
On Tue, Feb 6, 2024 at 2:40 PM veem v wrote: > Thank you Laurenz. Got it. > > So basically , you mean to say any DDL on a table won't allow the table to > be read by other processes. I was under the assumption that it should allow > the read queries to move ahead at least. I must be wrong here. T

Re: Question on partitioning

2024-02-06 Thread veem v
Thank you Laurenz. Got it. So basically , you mean to say any DDL on a table won't allow the table to be read by other processes. I was under the assumption that it should allow the read queries to move ahead at least. I must be wrong here. Thanks for correcting me. On Tue, 6 Feb 2024 at 15:46, L

Re: Question on partitioning

2024-02-06 Thread Laurenz Albe
On Tue, 2024-02-06 at 00:26 +0530, veem v wrote: > On Mon, 5 Feb 2024 at 17:52, Laurenz Albe wrote: > > On Mon, 2024-02-05 at 03:09 +0530, veem v wrote: > > > In postgresql, Is it possible to partition an existing nonpartitioned > > > table having data > > > already residing in it and indexes and

Re: Question on partitioning

2024-02-05 Thread veem v
On Mon, 5 Feb 2024 at 17:52, Laurenz Albe wrote: > On Mon, 2024-02-05 at 03:09 +0530, veem v wrote: > > In postgresql, Is it possible to partition an existing nonpartitioned > table having data > > already residing in it and indexes and constraints defined in it, > without the need of > > manuall

Re: Question on partitioning

2024-02-05 Thread Laurenz Albe
On Mon, 2024-02-05 at 03:09 +0530, veem v wrote: > In postgresql, Is it possible to partition an existing nonpartitioned table > having data > already residing in it and indexes and constraints defined in it, without the > need of > manually moving the data around, to make it faster? Similarly me

Re: Question on partitioning

2024-02-04 Thread Ron Johnson
On Sun, Feb 4, 2024 at 4:40 PM veem v wrote: > Hello All, > In postgresql, Is it possible to partition an existing nonpartitioned > table having data already residing in it and indexes and constraints > defined in it, without the need of manually moving the data around, to make > it faster? Simil

Question on partitioning

2024-02-04 Thread veem v
Hello All, In postgresql, Is it possible to partition an existing nonpartitioned table having data already residing in it and indexes and constraints defined in it, without the need of manually moving the data around, to make it faster? Similarly merging multiple partitions to one partition or spli

Re: Correct way of using complex expressions as partitioning key

2023-10-27 Thread Thomas Kellerer
+-+- > 1 | 100 | 100 > (1 row) > > test=# select * from test_2; > id | v1 | v2 > +-+- > 1 | 100 | 101 > (1 row) It seems you are trying to simulate hash partitioning using that expression. Why not use hash partitioning directly then? create table test

Re: Correct way of using complex expressions as partitioning key

2023-10-27 Thread Laurenz Albe
t=# explain analyze select * from test where ((v1 + v2) % 10) = 0 and v1 = > 100 and v2 = 100; > [partition pruning] Yes, you only get partition pruning if the WHERE clause contains a comparison with the partitioning key. There is no way around that. Yours, Laurenz Albe

Correct way of using complex expressions as partitioning key

2023-10-27 Thread Alexander Rumyantsev
Hello! Is there some correct way to use complex expressions as a key for partitioned table? Inserting works as expected, but select runs over all partitions until use complete partition key expression as predicate test=# create table test ( id text, v1 bigint, v2 bigint

Re: partitioning

2023-10-23 Thread David Rowley
On Tue, 24 Oct 2023 at 10:39, Torsten Förtsch wrote: > Then I added this constraint to the small table: > > ALTER TABLE original_small_table > ADD CONSTRAINT partition_boundaries > CHECK((false, '-infinity')<=(is_sold, purchase_time) > AND (is_sold, purchase_time)<(false, 'infinity')) > NOT VA

partitioning

2023-10-23 Thread Torsten Förtsch
one. Now I want to convert this to the declarative partitioning scheme and in that process introduce a new partition for finished processes. The fact that a process is finished is represented by the boolean column is_sold. The table has also a timestamp column called purchase_time. The new

Re: Efficient Partitioning Strategies for PostgreSQL Table with KSUID and High Volume

2023-09-05 Thread Lorusso Domenico
e table is unpartitioned and read performance is > sluggish. I'm contemplating partitioning the table by month using the > KSUID column, [leveraging its embedded uint32 timestamp][2], something > like this: > > ```sql > CREATE TABLE table_y2023m09 PARTITION OF ksuid > FOR VALUES FR

Efficient Partitioning Strategies for PostgreSQL Table with KSUID and High Volume

2023-09-05 Thread wheels
table. Currently, the table is unpartitioned and read performance is sluggish. I'm contemplating partitioning the table by month using the KSUID column, [leveraging its embedded uint32 timestamp][2], something like this: ```sql CREATE TABLE table_y2023m09 PARTITION OF ksuid FOR V

Re: suggestion about time based partitioning and hibernate

2023-07-25 Thread Marc Millas
On Tue, Jul 18, 2023 at 8:18 AM Luca Ferrari wrote: > Dear all, > I'm looking for ideas here, and it could be someone already stepped > into declarative partitioning of an existing database where Hibernate > (a Java ORM) handles the tables. > The situation is as follows: >

Re: suggestion about time based partitioning and hibernate

2023-07-20 Thread Ron
On 7/20/23 10:31, Luca Ferrari wrote: On Wed, Jul 19, 2023 at 6:45 PM Alvaro Herrera wrote: Therefore I suggest to avoid doing that. Either look at some other partitioning scheme that doesn't involve adding columns to the primary key, or disregard partitioning for this table entirely.

Re: suggestion about time based partitioning and hibernate

2023-07-20 Thread Luca Ferrari
On Wed, Jul 19, 2023 at 6:45 PM Alvaro Herrera wrote: > > Therefore I suggest to avoid doing > that. Either look at some other partitioning scheme that doesn't > involve adding columns to the primary key, or disregard partitioning for > this table entirely. What d

Re: suggestion about time based partitioning and hibernate

2023-07-19 Thread Alvaro Herrera
On 2023-Jul-18, Luca Ferrari wrote: > Dear all, > I'm looking for ideas here, and it could be someone already stepped > into declarative partitioning of an existing database where Hibernate > (a Java ORM) handles the tables. > The situation is as follows: > > create

RE: suggestion about time based partitioning and hibernate

2023-07-18 Thread n.kobzarev
От: Ron Отправлено: 18 июля 2023 г. 9:48 Кому: pgsql-general@lists.postgresql.org Тема: Re: suggestion about time based partitioning and hibernate On 7/18/23 01:18, Luca Ferrari wrote: Dear all, I'm looking for ideas here, and it could be someone already stepped into declar

Re: suggestion about time based partitioning and hibernate

2023-07-17 Thread Ron
On 7/18/23 01:18, Luca Ferrari wrote: Dear all, I'm looking for ideas here, and it could be someone already stepped into declarative partitioning of an existing database where Hibernate (a Java ORM) handles the tables. The situation is as follows: create table foo( id primary key, a_date

suggestion about time based partitioning and hibernate

2023-07-17 Thread Luca Ferrari
Dear all, I'm looking for ideas here, and it could be someone already stepped into declarative partitioning of an existing database where Hibernate (a Java ORM) handles the tables. The situation is as follows: create table foo( id primary key, a_date date, ... ); Now, the trivial w

Declarative partitioning and partition pruning/check

2022-04-11 Thread Mats Taraldsvik
Hi, I have tried to read about Oracle's spatial partitioning feature ( https://www.oracle.com/technetwork/database/enterprise-edition/spatial-twp-partitioningbp-10gr2-05-134277.pdf) and wondered if something like this is possible with PostgreSQL (with PostGIS): The first part, getting the

Re: Locks on FK Tables From Partitioning

2022-02-09 Thread Simon Riggs
On Tue, 8 Feb 2022 at 23:13, Tom Lane wrote: > > Aaron Sipser writes: > > I am fairly confused about the locks generated by some partitioning code I > > am using. The setup is that we have a partitioned table (call it P), and it > > has a foreign key constraint to anothe

Re: Locks on FK Tables From Partitioning

2022-02-08 Thread Tom Lane
Aaron Sipser writes: > I am fairly confused about the locks generated by some partitioning code I > am using. The setup is that we have a partitioned table (call it P), and it > has a foreign key constraint to another table F. I'm trying to figure out > why when I add partitio

Locks on FK Tables From Partitioning

2022-02-08 Thread Aaron Sipser
Hi, I am fairly confused about the locks generated by some partitioning code I am using. The setup is that we have a partitioned table (call it P), and it has a foreign key constraint to another table F. I'm trying to figure out why when I add partitions to P, it seems that a lock is also

Re: Partitioning a table by integer value (preferably in place)

2021-08-13 Thread Vijaykumar Jain
and only move the remainder of data to all four partitions. So at any point of time, if the range partition which was planned earlier does not work well for some periods, we split the partitions. and if we see the trend growing with increased volume, the new partitions are created qtry etc. I hav

Re: Partitioning a table by integer value (preferably in place)

2021-08-13 Thread Michael Lewis
On Fri, Aug 13, 2021 at 3:02 AM Vijaykumar Jain < vijaykumarjain.git...@gmail.com> wrote: > ... use binary split for large partitions, to avoid large row movements. > Would you expound on this?

Re: Partitioning a table by integer value (preferably in place)

2021-08-13 Thread Vijaykumar Jain
On Fri, 13 Aug 2021 at 14:13, Pól Ua Laoínecháin wrote: > Thanks again for your questions - they gave me pause for thought and I > will try to apply them in future partitioning scenarios. (Unfortunatly > :-) ) there is no magic number of partitions for, say, a given size of > table

Re: Partitioning a table by integer value (preferably in place)

2021-08-13 Thread Pól Ua Laoínecháin
Dear Ron, > > Just a quick follow-up - could I script the creation of 1000 > > partitions using bash > Sure. That's what scripting languages are for. Thank you so much for your helpful and expansive answer. No wonder everyone talks about how friendly and welcoming the PostgreSQL community is!

Re: Partitioning a table by integer value (preferably in place)

2021-08-13 Thread Pól Ua Laoínecháin
that delete based bloating can be skipped completely. It can be skipped! > How distributed is the data based on smallint keys, equally or unequally. Totally uniform - see my own answer to my question - if it wasn't uniform, I might have considered RANGE based partitioning? > What kind o

Re: Partitioning a table by integer value (preferably in place)

2021-08-12 Thread Pól Ua Laoínecháin
site very helpful for explaining the basics of partitioning (LIST, RANGE and HASH): https://www.enterprisedb.com/postgres-tutorials/how-use-table-partitioning-scale-postgresql I then found this absolute beaut of a site which was like manna from heaven: https://www.depesz.com/2021/01/17/are-there-

Re: Partitioning a table by integer value (preferably in place)

2021-08-09 Thread Vijaykumar Jain
equally or unequally. What kind of queries would be run and results returned ? Oltp or olap like ? Quick queries with few rows retuned or heavy queries with lot of rows returned. Partitioning has been ever improving, so the best option if possible would be to use the latest pg version is possible,.

Re: Partitioning a table by integer value (preferably in place)

2021-08-09 Thread Ron
samples on the web, but they are for partitioning by date - a quick sample by integer would be gratefully received! Is partman recommended by the community? -- Angular momentum makes the world go 'round.

  1   2   3   >