index row size 2720 exceeds btree version 4

2022-08-09 Thread Daulat
Hello Team, We are facing an error while uploading data in a table that has two B-tree indexes. As per the Postgres documentation there is a limitation of b-tree index size but I need to know if there is any alternative to overcome this issue. Error: " index row size 2720 exceeds btree version

Re: Creating A GIN index on JSONB column (large database)

2022-08-09 Thread Ron
800GB/20TB = 4%. You've got a 20TB database, and run out of space when *4% is added*. Either you're running way to close to the edge, or I misunderstand something. Either way, enable auto-scaling. https://aws.amazon.com/about-aws/whats-new/2019/06/rds-storage-auto-scaling/ On 8/9/22 10:29,

Re: Creating A GIN index on JSONB column (large database)

2022-08-09 Thread Ron
On 8/9/22 09:39, Rob Sargent wrote: On Aug 9, 2022, at 7:04 AM, Taylor Smith wrote: Hi all, I have a database that is in excess of 20TB in size, partitioned by date on a month to month basis. There is a column within that stores text (validated to be json but not stored as JSONB). I have

Re: Creating A GIN index on JSONB column (large database)

2022-08-09 Thread Taylor Smith
Yes, the database was always going to be a minimum of 20TB however we build the indexes on the partition which at max is about 1.3TB, which in temp files will use about 800GB temporary files. Note it doesn't have the same effect when doing non-concurrently. I don't think rolling off would help as i

Re: Creating A GIN index on JSONB column (large database)

2022-08-09 Thread Rob Sargent
> On Aug 9, 2022, at 7:45 AM, Taylor Smith wrote: > > Thanks Rob, > > This is what I was thinking. Would you say it presents any risk then having a > database unable to rebuild its own indexes if needed? > > No I would not. The scale-up + rebuild should tell you how much you actually ne

Re: Creating A GIN index on JSONB column (large database)

2022-08-09 Thread Taylor Smith
Thanks Rob, This is what I was thinking. Would you say it presents any risk then having a database unable to rebuild its own indexes if needed? On Tue, Aug 9, 2022 at 3:39 PM Rob Sargent wrote: > > > > On Aug 9, 2022, at 7:04 AM, Taylor Smith > wrote: > > > > Hi all, > > > > I have a databa

Re: Creating A GIN index on JSONB column (large database)

2022-08-09 Thread Rob Sargent
> On Aug 9, 2022, at 7:04 AM, Taylor Smith wrote: > > Hi all, > > I have a database that is in excess of 20TB in size, partitioned by date on a > month to month basis. > > There is a column within that stores text (validated to be json but not > stored as JSONB). I have a requirement to ma

Creating A GIN index on JSONB column (large database)

2022-08-09 Thread Taylor Smith
Hi all, I have a database that is in excess of 20TB in size, partitioned by date on a month to month basis. There is a column within that stores text (validated to be json but not stored as JSONB). I have a requirement to make the json searchable using GIN indexes. however when we try to create t