Re: How to do faster DML

2024-02-13 Thread Ron Johnson
On Tue, Feb 13, 2024 at 4:17 PM veem v wrote: [sni[] > One question here, if we have defined one column as a fixed length data > type "integer" and slowly we noticed the length of data keeps increasing > (in case of a sequence generated PK column which will keep increasing), and > we want to alte

Re: How to do faster DML

2024-02-13 Thread veem v
On Tue, 13 Feb 2024 at 20:32, Peter J. Holzer wrote: > Please do not conflate "char(n)" with native machine types like int or > float. These are very different things. A char(n) is string of fixed but > arbitrary length. This is not something a CPU can process in a single > instruction. It has to

Re: How should we design our tables and indexes

2024-02-13 Thread veem v
On Tue, 13 Feb 2024 at 20:59, Peter J. Holzer wrote: > For some kinds of queries a composite index can be dramatically faster. > While Postgres can combine indexes that means scanning both indexes and > combining the result, which may need a lot more disk I/O than scanning a > composite index. In

Re: FOR UPDATE SKIP LOCKED and get locked row/avoid updating other row(s)

2024-02-13 Thread Wiwwo Staff
On Tue, 13 Feb 2024 at 14:49, David G. Johnston wrote: > On Tuesday, February 13, 2024, Wiwwo Staff wrote: > >> Hi! >> I am implementing a queue using PostgreSQL. >> I am of course using "FOR UPDATE SKIP LOCKED". >> >> Is there any way I can tell PostgreSQL to only "operate" on the locked >> row

Re: Compressing large column by moving it to a unique table

2024-02-13 Thread Ron Johnson
On original_hashed, I think I'd try moving start_timestamp into its own index. On Tue, Feb 13, 2024 at 12:02 PM Adrian Garcia Badaracco < adr...@adriangb.com> wrote: > I'm using PostgreSQL 15.5. > > Here's a self-contained example. I included links to public pgMustard > query plans. > > Gist link

Re: Compressing large column by moving it to a unique table

2024-02-13 Thread Adrian Garcia Badaracco
I'm using PostgreSQL 15.5. Here's a self-contained example. I included links to public pgMustard query plans. Gist link: https://gist.github.com/adriangb/05a01cca99a438de2a286945903c44f4 Also copied below for archiving: ```sql CREATE OR REPLACE FUNCTION random_bytes(length integer) RETURNS byte

Re: How should we design our tables and indexes

2024-02-13 Thread Peter J. Holzer
On 2024-02-12 11:46:35 -0500, Greg Sabino Mullane wrote: > If PR_ID is a must in the Join criteria between these table tables table1, > table2 in all the queries, then is  it advisable to have a composite index > like (pr_id, mid), (pr_id,cid) etc rather than having index on individual

Re: How to do faster DML

2024-02-13 Thread Peter J. Holzer
On 2024-02-13 01:53:25 +0530, veem v wrote: > On Mon, 12 Feb 2024 at 03:40, Peter J. Holzer wrote: > > The fixed width types are those that the CPU can directly process: > Integers with 16, 32 and 64 bits, floating point numbers with 32 and 64 > bits. The CPU can read and write them w

Re: FOR UPDATE SKIP LOCKED and get locked row/avoid updating other row(s)

2024-02-13 Thread David G. Johnston
On Tuesday, February 13, 2024, Wiwwo Staff wrote: > Hi! > I am implementing a queue using PostgreSQL. > I am of course using "FOR UPDATE SKIP LOCKED". > > Is there any way I can tell PostgreSQL to only "operate" on the locked > row, and/or a way to reference it? > > Some explanations of what I me

Re: How to do faster DML

2024-02-13 Thread Peter J. Holzer
On 2024-02-12 11:28:41 -0500, Greg Sabino Mullane wrote: > On Mon, Feb 12, 2024 at 1:50 AM veem v wrote: > > So we were thinking, adding many column to a table should be fine in > postgres (as here we have a use case in which total number of columns may > go till ~500+). But then,  co

Re: MAT. VIEW security problems and PG 10-11 versions?

2024-02-13 Thread Ron Johnson
On Tue, Feb 13, 2024 at 3:44 AM Daniel Gustafsson wrote: > > On 13 Feb 2024, at 08:56, Durumdara wrote: > > > But maybe that's because PG 10 and 11 are no longer supported - and not > because they aren't affected by the issues. > > EOL versions do not recieve security updates and are not verifie

Re: Compressing large column by moving it to a unique table

2024-02-13 Thread Ron Johnson
1. Show us the PG version, view definition, the exact query that's slow, and the EXPLAIN (ANALYZE). 2. Presumably there's an index on each table's *_hash column? On Tue, Feb 13, 2024 at 8:48 AM Adrian Garcia Badaracco wrote: > Thank you for the reply Ron. > > Yes there are many fewer (<1%) the n

Re: FOR UPDATE SKIP LOCKED and get locked row/avoid updating other row(s)

2024-02-13 Thread Laurenz Albe
On Tue, 2024-02-13 at 11:17 +, Wiwwo Staff wrote: > I am implementing a queue using PostgreSQL. > I am of course using "FOR UPDATE SKIP LOCKED". > > Is there any way I can tell PostgreSQL to only "operate" on the locked row, > and/or a way to reference it? > > Some explanations of what I mea

Re: Compressing large column by moving it to a unique table

2024-02-13 Thread Adrian Garcia Badaracco
Thank you for the reply Ron. Yes there are many fewer (<1%) the number of rows in new_table. Thanks for making me think of normalization, I hadn’t seen it that way. Although there is no theoretical relationship between the rows in the other columns in the original table and the attributes column,

FOR UPDATE SKIP LOCKED and get locked row/avoid updating other row(s)

2024-02-13 Thread Wiwwo Staff
Hi! I am implementing a queue using PostgreSQL. I am of course using "FOR UPDATE SKIP LOCKED". Is there any way I can tell PostgreSQL to only "operate" on the locked row, and/or a way to reference it? Some explanations of what I mean: - I have a table with N rows - I lock row X with a PG F

Re: MAT. VIEW security problems and PG 10-11 versions?

2024-02-13 Thread Daniel Gustafsson
> On 13 Feb 2024, at 08:56, Durumdara wrote: > But maybe that's because PG 10 and 11 are no longer supported - and not > because they aren't affected by the issues. EOL versions do not recieve security updates and are not verified during security analysis and fixing, so hence they aren't listed