Re: Need help debugging slow logical replication

2023-02-07 Thread sunyuc...@gmail.com
ccing the mailist again in case someone else would have a idea how to debug: Here is what I see in the pg_locks on subscribe at all time: As you can see, it mostly concern following 3 tables, accounts, ledger, pending_ledger_fees, which I have tried analyze, vacuum them etc, none of them helped m

Re: Quoting issue from ODBC

2023-02-07 Thread Brad White
On 2/7/2023 6:19 PM, Adrian Klaver wrote: On 2/7/23 16:10, Brad White wrote: Front end: Access 365 Back end: Postgres 9.4 (I know, we are in the process of upgrading) I'm getting some cases where the SQL sent from MS-Access is failing. Looking at the postgres log shows that the field names and

Re: Need help debugging slow logical replication

2023-02-07 Thread sunyuc...@gmail.com
Hi Justin, thanks for the response! > REPLICA IDENTITY DEFAULT will only use primary keys, if the publisher > includes those tables, the subscriber when replaying the WAL will stop > throwing an error not knowing how to replay the UPDATE/DELETE. But I don't see any errors being thrown out

Re: Need help debugging slow logical replication

2023-02-07 Thread Justin
On Tue, Feb 7, 2023 at 8:07 PM sunyuc...@gmail.com wrote: > Hi Justin: > > - i checked that I have 2 tables using replication identity FULL, but > one table is empty and one table has only 1 row > - 7 tables using index > - overall I have ~100 tables in the publication: But I do find that > some

Re: Need help debugging slow logical replication

2023-02-07 Thread sunyuc...@gmail.com
Hi Justin: - i checked that I have 2 tables using replication identity FULL, but one table is empty and one table has only 1 row - 7 tables using index - overall I have ~100 tables in the publication: But I do find that some tables doesn't have PK , but using replica identity default, I'm abit co

Re: PostgreSQL

2023-02-07 Thread Stephen Frost
Greetings, * Joseph Kennedy (joseph.kennedy@gmail.com) wrote: > Thank you Laurenz, I just wanted to make sure. > > Do you know any alternative solution to RLS ? ... to do what? If you want to limit the set of rows that a given user can see, RLS is how to do that. If you want to limit the

Re: Quoting issue from ODBC

2023-02-07 Thread Rob Sargent
On 2/7/23 17:23, David G. Johnston wrote: On Tue, Feb 7, 2023 at 5:20 PM Brad White wrote: For example, this is the literal code in VBA Access: connection.Execute "UPDATE [" & strTable & "] SET [" & strTable & "].[InsertFlag] = Null" _     & " WHERE ((([" & strTable & "].[Inser

Re: Quoting issue from ODBC

2023-02-07 Thread David G. Johnston
On Tue, Feb 7, 2023 at 5:20 PM Brad White wrote: > For example, this is the literal code in VBA > > Access: connection.Execute "UPDATE [" & strTable & "] SET [" & strTable & > "].[InsertFlag] = Null" _ > & " WHERE ((([" & strTable & "].[InsertFlag])=" & lngCurrUID & "));", > , adCmdText Or ad

Fwd: Quoting issue from ODBC

2023-02-07 Thread Brad White
Front end: Access 365 Back end: Postgres 9.4 (I know, we are in the process of upgrading) I'm getting some cases where the SQL sent from MS-Access is failing. Looking at the postgres log shows that the field names and table names are not being quoted properly. It has been my experience that Access

Re: Need help debugging slow logical replication

2023-02-07 Thread Justin
On Tue, Feb 7, 2023 at 6:38 PM sunyuc...@gmail.com wrote: > Hi there, > > I am using PG 14.14 on both primary and secondary DB on AWS, setup > using a logical replication, I'm having trouble with huge replication > lag. > > My setup is as follows: > > P1 - physical - P1-R > | (logical) > P2 - p

Re: Quoting issue from ODBC

2023-02-07 Thread Adrian Klaver
On 2/7/23 16:10, Brad White wrote: Front end: Access 365 Back end: Postgres 9.4 (I know, we are in the process of upgrading) I'm getting some cases where the SQL sent from MS-Access is failing. Looking at the postgres log shows that the field names and table names are not being quoted properly.

Opportunity for clarification on ODBC options

2023-02-07 Thread Brad White
I'm setting my options for my DNS-less connection using https://odbc.postgresql.org/docs/config.html and https://odbc.postgresql.org/docs/config-opt.html I found it interesting that the sample at the bottom of the first page had options that don't exist on the second page. Namely, B4, B5, D4. I h

Re: A Small psql Suggestion

2023-02-07 Thread Brad White
1) Do not tack on to an existing thread, create a new post. Yup. Failure on my part. I'll repost.

Re: PostgreSQL

2023-02-07 Thread Brad White
Sorry about the reply failure. I was trying to start a new thread and failed. On Tue, Feb 7, 2023 at 5:42 PM Brad White wrote: > I'm setting my options for my DNS-less connection using > https://odbc.postgresql.org/docs/config.html > and > https://odbc.postgresql.org/docs/config-opt.html > > I

Quoting issue from ODBC

2023-02-07 Thread Brad White
Front end: Access 365 Back end: Postgres 9.4 (I know, we are in the process of upgrading) I'm getting some cases where the SQL sent from MS-Access is failing. Looking at the postgres log shows that the field names and table names are not being quoted properly. It has been my experience that Access

Re: A Small psql Suggestion

2023-02-07 Thread Adrian Klaver
On 2/7/23 16:00, Brad White wrote: Front end: Access 365 Back end: Postgres 9.4 (I know, we are in the process of upgrading) I'm getting some cases where the SQL sent from MS-Access is failing. Looking at the postgres log shows that the field names and table names are not being quoted properly.

Re: A Small psql Suggestion

2023-02-07 Thread Brad White
Front end: Access 365 Back end: Postgres 9.4 (I know, we are in the process of upgrading) I'm getting some cases where the SQL sent from MS-Access is failing. Looking at the postgres log shows that the field names and table names are not being quoted properly. It has been my experience that Access

Re: How do a user-defined function that returns a table executes a query?

2023-02-07 Thread David G. Johnston
On Tue, Feb 7, 2023 at 4:49 PM Katsuya Okizaki wrote: > In a normal SQL, we can use the EXPLAIN command to view the execution > plan. However, in this case, I am not sure how a user-defined function work. > > If anyone has faced a similar situation and found a way to view the > execution plan, I

How do a user-defined function that returns a table executes a query?

2023-02-07 Thread Katsuya Okizaki
Hi everyone, I hope this message finds you in good health and spirits. I am reaching out to this incredible community for some guidance and advice. I am currently trying to understand how a user-defined function that returns a table executes a query. In a normal SQL, we can use the EXPLAIN comman

Re: PostgreSQL

2023-02-07 Thread Brad White
I'm setting my options for my DNS-less connection using https://odbc.postgresql.org/docs/config.html and https://odbc.postgresql.org/docs/config-opt.html I found it interesting that the sample at the bottom of the first page had options that don't exist on the second page. Namely, B4, B5, D4. I h

Re: PostgreSQL

2023-02-07 Thread Peter J. Holzer
On 2023-02-07 12:36:57 +0100, Joseph Kennedy wrote: > RLS policy it only affects to command as SELECT, INSERT, UPDATE, DELETE or can > affects to operations on whole database, table like vacuum or reindex etc ? "Row level security" works, as the name implies at the row level. Tables, schemas and d

Re: ALTER COLUMN to change GENERATED ALWAYS AS expression?

2023-02-07 Thread Ron
Ignore this... On 2/7/23 13:05, Ron wrote: On 2/7/23 09:06, Adrian Klaver wrote: On 2/7/23 06:09, Philip Semanchuk wrote: On Feb 7, 2023, at 3:30 AM, Laurenz Albe wrote: On Mon, 2023-02-06 at 12:04 -0500, Philip Semanchuk wrote: I have a column defined GENERATED ALWAYS AS {my_expression}

Re: ALTER COLUMN to change GENERATED ALWAYS AS expression?

2023-02-07 Thread Ron
On 2/7/23 09:06, Adrian Klaver wrote: On 2/7/23 06:09, Philip Semanchuk wrote: On Feb 7, 2023, at 3:30 AM, Laurenz Albe wrote: On Mon, 2023-02-06 at 12:04 -0500, Philip Semanchuk wrote: I have a column defined GENERATED ALWAYS AS {my_expression} STORED. I’d like to change the {my_expressi

Re: ALTER COLUMN to change GENERATED ALWAYS AS expression?

2023-02-07 Thread Tom Lane
Ron writes: > On 2/7/23 09:06, Adrian Klaver wrote: >> To force immediate reclamation of space occupied by a dropped column, you >> can execute one of the forms of ALTER TABLE that performs a rewrite of the >> whole table. This results in reconstructing each row with the dropped >> column repla

Re: ALTER COLUMN to change GENERATED ALWAYS AS expression?

2023-02-07 Thread Laurenz Albe
On Tue, 2023-02-07 at 12:40 -0600, Ron wrote: > > "The DROP COLUMN form does not physically remove the column, but simply > > makes it > > invisible to SQL operations. Subsequent insert and update operations in the > > table > > will store a null value for the column. Thus, dropping a column is q

Re: ALTER COLUMN to change GENERATED ALWAYS AS expression?

2023-02-07 Thread Ron
On 2/7/23 09:06, Adrian Klaver wrote: On 2/7/23 06:09, Philip Semanchuk wrote: On Feb 7, 2023, at 3:30 AM, Laurenz Albe wrote: On Mon, 2023-02-06 at 12:04 -0500, Philip Semanchuk wrote: I have a column defined GENERATED ALWAYS AS {my_expression} STORED. I’d like to change the {my_expressi

Re: A Small psql Suggestion

2023-02-07 Thread Raymond Brinzer
I was really busy with work last week, so I didn't get around to thanking you, depesz. Setting d is a clever trick which hadn't occurred to me, and it has indeed made things nicer for me. I do think it would be a good thing to actually change in psql nevertheless, since I think the suggested beha

Re: Sequence vs UUID

2023-02-07 Thread Dominique Devienne
On Tue, Feb 7, 2023 at 3:47 PM Merlin Moncure wrote: > On Mon, Feb 6, 2023 at 1:22 PM Peter J. Holzer wrote: > >> On 2023-02-06 20:04:39 +0100, Julian Backes wrote: >> But UUIDs are random and that plays havoc with locality. > > > This is really key. [...] the databases I've seen that are writt

Re: ALTER COLUMN to change GENERATED ALWAYS AS expression?

2023-02-07 Thread Adrian Klaver
On 2/7/23 06:09, Philip Semanchuk wrote: On Feb 7, 2023, at 3:30 AM, Laurenz Albe wrote: On Mon, 2023-02-06 at 12:04 -0500, Philip Semanchuk wrote: I have a column defined GENERATED ALWAYS AS {my_expression} STORED. I’d like to change the {my_expression} part. After reading the documentati

Re: Sequence vs UUID

2023-02-07 Thread Merlin Moncure
On Mon, Feb 6, 2023 at 1:22 PM Peter J. Holzer wrote: > On 2023-02-06 20:04:39 +0100, Julian Backes wrote: > > I don't really understand what you mean by 'performance'. To me it is not > > surprising that incrementing (I know it is not just incrementing) a > > 64bit integer is faster than generat

Re: PostgreSQL

2023-02-07 Thread Joseph Kennedy
Thank you Laurenz, I just wanted to make sure. Do you know any alternative solution to RLS ? > Wiadomość napisana przez Laurenz Albe w dniu > 07.02.2023, o godz. 15:34: > > On Tue, 2023-02-07 at 12:36 +0100, Joseph Kennedy wrote: >> RLS policy it only affects to command as SELECT, INSERT, UP

Re: ERROR: deadlock detected

2023-02-07 Thread Laurenz Albe
On Tue, 2023-02-07 at 12:46 +0100, Matthias Apitz wrote: > We saw the following message in the file postgres-serverlog.error: > > 2023-01-24 17:16:16.578 CET [17468] ERROR:  deadlock detected > 2023-01-24 17:16:16.578 CET [17468] DETAIL:  Process 17468 waits for > ShareLock on transaction 9077664

Re: PostgreSQL

2023-02-07 Thread Laurenz Albe
On Tue, 2023-02-07 at 12:36 +0100, Joseph Kennedy wrote: > RLS policy it only affects to command as SELECT, INSERT, UPDATE, DELETE or > can affects to operations on whole database, table like vacuum or reindex etc > ? No, that wouldn't make any sense. Row Level Security determines who is allowed

Re: ALTER COLUMN to change GENERATED ALWAYS AS expression?

2023-02-07 Thread Philip Semanchuk
> On Feb 7, 2023, at 3:30 AM, Laurenz Albe wrote: > > On Mon, 2023-02-06 at 12:04 -0500, Philip Semanchuk wrote: >> I have a column defined GENERATED ALWAYS AS {my_expression} STORED. I’d like >> to change the >> {my_expression} part. After reading the documentation for ALTER TABLE >> (https:

Re: PostgreSQL

2023-02-07 Thread Giovanni Biscontini
Hi, RLS rely on Create policy command: https://www.postgresql.org/docs/15/sql-createpolicy.html it need a table_name, only Il giorno mar 7 feb 2023 alle ore 12:37 Joseph Kennedy < joseph.kennedy@gmail.com> ha scritto: > RLS policy it only affects to command as SELECT, INSERT, UPDATE, DELETE

Re: ERROR: deadlock detected

2023-02-07 Thread Torsten Förtsch
On Tue, Feb 7, 2023 at 12:47 PM Matthias Apitz wrote: > > > We saw the following message in the file postgres-serverlog.error: > > 2023-01-24 17:16:16.578 CET [17468] ERROR: deadlock detected > 2023-01-24 17:16:16.578 CET [17468] DETAIL: Process 17468 waits for > ShareLock on transaction 907766

ERROR: deadlock detected

2023-02-07 Thread Matthias Apitz
We saw the following message in the file postgres-serverlog.error: 2023-01-24 17:16:16.578 CET [17468] ERROR: deadlock detected 2023-01-24 17:16:16.578 CET [17468] DETAIL: Process 17468 waits for ShareLock on transaction 90776649; blocked by process 17724. Process 17724 waits for Sha

Re: PostgreSQL

2023-02-07 Thread Joseph Kennedy
RLS policy it only affects to command as SELECT, INSERT, UPDATE, DELETE or can affects to operations on whole database, table like vacuum or reindex etc ?Wiadomość napisana przez Giovanni Biscontini w dniu 07.02.2023, o godz. 12:16:Hi Joseph,   one way to filter without RLS is using functions to

Re: PostgreSQL

2023-02-07 Thread Giovanni Biscontini
Hi Joseph, one way to filter without RLS is using functions to make queries: this way you can cross check input parameters with login user (= user issuing query). Using functions slow an execution but incapsulate query letting you be able even to parse input parameters and filter results. Sorry

Re: PostgreSQL

2023-02-07 Thread Thomas Kellerer
Joseph Kennedy schrieb am 07.02.2023 um 12:02: > I would like restrict access to sensitive or restricted information > for some users (eg. hide data of one or more clients for some > database users). > > PostgreSQL allows to create security policy as Row-Level Security, > policies based on the quer

Re: Too coarse predicate locks granularity for B+ tree indexes

2023-02-07 Thread Rinat Shigapov
Thank you for your prompt reply! I've mentioned that I've generated ballast data to make the cost optimizer to switch to page-level locks. But my question is about more finer grained (less then page) predicate locks for indices. With page-level locks I could still get serialization failures if I

PostgreSQL

2023-02-07 Thread Joseph Kennedy
I would like restrict access to sensitive or restricted information for some users (eg. hide data of one or more clients for some database users). PostgreSQL allows to create security policy as Row-Level Security, policies based on the querying user. Are there any other alternatives to RLS, ar

Re: Get the sequence name corresponding to a GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY column

2023-02-07 Thread Sebastien Flaesch
Just sharing my conclusion so far, maybe this could do the job? Check the usage of double quotes, to manage upper/lower case in schema and table name. This is not needed for the column name, according to the doc of pg_get_serial_sequence(). with table_sequence as ( select '"'||ns.nspname||'"."

Re: Too coarse predicate locks granularity for B+ tree indexes

2023-02-07 Thread Laurenz Albe
On Tue, 2023-02-07 at 16:23 +0600, Rinat Shigapov wrote: > I have a concurrent testsuite that runs 14 test cases. Each test case operates > on a disjoint set of records, doesn't retry transactions and is run under > 'serializable' isolation level. The test data is small and likely fits within > a s

Re: Logical Replication - "invalid ordering of speculative insertion changes"

2023-02-07 Thread Rahila Syed
Hi Joe, On Fri, Feb 3, 2023 at 1:42 AM Joe Wildish wrote: > Just a bump on this --- perhaps the error is a bug with the DBMS? > > From what I can see "speculative insertion changes" in this context means > INSERT..ON CONFLICT DML. Although I have some experience writing > extensions and simple

Re: ALTER COLUMN to change GENERATED ALWAYS AS expression?

2023-02-07 Thread Laurenz Albe
On Mon, 2023-02-06 at 12:04 -0500, Philip Semanchuk wrote: > I have a column defined GENERATED ALWAYS AS {my_expression} STORED. I’d like > to change the > {my_expression} part. After reading the documentation for ALTER TABLE > (https://www.postgresql.org/docs/current/sql-altertable.html) and tryi