Re: Query performance issue

2024-10-22 Thread Jeff Ross
On 10/21/24 23:31, yudhi s wrote: On Thu, Oct 17, 2024 at 3:06 AM Peter J. Holzer wrote: The execution plan looks like a postgresql execution plan, not a mysql execution plan. Did you run this query on postgresql? That may be interesting for comparison purposese, but ultimately i

Re: Faster data load

2024-09-05 Thread Jeff Ross
On 9/5/24 14:14, Lok P wrote: Hi, We are having a requirement to create approx 50 billion rows in a partition table(~1 billion rows per partition, 200+gb size daily partitions) for a performance test. We are currently using ' insert into select.. From or ;' method . We have dropped all in

Re: After 10 -> 15 upgrade getting "cannot commit while a portal is pinned" on one python function

2024-03-27 Thread Jeff Ross
On 3/27/24 17:35, Rob Sargent wrote: On 3/27/24 17:05, Jeff Ross wrote: On 3/27/24 15:44, Tom Lane wrote: Perhaps "pinned" in the error message means "open"? No, it means "pinned" ... but I see that plpython pins the portal underlying any PLyCursor object i

Re: After 10 -> 15 upgrade getting "cannot commit while a portal is pinned" on one python function

2024-03-27 Thread Jeff Ross
On 3/27/24 17:41, Adrian Klaver wrote: On 3/27/24 16:35, Rob Sargent wrote: On 3/27/24 17:05, Jeff Ross wrote: On 3/27/24 15:44, Tom Lane wrote: Perhaps "pinned" in the error message means "open"? No, it means "pinned" ... but I see that plpython pins the p

Re: After 10 -> 15 upgrade getting "cannot commit while a portal is pinned" on one python function

2024-03-27 Thread Jeff Ross
On 3/27/24 15:44, Tom Lane wrote: Perhaps "pinned" in the error message means "open"? No, it means "pinned" ... but I see that plpython pins the portal underlying any PLyCursor object it creates. Most of our PLs do that too, to prevent a portal from disappearing under them (e.g. if you were to

Re: After 10 -> 15 upgrade getting "cannot commit while a portal is pinned" on one python function

2024-03-27 Thread Jeff Ross
On 3/20/24 17:04, Tom Lane wrote: Adrian Klaver writes: Haven't had a chance to go through this yet. I'm going to say though that Tom Lane is looking for a shorter generic case that anyone could run on their system. Yeah, it's a long way from that trigger function definition to a working (i.e

Re: After 10 -> 15 upgrade getting "cannot commit while a portal is pinned" on one python function

2024-03-21 Thread Jeff Ross
On 3/20/24 17:13, Adrian Klaver wrote: On 3/20/24 15:52, Jeff Ross wrote: On 3/20/24 16:25, Adrian Klaver wrote: On 3/20/24 15:18, Jeff Ross wrote: Greetings! I built a trigger fired process that copies an "order" from our production database to our dev database.  An order, in

Re: After 10 -> 15 upgrade getting "cannot commit while a portal is pinned" on one python function

2024-03-20 Thread Jeff Ross
On 3/20/24 16:50, Tom Lane wrote: Jeff Ross writes: I then get this error: NOTICE:  update cargotel_common.copy_orders_to_dev set copy_completed = 't', copy_completed_timestamp = clock_timestamp() where id = 21 ERROR:  cannot commit while a portal is pinned Would you mind supply

Re: After 10 -> 15 upgrade getting "cannot commit while a portal is pinned" on one python function

2024-03-20 Thread Jeff Ross
On 3/20/24 16:25, Adrian Klaver wrote: On 3/20/24 15:18, Jeff Ross wrote: Greetings! I built a trigger fired process that copies an "order" from our production database to our dev database.  An order, in this case, is an initial row from a table and all of the rows in all of the

After 10 -> 15 upgrade getting "cannot commit while a portal is pinned" on one python function

2024-03-20 Thread Jeff Ross
Greetings! I built a trigger fired process that copies an "order" from our production database to our dev database.  An order, in this case, is an initial row from a table and all of the rows in all of the tables in that database/schema that are needed to satisfy all of the foreign key constr

Re: Help diagnosing replication (copy) error

2024-03-08 Thread Jeff Ross
On 3/8/24 14:50, Steve Baldwin wrote: Hi, I'm in the process of migrating a cluster from 15.3 to 16.2. We have a 'zero downtime' requirement so I'm using logical replication to create the new cluster and then perform the switch in the application. I have a situation where all but one table

Re: No yum repo for CentOS 7 and postgres 16?

2023-10-05 Thread Jeff Ross
On 10/5/23 15:46, David G. Johnston wrote: On Thursday, October 5, 2023, Jeff Ross wrote: Hi all, CentOS 7 isn't quite dead yet but it appears that CentOS7 is not included in the new yum repo file including 16. Do those of us still on CentOS 7 wanting to upgrade to 1

No yum repo for CentOS 7 and postgres 16?

2023-10-05 Thread Jeff Ross
PGDG RPMs. * Tue Sep 12 2023 Devrim Gündüz - 42.0-35PGDG - Add v16 repos - Remove v16 repos from RHEL 7 Really?  Might one inquire as to why? Yes, CentOS 7 is headed for EOL but not until June 30, 2024. Do those of us still on CentOS 7 wanting to upgrade to 16 now have to build from source? Jeff Ross

Re: Question: Multiple pg clusters on one server can be reached with the standard port.

2023-06-16 Thread Jeff Ross
We have already looked at pgbouncer and it works with that but unfortunately you have to do the authentication in pgbouncer. Which we don't like so much. Regards, Michael You can set up pgbouncer to authenticate in postgres: https://www.2ndquadrant.com/en/blog/understanding-user-management

Re: 15 pg_upgrade with -j

2023-05-23 Thread Jeff Ross
On 5/22/23 5:43 PM, Adrian Klaver wrote: From docs: https://www.postgresql.org/docs/current/pgupgrade.html The --jobs option allows multiple CPU cores to be used for copying/linking of files and to dump and restore database schemas in parallel; a good place to start is the maximum of the n

Re: 15 pg_upgrade with -j

2023-05-23 Thread Jeff Ross
On 5/22/23 5:42 PM, Tom Lane wrote: Jeff Ross writes: On 5/22/23 5:24 PM, Adrian Klaver wrote: So is the 1400G mostly in one database in the cluster? Yes, one big database with about 80 schemas and several other smaller databases so -j should help, right? AFAICT from a quick look at the

Re: 15 pg_upgrade with -j

2023-05-22 Thread Jeff Ross
On 5/22/23 5:24 PM, Adrian Klaver wrote: On 5/22/23 16:20, Jeff Ross wrote: Hello! We are moving from 10 to 15 and are in testing now. Our development database is about 1400G and takes 12 minutes to complete a pg_upgrade with the -k (hard-links) version. This is on a CentOS 7 server with 80

15 pg_upgrade with -j

2023-05-22 Thread Jeff Ross
-k -j 80 Our production database is closer to 1900G.  If we're looking at a 30 minute pg_upgrade window we'll be okay but if there is anything we can do to knock that time down we will and any suggestions to do so would be greatly appreciated. Jeff Ross

Re: Logically replicated table has no visible rows

2022-06-03 Thread Jeff Ross
On 5/31/22 11:46 AM, Jeff Ross wrote: Hello, We have a logically replicated table on RDS that is 39 G in size on both the publisher (10.21) and the subscriber (12.8). The replication slots on the publisher are all marked as active and the lsns are current so no lag. Other tables on the

Logically replicated table has no visible rows

2022-05-31 Thread Jeff Ross
Hello, We have a logically replicated table on RDS that is 39 G in size on both the publisher (10.21) and the subscriber (12.8). The replication slots on the publisher are all marked as active and the lsns are current so no lag. Other tables on the subscriber side are also identical in size

Re: Getting more detail in plpython error messages

2022-03-07 Thread Jeff Ross
On 3/7/22 11:06 AM, Tom Lane wrote: Jeff Ross writes: Is there a way to get the DETAIL part as well? It's not very well documented AFAICS, but a SPIError object has a "detail" attribute, so "e.detail" should help you. It looks like you might prefer to print &q

Getting more detail in plpython error messages

2022-03-07 Thread Jeff Ross
Hi all, In psql a database error will print both ERROR: and DETAIL: lines. postgres@testdb# delete from inspection where bundle_id in (select id from test_archive_20170401.load order by id); ERROR:  update or delete on table "inspection" violates foreign key constraint "inspection_weather_insp

Re: NOTIFY queue is at 66% and climbing...

2021-10-14 Thread Jeff Ross
On 10/13/21 5:50 PM, Tom Lane wrote: Jeff Ross writes: On 10.15 I'm getting the following on a logically replicated server. 2021-10-13 18:49:39.792 EDT,,,213601,,6143c257.34261,64243,,2021-09-16 18:16:55 EDT,4/3914851,60709901,WARNING,01000,"NOTIFY queue is 66% full",,,

NOTIFY queue is at 66% and climbing...

2021-10-13 Thread Jeff Ross
Hi all, On 10.15 I'm getting the following on a logically replicated server. From the CSV logs: 2021-10-13 18:49:39.792 EDT,,,213601,,6143c257.34261,64243,,2021-09-16 18:16:55 EDT,4/3914851,60709901,WARNING,01000,"NOTIFY queue is 66% full","" 2021-10-13 18:49:46.058 EDT,,,213601,,6143

Re: Removing a subscription that does not exist

2021-07-12 Thread Jeff Ross
On 7/11/21 7:38 PM, Kyotaro Horiguchi wrote: At Fri, 9 Jul 2021 10:49:46 -0600, Jeff Ross wrote in Hello, I'm working with an RDS instance running 12 that has an old subscription that I can't seem to drop. ... Look at the subdbid field in the first query result. You were loggin

Removing a subscription that does not exist

2021-07-09 Thread Jeff Ross
nt. What else can I try to remove this old non-functional subscription? Thanks, Jeff Ross

Re: Question: what is proper way to define python function as event_trigger?

2019-12-22 Thread Jeff Ross
On 2019-12-22 16:07, Jeff Ross wrote: On 2019-12-22 15:27, Andrei Pozolotin wrote: Hello. Problem: 1. any attempt to define python function as an event_trigger, i.e.: CREATE FUNCTION public.verify() RETURNS event_trigger LANGUAGE 'plpython3u' AS $$ print("hello-kitty"

Re: Question: what is proper way to define python function as event_trigger?

2019-12-22 Thread Jeff Ross
On 2019-12-22 15:27, Andrei Pozolotin wrote: Hello. Problem: 1. any attempt to define python function as an event_trigger, i.e.: CREATE FUNCTION public.verify() RETURNS event_trigger LANGUAGE 'plpython3u' AS $$ print("hello-kitty") $$; 2. fails with message: ERROR: trigger functions can onl

Re: Query which shows FK child columns?

2019-11-14 Thread Jeff Ross
On 11/14/19 11:49 AM, Ron wrote: v9.6.16 I have a query which shows the parents and children in FK relations, along with the parent column name, but can't seem to find the child column names. Is there a way to find the child column names without having to dig into pg_constraint? Thanks

Re: Dubugging an intermittent foreign key insert error with csvlog

2019-09-16 Thread Jeff Ross
On 9/16/19 4:07 PM, Adrian Klaver wrote: On 9/16/19 1:46 PM, Jeff Ross wrote: The csvlog snip shows what I believe are 2 simultaneous but separate sessions and the session that attempts to insert into the cargo_det table is not the same session that inserted into the load_det table

Dubugging an intermittent foreign key insert error with csvlog

2019-09-16 Thread Jeff Ross
ence / last_value hack but not quite there yet. Thanks, Jeff Ross

Re: COPY FROM - to avoid WAL generation

2018-08-21 Thread Jeff Ross
On 8/21/18 9:00 AM, Ravi Krishna wrote: In a recent thread of mine I learned something very interesting. If a table is created and data is loaded via COPY FROM within the same transaction, then PG will be smart enough to not generate WAL logs because all it needs to do is to track the status o