Re: Need help in logical replication

2025-01-13 Thread Justin
Hi Divyansh, Go to the subscriber and look for errors in the PostgreSQL logs. When creating a subscription the default action is to sync the tables. Is the subscriber table empty?? Thank you, On Mon, Jan 13, 2025 at 7:30 AM Divyansh Gupta JNsThMAudy < ag1567...@gmail.com> wrote: > Hii PostgreS

Re: Suddenly all queries moved to seq scan

2024-11-20 Thread Justin
On Wed, Nov 20, 2024 at 9:09 AM Sreejith P wrote: > > > > Queries were taking 20 ms started taking 60 seconds. So have done SQL > analyse to understand about query plan. There we found that query planner > taking seq scan instead in index scan. > > I would like to add one ore point. A delete que

Re: Synchronize the dump with a logical slot with --snapshot

2024-09-28 Thread Justin
ql.org/docs/17/functions-admin.html#FUNCTIONS-SNAPSHOT-SYNCHRONIZATION Then you can create the logical replication slot with using that slotname option https://www.postgresql.org/docs/17/sql-createsubscription.html#SQL-CREATESUBSCRIPTION-PARAMS-WITH-SLOT-NAME and no sync option. Then you tell pg_dump to use that snapshot name snapshot with this option --snapshot=snapshotname https://www.postgresql.org/docs/current/app-pgdump.html Once pg_restore is done on the destination , you can create a subscription using that slotname option probably and specify copy_data = false. Keep in mind the WAL will build up during this process, not sure what the benefit would be just allowing logical replication to do the initial sync. Thanks Justin

Re: Logical Replication Delay

2024-09-23 Thread Justin
ng LR will execute it. I can see an LR worker being AHEAD of other workers trying to reference ROWs that do not exist yet. Which can be dealt with by making sure the Triggers that reference other tables are in the same publication and subscription. Thanks Justin On Mon, Sep 23, 2024 at 12:

Re: Logical Replication Delay

2024-09-21 Thread Justin
Hi Ramakrishna, 4GB of WAL generated per minute is a lot. I would expect the replay on the subscriber to lag behind because it is a single process. PostgreSQL 16 can create parallel workers for large transactions, however if there is a flood of small transactions touching many tables the single

Re: Upgrade Ubuntu 22 -> 24 may break PostgreSQL

2024-08-31 Thread Justin Clift
sions and stuff like that. Thoughts? Regards and best wishes, Justin Clift

Re: Significant Execution Time Difference Between PG13.14 and PG16.4 for Query on information_schema Tables.

2024-08-27 Thread Justin Clift
On 2024-08-27 20:14, David Rowley wrote: On Tue, 27 Aug 2024 at 18:00, Justin Clift wrote: As a general thought, seeing that this might be an actual problem should some kind of automated testing be added that checks for performance regressions like this? We normally try to catch these sorts

Re: Significant Execution Time Difference Between PG13.14 and PG16.4 for Query on information_schema Tables.

2024-08-26 Thread Justin Clift
ng commit as at first I thought it might be related to Memoize. It does not seem to be. As a general thought, seeing that this might be an actual problem should some kind of automated testing be added that checks for performance regressions like this? Regards and best wishes, Justin Clift

On exclusion constraints and validity dates

2024-08-24 Thread Justin Giacobbi
Hello, I have an issue that on the surface seems orthogonal to existing functionality. I'm trying to dynamically update validity ranges as new s replace old s. In a nutshell the problem looks like this: psqlprompt=# select * from rangetest; id | rangecol +-

Re: Planet Postgres and the curse of AI

2024-08-20 Thread Justin Clift
cally and factually correct. Use of AI should be used for minor editing, not primary generation" Sounds pretty sensible. :) + Justin

Re: --frokbackend process

2024-06-26 Thread Justin
Hi Satyajit: Can't tell what is going on from the task manager list. Looks like a normal task list for PostgreSQL. Keep in mind PG is a process based application NOT a threaded application. Meaning postgresql starts/forks a new process for every connection. This means we can see each sessions

Re: Questions on logical replication

2024-06-13 Thread Justin
o low on resources that Logical Replication is problematic one can create a binary replica, promote it and convert it to logical replication skipping the initial sync. Then upgrade that server. There is a minor outage required to convert a binary replica to a logical replica. I've done it in u

Re: Questions on logical replication

2024-06-12 Thread Justin
On Tue, Jun 11, 2024 at 5:43 PM Koen De Groote wrote: > > If there are any errors during the replay of WAL such as missing indexes > for Replica Identities during an Update or Delete this will cause the main > subscriber worker slot on the publisher to start backing up WAL files > > And also if

Re: Questions on logical replication

2024-06-08 Thread Justin
On Sat, Jun 8, 2024 at 1:41 PM Koen De Groote wrote: > What I'm trying to do is upgrade a PG11 database to PG16, using logical > replication. > > The PG11 has an active and a standby, there are a handful of databases. On > particular one has a few tables just over 100GB, then a few 100 tables nea

Re: Table data migration from single server to Flexi server

2024-05-02 Thread Justin Clift
On 2024-05-02 13:24, Bagesh kamar singh wrote: Recently we migrated our postgreSQL single server to flexi server. Hmmm, what's "Flexi server"? Doing a quick online search just now isn't showing things that seem to be PostgreSQL related. Regards and best wishes, Justin Clift

Re: SSPI Feature Request

2024-04-19 Thread Justin Clift
ght person. ? Regards and best wishes, Justin Clift

Re: replication slot "pg_1015733_sync_1014718_7358407382484881476" does not exist

2024-04-17 Thread Justin
Hi Avi, Based on the slot name this is an initial sync worker being created by the Logical Replication supervisor. Subscriber started an initial sync either failed to create the slot and now thinks it exists and keeps trying to drop it on the publisher or another process dropped the slot on the p

Re: constant crashing hardware issue and thank you TAKE AWAY

2024-04-17 Thread Justin Clift
n" test of all the things (memory, hard disks/ssds, cpu, gpu, etc) just to make sure everything is ok before you start using it for important stuff. Regards and best wishes, Justin Clift

Re: Storing and comparing columns of cryptographic hashes?

2024-04-09 Thread Justin
has many drawbacks compared to btree. None of the above queries are possible with GIN indexes or using array columns without a lot more code. Arrays are not data sets if the design needs to access a specific hash value for update,delete, append new values, an array probably not the best solution. Hope this helps Justin

Re: Please recommend postgresql.conf improvements for osm2pgsql loading Europe

2024-03-31 Thread Justin Clift
ace (depends on the storage you're using), you could potentially load things from that backup / snapshot (etc) instead of having to do the import all over again each time. Regards and best wishes, Justin Clift

Re: Please recommend postgresql.conf improvements for osm2pgsql loading Europe

2024-03-29 Thread Justin Clift
27;t seem to be optimised for loading into a database. (?) It kind of looks like they'd be stored into individual records, which probably means they'd be getting imported as individual INSERT statements rather than something that's optimised for bulk loading. :( Regards and best wishes, Justin Clift

Re: PostgreSQL as advanced job queuing system

2024-03-25 Thread Justin Clift
t-ize this. Stay tuned Hopefully it get approved. More battle tested queue systems are welcome, especially those that have been used at non-trivial scales. :D + Justin

Re: Local replication "slot does not exist" after initial sync

2024-02-25 Thread Justin
On Sun, Feb 25, 2024 at 1:11 PM Mike Lissner wrote: > Sorry, two more little things here. The publisher logs add much, but > here's what we see: > > STATEMENT: START_REPLICATION SLOT > "pg_20031_sync_17418_7324846428853951375" LOGICAL F1D0/346C6508 > (proto_version '2', publication_names '"compas

Re: Partitioning options

2024-02-20 Thread Justin
om > > Alec > > > Hi Alec, would need to see the DDL of the partitions and the queries accessing these partitions to have an opinion Thank you Justin

Re: Safest pgupgrade jump distance

2024-02-12 Thread Justin Clift
As always though, make sure you have a backup (that's known to work) before you try it all out though. :) + Justin

Re: Partitioning options

2024-02-11 Thread Justin
, 2024 at 10:25 PM Justin wrote: > >> Hi Sud, >> >> Would not look at HASH partitioning as it is very expensive to add or >> subtract the number of partitions. >> >> Would probably look at a nested partitioning using customer ID using >> range o

Re: Partitioning options

2024-02-08 Thread Justin
Hi Sud, Would not look at HASH partitioning as it is very expensive to add or subtract the number of partitions. Would probably look at a nested partitioning using customer ID using range or list of IDs then by transaction date, Its easy to add partitions and balance the partitions segments.

Re: building a singularity image from docker hub postgres image

2024-01-30 Thread Justin Clift
pure guess work though. :) Regards and best wishes, Justin Clift

Re: Logical replication claims to work, not working - new tables

2024-01-21 Thread Justin
On Sat, Jan 20, 2024, 5:43 PM Chris Angelico wrote: > PostgreSQL 15 on Debian, both ends of replication. > > I'm doing logical replication in a bit of a complex setup. Not sure > how much of this is relevant so I'll give you a lot of detail; sorry > if a lot of this is just noise. > > * Bidirecti

Re: Moving to Postgresql database

2024-01-15 Thread Justin Clift
le-to-postgresql-migration-guide-tutorial-move-convert-database-oracle-alternative Regards and best wishes, Justin Clift

Re: Filled Postgres server as Docker image

2024-01-06 Thread Justin Clift
ot;myimage:latest", which you can then use as you need later on. Does that make sense? Regards and best wishes, Justin Clift

Re: Properly handle OOM death?

2023-11-13 Thread Justin Pryzby
re aren't more, similar complaints about this. It's as Peter said: it (sometimes) causes systemd to actively *stop* the cluster after OOM, when it would've come back online on its own if the init (supervisor) process didn't interfere. My solution was to set: /usr/lib/systemd/system/postgresql@.service OOMPolicy=continue I suggest that the default unit files should do likewise. -- Justin

Re: Server process exited with exit code 4

2023-11-11 Thread Justin Clift
On 2023-11-09 11:41, yangsr3411 wrote: Finally, we used the Windows tool Gflags.exe and found that other software terminated the postgres process. Out of curiosity, what was the other software? :) Regards and best wishes, Justin Clift

Re: Server process exited with exit code 4

2023-10-29 Thread Justin Clift
that can show a log of any recent weirdness that occurred at a hardware level. If yours can, take a look for things like ECC errors or any other strange stuff. :) Regards and best wishes, Justin Clift

Re: Disk wait problem... may not be hardware...

2023-10-26 Thread Justin Clift
https://github.com/openzfs/zfs/discussions/14793 Note - that's not PostgreSQL specific or anything, but more of a "weird stuff showing up with NVMe drives" thing. Regards and best wishes, Justin Clift

Re: Driver Postgresql HP-Unix

2023-10-21 Thread Justin Clift
On 2023-10-21 21:13, Dave Cramer wrote: On Sat, 21 Oct 2023 at 05:50, Justin Clift wrote: On 2023-10-19 04:45, Abelardo Erazo Lopez wrote: > Hi, Everyone > > I have an Oracle database Oracle 19c and I need to access a PostgreSQL > database 15.4 that resides on a different server

Re: Driver Postgresql HP-Unix

2023-10-21 Thread Justin Clift
*sure* it's PostgreSQL 15.4, and not PostgreSQL 12.4? Asking because I'm not seeing version PG 15.4 in the list here, though PG 12.4 and 16 are: http://hpux.connect.org.uk/hppd/cgi-bin/search?term=postgresql Regards and best wishes, Justin Clift

Re: [EXTERNAL] Oracle FDW version

2023-08-22 Thread Justin Clift
using PostgreSQL, but are instead using a special proprietary spin-off called Greenplum. If that's the case, then you'll need to have a chat with the Greenplum support people. They'll know their product better than we do. :) Regards and best wishes, Justin Clift

Re: PG minor version in data directory?

2023-08-19 Thread Justin Clift
On 2023-08-19 19:14, Peter J. Holzer wrote: On 2023-08-19 14:22:25 +1000, Justin Clift wrote: Thanks, that's what I figured. I'll have to keep state in a PG_VERSION_MINOR there or something. :) Wouldn't it be better to ask whatever system you use to install the software

Re: PG minor version in data directory?

2023-08-18 Thread Justin Clift
On 2023-08-19 14:10, David G. Johnston wrote: On Fri, Aug 18, 2023 at 6:36 PM Justin Clift wrote: If the minor version is already tracked somewhere as well, that would be extremely useful for my use case. The data directory doesn't have a concept of "minor version". Onl

PG minor version in data directory?

2023-08-18 Thread Justin Clift
r version is already tracked somewhere as well, that would be extremely useful for my use case. Otherwise, I'll have to start manually adding info to track it. Regards and best wishes, Justin Clift

Re: Trigger questions

2023-05-04 Thread Justin
On Thu, May 4, 2023 at 9:49 AM DAVID ROTH wrote: > 1) Can I create a trigger on a view? > 2) Do triggers cascade? > > Say I have an insert trigger on a table. > And, I have an insert trigger on a view that references this table > If I do an insert on the view, will both triggers fire? > Can not

Re: Need help debugging slow logical replication

2023-02-07 Thread Justin
On Tue, Feb 7, 2023 at 8:07 PM sunyuc...@gmail.com wrote: > Hi Justin: > > - i checked that I have 2 tables using replication identity FULL, but > one table is empty and one table has only 1 row > - 7 tables using index > - overall I have ~100 tables in the publication:

Re: Need help debugging slow logical replication

2023-02-07 Thread Justin
On Tue, Feb 7, 2023 at 6:38 PM sunyuc...@gmail.com wrote: > Hi there, > > I am using PG 14.14 on both primary and secondary DB on AWS, setup > using a logical replication, I'm having trouble with huge replication > lag. > > My setup is as follows: > > P1 - physical - P1-R > | (logical) > P2 - p

Re: Supported RHEL version for PostgreSQL 11

2021-08-28 Thread Justin Pryzby
e from postgres 9.6 to 11; You may need to install updated PGDG packages for the RH8. How are you planning to upgrade postgres ? Using pg_upgrade, or pg_dump, or ?? > Postgres is installed using PGDG packages. > > On Thu, Aug 26, 2021 at 1:15 AM Justin Pryzby wrote: > > > On We

Re: Supported RHEL version for PostgreSQL 11

2021-08-25 Thread Justin Pryzby
e new locale data. Therefore, caution will be necessary when upgrading. > Is PostgreSQL 11 compatible with RHEL 8.4 How did you install postgres ? Compiled from source ? Using RHEL packages ? Using PGDG packages ? PDDG has pacakges for PG11/RH8 https://yum.postgresql.org/rpmchart/#pg11 -- Justin

Re: Row estimates for empty tables

2020-08-24 Thread Justin Pryzby
org/message-id/20200427181034.ga28...@telsasoft.com |... 1) create an child table: CREATE TABLE x_child() INHERITS(x) |and, 2) change the query to use "select from ONLY". | |(1) allows the planner to believe that the table really is empty, a conclusion |it otherwise avoids and (2) avoids decending into the child (for which the |planner would likewise avoid the conclusion that it's actually empty). -- Justin

Re: walreceiver termination

2020-05-04 Thread Justin King
ystem in common -- particularly ZFS (or btrfs, in the bottom case). Is there anything more we can do here to help narrow down this issue? I'm happy to help, but I honestly wouldn't even know where to begin. Thanks- Justin King flightaware.com On Thu, Apr 23, 2020 at 4:40 PM Justin King

Re: walreceiver termination

2020-04-23 Thread Justin King
On Thu, Apr 23, 2020 at 3:06 PM Tom Lane wrote: > > Justin King writes: > > I assume it would be related to the following: > > LOG: incorrect resource manager data checksum in record at 2D6/C259AB90 > > since the walreceiver terminates just after this - but I'm un

Re: walreceiver termination

2020-04-23 Thread Justin King
seemingly random times. Also, just to clarify, this will only happen on a single replica at a time. On Thu, Apr 23, 2020 at 2:46 PM Justin King wrote: > > On Thu, Apr 23, 2020 at 12:47 PM Tom Lane wrote: > > > > Justin King writes: > > > We've seen unexpect

Re: walreceiver termination

2020-04-23 Thread Justin King
On Thu, Apr 23, 2020 at 12:47 PM Tom Lane wrote: > > Justin King writes: > > We've seen unexpected termination of the WAL receiver process. This > > stops streaming replication, but the replica stays available -- > > restarting the server resumes streaming

walreceiver termination

2020-04-23 Thread Justin King
DEBUG: shmem_exit(1): 6 on_shmem_exit callbacks to make", 140, MSG_NOSIGNAL, NULL, 0) = 140 sendto(4, "\x58\x00\x00\x00\x04", 5, MSG_NOSIGNAL, NULL, 0) = 5 Any suggestions? Thanks- Justin

Re: PG12 autovac issues

2020-03-27 Thread Justin King
On Fri, Mar 27, 2020 at 12:12 AM Michael Paquier wrote: > > On Thu, Mar 26, 2020 at 09:46:47AM -0500, Justin King wrote: > > Nope, it was just these tables that were looping over and over while > > nothing else was getting autovac'd. I'm happy to share the full log &

Re: PG12 autovac issues

2020-03-26 Thread Justin King
On Wed, Mar 25, 2020 at 8:43 PM Michael Paquier wrote: > > On Wed, Mar 25, 2020 at 10:39:17AM -0500, Justin King wrote: > > This started happening again. DEBUG1 is enabled: > > Thanks for enabling DEBUG1 logs while this happened. > > > Mar 25 14:48:26 cowtn postgres[3

Re: PG12 autovac issues

2020-03-25 Thread Justin King
) DESC LIMIT 3; slot_name | xmin | age | catalog_xmin | age ---+--+-+--+- (0 rows) postgres=# SELECT gid, database, transaction FROM pg_prepared_xacts ORDER BY age(transaction) LIMIT 3; gid | database | transaction -+--+- (0 rows) Let me know if there's anyt

Re: PG12 autovac issues

2020-03-23 Thread Justin King
On Mon, Mar 23, 2020 at 4:31 PM Justin King wrote: > > On Mon, Mar 23, 2020 at 3:00 PM Andres Freund wrote: > > > > Hi, > > > > On 2020-03-23 20:47:25 +0100, Julien Rouhaud wrote: > > > > - relfrozenxid, age(relfrozenxid) for the oldest table in the old

Re: PG12 autovac issues

2020-03-23 Thread Justin King
On Mon, Mar 23, 2020 at 3:00 PM Andres Freund wrote: > > Hi, > > On 2020-03-23 20:47:25 +0100, Julien Rouhaud wrote: > > > - relfrozenxid, age(relfrozenxid) for the oldest table in the oldest > > > database > > > SELECT oid::regclass, age(relfrozenxid), relfrozenxid FROM pg_class > > > WHERE r

Re: PG12 autovac issues

2020-03-20 Thread Justin King
On Thu, Mar 19, 2020 at 6:56 PM Andres Freund wrote: > > Hi, > > On 2020-03-19 18:07:14 -0500, Justin King wrote: > > On Thu, Mar 19, 2020 at 5:35 PM Andres Freund wrote: > > > > > > Hi, > > > > > > On 2020-03-19 10:23:48 -0500, Justin King

Re: PG12 autovac issues

2020-03-19 Thread Justin King
On Thu, Mar 19, 2020 at 5:35 PM Andres Freund wrote: > > Hi, > > On 2020-03-19 10:23:48 -0500, Justin King wrote: > > > From a single stats snapshot we can't actually understand the actual xid > > > consumption - is it actually the xid usage that triggers the

Re: Fwd: PG12 autovac issues

2020-03-19 Thread Justin King
On Thu, Mar 19, 2020 at 11:02 AM Michael Lewis wrote: > > On Thu, Mar 19, 2020 at 9:31 AM Justin King wrote: >> >> On Wed, Mar 18, 2020 at 1:40 PM Michael Lewis wrote: >> > >> > Do you have default fillfactor set on this table? If not, I would wonder &

Re: Fwd: PG12 autovac issues

2020-03-19 Thread Justin King
On Wed, Mar 18, 2020 at 1:40 PM Michael Lewis wrote: > > Do you have default fillfactor set on this table? If not, I would wonder if > reducing it to 50% or even 20% would allow many more HOT updates that would > reduce bloat. I don't believe we have a default fillfactor, but I'm still trying t

Re: Re: PG12 autovac issues

2020-03-19 Thread Justin King
On Wed, Mar 18, 2020 at 10:13 AM Adrian Klaver wrote: > > On 3/18/20 6:57 AM, Justin King wrote: > Please reply to list also > Ccing list > > > >>> Here are the settings, these are the only ones that are not set to > >>> default with the exception

RE: PG12 autovac issues

2020-03-19 Thread Justin King
Hi Andres- Thanks for the reply, answers below. On Tue, Mar 17, 2020 at 8:19 PM Andres Freund wrote: > > Hi, > > On 2020-03-17 17:18:57 -0500, Justin King wrote: > > As you can see in this table, there are only ~80K rows, but billions > > of updates. What we have observ

Re: Fwd: PG12 autovac issues

2020-03-17 Thread Justin King
On Tue, Mar 17, 2020 at 5:39 PM Adrian Klaver wrote: > > On 3/17/20 3:22 PM, Justin King wrote: > > Apologies, I accidentally sent this to the pgsql-admin list initially > > but intended it go here: > > > > We have a database that isn't overly large (~20G

Fwd: PG12 autovac issues

2020-03-17 Thread Justin King
and why it suddenly started when we moved from PG10 > PG12. The configs and workload are essentially the same between versions. We realize we could simply increase the autovacuum_freeze_max_age, but that doesn't seem to actually resolve anything -- it just pushes the problem out. Has anyone seen anything similar to this? Thanks very much for the consideration. Justin King http://flightaware.com/

Re: Force WAL cleanup on running instance

2020-03-11 Thread Justin
Question everyone isn't this a problem with the order of operations? switching the wal files then running checkpoint means the Checkpoint can cross wal files, so the previous wal file can not be deleted??? To my understanding the order operations should be Checkpoint which flushes everything

Re: Who mades the inserts?

2020-03-09 Thread Justin
Hi DD By default Postgresql does not collect this level of detail information to tell you which database has a high load at X point in time. You can infer which database has this high load without increasing logging Select * from pg_stat_database this dumps total inserts, update, scans etc..

Re: Rules versus triggers

2020-03-07 Thread Justin
9AM -0500, Justin wrote: > > Hi Stan > > > > Rules actual are able to rewrite the SQL query sent to postgresql. Most > > everyone suggestion is avoid rules. > > > > Triggers are just like every other databases Triggers firing off code for > >

Re: Rules versus triggers

2020-03-07 Thread Justin
Hi Stan Rules actual are able to rewrite the SQL query sent to postgresql. Most everyone suggestion is avoid rules. Triggers are just like every other databases Triggers firing off code for Insert/Update/Delete/Truncate event https://www.postgresql.org/docs/current/sql-createtrigger.html On Sa

Re: Performance Problem

2020-03-04 Thread Justin
taking a quick glance at config file I do not see any modifications to any key settings shared_buffers, efffecttive cache size work_mem meaning the server is running at the default settings which results in horrible performance here is a website that gives suggested config changes based on serve

Re: Postgresql 12.x on Windows (vs Linux)

2020-03-02 Thread Justin
Hi Robert I've used Postgresql on windows for years. Yes there are performance differences between windows and linux and the gap has gotten bigger with JIT Common performance hits Shared Buffers JIT not supported Windows can be aggressive flushing its disk cache, https://docs.microsoft.com/en-

Re: Connections dropping while using Postgres backend DB with Ejabberd

2020-02-25 Thread Justin
resources long before reaching 50K Something is off here I would be looking into how this test actually works, how the connections are opened, and commands it sends to Postgresql On Tue, Feb 25, 2020 at 2:12 PM Dipanjan Ganguly wrote: > Hi Justin, > > Thanks for your insight. > &g

Re: Connections dropping while using Postgres backend DB with Ejabberd

2020-02-25 Thread Justin
Hi Dipanjan Please do not post to all the postgresql mailing list lets keep this on one list at a time, Keep this on general list Am i reading this correctly 10,000 to 50,000 open connections. Postgresql really is not meant to serve that many open connections. Due to design of Postgresql each c

Re: How to fix 0xC0000005 exception in Postgres 9.0

2020-02-21 Thread Justin
Yes publish the ODBC logs seeing the Linux logs tells us the ODBC client crashing NOT the client process. if the linux postgrsql client process crashes it typically will cause the postgresql postmaster to panic and restart as it has to assume it corrupt share memory. On Fri, Feb 21, 2020 at

Re: How to fix 0xC0000005 exception in Postgres 9.0

2020-02-21 Thread Justin
forgot to say publish the Linux logs it may have more details what is going on On Fri, Feb 21, 2020 at 12:27 PM Justin wrote: > is this happening from any client or just a specific client running ODBC? > are the clients running AV if so are the AV versions the same? > > Given this

Re: How to fix 0xC0000005 exception in Postgres 9.0

2020-02-21 Thread Justin
is this happening from any client or just a specific client running ODBC? are the clients running AV if so are the AV versions the same? Given this is killing a Linux server, sounds like ODBC is sending back garabage data to the server crashing it. There are several settings in OBDC, to change

Re: How to fix 0xC0000005 exception in Postgres 9.0

2020-02-20 Thread Justin
having chased this error several times over the last 20 years, if its not windows update, AV update, firewall config change, UAC or a driver update it is nearly impossible to to figure out what is causing it. One can try to run sysinternal app such as process explorer or depends to look over the

Re: How to fix 0xC0000005 exception in Postgres 9.0

2020-02-20 Thread Justin
This error is really common and is caused by windows via UAC, SFC, Firewarll, AV etc.. for it spontaneously to appear means system config change happened or update occurred. Common sources windows updates, anti-viruses changes, firewall changes. If its not the above 4 figuring out what causin

Re: JIT on Windows with Postgres 12.1

2020-02-12 Thread Justin
Here is a link to build LLVM on windows http://llvm.org/docs/GettingStartedVS.html On Wed, Feb 12, 2020 at 2:55 PM Tom Lane wrote: > Andres Freund writes: > > On 2020-02-12 09:39:19 +0100, Josef Šimánek wrote: > >> I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled. > > > It's

Re: How to restore roles without changing postgres password

2020-02-11 Thread Justin
HI Tom Not a bad idea, would want to extend this to all the roles on the server not just postgres I've edited the global dump many times removing/editing table spaces, comment old users, etc.. On Tue, Feb 11, 2020 at 5:46 PM Tom Lane wrote: > "Andrus" writes: > > How to create backup scri

Re: How to restore roles without changing postgres password

2020-02-11 Thread Justin
pg_dumpall creates an SQL file which is just a simple text file you can then edit sql removing postgres user from the file This can be automated in a script that searches the generated sql file for the postgres user replacing it with a blank/empty line or adds -- to the bringing of the line whi

Re: Fwd: sensible configuration of max_connections

2020-02-07 Thread Justin
On Fri, Feb 7, 2020 at 1:56 PM Sam Gendler wrote: > Benchmarks, at the time, showed that performance started to fall off due > to contention if the number of processes got much larger. I imagine that > the speed of storage today would maybe make 3 or 4x core count a pretty > reasonable place to

Re: sensible configuration of max_connections

2020-02-07 Thread Justin
Hi Chris Withers As stated each connection uses X amount of resources and its very easy to configure Postgresql where even small number of connections will each up all the RAM WorkMem is the biggest consumer of resources lets say its set to 5 megs per connection at 1000 connections that 5,000 me

Re: calculating the MD5 hash of role passwords in C

2020-01-22 Thread Justin
Not sure what your after but here is more information regarding how to store passwords in Postgresql, not related to database roles but for storing passwords for things like websites... https://www.postgresql.org/docs/current/pgcrypto.html section F.25.2.XXX On Wed, Jan 22, 2020 at 2:41 PM Ma

Re: @@TRANCOUNT ?

2020-01-22 Thread Justin
Postgresql does not support Transaction in the same way mssql, it does support nesting transactions in a limited way from version 11+ https://docs.microsoft.com/en-us/sql/t-sql/functions/trancount-transact-sql?view=sql-server-ver15 https://www.postgresql.org/docs/current/sql-begin.html https://w

Re: Is there a GoTo ?

2020-01-16 Thread Justin
Hi İlyas As noted by other there is no GOTO you can move the goto code into the else statement or move that code into a new function call and return that function or just return null in the else if that is all the it is doing CREATE OR REPLACE FUNCTION test(i integer) RETURNS integer AS $$BE

Re: Can I do this?

2020-01-16 Thread Justin
it does not, but the odds the same user will run this command by this id in two different sessions at the same time are very low. this type of code exist for PO,SO, Invoices, to assign the next line item # in many apps. On Thu, Jan 16, 2020 at 10:40 AM Michael Nolan wrote: > > > On Thu, Jan

Re: Can I drop a NOT NUL constrain on an existing table?

2020-01-16 Thread Justin
Alter table mytable drop constraint name_of_constraint https://www.postgresql.org/docs/current/sql-altertable.html On Thu, Jan 16, 2020 at 9:43 AM stan wrote: > I see how to do this if it is a "dcleared" constraint, but this was just > defined in the table createion as inL > > report_no

Re: Can I do this?

2020-01-16 Thread Justin
Hi Stan in you code sample there are "(" mis-matched, "MAX(" matches to "= project_key)"; it should be MAX(NULLIF(regexp_replace(report_no, '\D','','g'), '')::numeric) I do exactly what you do, and you are correct sequences are not a good fit I typically do something like this for selec

Re: Is it possible to replicate through an http proxy?

2020-01-14 Thread Justin
Another solution to the problem instead of logical replication would be utilize wal_shipping and have the edge servers replay the wal using the restore_command The wal files can be downloaded from from HTTP server via a proxy and placed on the edge servers wal_archive directory to be replayed se

Re: Worse performance with higher work_mem?

2020-01-14 Thread Justin
Updating the stats can be done via vacuum or analyze command, https://www.postgresql.org/docs/12/sql-analyze.html. To just analyze a table typically does not take much time. and can be scheduled to run so the stats update instead of waiting on auto-vacuum to deal with it which could be some tim

Re: Encrypted connection SQL server fdw

2020-01-10 Thread Justin
By loading data meaning this is a one time deal or only used to refresh data stored in the postgresql database??? A possible solution would be to setup a vpn tunnel, or ipsec connection to server. then run FDW through that connection. Not idea and will slow things down. The other option is to u

Re: UPDATE many records

2020-01-06 Thread Justin
worth looking into If its a one time run or every 12 months who cares, Start the update on friday night, go in on Saturday to check it On Mon, Jan 6, 2020 at 5:38 PM Israel Brewster wrote: > > On Jan 6, 2020, at 12:49 PM, Justin wrote: > > What was the HD wait time ? What tool is bei

Re: UPDATE many records

2020-01-06 Thread Justin
What was the HD wait time ? What tool is being use to monitor the server resources?? It appears based on this information there is allot more going on than a simple Update command Moving code out of the trigger probably not going to improve performance, unless there is allot of code that does

Re: UPDATE many records

2020-01-06 Thread Justin
7320 > Work: 907-474-5172 > cell: 907-328-9145 > > On Jan 6, 2020, at 10:05 AM, Justin wrote: > > There are several ways to actually do this > > If you have Postgresql 11 or higher we now have Create Procedure that > allows committing transactions, one d

Re: UPDATE many records

2020-01-06 Thread Justin
There are several ways to actually do this If you have Postgresql 11 or higher we now have Create Procedure that allows committing transactions, one draw back is it can not parallel from inside the procedure https://www.postgresql.org/docs/11/sql-createprocedure.html https://severalnines.com/dat

Re: When should parameters be passed as text v. binary?

2020-01-04 Thread Justin
As noted by Adrian what is the USE CASE As a general rule one wants to use the format the data is being stored in. every time data is cast to another type its going to eat those all so precious CPU cycles. (all the horror of electrons turned into infrared beams) converting Bytea type to a string

Re: unanalyze a foreign table

2019-12-23 Thread Justin
I do not know of way to undo an analyze once its committed. I do not know the danger in deleting an entry in pg_statistic What you can do in the future is make copy of the Statics for this table, analyze, if it negatively affect results put the copy back. Another option is to do begin ; ANALYZE

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Justin
:12 PM Tom Lane wrote: > Justin writes: > > I now see what is causing this specific issue... > > The update and row versions is happening on 2kb chunk at a time, That's > > going to make tracking what other clients are doing a difficult task. > > Yeah, it

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Justin
y X chunks" On Wed, Dec 18, 2019 at 11:12 AM Tom Lane wrote: > Justin writes: > > I have a question reading through this email chain. Does Large Objects > > table using these functions work like normal MVCC where there can be two > > versions of a large object in pg_

  1   2   >