Re: Back Port Request for INVALID Startup Packet

2020-03-11 Thread Laurenz Albe
On Thu, 2020-03-12 at 01:16 +, Virendra Kumar wrote: > This is simple patch, would that impact badly if patched to prior versions or > some other > constraints forced to not do that. I am just trying to understand this a bit. It is not that this patch would have a terrible impact. There is a

Re: Back Port Request for INVALID Startup Packet

2020-03-11 Thread Virendra Kumar
Hi Tom, Thank you for your reply! This is simple patch, would that impact badly if patched to prior versions or some other constraints forced to not do that. I am just trying to understand this a bit. On AWS RDS we have primary and secondary hosts known in advance in most cases. So if a primary

encrypt/decrypt between javascript and postgresql.

2020-03-11 Thread AC Gomez
I'm trying to encrypt/decrypt between javascript and postgresql. I'm using this: https://gist.github.com/vlucas/2bd40f62d20c1d49237a109d491974eb algorithm to encrypt my text, and then in PostgreSQL I use PGCRYPTO.decrypt_iv to decrypt the text. I pass in 'ThisISMySign' to the Encrypt function. E

Re: Back Port Request for INVALID Startup Packet

2020-03-11 Thread Tom Lane
Virendra Kumar writes: > Can you please back port patch where if a 0 byte packet sent to PG instance > (Health Checks), it starts complaining about invalid startup packet and flood > the log which increases log size considerably if the health checks are every > 3 seconds or something like that.

Re: pg_restore restores out of order

2020-03-11 Thread Adrian Klaver
On 3/11/20 4:11 PM, Kevin Brannen wrote: Adrian Klaver wrote: On 3/11/20 2:46 PM, Kevin Brannen wrote: I'm working thru our system to upgrade from 9.6.5 to 12.2. One of the last things I'm verifying is backup/restore and it's failing for no reason I can figure out. So I'm looking for pointers

Back Port Request for INVALID Startup Packet

2020-03-11 Thread Virendra Kumar
Hi Team, Can you please back port patch where if a 0 byte packet sent to PG instance (Health Checks), it starts complaining about invalid startup packet and flood the log which increases log size considerably if the health checks are every 3 seconds or something like that. Patch Requested - htt

RE: pg_restore restores out of order

2020-03-11 Thread Kevin Brannen
>Adrian Klaver wrote: >On 3/11/20 2:46 PM, Kevin Brannen wrote: >> I'm working thru our system to upgrade from 9.6.5 to 12.2. One of the last >> things I'm verifying is backup/restore and it's failing for no reason I can >> figure out. So I'm looking for pointers on this. >> >> If it matters, the

Re: Query to retrieve the index columns when a function is used.

2020-03-11 Thread Tom Lane
David Rowley writes: > On Tue, 10 Mar 2020 at 02:16, Sterpu Victor wrote: >> How can I retrieve this detail? > Those details are in the indexprs column. There's an item there for > each 0 valued indkey. It's not going to be particularly easy for you > to parse that from SQL. Internally in Post

Re: Streaming replication - 11.5

2020-03-11 Thread Adrian Klaver
On 3/11/20 2:12 PM, Nicola Contu wrote: CPU load on the server to be built? No. CPU load, I/O load on the servers in the replication chain. Basically you just recently, it seems, imposed extra overhead to the process by encrypting/decrypting. From what I gather from earlier post then your re

Re: pg_restore restores out of order

2020-03-11 Thread Adrian Klaver
On 3/11/20 2:46 PM, Kevin Brannen wrote: I'm working thru our system to upgrade from 9.6.5 to 12.2. One of the last things I'm verifying is backup/restore and it's failing for no reason I can figure out. So I'm looking for pointers on this. If it matters, the code is from the version 12.2 from

Re: Querying an index's btree version

2020-03-11 Thread Laurenz Albe
On Wed, 2020-03-11 at 17:12 -0400, Darren Lafreniere wrote: > Separate but related follow-up question: when you restore a DB from a backup, > does the restored index use the old format or the latest one? If you restore a pg_dump, you will have the latest version. If you restore a file system bac

pg_restore restores out of order

2020-03-11 Thread Kevin Brannen
I'm working thru our system to upgrade from 9.6.5 to 12.2. One of the last things I'm verifying is backup/restore and it's failing for no reason I can figure out. So I'm looking for pointers on this. If it matters, the code is from the version 12.2 from the Pg site, RPMs for Centos 6 (.10). Th

Re: strange locks on PG 11 with Golang programs

2020-03-11 Thread Peter J. Holzer
On 2020-03-09 11:02:37 +0200, Achilleas Mantzios wrote: > Fully review your programs for connection / xaction leaks. Do you use a > connection pool? Go's sql package encourages the use of connection pools (type DB) over single connections (type Conn): | Prefer running queries from DB unless there

Re: Querying an index's btree version

2020-03-11 Thread Peter Geoghegan
On Wed, Mar 11, 2020 at 2:13 PM Darren Lafreniere wrote: > when you restore a DB from a backup, does the restored index use the old > format or the latest one? If you use pg_restore, it uses the latest index format. If you're using pg_upgrade, the version won't change unless and until you REIND

Re: Querying an index's btree version

2020-03-11 Thread Darren Lafreniere
Thank you for the info, Peter. Separate but related follow-up question: when you restore a DB from a backup, does the restored index use the old format or the latest one? Thank you, Darren Lafreniere On Wed, Mar 11, 2020 at 4:30 PM Peter Geoghegan wrote: > On Wed, Mar 11, 2020 at 1:26 PM Dar

Re: Streaming replication - 11.5

2020-03-11 Thread Nicola Contu
CPU load on the server to be built? No. System logs don't show anything relevant unfortunately Il mer 11 mar 2020, 21:34 Adrian Klaver ha scritto: > On 3/11/20 11:59 AM, Nicola Contu wrote: > > I am actually cascading. > > The master is in nyh, the first slave is in Dallas and the one having > >

Re: Query to retrieve the index columns when a function is used.

2020-03-11 Thread David Rowley
On Tue, 10 Mar 2020 at 02:16, Sterpu Victor wrote: > I'm testing on Postgresql 12.1 and I have a index like this: > "check_dates_gist" EXCLUDE USING gist (id_test1 WITH =, id_test2 WITH =, > tsrange(valid_from::timestamp without time zone, valid_to::timestamp without > time zone) WITH &&) > > Wh

Re: Streaming replication - 11.5

2020-03-11 Thread Adrian Klaver
On 3/11/20 11:59 AM, Nicola Contu wrote: I am actually cascading. The master is in nyh, the first slave is in Dallas and the one having problems is in Dallas as well on the same switch of the one replicating from the master. It always worked not sure what is wrong now. We just encrypted disks

Re: Querying an index's btree version

2020-03-11 Thread Peter Geoghegan
On Wed, Mar 11, 2020 at 1:26 PM Darren Lafreniere wrote: > We've read that PG 12 has improved btree index support, and that the latest > internal btree version was bumped from 3 to 4. Is it possible to query the > btree version that a particular index is using? We'd like to automatically > star

Querying an index's btree version

2020-03-11 Thread Darren Lafreniere
Hello, We've read that PG 12 has improved btree index support, and that the latest internal btree version was bumped from 3 to 4. Is it possible to query the btree version that a particular index is using? We'd like to automatically start a concurrent re-index if we detect any btree indexes are st

Re: Web users as database users?

2020-03-11 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > I'm not aware that anyone's done formal performance testing, > but if you want to have a lot of roles in the system, my > expectation is that you'd be better off granting privileges > to a small number of group roles and then granting group > rol

Re: Streaming replication - 11.5

2020-03-11 Thread Nicola Contu
I am actually cascading. The master is in nyh, the first slave is in Dallas and the one having problems is in Dallas as well on the same switch of the one replicating from the master. It always worked not sure what is wrong now. We just encrypted disks on all servers Il mer 11 mar 2020, 18:57 Ad

Re: Streaming replication - 11.5

2020-03-11 Thread Adrian Klaver
On 3/11/20 2:54 AM, Nicola Contu wrote: These are the lines before 2020-03-11 09:05:08 GMT [127.0.0.1(40214)] [43853]: [1-1] db=cmdv3,user=zabbix_check ERROR:  recovery is in progress 2020-03-11 09:05:08 GMT [127.0.0.1(40214)] [43853]: [2-1] db=cmdv3,user=zabbix_check HINT:  WAL control functi

Re: How to discover what table is

2020-03-11 Thread Adrian Klaver
On 3/11/20 1:22 AM, PegoraroF10 wrote: Well, for now it´s solved but I´ll explain what happens to solve it better on future. Suppose on Master you have a database with hundreds of schemas with same structure, so table Customer happens 500 times on that DB. That database being replicated with publ

Re: Force WAL cleanup on running instance

2020-03-11 Thread Torsten Krah
Am Mittwoch, den 11.03.2020, 11:46 -0500 schrieb Jerry Sievers: > If your site can afford a restart after the bulk load, > > 1. Clean shutdown. > 2. pg_resetwal > 3. Start > > That should leave you with a very small N WAL files, perhaps just 1, > though I've not run it lately to reverify. Thank

Re: Force WAL cleanup on running instance

2020-03-11 Thread Paul Förster
Hi Justin, that came to my mind also. Then I tried and found that not always a new WAL is created. I admit I tried on a test DC with no other transactions going on. Maybe I should have done that. Anyway, I also always do the checkpoint first and then the WAL switch, which in my case is also an

Re: Force WAL cleanup on running instance

2020-03-11 Thread Jerry Sievers
Torsten Krah writes: > Am Mittwoch, den 11.03.2020, 15:45 + schrieb Simon Riggs: > >> The size of the task varies, so sometimes takes longer than 60s, >> depending >> upon your hardware. > > Yes that's what I am observing and why I am asking if there is some > select statement or command whic

Re: Force WAL cleanup on running instance

2020-03-11 Thread Torsten Krah
Am Mittwoch, den 11.03.2020, 15:45 + schrieb Simon Riggs: > The size of the task varies, so sometimes takes longer than 60s, > depending > upon your hardware. Yes that's what I am observing and why I am asking if there is some select statement or command which triggers that task and returns af

Re: Force WAL cleanup on running instance

2020-03-11 Thread Justin
Question everyone isn't this a problem with the order of operations? switching the wal files then running checkpoint means the Checkpoint can cross wal files, so the previous wal file can not be deleted??? To my understanding the order operations should be Checkpoint which flushes everything

Re: Force WAL cleanup on running instance

2020-03-11 Thread Simon Riggs
On Wed, 11 Mar 2020 at 08:59, Torsten Krah wrote: > Hi, > > I am building a docker image with a postgresql 12.2 instance and while > doing so and importing a dump and running some update scripts wal size > is increasing. > > When finished I don't need all those wal files anymore and tried to > fo

Re: ERROR: invalid memory alloc request size 1073741824

2020-03-11 Thread Tom Lane
Stefan Blanke writes: > We've upgraded to PostgreSQL 11.5 (postgresql.org rhel 6 rpm) and I have > had another occurrence of this invalid alloc of 1GB. Apologies for never > providing a query plan when discussing this two years ago; we decided to > move to a newer PostgreSQL to see if the issue

Re: ERROR: invalid memory alloc request size 1073741824

2020-03-11 Thread Stefan Blanke
Hi, We've upgraded to PostgreSQL 11.5 (postgresql.org rhel 6 rpm) and I have had another occurrence of this invalid alloc of 1GB. Apologies for never providing a query plan when discussing this two years ago; we decided to move to a newer PostgreSQL to see if the issue went away but took a wh

Re: Web users as database users?

2020-03-11 Thread Tom Lane
Michael Lewis writes: > On Fri, Sep 20, 2019 at 8:19 AM Tom Lane wrote: >> There is a restriction on how many distinct GRANTs you can >> issue against any one object --- performance will get bad if the ACL >> list gets too large. > Any ballpark numbers here? Are we talking 50 or 8000? More like

Re: Force WAL cleanup on running instance

2020-03-11 Thread Torsten Krah
Am Mittwoch, den 11.03.2020, 08:42 -0600 schrieb Michael Lewis: > I don't know the answer to your stated question. I am curious if you > have > set wal_level = minimal and if not, if that would be appropriate for > your > use case and might render your concern a non-issue. Hi Micheal, I am alread

Re: Web users as database users?

2020-03-11 Thread Andrei Zhidenkov
I used to use a different approach: 1. Create auth() pl/python procedure as follows: create or replace function auth(auser_id integer) returns void as $$ GD['user_id'] = auser_id $$ language plpythonu; This procedure is supposed to be called after a sucesseful authorisation (in a database o

Re: Web users as database users?

2020-03-11 Thread Michael Lewis
On Fri, Sep 20, 2019 at 8:19 AM Tom Lane wrote: > There is a restriction on how many distinct GRANTs you can > issue against any one object --- performance will get bad if the ACL > list gets too large. > Any ballpark numbers here? Are we talking 50 or 8000?

Re: Force WAL cleanup on running instance

2020-03-11 Thread Michael Lewis
I don't know the answer to your stated question. I am curious if you have set wal_level = minimal and if not, if that would be appropriate for your use case and might render your concern a non-issue.

Re: How to set a value when NULL

2020-03-11 Thread David G. Johnston
On Wednesday, March 11, 2020, sivapostg...@yahoo.com wrote: > Hello, > Need to set a value of Zero when the field value is NULL in trigger > function. > > Tried with, > NEW.fieldname = NULLIF(NEW.fieldname, 0) > > in before insert/update trigger. > > Looks like it's not working. I'm doing anythi

RE: Patterns to look for in the PostgreSQL server log

2020-03-11 Thread Kevin Brannen
>From: Mageshwaran Janarthanam > >Hi Team...I am trying to setup some monitoring over the PostgreSQL server log. >I am not clear which error I should be most concerned about. Could you please >share your thoughts on what pattern I should search in the log file? I'd treat it like any other log a

Re: How to set a value when NULL

2020-03-11 Thread sivapostg...@yahoo.com
Hello,Need to set a value of Zero when the field value is NULL in trigger function. Tried with,NEW.fieldname = NULLIF(NEW.fieldname, 0) in before insert/update trigger. Looks like it's not working.  I'm doing anything wrong. Happiness AlwaysBKR Sivaprakash

Re: Streaming replication - 11.5

2020-03-11 Thread Nicola Contu
These are the lines before 2020-03-11 09:05:08 GMT [127.0.0.1(40214)] [43853]: [1-1] db=cmdv3,user=zabbix_check ERROR: recovery is in progress 2020-03-11 09:05:08 GMT [127.0.0.1(40214)] [43853]: [2-1] db=cmdv3,user=zabbix_check HINT: WAL control functions cannot be executed during recovery. 2020

Force WAL cleanup on running instance

2020-03-11 Thread Torsten Krah
Hi, I am building a docker image with a postgresql 12.2 instance and while doing so and importing a dump and running some update scripts wal size is increasing. When finished I don't need all those wal files anymore and tried to force the daemon to clean them up and tried this: select pg_swi

Re: How to discover what table is

2020-03-11 Thread PegoraroF10
Well, for now it´s solved but I´ll explain what happens to solve it better on future. Suppose on Master you have a database with hundreds of schemas with same structure, so table Customer happens 500 times on that DB. That database being replicated with publication/subscription for all tables model