Re: Can the current session be notified and refreshed with a new credentials context?

2020-06-22 Thread Tom Lane
AC Gomez writes: > OK, here goes again: Again, you're just asserting some claims without showing us what you did. As an example of the kind of detail I'm asking for, I ran this script (partially based on the example in the dblink docs), starting as a superuser so I had permissions to create ever

Re: Can the current session be notified and refreshed with a new credentials context?

2020-06-22 Thread David G. Johnston
On Mon, Jun 22, 2020 at 5:41 PM AC Gomez wrote: > But what I understand you to say is that, one can start running a function > in PG, change all security context from under it, and it will still work > under the original login context, despite the changes. > https://www.postgresql.org/docs/12/tu

Re: Can the current session be notified and refreshed with a new credentials context?

2020-06-22 Thread AC Gomez
Thanks Tom, OK, here goes again: Inside a PG database there's a master function. Inside this master function there are several calls to external databases using DBLINK. This master function works perfectly fine when not rotating the password. An outside application connects to the database and e

Re: scram-sha-256 encrypted password in pgpass

2020-06-22 Thread Adrian Klaver
On 6/22/20 3:54 PM, Stephen Frost wrote: Greetings, * Pavan Kumar (pavan.db...@gmail.com) wrote: What would be the point of storing the encrypted password instead of the plaintext one? As per our organization security policies, we can 't keep any passwords in plain text format. Then you nee

Re: scram-sha-256 encrypted password in pgpass

2020-06-22 Thread Ravi Krishna
> > But if you want to log in with encrypted password and someone can grab > it from the file not sure what the difference is from grabbing the plain > text one if they both end up logging the user in? Exactly. saved me the trouble of typing this.

Re: scram-sha-256 encrypted password in pgpass

2020-06-22 Thread Stephen Frost
Greetings, * Pavan Kumar (pavan.db...@gmail.com) wrote: > > What would be the point of storing the encrypted password instead of the > > plaintext one? > As per our organization security policies, we can 't keep any passwords in > plain text format. Then you need to *actually* encrypt the passwo

Re: scram-sha-256 encrypted password in pgpass

2020-06-22 Thread David G. Johnston
On Mon, Jun 22, 2020 at 3:32 PM Pavan Kumar wrote: > Adrian, David, > > Thank you so much for the quick response. > > What would be the point of storing the encrypted password instead of the > plaintext one? > As per our organization security policies, we can 't keep any passwords > in plain tex

Re: scram-sha-256 encrypted password in pgpass

2020-06-22 Thread Adrian Klaver
On 6/22/20 3:32 PM, Pavan Kumar wrote: Adrian, David, Thank you so much for the quick response. What would be the point of storing the encrypted password instead of the plaintext one? As per our organization security policies, we can 't keep any  passwords in plain text format. But if you w

Re: scram-sha-256 encrypted password in pgpass

2020-06-22 Thread Pavan Kumar
Adrian, David, Thank you so much for the quick response. What would be the point of storing the encrypted password instead of the plaintext one? As per our organization security policies, we can 't keep any passwords in plain text format. I am working on postgres + pgbouncer setup, tested pgboun

Re: Feature suggestion: auto-prefixing SELECT query column names with table/alias names

2020-06-22 Thread David G. Johnston
On Sunday, June 21, 2020, Guy Burgess wrote: > > a.id, a.title, b.id, b.title You are missing some double-quotes there. Of course, this can be achieved by avoiding the (often frowned-upon) SELECT > * syntax in the first place and using explicit column names, Or choose better, distinguishi

Re: Can the current session be notified and refreshed with a new credentials context?

2020-06-22 Thread Tom Lane
AC Gomez writes: > We do hold the original session open. The problem comes when we change the > password while that session is open, now the session and the User Mappings > are out of synch and we have failure. Well, there's no obvious reason for that to be a problem. As another respondent said,

Re: Can the current session be notified and refreshed with a new credentials context?

2020-06-22 Thread AC Gomez
We do hold the original session open. The problem comes when we change the password while that session is open, now the session and the User Mappings are out of synch and we have failure. On Mon, Jun 22, 2020, 6:08 PM Tom Lane wrote: > AC Gomez writes: > > Suppose you have the following scenari

Re: Can the current session be notified and refreshed with a new credentials context?

2020-06-22 Thread Tom Lane
AC Gomez writes: > Suppose you have the following scenario: > 1: Call some function with a certain user and password > 2: From inside that function, have several calls using DBLink > 3: At some point during the running of that function a password rotation(a > separate process) comes along and upda

Re: scram-sha-256 encrypted password in pgpass

2020-06-22 Thread David G. Johnston
Please don't cross-post. On Mon, Jun 22, 2020 at 1:35 PM Pavan Kumar wrote: > scram-sha-256 encrypted passwords are supported in .pgpass file ? If yes > kindly provide us an example. > > I am using below format and it is not working for me > > *pglnx1*:*5432*:pgbouncer:*pgadmin*:"SCRAM-SHA-256$4

Re: DISTINCT on jsonb fields and Indexes

2020-06-22 Thread Tom Lane
Michael Lewis writes: >> In the example of "select distinct expression", the planner will never >> notice that that expression has anything to do with an index. > Thanks for that explanation. I assume re-writing as a 'group by' would have > no bearing on that planner decision. Hmm ... actually,

Re: DISTINCT on jsonb fields and Indexes

2020-06-22 Thread Tom Lane
Michael Lewis writes: >> In the example of "select distinct expression", the planner will never >> notice that that expression has anything to do with an index. > Thanks for that explanation. I assume re-writing as a 'group by' would have > no bearing on that planner decision. Hmm ... actually,

RE: autovacuum failing on pg_largeobject and disk usage of the pg_largeobject growing unchecked

2020-06-22 Thread Daniel Verite
Jim Hurne wrote: > We are of course going to continue to try different things, but does > anyone have any other suggestions on what we should be looking at or what > settings we might want to adjust? If you can arrange a maintenance window, a faster way to rebuild pg_largeobject when it

Re: scram-sha-256 encrypted password in pgpass

2020-06-22 Thread Adrian Klaver
On 6/22/20 1:34 PM, Pavan Kumar wrote: Hello expertes, scram-sha-256 encrypted passwords are supported in .pgpass file ? If yes kindly provide us an example. I am using below format and it is not working for me /|pglnx1|/:/|5432|/:pgbouncer:/|pgadmin|/:"SCRAM-SHA-256$4096:6IDsjfedwsdpymp0Za7

Re: Feature suggestion: auto-prefixing SELECT query column names with table/alias names

2020-06-22 Thread Guy Burgess
On 22/06/2020 3:25 pm, Laurenz Albe wrote: Then there is the case of "a JOIN b USING (somecol)". Here, "somecol" will appear in the output only once. How should it be labeled? If you say "not at all", then what do you want to happen for SELECT * FROM a JOIN b USING (id) JOIN c USING (x) wh

Re: Can the current session be notified and refreshed with a new credentials context?

2020-06-22 Thread David G. Johnston
On Mon, Jun 22, 2020 at 2:21 PM Ron wrote: > On 6/22/20 4:07 PM, AC Gomez wrote: > > Suppose you have the following scenario: > > 1: Call some function with a certain user and password > 2: From inside that function, have several calls using DBLink > 3: At some point during the running of that fu

Re: Can the current session be notified and refreshed with a new credentials context?

2020-06-22 Thread Ron
On 6/22/20 4:07 PM, AC Gomez wrote: Suppose you have the following scenario: 1: Call some function with a certain user and password 2: From inside that function, have several calls using DBLink 3: At some point during the running of that function a password rotation(a separate process) comes al

Re: autovacuum failing on pg_largeobject and disk usage of the pg_largeobject growing unchecked

2020-06-22 Thread Michael Lewis
On Tue, Jun 16, 2020 at 2:41 PM Michael Lewis wrote: > On Tue, Jun 16, 2020 at 1:45 PM Jim Hurne wrote: > >> Thanks Michael, >> >> Here are our current autovacuum settings: >> >> autovacuum | on >> autovacuum_analyze_scale_factor | 0.1 >> autovacuum_analyze_thresh

Can the current session be notified and refreshed with a new credentials context?

2020-06-22 Thread AC Gomez
Suppose you have the following scenario: 1: Call some function with a certain user and password 2: From inside that function, have several calls using DBLink 3: At some point during the running of that function a password rotation(a separate process) comes along and updates the session user passwo

Re: autovacuum failing on pg_largeobject and disk usage of the pg_largeobject growing unchecked

2020-06-22 Thread Magnus Hagander
On Mon, Jun 22, 2020 at 10:01 PM Jim Hurne wrote: > We're still struggling to figure out why autovacuum seems to be failing or > is ineffective sometimes against the pg_largeobject table. > > We decided to try a VACUUM FULL VERBOSE on one of our problematic > databases. It did complete, but it to

scram-sha-256 encrypted password in pgpass

2020-06-22 Thread Pavan Kumar
Hello expertes, scram-sha-256 encrypted passwords are supported in .pgpass file ? If yes kindly provide us an example. I am using below format and it is not working for me *pglnx1*:*5432*:pgbouncer:*pgadmin*:"SCRAM-SHA-256$4096:6IDsjfedwsdpymp0Za7jaMew==$rzSoYL4ZYsW1WJAj7Lt3JtNLNR73AVY7sfsauikwe

Re: DISTINCT on jsonb fields and Indexes

2020-06-22 Thread Michael Lewis
> > In the example of "select distinct expression", the planner will never > notice > that that expression has anything to do with an index. > Thanks for that explanation. I assume re-writing as a 'group by' would have no bearing on that planner decision.

RE: autovacuum failing on pg_largeobject and disk usage of the pg_largeobject growing unchecked

2020-06-22 Thread Jim Hurne
We're still struggling to figure out why autovacuum seems to be failing or is ineffective sometimes against the pg_largeobject table. We decided to try a VACUUM FULL VERBOSE on one of our problematic databases. It did complete, but it took a surprisingly long time: INFO: vacuuming "pg_catal

Re: DISTINCT on jsonb fields and Indexes

2020-06-22 Thread Tom Lane
Michael Lewis writes: > On Sun, Jun 21, 2020 at 10:43 PM Sankar P > wrote: >> 2) CREATE INDEX idx_btree_records ON fluent USING BTREE ((record ->> >> 'destinationServiceName')); > If it is rarely used, create a partial index perhaps. I am a little > surprised that the plain btree index wasn't us

Re: Netapp SnapCenter

2020-06-22 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > > On 22. Jun, 2020, at 17:46, Wolff, Ken L wrote: > > So apologies if this is a stupid question but there's obviously been a lot > > of discussion on this issue. Was a consensus ever reached on the following? > > > > If a Postgres da

Re: ERROR: invalid memory alloc request size 18446744073709551613

2020-06-22 Thread Tom Lane
Flaris Feller writes: > Em seg., 22 de jun. de 2020 às 12:33, Peter J. Holzer > escreveu: >> On 2020-06-22 11:13:33 -0300, Flaris Feller wrote: >>> When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on >>> Intel x86_64 I noticed "invalid memory alloc request size" error at >>>

Re: Netapp SnapCenter

2020-06-22 Thread Paul Förster
Hi Ken, please don't put your answer at the top. > On 22. Jun, 2020, at 17:46, Wolff, Ken L wrote: > > So apologies if this is a stupid question but there's obviously been a lot of > discussion on this issue. Was a consensus ever reached on the following? > > If a Postgres database (both dat

Re: DISTINCT on jsonb fields and Indexes

2020-06-22 Thread Michael Lewis
On Sun, Jun 21, 2020 at 10:43 PM Sankar P wrote: > I have a table with the schema: > > CREATE TABLE fluent (id BIGSERIAL, record JSONB); > > Then I created a couple of indexes: > 1) CREATE INDEX idx_records ON fluent USING GIN (record); > What about using non-default jsonb_path_ops? > 2) CREA

Re: ERROR: invalid memory alloc request size 18446744073709551613

2020-06-22 Thread Flaris Feller
Hi Peter, The error happens with SELECT command too and the COPY command was used to extract data out of the database. Thanks in advance, Flaris Feller. Em seg., 22 de jun. de 2020 às 12:33, Peter J. Holzer escreveu: > On 2020-06-22 11:13:33 -0300, Flaris Feller wrote: > > When using Postgres

Re: Hiding a GUC from SQL

2020-06-22 Thread Michel Pelletier
On Sun, Jun 21, 2020 at 10:21 PM raf wrote: > Laurenz Albe wrote: > > > > But only mostly useless. :-) There are ways to limit the power of the > > > superuser. On Linux, for instance, "sysctl kernel.yama.ptrace_scope=3" > > > prevents tracing, debugging, and reading another process's memory, eve

Re: Netapp SnapCenter

2020-06-22 Thread Wolff, Ken L
So apologies if this is a stupid question but there's obviously been a lot of discussion on this issue. Was a consensus ever reached on the following? If a Postgres database (both data and WAL) is located on one NetApp volume, meaning a snapshot should capture everything at exactly the same tim

Re: ERROR: invalid memory alloc request size 18446744073709551613

2020-06-22 Thread Peter J. Holzer
On 2020-06-22 11:13:33 -0300, Flaris Feller wrote: > When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on Intel > x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL logs. > This is the postgresq.log file's fragment of log where the error was found. > > 2020

Re: ERROR: invalid memory alloc request size 18446744073709551613

2020-06-22 Thread Rob Sargent
> On Jun 22, 2020, at 8:13 AM, Flaris Feller wrote: > > Hello all, > > When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on Intel > x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL logs. > This is the postgresq.log file's fragment of log where the er

ERROR: invalid memory alloc request size 18446744073709551613

2020-06-22 Thread Flaris Feller
Hello all, When using Postgres 9.6.15 on "CentOS Linux release 7.5.1804 (Core)" on Intel x86_64 I noticed "invalid memory alloc request size" error at PostgreSQL logs. This is the postgresq.log file's fragment of log where the error was found. 2020-06-22 00:29:18 BRT [16987]: [1-1] db=bxs,user=po

Re: Netapp SnapCenter

2020-06-22 Thread Stephen Frost
Greetings, * Paul Förster (paul.foers...@gmail.com) wrote: > > On 22. Jun, 2020, at 13:08, Magnus Hagander wrote: > > It does not work off *that* base backup. But if you start from the *prior* > > be backup (one that did complete with a successful pg_stop_backup) then you > > can still use the

Re: Netapp SnapCenter

2020-06-22 Thread Paul Förster
Hi Magnus, > On 22. Jun, 2020, at 13:08, Magnus Hagander wrote: > > It does not work off *that* base backup. But if you start from the *prior* be > backup (one that did complete with a successful pg_stop_backup) then you can > still use the archived wal to recover to any point in time. ok, t

Re: Netapp SnapCenter

2020-06-22 Thread Paul Förster
Hi Peter, > On 22. Jun, 2020, at 13:01, Peter J. Holzer wrote: > Restore the previous backup and replay WALs from there (that assumes of > course that you are archiving WALs continuously, but if you don't, you > can't do PITR in general, so if you have that requirement you are doing > it). ok, t

Re: The backup API and general purpose backup software

2020-06-22 Thread Peter J. Holzer
On 2020-06-21 17:35:41 -0500, Ron wrote: > On 6/21/20 10:45 AM, Peter J. Holzer wrote: > > On 2020-06-21 10:32:16 -0500, Ron wrote: > > > On 6/21/20 8:28 AM, Peter J. Holzer wrote: > > > > To make a full backup with the "new" (non-exclusive) API, a software > > > > must do the following > > > > >

Re: Netapp SnapCenter

2020-06-22 Thread Magnus Hagander
On Mon, Jun 22, 2020 at 8:02 AM Paul Förster wrote: > Hi Stephen, > > > On 22. Jun, 2020, at 07:36, Stephen Frost wrote: > > That's not the only case that I, at least, have heard of- folks aren't > > really very happy with their backups fail when they could have just as > > well completed, even

Re: Netapp SnapCenter

2020-06-22 Thread Peter J. Holzer
On 2020-06-22 08:02:06 +0200, Paul Förster wrote: > > On 22. Jun, 2020, at 07:36, Stephen Frost wrote: > > The thing about this is though that the new API avoids *other* issues, > > like what happens if the system crashes during a backup (which is an > > entirely common thing that happens, conside

Re: A query in Streaming Replication

2020-06-22 Thread Sreerama Manoj
Hello, Thanks for you reply >> If files are assumed to > >> be sound, pg_rewind will adjust the old master as a new standby > To use pg_rewind, we should either enable *wal_log_hints *or initialize data directory with checksum enabled. These are not default and will not be enabled in most of t

Re: DISTINCT on jsonb fields and Indexes

2020-06-22 Thread Sankar P
> > There is some work in progress to improve this type of query, but > it'll be at least PG14 before we see that. oh okay. > > For your version, you might want to look at > https://wiki.postgresql.org/wiki/Loose_indexscan and try some of the > proposed solutions from there. Thanks a lot :) --

Re: DISTINCT on jsonb fields and Indexes

2020-06-22 Thread David Rowley
On Mon, 22 Jun 2020 at 16:44, Sankar P wrote: > select distinct record ->> 'destinationServiceName' from fluent; > This query takes a lot of time, and does full table scan. The query planning > is: > I see that none of the indexes are used. I want to do a few > aggregations, like "what are the