Re: [bug fix] Produce a crash dump before main() on Windows

2018-07-18 Thread Craig Ringer
On 18 July 2018 at 14:34, Michael Paquier wrote: > On Wed, Jul 18, 2018 at 06:09:57AM +, Tsunakawa, Takayuki wrote: > > From: Haribabu Kommi [mailto:kommi.harib...@gmail.com] > >> I reviewed patch and it works as per the subject, but I am not able to > verify > >> the actual > >> bug that is

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-07-18 Thread Konstantin Knizhnik
On 18.07.2018 02:58, Tomas Vondra wrote: On 07/18/2018 12:41 AM, Konstantin Knizhnik wrote: ... Teodor Sigaev has proposed an alternative approach for calculating selectivity of multicolumn join or compound index search. Usually DBA creates compound indexes which can be used  by optimizer to

RE: Temporary tables prevent autovacuum, leading to XID wraparound

2018-07-18 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:mich...@paquier.xyz] > + /* Does the backend own the temp schema? */ > + if (proc->tempNamespaceId != namespaceID) > + return false; > I have a very hard time believing that this is safe lock-less, and a spin > lock would be enough it seems. The lwlock in Ba

Re: [bug fix] Produce a crash dump before main() on Windows

2018-07-18 Thread Haribabu Kommi
On Wed, Jul 18, 2018 at 4:10 PM Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > From: Haribabu Kommi [mailto:kommi.harib...@gmail.com] > > I reviewed patch and it works as per the subject, but I am not able to > verify > > the actual > > bug that is reported in the upthread. The mov

Re: missing toast table for pg_policy

2018-07-18 Thread John Naylor
On 7/17/18, Michael Paquier wrote: > [... digging ...] > This comes from get_rel_infos where large objects are treated as user > data. Rather than the comment you added, I would rather do the > following: > "Large object catalogs and toast tables are mutually exclusive and large > object data is

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 01:43, Fabien COELHO wrote: The more reasonable alternative could be to always last 2 seconds under -T 2, even if the execution can be shorten because there is nothing to do at all, i.e. remove the environment-based condition but keep the sleep. That sounds reasonable. It's a bit si

Re: Incorrect error handling for two-phase state files resulting in data loss

2018-07-18 Thread Michael Paquier
On Mon, Jul 09, 2018 at 02:03:09PM +0900, Michael Paquier wrote: > I think that we really need to harden things, by making > ReadTwoPhaseFile() fail hard is it finds something unexpected, which is > in this case anything except trying to open a file which fails on > ENOENT, and that this stuff shou

Re: cursors with prepared statements

2018-07-18 Thread Heikki Linnakangas
On 16/07/18 15:56, Peter Eisentraut wrote: On 11.07.18 19:07, Heikki Linnakangas wrote: It's confusing, and risks conflicting with future additions to the standard. ECPG supports the actual standard syntax, with OPEN, right? So this wouldn't be consistent with ECPG, either. It would be consist

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-07-18 Thread Amit Langote
Hi David, Thanks for taking a look. On 2018/07/15 17:34, David Rowley wrote: > I've looked over the code and the ExecUseUpdateResultRelForRouting() > function is broken. Your while loop only skips partitions for the > current partitioned table, it does not skip ModifyTable subnodes that > belong

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-07-18 Thread Kyotaro HORIGUCHI
Hello. At Wed, 18 Jul 2018 14:02:47 +1200, Thomas Munro wrote in > On Sat, Jul 14, 2018 at 5:34 AM, Heikki Linnakangas wrote: > > On 18/04/18 09:55, Thomas Munro wrote: > >> Here's a draft patch that does that. One contentious question is: > >> should you have to opt *in* to auto-exit-on-pos

RE: [bug fix] Produce a crash dump before main() on Windows

2018-07-18 Thread Tsunakawa, Takayuki
From: Haribabu Kommi [mailto:kommi.harib...@gmail.com] > May be I can give a try by modifying the source code to get the crash. Thank you, that would be great if you could come up with a good way! > My point is, With this patch, in case if the postgres crashses > before reaching main(), does it

Re: Bug in gin insert redo code path during re-compression of empty gin data leaf pages

2018-07-18 Thread Alexander Korotkov
Hi! On Wed, Jul 18, 2018 at 1:40 AM R, Siva wrote: > We came across an issue during replay of a gin insert record on a pre-9.4 > uncompressed data leaf page that does not have any items in it. The engine > version where the replay is done is 9.6.3. The redo logic attempts to > compress the exist

Re: why partition pruning doesn't work?

2018-07-18 Thread Amit Langote
On 2018/07/16 2:02, Tom Lane wrote: > Amit Langote writes: >> On 2018/06/19 2:05, Tom Lane wrote: >>> Or maybe what we should do is drop ExecLockNonLeafAppendTables/ >>> ExecOpenAppendPartitionedTables entirely and teach InitPlan to do it. > >> Hmm, for InitPlan to do what ExecOpenAppendPartition

Re: PG 10: could not generate random cancel key

2018-07-18 Thread Dean Rasheed
On 18 July 2018 at 03:17, Michael Paquier wrote: >> [1] https://wiki.openssl.org/index.php/Random_Numbers > > This quote from the wiki is scary so that's not quite clean either for > Windows: > "Be careful when deferring to RAND_poll on some Unix systems because it > does not seed the generator. S

Re: Alter index rename concurrently to

2018-07-18 Thread Peter Eisentraut
On 17.07.18 13:48, Andrey Klychkov wrote: > Please, have a look at previous discussions on the subject: > - 2012 > > https://www.postgresql.org/message-id/cab7npqtys6juqdxuczbjb0bnw0kprw8wdzuk11kaxqq6o98...@mail.gmail.com > - > 2013  > https://www.postgresql.org/message-id/cab7

Re: documentation about explicit locking

2018-07-18 Thread Peter Eisentraut
On 06.07.18 04:00, Amit Langote wrote: > On 2018/07/05 23:02, Robert Haas wrote: >> On Wed, Jul 4, 2018 at 3:09 AM, Amit Langote >> wrote: >>> I wonder why we mention on the following page that CREATE COLLATION >>> requires SHARE ROW EXCLUSIVE lock >>> >>> https://www.postgresql.org/docs/devel/sta

RE: Speeding up INSERTs and UPDATEs to partitioned tables

2018-07-18 Thread Kato, Sho
On 2018-Jul-11, Alvaro Herrera wrote: > That commit is also in pg11, though -- just not in beta2. So we still don't > know how much of an improvement patch2 is by itself :-) Oops! I benchmarked with 11beta2 + 0001-Speed​​-up-INSERT-and-UPDATE-on-partitioned-tables.patch. Results are as follows.

Re[2]: Alter index rename concurrently to

2018-07-18 Thread Andrey Klychkov
>Среда, 18 июля 2018, 12:21 +03:00 от Peter Eisentraut >: > >If we think the lower lock level is OK, then we should just use >it always. > Hi, I absolutely agree with you. If lower locking is safe and possible to be used by default in renaming it will be great. What stage is solving of this issu

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-07-18 Thread David Rowley
On 18 July 2018 at 21:44, Kato, Sho wrote: > part_num | latency_avg | tps_ex | update_latency | select_latency | > insert_latency > --+-++++ > 100 |2.09 | 478.379516 | 1.407 | 0.36

Re: untrusted PLs should be GRANTable

2018-07-18 Thread Robert Haas
On Tue, Jul 17, 2018 at 1:20 AM, Craig Ringer wrote: > Forcing users to create their PLs as a superuser increases the routine use > of superuser accounts. Most users' DDL deploy scripts will get be run as a > superuser if they have to use a superuser for PL changes; they're not going > to SET ROLE

Re: [HACKERS] WAL logging problem in 9.4.3?

2018-07-18 Thread Robert Haas
On Tue, Jul 17, 2018 at 8:28 AM, Andrew Dunstan wrote: > Well, we'd be getting rid of it because of a danger of data loss which we > can't otherwise mitigate. Maybe it does need to be backpatched, even if we > haven't had complaints. What's wrong with the approach proposed in http://postgr.es/m/5

Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

2018-07-18 Thread Etsuro Fujita
(2018/07/04 11:00), Etsuro Fujita wrote: (2018/07/04 1:35), Andres Freund wrote: What's the plan forward here? I still think that this approach would be the right way to go, so I plan to polish the patch. The approach would not require extra cycles where partitioning is not involved as disc

Re: GiST VACUUM

2018-07-18 Thread Heikki Linnakangas
On 17/07/18 21:41, Andrey Borodin wrote: I was checking WAL replay of new scheme to log page deletes and found a bug there (incorrect value of deleted downlink in WAL record). Here's fixed patch v10. Also I've added support to WAL identification for new record, done some improvements to comments

Re: Make foo=null a warning by default.

2018-07-18 Thread Fabien COELHO
Hello David, I assure you that you expression yourself in English a good deal better than I do in Portuguese. Alas, despite a Portuguese "rabbit" name, I cannot speak the language which got lost between generations. About this v3: Patch applies, compiles, "make check" ok. A few minor com

Re: Libpq support to connect to standby server as priority

2018-07-18 Thread Robert Haas
On Wed, Jul 4, 2018 at 9:14 AM, Laurenz Albe wrote: > What about keeping the first successful connection open and storing it in a > variable if we are in "prefer-read" mode. > If we get the read-only connection we desire, close that cached connection, > otherwise use it. I like this idea.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Fabien COELHO
Hello Heikki, I did that in the attached version: no more environment variable hack, and no execution shortcut even if there is nothing to do. I also had to reproduce the progress logic to keep on printing report of (no) progress in this tailing phase. On second thoughts, there's one problem

Re: PG 10: could not generate random cancel key

2018-07-18 Thread Michael Paquier
On Wed, Jul 18, 2018 at 10:14:56AM +0100, Dean Rasheed wrote: > OK, I guess that it is possible that an older version of OpenSSL > requires RAND_poll() to be called multiple times. Here's an updated > patch doing that (with up to 8 retries, based on the old OpenSSL > code). Thanks for the updated

Re: [HACKERS] WAL logging problem in 9.4.3?

2018-07-18 Thread Michael Paquier
On Wed, Jul 18, 2018 at 06:42:10AM -0400, Robert Haas wrote: > On Tue, Jul 17, 2018 at 8:28 AM, Andrew Dunstan > wrote: >> Well, we'd be getting rid of it because of a danger of data loss which we >> can't otherwise mitigate. Maybe it does need to be backpatched, even if we >> haven't had complain

Re: PG 10: could not generate random cancel key

2018-07-18 Thread Dean Rasheed
On 18 July 2018 at 14:01, Michael Paquier wrote: > Thanks for the updated version. This looks safer to me. It is possible > to simplify the code by removing the external RAND_status() call and > check for RAND_status() first in the loop as per the attached. OK, thanks. Barring any further comm

Re: [HACKERS] WAL logging problem in 9.4.3?

2018-07-18 Thread Robert Haas
On Wed, Jul 18, 2018 at 9:06 AM, Michael Paquier wrote: >> What's wrong with the approach proposed in >> http://postgr.es/m/55afc302.1060...@iki.fi ? > > For back-branches that's very invasive so that seems risky to me > particularly seeing the low number of complaints on the matter. Hmm. I think

Re: Alter index rename concurrently to

2018-07-18 Thread Peter Eisentraut
On 18.07.18 11:44, Andrey Klychkov wrote: > If lower locking is safe and possible to be used by default in renaming > it will be great. > What stage is solving of this issue? Does anybody do it? We wait to see if anyone has any concerns about this change. -- Peter Eisentraut http://

Re: why partition pruning doesn't work?

2018-07-18 Thread Tom Lane
Amit Langote writes: > On 2018/07/16 2:02, Tom Lane wrote: >> BTW, there'd be a lot to be said for having InitPlan just open all >> the rels and build an array of Relation pointers that parallels the >> RTE list, rather than doing heap_opens in random places elsewhere. > +1 to this. Actually I h

Re: [HACKERS] PoC: full merge join on comparison clause

2018-07-18 Thread Ashutosh Bapat
On Tue, Jul 10, 2018 at 10:06 PM, Alexander Kuzmenkov wrote: > I tried to fix the things you mentioned and improve the comments. Among > other changes, there is now a description of how merge join works with > inequalities at the top of nodeMergejoin.c. It also explains why we only > support one i

Re: [HACKERS] logical decoding of two-phase transactions

2018-07-18 Thread Tomas Vondra
On 07/17/2018 08:10 PM, Robert Haas wrote: On Mon, Jul 16, 2018 at 3:25 PM, Tomas Vondra wrote: Oh, right, I forgot the patch also adds the leader into the group, for some reason (I agree it's unclear why that would be necessary, as you pointed out later). But all this is happening while hol

Re: ENOSPC FailedAssertion("!(RefCountErrors == 0)"

2018-07-18 Thread Tom Lane
Kyotaro HORIGUCHI writes: > Hello. I confirmed that this patch fixes the crash. Thanks for double-checking. > At Tue, 17 Jul 2018 20:01:05 -0400, Tom Lane wrote in > <14892.1531872...@sss.pgh.pa.us> >> Further investigation showed that the part of that code that was >> actually needed was not

Re: Segfault logical replication PG 10.4

2018-07-18 Thread Mai Peng
Hello, Here the backtrace ```Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `postgres: bgworker: logical replication worker for subscription'. Program terminated with signal SIGSEGV, Segmentation fault. #0 GetActiveSnapshot () at ./build/../src/b

Re: partition tree inspection functions

2018-07-18 Thread Jesper Pedersen
Hi Amit, On 06/28/2018 01:49 AM, Amit Langote wrote: OK, I've added an example below the table of functions added by the patch. Attached updated patch. You forgot to remove the test output in create_table.out, so check-world is failing. In pg_partition_parent + else + /* Not

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 16:01, Fabien COELHO wrote: I don't think you want to wait in that situation. I think we should wait at the end only if there some threads still alive, with nothing to do only because of --rate. Yep. The attached version does only the tailing stuff under -R and not all threads were

Re: [HACKERS] logical decoding of two-phase transactions

2018-07-18 Thread Robert Haas
On Wed, Jul 18, 2018 at 10:08 AM, Tomas Vondra wrote: > The problem is you don't know if a transaction does DDL sometime later, in > the part that you might not have decoded yet (or perhaps concurrently with > the decoding). So I don't see how you could easily exclude such transactions > from the

Re: More consistency for some file-related error message

2018-07-18 Thread Tom Lane
Michael Paquier writes: > While looking at the source code for more consistency work with error > messages, I have bumped into a couple of messages which could be > simplified, as those include in the name of the file manipulated > basically the same information as the context added. > I have fin

Re: [HACKERS] WAL logging problem in 9.4.3?

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 16:29, Robert Haas wrote: On Wed, Jul 18, 2018 at 9:06 AM, Michael Paquier wrote: What's wrong with the approach proposed in http://postgr.es/m/55afc302.1060...@iki.fi ? For back-branches that's very invasive so that seems risky to me particularly seeing the low number of complain

Re: Segfault logical replication PG 10.4

2018-07-18 Thread Tom Lane
Mai Peng writes: > Here the backtrace Hmm .. so this can be summarized as "logical replication workers should provide an ActiveSnapshot in case the user functions they call want one". Makes me wonder how much other transactional infrastructure is needed but not present. r

Re: Make foo=null a warning by default.

2018-07-18 Thread David Fetter
On Wed, Jul 18, 2018 at 08:25:46AM -0400, Fabien COELHO wrote: > > Hello David, > > >I assure you that you expression yourself in English a good deal > >better than I do in Portuguese. > > Alas, despite a Portuguese "rabbit" name, I cannot speak the language which > got lost between generations.

Re: [HACKERS] logical decoding of two-phase transactions

2018-07-18 Thread Tomas Vondra
On 07/18/2018 04:56 PM, Robert Haas wrote: On Wed, Jul 18, 2018 at 10:08 AM, Tomas Vondra wrote: The problem is you don't know if a transaction does DDL sometime later, in the part that you might not have decoded yet (or perhaps concurrently with the decoding). So I don't see how you could easi

One transaction and several processes

2018-07-18 Thread Valery Kuzmin
Hi, Can multiple processes participate in a single transaction's execution? I need to do the following actions sequence: 1. Calling service begins transaction. At this step transaction manager generate new XID. 2. Calling service updates some data. 3. Need to detach a transaction from a service co

Re: One transaction and several processes

2018-07-18 Thread Tomas Vondra
On 07/18/2018 05:31 PM, Valery Kuzmin wrote: Hi, Can multiple processes participate in a single transaction's execution? I need to do the following actions sequence: 1. Calling service begins transaction. At this step transaction manager generate new XID. 2. Calling service updates some data. 3.

Re: One transaction and several processes

2018-07-18 Thread David G. Johnston
On Wed, Jul 18, 2018 at 8:31 AM, Valery Kuzmin wrote: > Hi, > > Can multiple processes participate in a single transaction's execution? > I need to do the following actions sequence: > 1. Calling service begins transaction. At this step transaction > manager generate new XID. > 2. Calling service

Re: [HACKERS] logical decoding of two-phase transactions

2018-07-18 Thread Robert Haas
On Wed, Jul 18, 2018 at 11:27 AM, Tomas Vondra wrote: >> One idea is that maybe the running transaction could communicate with >> the decoding process through shared memory. For example, suppose that >> before you begin decoding an ongoing transaction, you have to send >> some kind of notificatio

Re: Add SKIP LOCKED to VACUUM and ANALYZE

2018-07-18 Thread Bossart, Nathan
Hi Michael, Thanks for taking a look. On 7/17/18, 1:22 AM, "Michael Paquier" wrote: > The first thing which is striking me is that we may actually *not* want > to check for lock skipping within expand_vacuum_rel() as that's mainly a > function aimed at building the relations which are going to b

Re: ENOSPC FailedAssertion("!(RefCountErrors == 0)"

2018-07-18 Thread Robert Haas
On Wed, Jul 18, 2018 at 10:33 AM, Tom Lane wrote: >> So couldn't we use TopTransactionResourceOwner instead of >> AuxProcessResrouceOwner? I feel a bit uneasy that bootstrap and >> standalone-backend have *AuxProcess*ResourceOwner. > > Since the aux processes aren't running transactions, I didn't

Have an encrypted pgpass file

2018-07-18 Thread Marco van Eck
Hi, Since .pgpass files contain plain-text passwords, I searched for an alternative. In the attached patch I've added the possibility to run a command to produce the content of the pgpass file, in exactly the same format. In this way I could use gpg or any other command to decrypt a pgpass file. I

Re: ENOSPC FailedAssertion("!(RefCountErrors == 0)"

2018-07-18 Thread Tom Lane
Robert Haas writes: > On Wed, Jul 18, 2018 at 10:33 AM, Tom Lane wrote: >>> So couldn't we use TopTransactionResourceOwner instead of >>> AuxProcessResrouceOwner? I feel a bit uneasy that bootstrap and >>> standalone-backend have *AuxProcess*ResourceOwner. >> Since the aux processes aren't runni

Re: Subplan result caching

2018-07-18 Thread Robert Haas
On Mon, Jul 9, 2018 at 5:08 AM, David Rowley wrote: > From looking at the code I see that the caching is being done inside > nodeSubplan.c. I don't think this is the right approach to the > problem. The problem exists for any parameterized path, so I think a > more general approach would be much b

Re: GiST VACUUM

2018-07-18 Thread Andrey Borodin
Hi! > 18 июля 2018 г., в 16:02, Heikki Linnakangas написал(а): > > In the corresponding B-tree code, we use don't do actual recursion, but a > hand-optimized "tail recursion", to avoid stack overflow if there are a lot > of splits. I think we need to do something like tha there, too. I don't t

Re: Possible bug in logical replication.

2018-07-18 Thread Alvaro Herrera
On 2018-Jul-12, Michael Paquier wrote: > On Wed, Jul 04, 2018 at 10:50:28AM +0900, Michael Paquier wrote: > > On Tue, Jul 03, 2018 at 01:17:48AM -0400, Alvaro Herrera wrote: > > > Let me review tomorrow. > > > > Of course, please feel free. > > Alvaro, are you planning to look at that to close t

Re: Faster str to int conversion (was Table with large number of int columns, very slow COPY FROM)

2018-07-18 Thread Robert Haas
On Sat, Jul 7, 2018 at 4:01 PM, Andres Freund wrote: > FWIW, here's a rebased version of this patch. Could probably be polished > further. One might argue that we should do a bit more wide ranging > changes, to convert scanint8 and pg_atoi to be also unified. But it > might also just be worthwhile

Background worker/idle sessions and caching

2018-07-18 Thread Jeremy Finzel
I have a background worker running SQL functions, and I believe I have noticed that when I do things like change function definitions, or even add tables, the background worker does not pick up the schema changes until I restart the worker. Is this expected behavior? If I use background workers i

Re: [PG-11] Potential bug related to INCLUDE clause of CREATE INDEX

2018-07-18 Thread Tom Lane
Yugo Nagata writes: > To fix this, we agree with Tom about getting rid of "must not intersect" > restriction. > A patch is attached for this Pushed, after fixing documentation and regression tests to match. regards, tom lane

Re: patch to allow disable of WAL recycling

2018-07-18 Thread Jerry Jelinek
I've gotten a wide variety of feedback on the proposed patch. The comments range from rough approval through various discussion about alternative solutions. At this point I am unsure if this patch is rejected or if it would be accepted once I had the updated man page changes that were discussed las

Re: Postgres, fsync, and OSs (specifically linux)

2018-07-18 Thread Robert Haas
On Tue, May 29, 2018 at 4:53 AM, Craig Ringer wrote: > I've revised the fsync patch with the cleanups discussed and gone through > the close() calls. > > AFAICS either socket closes, temp file closes, or (for WAL) already PANIC on > close. It's mainly fd.c that needs amendment. Which I've done pe

Re: patch to allow disable of WAL recycling

2018-07-18 Thread Robert Haas
On Wed, Jul 18, 2018 at 3:22 PM, Jerry Jelinek wrote: > I've gotten a wide variety of feedback on the proposed patch. The comments > range from rough approval through various discussion about alternative > solutions. At this point I am unsure if this patch is rejected or if it > would be accepted

Re: patch to allow disable of WAL recycling

2018-07-18 Thread Robert Haas
On Tue, Jul 17, 2018 at 4:47 PM, Tomas Vondra wrote: > Makes sense, I guess. But I think many claims made in this thread are > mostly just assumptions at this point, based on our beliefs how CoW or > non-CoW filesystems work. The results from ZFS (showing positive impact) > are an exception, but t

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Fabien COELHO
Hello Heikki, Yep. The attached version does only the tailing stuff under -R and not all threads were stopped on errors, with comments to tell about the why. Hmm. How about we just remove this special case from doCustom(): case CSTATE_START_THROTTLE: // ... if (duration > 0 && st->t

psql's \d versus included-index-column feature

2018-07-18 Thread Tom Lane
I noticed that psql's \d command doesn't do very well with included index columns. Given the regression db's test case, CREATE INDEX tbl_include_reg_idx ON tbl_include_reg (c1, c2) INCLUDE (c3, c4); we get regression=# \d tbl_include_reg_idx Index "public.tbl_include_reg_idx" Column | Type

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 22:56, Fabien COELHO wrote: Hello Heikki, Yep. The attached version does only the tailing stuff under -R and not all threads were stopped on errors, with comments to tell about the why. Hmm. How about we just remove this special case from doCustom(): case CSTATE_START_THROTTL

Re: Background worker/idle sessions and caching

2018-07-18 Thread Tom Lane
Jeremy Finzel writes: > I have a background worker running SQL functions, and I believe I have > noticed that when I do things like change function definitions, or even add > tables, the background worker does not pick up the schema changes until I > restart the worker. Maybe you need some Accept

Re: psql's \d versus included-index-column feature

2018-07-18 Thread David G. Johnston
On Wed, Jul 18, 2018 at 12:55 PM, Tom Lane wrote: > > regression=# \d tbl_include_reg_idx > Index "public.tbl_include_reg_idx" > Column | Type | Key | Definition > +-+-- > c1 | integer | t | c1 > c2 | integer | t | c2 > c3 | integer | f | c

Re: Make foo=null a warning by default.

2018-07-18 Thread Fabien COELHO
[...] Done. All looks well, but I just noticed a warning: gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -O2 -I. -I. -I

Re: psql's \d versus included-index-column feature

2018-07-18 Thread Alvaro Herrera
On 2018-Jul-18, David G. Johnston wrote: > ​-1 for printing a boolean t/f; would rather spell it out: > > CASE WHEN "Key" THEN 'Key' ELSE 'Included' END AS "Data" +1 -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Servi

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Fabien COELHO
Hmm. How about we just remove this special case from doCustom(): case CSTATE_START_THROTTLE: // ... if (duration > 0 && st->txn_scheduled > end_time) { st->state = CSTATE_FINISHED; break; } That way, we let the client go into CSTATE_THROTTLE state, even tho

Re: Background worker/idle sessions and caching

2018-07-18 Thread Jeremy Finzel
On Wed, Jul 18, 2018 at 3:05 PM, Tom Lane wrote: > Jeremy Finzel writes: > > I have a background worker running SQL functions, and I believe I have > > noticed that when I do things like change function definitions, or even > add > > tables, the background worker does not pick up the schema chan

Re: psql's \d versus included-index-column feature

2018-07-18 Thread Tom Lane
Alvaro Herrera writes: > On 2018-Jul-18, David G. Johnston wrote: >> -1 for printing a boolean t/f; would rather spell it out: >> >> CASE WHEN "Key" THEN 'Key' ELSE 'Included' END AS "Data" > +1 I can sympathize with the eyestrain argument against t/f, but the above doesn't seem like an improve

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 23:29, Fabien COELHO wrote: Hmm. How about we just remove this special case from doCustom(): case CSTATE_START_THROTTLE: // ... if (duration > 0 && st->txn_scheduled > end_time) { st->state = CSTATE_FINISHED; break; } That way, we let the cli

Re: psql's \d versus included-index-column feature

2018-07-18 Thread Alvaro Herrera
On 2018-Jul-18, Tom Lane wrote: > I can sympathize with the eyestrain argument against t/f, but the > above doesn't seem like an improvement --- in particular, "Data" > as the column header seems quite content-free. My counterproposal > is to keep "Key" as the header and use "Yes"/"No" as the val

Re: GiST VACUUM

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 21:27, Andrey Borodin wrote: Hi! 18 июля 2018 г., в 16:02, Heikki Linnakangas написал(а): , but I think it would be better to split this into two patches as follows: 1st patch: Scan the index in physical rather than logical order. No attempt at deleting empty pages yet. 2nd patc

RE: GSOC 2018 Project - A New Sorting Routine

2018-07-18 Thread Kefan Yang
Hey Tomas! I am trying to reproduce the results on my machine. Could you please share the script to generate .ods files? Regards, Kefan From: Tomas Vondra Sent: July 18, 2018 2:05 AM To: Andrey Borodin Cc: Peter Geoghegan; Kefan Yang; PostgreSQL Hackers Subject: Re: GSOC 2018 Project - A New So

Re: GSOC 2018 Project - A New Sorting Routine

2018-07-18 Thread Tomas Vondra
I don't have any script for that - load the files into a spreadsheet, create pivot tables and you're done. regards On 07/18/2018 11:13 PM, Kefan Yang wrote: > Hey Tomas! > >   > > I am trying to reproduce the results on my machine. Could you please > share the script to generate .ods files? >

Re: Have an encrypted pgpass file

2018-07-18 Thread Thomas Munro
On Thu, Jul 19, 2018 at 5:46 AM, Marco van Eck wrote: > Since .pgpass files contain plain-text passwords, I searched for an > alternative. > In the attached patch I've added the possibility to run a command to produce > the content of the pgpass file, in exactly the same format. In this way I > co

Re: Have an encrypted pgpass file

2018-07-18 Thread Christophe Pettus
> On Jul 18, 2018, at 14:33, Thomas Munro wrote: > Here you side step those questions completely and make that the end > user's problem. I like it. +1. This is a clever solution, since any kind of key vault or other system could be dropped in there. -- -- Christophe Pettus x...@thebuild

Re: YA race condition in 001_stream_rep.pl (was Re: pgsql: Allow using the updated tuple while moving it to a different par)

2018-07-18 Thread Tom Lane
Amit Kapila writes: > On Sat, Jul 14, 2018 at 11:46 PM, Tom Lane wrote: >> So the issue boils down to this: the test script is, effectively, >> assuming that it's guaranteed that the walreceiver will send a feedback >> message before it shuts down; but there is no such guarantee. Is this >> a bu

Re: Have an encrypted pgpass file

2018-07-18 Thread Tom Lane
Thomas Munro writes: > On Thu, Jul 19, 2018 at 5:46 AM, Marco van Eck wrote: >> Since .pgpass files contain plain-text passwords, I searched for an >> alternative. >> In the attached patch I've added the possibility to run a command to produce >> the content of the pgpass file, in exactly the sam

Re: GSOC 2018 Project - A New Sorting Routine

2018-07-18 Thread Alvaro Herrera
On 2018-Jul-18, Tomas Vondra wrote: > I don't have any script for that - load the files into a spreadsheet, > create pivot tables and you're done. What!? You don't use psql's \crosstabview !? ... walks away disappointed ... -- Álvaro Herrerahttps://www.2ndQuadrant.com/ Postgre

Re: Have an encrypted pgpass file

2018-07-18 Thread Thomas Munro
On Thu, Jul 19, 2018 at 9:52 AM, Tom Lane wrote: > Thomas Munro writes: >> On Thu, Jul 19, 2018 at 5:46 AM, Marco van Eck >> wrote: >>> Since .pgpass files contain plain-text passwords, I searched for an >>> alternative. >>> In the attached patch I've added the possibility to run a command to p

Re: psql's \d versus included-index-column feature

2018-07-18 Thread Alexander Korotkov
On Wed, Jul 18, 2018 at 11:14 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Wed, Jul 18, 2018 at 12:55 PM, Tom Lane wrote: > >> >> regression=# \d tbl_include_reg_idx >> Index "public.tbl_include_reg_idx" >> Column | Type | Key | Definition >> +-+-

Re: Have an encrypted pgpass file

2018-07-18 Thread Alvaro Herrera
On 2018-Jul-18, Marco van Eck wrote: > Since .pgpass files contain plain-text passwords, I searched for an > alternative. > In the attached patch I've added the possibility to run a command to > produce the content of the pgpass file, in exactly the same format. In this > way I could use gpg or an

Re: Postgres, fsync, and OSs (specifically linux)

2018-07-18 Thread Thomas Munro
On Thu, Jul 19, 2018 at 7:23 AM, Robert Haas wrote: > 2. I don't like promote_ioerr_to_panic() very much, partly because the > same pattern gets repeated over and over, and partly because it would > be awkwardly-named if we discovered that another 2 or 3 errors needed > similar handling (or some o

Re: Have an encrypted pgpass file

2018-07-18 Thread Tom Lane
Alvaro Herrera writes: > Seems to me that passing %-specifiers to the command would make it more > useful (%u for "user", "host" etc) -- your command could refuse to give > you a password for the superuser account for instance but grant one for > a read-only user. It would also provide a *very* f

Re: print_path is missing GatherMerge and CustomScan support

2018-07-18 Thread Michael Paquier
On Wed, Jul 18, 2018 at 12:15:25PM +0530, Ashutosh Bapat wrote: > Yes that's right. Thanks for taking care of it. Okay, I have pushed a fix for this one as that's wrong and back-patched to v11. The coverage of reparameterize_path_by_child is actually quite poor if you look at the reports: https:/

Re: untrusted PLs should be GRANTable

2018-07-18 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 17.07.18 07:20, Craig Ringer wrote: > > A user has raised the point that our refusal to GRANT rights to > > untrusted PLs is counterproductive and inconsistent with how we behave > > elsewhere. > > Previous discussion: >

Re: Have an encrypted pgpass file

2018-07-18 Thread Joshua D. Drake
On 07/18/2018 04:25 PM, Tom Lane wrote: Alvaro Herrera writes: Seems to me that passing %-specifiers to the command would make it more useful (%u for "user", "host" etc) -- your command could refuse to give you a password for the superuser account for instance but grant one for a read-only user

Re: untrusted PLs should be GRANTable

2018-07-18 Thread Stephen Frost
Greetings, * Craig Ringer (cr...@2ndquadrant.com) wrote: > Untrusted PLs should be GRANTable with a NOTICE or WARNING telling the > admin that GRANTing an untrusted PL effectively gives the user the ability > to escape to superuser. I don't know that we really want to get into the business of iss

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Fabien COELHO
Hello Heikki, [...] So threadRun() would not have the opportunity to stop the scheduled transaction, even if beyond the end of run, because it would not have got out of doCustom, in the case I outlined above. I see. Instead of moving to FINISHED state, then, we could stay in THROTTLE state,

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-18 Thread Asim R P
On Thu, Jun 22, 2017 at 10:50 AM, Andres Freund wrote: > > Or, probably more robust: Simply _exit(2) without further ado, and rely > on postmaster to output an appropriate error message. Arguably it's not > actually useful to see hundreds of "WARNING: terminating connection because of > crash of a

RE: Recovery performance of standby for multiple concurrent truncates on large tables

2018-07-18 Thread Jamison, Kirk
Hi, Thank you for your replies. On Tue, July 10, 2018 4:15 PM, Andres Freund wrote: >I think you'd run into a lot of very hairy details with this approach. >Consider what happens if client processes need fresh buffers and need to write >out a victim buffer. You'll need to know that the relevant

Re: print_path is missing GatherMerge and CustomScan support

2018-07-18 Thread Michael Paquier
On Wed, Jul 18, 2018 at 03:22:02PM +0900, Michael Paquier wrote: > Good catch. Those should be backpatched. While I am looking at this > stuff, I have noticed that pathnode.c/reparameterize_path_by_child uses > T_MergeAppend and not T_MergeAppendPath. Okay, I have checked the full list of path n

Re: YA race condition in 001_stream_rep.pl (was Re: pgsql: Allow using the updated tuple while moving it to a different par)

2018-07-18 Thread Michael Paquier
On Wed, Jul 18, 2018 at 05:42:40PM -0400, Tom Lane wrote: > Hearing nobody speaking in favor of the other alternatives, > I've removed the test. I had this problem running in the background for a couple of days, but I could not come up with a solution cleaner than just removing the test. Thanks fo

Re: [PATCH] pg_hba.conf : new auth option : clientcert=verify-full

2018-07-18 Thread Thomas Munro
On Sun, Jul 15, 2018 at 12:47 AM, Julian Markwort wrote: > Also, while writing this part of the docs, I tried to stay below 80 > characters, but I've exceeded it in some places. > There are several other places (several in the .sgml files touched by this > patch), where 80 characters are exceede

Re: Possible bug in logical replication.

2018-07-18 Thread Michael Paquier
On Wed, Jul 18, 2018 at 02:30:53PM -0400, Alvaro Herrera wrote: > In the immortal words of Julian Bream: "yeah, I didn't like any of > that". One wikipedia lookup later, I still don't know where this quote comes from, but at least I understand who the man is. I may be missing something, but I can

Re: Add SKIP LOCKED to VACUUM and ANALYZE

2018-07-18 Thread Michael Paquier
On Wed, Jul 18, 2018 at 04:58:48PM +, Bossart, Nathan wrote: > On 7/17/18, 1:22 AM, "Michael Paquier" wrote: > Perhaps we could extend RangeVarGetRelidExtended() to only lock if > has_subclass() is true. However, I also understand Robert's position > on calling RangeVarGetRelidExtended() with

Re: Possible bug in logical replication.

2018-07-18 Thread Alvaro Herrera
On 2018-Jul-19, Michael Paquier wrote: > On Wed, Jul 18, 2018 at 02:30:53PM -0400, Alvaro Herrera wrote: > > In the immortal words of Julian Bream: "yeah, I didn't like any of > > that". > > One wikipedia lookup later, I still don't know where this quote comes > from, but at least I understand wh

  1   2   >