Re: csv format for psql

2018-03-07 Thread David Fetter
On Wed, Mar 07, 2018 at 09:37:26PM +0100, Pavel Stehule wrote: > 2018-03-07 21:31 GMT+01:00 Daniel Verite : > > > David Fetter wrote: > > > > > We have some inconsistency here in that fewer table formats are > > > supported, but I think asciidoc, etc., do this correctly via > > > invocatio

Re: [HACKERS] SERIALIZABLE with parallel query

2018-03-07 Thread Robert Haas
On Wed, Feb 28, 2018 at 11:35 PM, Thomas Munro wrote: > On Mon, Feb 26, 2018 at 6:37 PM, Thomas Munro > wrote: >> I've now broken it into two patches. > > Rebased. +SerializableXactHandle +ShareSerializableXact(void) +{ +Assert(!IsParallelWorker()); + +return MySerializableXact; +} Uh,

Re: public schema default ACL

2018-03-07 Thread Peter Eisentraut
On 3/7/18 10:05, Stephen Frost wrote: > I liken this to a well-known and well-trodden feature for auto creating > user home directories on Unix. I don't think likening schemas to home directories is really addressing the most typical use cases. Database contents are for the most part carefully co

Re: faster testing with symlink installs

2018-03-07 Thread Robert Haas
On Wed, Feb 28, 2018 at 9:34 PM, Peter Eisentraut wrote: > Except ... this doesn't actually work. find_my_exec() resolves symlinks > to find the actual program installation, and so for example the > installed initdb will look for postgres in src/bin/initdb/. I would > like to revert this behavio

Re: public schema default ACL

2018-03-07 Thread David G. Johnston
On Wed, Mar 7, 2018 at 2:48 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 3/7/18 10:05, Stephen Frost wrote: > > I liken this to a well-known and well-trodden feature for auto creating > > user home directories on Unix. > > I don't think likening schemas to home directories

Re: unused includes in test_decoding

2018-03-07 Thread Robert Haas
On Wed, Feb 28, 2018 at 6:59 PM, Euler Taveira wrote: > This is a cosmetic patch that removes some unused includes in > test_decoding. It seems to be like this since day 1 (9.4). Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Two-phase update of restart_lsn in LogicalConfirmReceivedLocation

2018-03-07 Thread Robert Haas
On Thu, Mar 1, 2018 at 2:03 AM, Craig Ringer wrote: > So I can't say it's definitely impossible. It seems astonishingly unlikely, > but that's not always good enough. Race conditions tend to happen a lot more often than one might think. If there's a theoretical opportunity for this to go wrong, i

Re: faster testing with symlink installs

2018-03-07 Thread Tom Lane
Robert Haas writes: > On Wed, Feb 28, 2018 at 9:34 PM, Peter Eisentraut > wrote: >> Except ... this doesn't actually work. find_my_exec() resolves symlinks >> to find the actual program installation, and so for example the >> installed initdb will look for postgres in src/bin/initdb/. I would >

Re: Two-phase update of restart_lsn in LogicalConfirmReceivedLocation

2018-03-07 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 1, 2018 at 2:03 AM, Craig Ringer wrote: >> So I can't say it's definitely impossible. It seems astonishingly unlikely, >> but that's not always good enough. > Race conditions tend to happen a lot more often than one might think. Just to back that up --- we've s

Re: Server won't start with fallback setting by initdb.

2018-03-07 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 6, 2018 at 10:51 PM, Stephen Frost wrote: >> Changing the defaults to go back down strikes me as an entirely wrong >> approach after we've had a release with the higher defaults without >> seriously compelling arguments against, and I don't agree that we've had >

Re: Server won't start with fallback setting by initdb.

2018-03-07 Thread Michael Paquier
On Wed, Mar 07, 2018 at 06:39:32PM -0500, Tom Lane wrote: > OK, seems like I'm on the short end of that vote. I propose to push the > GUC-crosschecking patch I posted yesterday, but not the default-value > change, and instead push Kyotaro-san's initdb change. Should we back-patch > these things t

RE: Protect syscache from bloating with negative cache entries

2018-03-07 Thread Tsunakawa, Takayuki
From: Alvaro Herrera [mailto:alvhe...@alvh.no-ip.org] > The thing that comes to mind when reading this patch is that some time ago > we made fun of other database software, "they are so complicated to configure, > they have some magical settings that few people understand how to set". > Postgres wa

Re: [HACKERS] Lazy hash table for XidInMVCCSnapshot (helps Zipfian a bit)

2018-03-07 Thread Tomas Vondra
On 03/06/2018 06:23 AM, Yura Sokolov wrote: > 05.03.2018 18:00, Tom Lane пишет: >> Tomas Vondra writes: >>> Snapshots are static (we don't really add new XIDs into existing ones, >>> right?), so why don't we simply sort the XIDs and then use bsearch to >>> lookup values? That should fix the linear

Re: [HACKERS] Subscription code improvements

2018-03-07 Thread Masahiko Sawada
On Wed, Mar 7, 2018 at 11:13 PM, Alvaro Herrera wrote: > 0001: > > there are a bunch of other messages of the same ilk in the file. I > don't like how the current messages are worded; maybe Peter or Petr had > some reason why they're like that, but I would have split out the reason > for not star

Re: INOUT parameters in procedures

2018-03-07 Thread Peter Eisentraut
On 3/6/18 04:22, Pavel Stehule wrote: > why just OUT variables are disallowed? > > The oracle initializes these values to NULL - we can do same? The problem is function call resolution. If we see a call like CALL foo(a, b, c); the this could be foo() with zero input and three output parameters

Re: Add default role 'pg_access_server_files'

2018-03-07 Thread Michael Paquier
On Wed, Mar 07, 2018 at 07:00:03AM -0500, Stephen Frost wrote: > * Michael Paquier (mich...@paquier.xyz) wrote: >> First, could it be possible to do a split for 1) and 2)? > > Done, because it was less work than arguing about it, but I'm not > convinced that we really need to split out patches to

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-03-07 Thread Tomas Vondra
On 03/07/2018 03:21 PM, Robert Haas wrote: > On Wed, Mar 7, 2018 at 8:59 AM, Prabhat Sahu > mailto:prabhat.s...@enterprisedb.com>> > wrote: > > 2018-03-07 19:24:44.263 IST [54400] LOG:  background worker > "parallel worker" (PID 54482) was terminated by signal 9: Killed > > > That looks

Re: Add default role 'pg_access_server_files'

2018-03-07 Thread Michael Paquier
On Thu, Mar 08, 2018 at 10:15:11AM +0900, Michael Paquier wrote: > Other than that the patch looks in pretty good shape to me. The regression tests of file_fdw are blowing up because of an error string patch 2 changes. -- Michael signature.asc Description: PGP signature

archive_command

2018-03-07 Thread Rui DeSousa
Hi, I’ve been encouraged to submit this code as there has been talk in the past about a simple pgcopy command to use with the archive_command. Currently there is really no good solution in most base systems without having to introduce a dedicated third party Postgres solution. The best base

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-03-07 Thread Peter Geoghegan
On Wed, Mar 7, 2018 at 5:16 PM, Tomas Vondra wrote: > FWIW that's usually written to the system log. Does dmesg say something > about the kill? While it would be nice to confirm that it was indeed the OOM killer, either way the crash happened because SIGKILL was sent to a parallel worker. There i

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-03-07 Thread Andres Freund
On March 7, 2018 5:40:18 PM PST, Peter Geoghegan wrote: >On Wed, Mar 7, 2018 at 5:16 PM, Tomas Vondra > wrote: >> FWIW that's usually written to the system log. Does dmesg say >something >> about the kill? > >While it would be nice to confirm that it was indeed the OOM killer, >either way the cr

Re: Two-phase update of restart_lsn in LogicalConfirmReceivedLocation

2018-03-07 Thread Craig Ringer
On 8 March 2018 at 07:32, Tom Lane wrote: > Robert Haas writes: > > On Thu, Mar 1, 2018 at 2:03 AM, Craig Ringer > wrote: > >> So I can't say it's definitely impossible. It seems astonishingly > unlikely, > >> but that's not always good enough. > > > Race conditions tend to happen a lot more of

Re: PATCH: Configurable file mode mask

2018-03-07 Thread Michael Paquier
On Wed, Mar 07, 2018 at 10:56:32AM -0500, David Steele wrote: > On 3/6/18 10:04 PM, Michael Paquier wrote: >> Seems like you forgot to re-add the chmod calls in initdb.c. > > Hmmm, I thought we were talking about moving the position of umask(). > > I don't think the chmod() calls are needed becau

Re: RFC: Add 'taint' field to pg_control.

2018-03-07 Thread Craig Ringer
On 8 March 2018 at 04:58, Robert Haas wrote: > On Wed, Feb 28, 2018 at 8:03 PM, Craig Ringer > wrote: > > A huge +1 from me for the idea. I can't even count the number of black > box > > "WTF did you DO?!?" servers I've looked at, where bizarre behaviour has > > turned out to be down to the user

Re: pgstat_report_activity() and parallel CREATE INDEX (was: Parallel index creation & pg_stat_activity)

2018-03-07 Thread Peter Geoghegan
On Thu, Mar 1, 2018 at 2:47 PM, Peter Geoghegan wrote: > No. Just an oversight. Looks like _bt_parallel_build_main() should > call pgstat_report_activity(), just like ParallelQueryMain(). > > I'll come up with a patch soon. Attached patch has parallel CREATE INDEX propagate debug_query_string to

Re: FOR EACH ROW triggers on partitioned tables

2018-03-07 Thread Alvaro Herrera
Alvaro Herrera wrote: > I reserve the right to revise this further, as I'm going to spend a > couple of hours looking at it this afternoon, particularly to see how > concurrent DDL behaves, but I don't see anything obviously wrong with > it. I do now. TLDR; I'm afraid this cute idea crashed and

Re: RFC: Add 'taint' field to pg_control.

2018-03-07 Thread Andres Freund
On March 7, 2018 5:51:29 PM PST, Craig Ringer wrote: >My favourite remains an organisation that kept "fixing" an issue by >kill >-9'ing the postmaster and removing postmaster.pid to make it start up >again. Without killing all the leftover backends. Of course, the system >kept getting more unsta

Re: Some message fixes

2018-03-07 Thread Kyotaro HORIGUCHI
At Wed, 7 Mar 2018 07:10:34 -0300, Alvaro Herrera wrote in <20180307101034.l7z7kqwqfkjg6c2p@alvherre.pgsql> > Kyotaro HORIGUCHI wrote: > > > 1. some messages are missing partitioned table/index .. > I *think* the idea here is that a partitioned table is a table, so there > is no need to say "foo

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-07 Thread David Gould
On Tue, 06 Mar 2018 11:16:04 -0500 Tom Lane wrote: > so that we can decide whether this bug is bad enough to justify > back-patching a behavioral change. I remain concerned that the proposed > fix is too simplistic and will have some unforeseen side-effects, so > I'd really rather just put it in

Re: Server won't start with fallback setting by initdb.

2018-03-07 Thread Robert Haas
On Wed, Mar 7, 2018 at 6:43 PM, Michael Paquier wrote: > On Wed, Mar 07, 2018 at 06:39:32PM -0500, Tom Lane wrote: >> OK, seems like I'm on the short end of that vote. I propose to push the >> GUC-crosschecking patch I posted yesterday, but not the default-value >> change, and instead push Kyotar

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-07 Thread Robert Haas
On Fri, Mar 2, 2018 at 5:17 PM, Tom Lane wrote: > (1) do we really want to go over to treating ANALYZE's tuple density > result as gospel, contradicting the entire thrust of the 2011 discussion? > >> This tables reltuples is 18 times the actual row count. It will never >> converge >> because with

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-07 Thread Pavan Deolasee
On Wed, Feb 28, 2018 at 12:38 PM, Rajkumar Raghuwanshi < rajkumar.raghuwan...@enterprisedb.com> wrote: > On Wed, Feb 14, 2018 at 5:44 PM, Amit Kapila > wrote: > >> +# Concurrency error from GetTupleForTrigger >> +# Concurrency error from ExecLockRows >> >> I think you don't need to mention above

Re: Protect syscache from bloating with negative cache entries

2018-03-07 Thread Kyotaro HORIGUCHI
Hello, At Thu, 8 Mar 2018 00:28:04 +, "Tsunakawa, Takayuki" wrote in <0A3221C70F24FB45833433255569204D1F8FF0D9@G01JPEXMBYT05> > From: Alvaro Herrera [mailto:alvhe...@alvh.no-ip.org] > > The thing that comes to mind when reading this patch is that some time ago > > we made fun of other datab

Re: RFC: Add 'taint' field to pg_control.

2018-03-07 Thread Craig Ringer
On 8 March 2018 at 10:18, Andres Freund wrote: > > > On March 7, 2018 5:51:29 PM PST, Craig Ringer > wrote: > >My favourite remains an organisation that kept "fixing" an issue by > >kill > >-9'ing the postmaster and removing postmaster.pid to make it start up > >again. Without killing all the le

Re: Protect syscache from bloating with negative cache entries

2018-03-07 Thread Tom Lane
Kyotaro HORIGUCHI writes: > At Thu, 8 Mar 2018 00:28:04 +, "Tsunakawa, Takayuki" > wrote in > <0A3221C70F24FB45833433255569204D1F8FF0D9@G01JPEXMBYT05> >> Yes. We are now facing the problem of too much memory use by PostgreSQL, >> where about some applications randomly access about 200,000

Re: PATCH: psql tab completion for SELECT

2018-03-07 Thread Edmund Horner
New patch that fixes a little bug with appending NULL addons to schema queries. psql-select-tab-completion-v6.patch Description: Binary data

Re: RFC: Add 'taint' field to pg_control.

2018-03-07 Thread Tom Lane
Craig Ringer writes: > As I understand it, because we allow multiple Pg instances on a system, we > identify the small sysv shmem segment we use by the postmaster's pid. If > you remove the DirLockFile (postmaster.pid) you remove the interlock > against starting a new postmaster. It'll think it's

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2018-03-07 Thread Ashutosh Bapat
On Wed, Mar 7, 2018 at 8:55 AM, Stephen Frost wrote: > Greetings Robert, Ashutosh, Arthur, Etsuro, all, > > * Arthur Zakirov (a.zaki...@postgrespro.ru) wrote: >> On Tue, Mar 06, 2018 at 08:09:50PM +0900, Etsuro Fujita wrote: >> > Agreed. I added a comment to that function. I think that that comm

Testbed for predtest.c ... and some arguable bugs therein

2018-03-07 Thread Tom Lane
In the thread about being able to prove constraint exclusion from an IN clause mentioning a NULL, https://www.postgresql.org/message-id/flat/3bad48fc-f257-c445-feeb-8a2b2fb62...@lab.ntt.co.jp I expressed concern about whether there were existing bugs in predtest.c given the lack of clarity of the c

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-07 Thread Pavan Deolasee
On Tue, Feb 13, 2018 at 12:41 PM, amul sul wrote: > > Thanks for the confirmation, updated patch attached. > > I am actually very surprised that 0001-Invalidate-ip_blkid-v5.patch does not do anything to deal with the fact that t_ctid may no longer point to itself to mark end of the chain. I just

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-07 Thread Jeff Janes
On Sun, Mar 4, 2018 at 3:18 PM, David Gould wrote: > On Sun, 4 Mar 2018 07:49:46 -0800 > Jeff Janes wrote: > > > On Wed, Jan 17, 2018 at 4:49 PM, David Gould wrote: > ... > > > > Maybe a well-timed crash caused n_dead_tup to get reset to zero and that > is > > why autovac is not kicking in? Wh

Re: ALTER TABLE ADD COLUMN fast default

2018-03-07 Thread Andrew Dunstan
On Tue, Mar 6, 2018 at 8:15 PM, David Rowley wrote: > On 6 March 2018 at 22:40, David Rowley wrote: >> Okay, it looks like the patch should disable physical tlists when >> there's a missing column the same way as we do for dropped columns. >> Patch attached. > > Please disregard the previous patc

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-07 Thread Amit Khandekar
On 8 March 2018 at 09:15, Pavan Deolasee wrote: > For example, with your patches applied: > > CREATE TABLE pa_target (key integer, val text) > PARTITION BY LIST (key); > CREATE TABLE part1 PARTITION OF pa_target FOR VALUES IN (1); > CREATE TABLE part2 PARTITION OF pa_target FOR VALUES IN (2);

Re: RFC: Add 'taint' field to pg_control.

2018-03-07 Thread Craig Ringer
On 8 March 2018 at 12:34, Tom Lane wrote: > Craig Ringer writes: > > As I understand it, because we allow multiple Pg instances on a system, > we > > identify the small sysv shmem segment we use by the postmaster's pid. If > > you remove the DirLockFile (postmaster.pid) you remove the interlock

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-07 Thread Amit Kapila
On Thu, Mar 8, 2018 at 11:04 AM, Pavan Deolasee wrote: > > On Tue, Feb 13, 2018 at 12:41 PM, amul sul wrote: >> >> Thanks for the confirmation, updated patch attached. >> > > I am actually very surprised that 0001-Invalidate-ip_blkid-v5.patch does not > do anything to deal with the fact that t_ct

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-07 Thread Amit Kapila
On Thu, Mar 8, 2018 at 11:57 AM, Amit Khandekar wrote: > On 8 March 2018 at 09:15, Pavan Deolasee wrote: >> For example, with your patches applied: >> >> CREATE TABLE pa_target (key integer, val text) >> PARTITION BY LIST (key); >> CREATE TABLE part1 PARTITION OF pa_target FOR VALUES IN (1);

Re: PATCH: psql tab completion for SELECT

2018-03-07 Thread Edmund Horner
Some updates on patch status: - "version-dependent-tab-completion-1.patch" by Tom Lane was committed in 722408bcd. - "psql-tab-completion-savepoint-v1.patch" by Edmund Horner is probably not needed. - "psql-select-tab-completion-v6.patch" (the latest) is still under development/review.

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-07 Thread Amit Khandekar
On 8 March 2018 at 12:34, Amit Kapila wrote: > On Thu, Mar 8, 2018 at 11:57 AM, Amit Khandekar > wrote: >> On 8 March 2018 at 09:15, Pavan Deolasee wrote: >>> For example, with your patches applied: >>> >>> CREATE TABLE pa_target (key integer, val text) >>> PARTITION BY LIST (key); >>> CREA

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-03-07 Thread Pavan Deolasee
On Thu, Mar 8, 2018 at 12:31 PM, Amit Kapila wrote: > On Thu, Mar 8, 2018 at 11:04 AM, Pavan Deolasee > wrote: > > > > On Tue, Feb 13, 2018 at 12:41 PM, amul sul wrote: > >> > >> Thanks for the confirmation, updated patch attached. > >> > > > > I am actually very surprised that 0001-Invalidate-

Re: INOUT parameters in procedures

2018-03-07 Thread Pavel Stehule
Hi 2018-03-08 1:53 GMT+01:00 Peter Eisentraut : > On 3/6/18 04:22, Pavel Stehule wrote: > > why just OUT variables are disallowed? > > > > The oracle initializes these values to NULL - we can do same? > > The problem is function call resolution. If we see a call like > > CALL foo(a, b, c); > > t

Re: csv format for psql

2018-03-07 Thread Pavel Stehule
2018-03-07 22:16 GMT+01:00 David Fetter : > On Wed, Mar 07, 2018 at 09:37:26PM +0100, Pavel Stehule wrote: > > 2018-03-07 21:31 GMT+01:00 Daniel Verite : > > > > > David Fetter wrote: > > > > > > > We have some inconsistency here in that fewer table formats are > > > > supported, but I thi

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-07 Thread Ashutosh Bapat
On Wed, Mar 7, 2018 at 8:07 PM, Jeevan Chalke wrote: Here are some more review comments esp. on try_partitionwise_grouping() function. BTW name of that function doesn't go in sync with enable_partitionwise_aggregation (which btw is in sync with enable_fooagg GUCs). But it goes in sync with create_

Re: parallel append vs. simple UNION ALL

2018-03-07 Thread Rajkumar Raghuwanshi
On Thu, Mar 8, 2018 at 12:27 AM, Robert Haas wrote: > New patches attached, fixing all 3 of the issues you reported: > Thanks. new patches applied cleanly on head and fixing all reported issue. Thanks & Regards, Rajkumar Raghuwanshi QMG, EnterpriseDB Corporation

<    1   2