Re: Fw: Documentation fix for adding a column with a default value

2019-07-16 Thread Ian Barwick
On Wed, 17 Jul 2019 at 15:42, Daniel Westermann (DWE) < daniel.westerm...@dbi-services.com> wrote: > >__ > >From: Daniel Westermann (DWE) > >Sent: Monday, July 15, 2019 13:01 > >To: pgsql-hack...@postgresql.org > >Subject: Documentation fix for adding a column w

Fw: Documentation fix for adding a column with a default value

2019-07-16 Thread Daniel Westermann (DWE)
>__ >From: Daniel Westermann (DWE) >Sent: Monday, July 15, 2019 13:01 >To: pgsql-hack...@postgresql.org >Subject: Documentation fix for adding a column with a default value > >Hi, > >the tip in the "Adding a column" section is not true anymore since PostgreSQL >

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Amit Kapila
On Tue, Jul 16, 2019 at 9:52 PM Robert Haas wrote: > > On Tue, Jul 16, 2019 at 7:13 AM Amit Kapila wrote: > > > I also strongly suspect it is altogether wrong to do > > > this before CommitSubTransaction sets s->state to TRANS_COMMIT; what > > > if a subxact callback throws an error? > > > > Are

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Amit Kapila
On Wed, Jul 17, 2019 at 3:53 AM Andres Freund wrote: > On 2019-07-15 12:26:21 -0400, Robert Haas wrote: Responding again with some more details. > > > > But, my understanding of the current design being implemented is that > > there is a hard limit on the number of transactions that can be > > p

Re: make \d pg_toast.foo show its indices ; and, \d toast show its main table ; and \d relkind=I show its partitions

2019-07-16 Thread Justin Pryzby
Find attached updated patches which also work against old servers. 1) avoid ::regnamespace; 2) don't PQgetvalue() fields which don't exist and then crash. >From 16b31dc1e4142ed6d0f5f7ed6d65c6184f546a3c Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 30 Apr 2019 19:05:53 -0500 Subject: [P

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Amit Kapila
On Tue, Jul 16, 2019 at 9:44 PM Robert Haas wrote: > > On Tue, Jul 16, 2019 at 12:32 AM Amit Kapila wrote: > > The idea is that the queues can get full, but not rollback hash table. > > In the case where the error queue gets full, we mark the entry as > > Invalid in the hash table and later when

Re: [PATCH] Implement INSERT SET syntax

2019-07-16 Thread Marko Tiikkaja
On Wed, Jul 17, 2019 at 7:30 AM Gareth Palmer wrote: > Attached is a patch that adds the option of using SET clause to specify > the columns and values in an INSERT statement in the same manner as that > of an UPDATE statement. > Cool! Thanks for working on this, I'd love to see the syntax in P

Re: pg_receivewal documentation

2019-07-16 Thread Laurenz Albe
On Wed, 2019-07-17 at 10:38 +0900, Michael Paquier wrote: > + > +Note that while WAL will be flushed with this setting, > +pg_receivewal never applies it, so > + must not be set to > +remote_apply if > pg_receivewal > +is the only synchronous standby.

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Dilip Kumar
On Wed, Jul 17, 2019 at 9:27 AM Thomas Munro wrote: > > On Wed, Jul 17, 2019 at 3:44 PM Dilip Kumar wrote: > > Right, actually I got that point. But, I was thinking that we are > > wasting one logno from undo log addressing space no?. Instead, if we > > can keep it attached to the slot and some

Re: psql ctrl+f skips displaying of one record and displays skipping one line

2019-07-16 Thread vignesh C
Thanks Tom. That sounds good to me. On Wed, Jul 17, 2019 at 10:17 AM Tom Lane wrote: > vignesh C writes: > > I'm able to get the same behaviour in centos as well. > > Should we do anything to handle this in Postgres or any documentation > > required? > > It already is documented: > > PSQL_P

Re: buildfarm's typedefs list has gone completely nutso

2019-07-16 Thread Andres Freund
Hi, On 2019-07-16 19:35:39 -0700, Andres Freund wrote: > Hi, > > On 2019-07-10 17:24:41 -0700, Andres Freund wrote: > > Not yet sure what's actually going on, but there's something odd with > > debug information afaict: > > > > objdump -W spits out warnings for a few files, all static libraries:

Re: block-level incremental backup

2019-07-16 Thread Jeevan Chalke
On Thu, Jul 11, 2019 at 5:00 PM Jeevan Chalke < jeevan.cha...@enterprisedb.com> wrote: > Hi Anastasia, > > On Wed, Jul 10, 2019 at 11:47 PM Anastasia Lubennikova < > a.lubennik...@postgrespro.ru> wrote: > >> 23.04.2019 14:08, Anastasia Lubennikova wrote: >> > I'm volunteering to write a draft patc

Re: Minimal logical decoding on standbys

2019-07-16 Thread Amit Khandekar
On Tue, 16 Jul 2019 at 22:56, Andres Freund wrote: > > Hi, > > On 2019-07-12 14:53:21 +0530, tushar wrote: > > On 07/10/2019 05:12 PM, Amit Khandekar wrote: > > > All right. Will do that in the next patch set. For now, I have quickly > > > done the below changes in a single patch again (attached),

Re: psql ctrl+f skips displaying of one record and displays skipping one line

2019-07-16 Thread Tom Lane
vignesh C writes: > I'm able to get the same behaviour in centos as well. > Should we do anything to handle this in Postgres or any documentation > required? It already is documented: PSQL_PAGER PAGER If a query's results do not fit on the screen, they are piped through

Re: A little report on informal commit tag usage

2019-07-16 Thread Michael Paquier
On Tue, Jul 16, 2019 at 04:33:07PM -0700, Andres Freund wrote: > On 2019-07-16 19:26:59 -0400, Tom Lane wrote: >> I've wondered for some time what you think the "-" means in this. > > Up to master. Occasionally there's bugs that only need to be fixed in > some back branches etc. Is "-" most commo

Re: psql ctrl+f skips displaying of one record and displays skipping one line

2019-07-16 Thread vignesh C
I'm able to get the same behaviour in centos as well. Should we do anything to handle this in Postgres or any documentation required? On Wed, Jul 17, 2019 at 10:05 AM Tom Lane wrote: > Thomas Munro writes: > > Pretty sure this is coming from your system's pager. You can see the > > same thing

Re: psql ctrl+f skips displaying of one record and displays skipping one line

2019-07-16 Thread Tom Lane
Thomas Munro writes: > Pretty sure this is coming from your system's pager. You can see the > same thing when you run this on a RHEL box: > seq 1 1 | more > It skips a line each time you press ^F. Yeah, duplicated on RHEL6. It seems to behave the same as the documented "s" command. Not sur

[PATCH] Implement INSERT SET syntax

2019-07-16 Thread Gareth Palmer
Hello, Attached is a patch that adds the option of using SET clause to specify the columns and values in an INSERT statement in the same manner as that of an UPDATE statement. A simple example that uses SET instead of a VALUES() clause: INSERT INTO t SET c1 = 'foo', c2 = 'bar', c3 = 'baz'; Valu

Re: psql ctrl+f skips displaying of one record and displays skipping one line

2019-07-16 Thread Thomas Munro
On Wed, Jul 17, 2019 at 4:07 PM vignesh C wrote: > One observation when we execute a select query having results more than the > screen space available and press ctrl+f to display the remaining records, one > of the record was not displayed and the message "...skipping one line" was > displayed

psql ctrl+f skips displaying of one record and displays skipping one line

2019-07-16 Thread vignesh C
Hi, One observation when we execute a select query having results more than the screen space available and press ctrl+f to display the remaining records, one of the record was not displayed and the message "...skipping one line" was displayed. I'm not sure if this is intentional behaviour. Steps

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Thomas Munro
On Wed, Jul 17, 2019 at 3:44 PM Dilip Kumar wrote: > Right, actually I got that point. But, I was thinking that we are > wasting one logno from undo log addressing space no?. Instead, if we > can keep it attached to the slot and somehow manage to add to the free > list then the same logno can be

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Dilip Kumar
On Wed, Jul 17, 2019 at 3:48 AM Thomas Munro wrote: > > On Tue, Jul 16, 2019 at 11:33 PM Dilip Kumar wrote: > > On Mon, Jul 1, 2019 at 1:24 PM Thomas Munro wrote: > > /* If we discarded everything, the slot can be given up. */ > > + if (entirely_discarded) > > + free_undo_log_slot(slot); > > > >

[PATCH] Make configuration file "include" directive handling more robust

2019-07-16 Thread Ian Barwick
Hi While poking about with [1], I noticed a few potential issues with the inclusion handling for configuration files; another issue is demonstrated in [2]. [1] https://www.postgresql.org/message-id/aed6cc9f-98f3-2693-ac81-52bb0052307e%402ndquadrant.com ("Stop ALTER SYSTEM from making bad as

Re: SegFault on 9.6.14

2019-07-16 Thread Thomas Munro
On Wed, Jul 17, 2019 at 12:57 PM Thomas Munro wrote: > On Wed, Jul 17, 2019 at 12:44 PM Thomas Munro wrote: > > > #11 0x55666e0359df in ExecShutdownNode > > > (node=node@entry=0x55667033a6c8) > > > at > > > /build/postgresql-9.6-5O8OLM/postgresql-9.6-9.6.14/build/../src/backend/executor

Re: refactoring - share str2*int64 functions

2019-07-16 Thread Michael Paquier
On Tue, Jul 16, 2019 at 01:04:38PM -0700, Andres Freund wrote: > There is the issue that there already is pg_strtoint16 and > pg_strtoint32, which do not have the option to not raise an error. I'd > probably name the non-error throwing ones something like pg_strtointNN_e > (for extended, or error

Re: refactoring - share str2*int64 functions

2019-07-16 Thread Michael Paquier
On Tue, Jul 16, 2019 at 01:18:38PM -0700, Andres Freund wrote: > Hi, > > On 2019-07-16 16:11:44 +0900, Michael Paquier wrote: > Yea, consistent naming seems like a strong requirement > here. Additionally I think we should just provide a consistent set > rather than what's needed just now. That'll

Re: POC: converting Lists into arrays

2019-07-16 Thread David Rowley
On Wed, 17 Jul 2019 at 11:06, Tom Lane wrote: > 0002 changes some additional places where it's maybe a bit less safe, > ie there's a potential for user-visible behavioral change because > processing will occur in a different order. In particular, the proposed > change in execExpr.c causes aggrega

Re: Adding SMGR discriminator to buffer tags

2019-07-16 Thread Thomas Munro
On Tue, Jul 16, 2019 at 10:49 AM Thomas Munro wrote: > I'll go and commit the simple refactoring bits of this work, which > just move some stuff belonging to md.c out of smgr.c (see attached). Pushed. The rest of that earlier patch set is hereby abandoned (at least for now). I'll be posting a n

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Amit Kapila
On Wed, Jul 17, 2019 at 3:53 AM Andres Freund wrote: > > Hi, > > On 2019-07-15 12:26:21 -0400, Robert Haas wrote: > > Yeah. I didn't understand that explanation. It seems to me that one > > of the fundamental design questions for this system is whether we > > should allow there to be an unbounded

RE: Bug: ECPG: Cannot use CREATE AS EXECUTE statemnt

2019-07-16 Thread Matsumura, Ryo
Meskes-san Thank you for your comment. I attach a patch. It doesn't include tests, but it passed some test(*1). Explanation about the patch: - Add a new ECPGst_exec_embedded_in_other_stmt whether EXECUTE statement has exprlist or not. This type name may not be good. It is a type for [CRE

Re: buildfarm's typedefs list has gone completely nutso

2019-07-16 Thread Andres Freund
Hi, On 2019-07-10 17:24:41 -0700, Andres Freund wrote: > Not yet sure what's actually going on, but there's something odd with > debug information afaict: > > objdump -W spits out warnings for a few files, all static libraries: > > ../install/lib/libpgcommon.a > objdump: Warning: Location list s

pg_stat_replication lag fields return non-NULL values even with NULL LSNs

2019-07-16 Thread Michael Paquier
Hi all, (Thomas in CC as per 6912acc0) I got surprised by the following behavior from pg_stat_get_wal_senders when connecting for example pg_receivewal to a primary: =# select application_name, flush_lsn, replay_lsn, flush_lag, replay_lag from pg_stat_replication; application_name | flush_lsn | r

Re: Extracting only the columns needed for a query

2019-07-16 Thread Melanie Plageman
We implemented Approach B in the attached patch set (patch 0001) and then implemented Approach A (patch 0002) to sanity check the pruned list of columns to scan we were getting at plan-time. We emit a notice in SeqNext() if the two column sets differ. Currently, for all of the queries in the regres

Re: pg_receivewal documentation

2019-07-16 Thread Michael Paquier
On Tue, Jul 16, 2019 at 01:03:12PM -0400, Jesper Pedersen wrote: > Here is the patch for that. + +Note that while WAL will be flushed with this setting, +pg_receivewal never applies it, so + must not be set to +remote_apply if pg_receivewal +is the o

Re: Parallel Append subplan order instability on aye-aye

2019-07-16 Thread David Rowley
On Wed, 17 Jul 2019 at 07:23, Andres Freund wrote: > > Hi, > > On 2019-07-15 21:12:32 -0400, Tom Lane wrote: > > But I bet that these tables forming > > an inheritance hierarchy (with multiple inheritance even) does > > have something to do with it somehow, because if this were a > > generic VACUU

Re: SegFault on 9.6.14

2019-07-16 Thread Thomas Munro
On Wed, Jul 17, 2019 at 12:44 PM Thomas Munro wrote: > > #11 0x55666e0359df in ExecShutdownNode (node=node@entry=0x55667033a6c8) > > at > > /build/postgresql-9.6-5O8OLM/postgresql-9.6-9.6.14/build/../src/backend/executor/execProcnode.c:830 > > #12 0x55666e04d0ff in ExecLimit (node=nod

Re: SegFault on 9.6.14

2019-07-16 Thread Jerry Sievers
Thomas Munro writes: > On Wed, Jul 17, 2019 at 12:26 PM Jerry Sievers wrote: > >> Is this the right sequencing? >> >> 1. Start client and get backend pid >> 2. GDB; handle SIGUSR1, break, cont >> 3. Run query >> 4. bt > > Perfect, thanks. I think I just spotted something: Dig that! Great big

Re: SegFault on 9.6.14

2019-07-16 Thread Thomas Munro
On Wed, Jul 17, 2019 at 12:26 PM Jerry Sievers wrote: > Is this the right sequencing? > > 1. Start client and get backend pid > 2. GDB; handle SIGUSR1, break, cont > 3. Run query > 4. bt Perfect, thanks. I think I just spotted something: > #11 0x55666e0359df in ExecShutdownNode (node=node@

Re: SegFault on 9.6.14

2019-07-16 Thread Jerry Sievers
Thomas Munro writes: > On Wed, Jul 17, 2019 at 12:05 PM Jerry Sievers wrote: > >> Program received signal SIGUSR1, User defined signal 1. > > Oh, we need to ignore those pesky signals with "handle SIGUSR1 noprint > nostop". Is this the right sequencing? 1. Start client and get backend pid 2.

Re: SegFault on 9.6.14

2019-07-16 Thread Thomas Munro
On Wed, Jul 17, 2019 at 12:05 PM Jerry Sievers wrote: > Program received signal SIGUSR1, User defined signal 1. Oh, we need to ignore those pesky signals with "handle SIGUSR1 noprint nostop". -- Thomas Munro https://enterprisedb.com

Re: SegFault on 9.6.14

2019-07-16 Thread Jerry Sievers
Thomas Munro writes: > On Wed, Jul 17, 2019 at 11:33 AM Jerry Sievers wrote: > >> -> Nested Loop Left Join (cost=251621.81..12300177.37 rows=48 >> width=44) >>-> Gather (cost=1001.55..270403.27 rows=48 width=40) > >>-> Limit (cost=250620.25..250620

Re: SegFault on 9.6.14

2019-07-16 Thread Thomas Munro
On Wed, Jul 17, 2019 at 11:33 AM Jerry Sievers wrote: > -> Nested Loop Left Join (cost=251621.81..12300177.37 rows=48 > width=44) >-> Gather (cost=1001.55..270403.27 rows=48 width=40) >-> Limit (cost=250620.25..250620.27 rows=1 width=20) >

Re: SegFault on 9.6.14

2019-07-16 Thread Jerry Sievers
Thomas Munro writes: > On Wed, Jul 17, 2019 at 11:06 AM Jerry Sievers wrote: > >> (gdb) p *scan->rs_parallel >> Cannot access memory at address 0x7fa673a54108 > > So I guess one question is: was it a valid address that's been > unexpectedly unmapped, or is the pointer corrupted? Any chance you

Re: SegFault on 9.6.14

2019-07-16 Thread Jerry Sievers
Thomas Munro writes: > On Wed, Jul 17, 2019 at 11:06 AM Jerry Sievers wrote: > >> (gdb) p *scan->rs_parallel >> Cannot access memory at address 0x7fa673a54108 > > So I guess one question is: was it a valid address that's been > unexpectedly unmapped, or is the pointer corrupted? Any chance you

Re: A little report on informal commit tag usage

2019-07-16 Thread Andres Freund
Hi, On 2019-07-16 19:26:59 -0400, Tom Lane wrote: > Andres Freund writes: > > They don't preclude each other though. E.g. it'd be sensible to have both > > >> Per gripe from Ken Tanzer. Back-patch to 9.6. The issue exists > >> further back, but before 9.6 the code looks very different and it >

Re: A little report on informal commit tag usage

2019-07-16 Thread Tom Lane
Andres Freund writes: > They don't preclude each other though. E.g. it'd be sensible to have both >> Per gripe from Ken Tanzer. Back-patch to 9.6. The issue exists >> further back, but before 9.6 the code looks very different and it >> doesn't actually know whether the "var" name matches anythi

Re: A little report on informal commit tag usage

2019-07-16 Thread Andres Freund
Hi, On 2019-07-16 10:33:06 -0400, Tom Lane wrote: > Michael Paquier writes: > > As mentioned on different threads, "Discussion" is the only one we had > > a strong agreement with. Could it be possible to consider things like > > Author, Reported-by, Reviewed-by or Backpatch-through for example a

Re: SegFault on 9.6.14

2019-07-16 Thread Thomas Munro
On Wed, Jul 17, 2019 at 11:11 AM Thomas Munro wrote: > map, unmap mmap, munmap -- Thomas Munro https://enterprisedb.com

Re: SegFault on 9.6.14

2019-07-16 Thread Thomas Munro
On Wed, Jul 17, 2019 at 11:06 AM Jerry Sievers wrote: > (gdb) p *scan->rs_parallel > Cannot access memory at address 0x7fa673a54108 So I guess one question is: was it a valid address that's been unexpectedly unmapped, or is the pointer corrupted? Any chance you can strace the backend and pull ou

Re: POC: converting Lists into arrays

2019-07-16 Thread Tom Lane
I wrote: > * Look at places using lcons/list_delete_first to maintain FIFO lists. > The patch makes these O(N^2) for long lists. If we can reverse the list > order and use lappend/list_truncate instead, it'd be better. Possibly in > some places the list ordering is critical enough to make this im

Re: SegFault on 9.6.14

2019-07-16 Thread Jerry Sievers
Tomas Vondra writes: > On Mon, Jul 15, 2019 at 08:20:00PM -0500, Jerry Sievers wrote: > >>Tomas Vondra writes: >> >>> On Mon, Jul 15, 2019 at 07:22:55PM -0500, Jerry Sievers wrote: >>> Tomas Vondra writes: > On Mon, Jul 15, 2019 at 06:48:05PM -0500, Jerry Sievers wrote: > >

Re: Allow simplehash to use already-calculated hash values

2019-07-16 Thread Andres Freund
Hi, On 2019-07-16 15:20:33 -0700, Jeff Davis wrote: > The attached small patch adds new entry points to simplehash.h that > allow the caller to pass in the already-calculated hash value, so that > simplehash doesn't need to recalculate it. > > This is helpful for Memory-Bounded Hash Aggregation[1

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Andres Freund
Hi, On 2019-07-15 12:26:21 -0400, Robert Haas wrote: > Yeah. I didn't understand that explanation. It seems to me that one > of the fundamental design questions for this system is whether we > should allow there to be an unbounded number of transactions that are > pending undo application, or whe

Allow simplehash to use already-calculated hash values

2019-07-16 Thread Jeff Davis
The attached small patch adds new entry points to simplehash.h that allow the caller to pass in the already-calculated hash value, so that simplehash doesn't need to recalculate it. This is helpful for Memory-Bounded Hash Aggregation[1], which uses the hash value for multiple purposes. For instanc

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Thomas Munro
On Tue, Jul 16, 2019 at 11:33 PM Dilip Kumar wrote: > On Mon, Jul 1, 2019 at 1:24 PM Thomas Munro wrote: > /* If we discarded everything, the slot can be given up. */ > + if (entirely_discarded) > + free_undo_log_slot(slot); > > I have noticed that when the undo log was detached and it was full >

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Andres Freund
Hi, On 2019-07-13 15:55:51 +0530, Amit Kapila wrote: > On Fri, Jul 12, 2019 at 7:08 PM Robert Haas wrote: > > > I think even if we currently go with a binary heap, it will be > > > possible to change it to rbtree later, but I am fine either way. > > > > Well, I don't see much point in revising al

Re: Detailed questions about pg_xact_commit_timestamp

2019-07-16 Thread Morris de Oryx
Adrien, thanks a lot for taking the time to try and explain all of these details to me. I'm looking at incremental rollups, and thinking through various alternative designs. It sounds like pg_xact_commit_timestamp just isn't the right tool for my purposes, so I'll go in another direction. All the

Re: refactoring - share str2*int64 functions

2019-07-16 Thread Andres Freund
Hi, On 2019-07-16 16:11:44 +0900, Michael Paquier wrote: > numutils.c also has pg_strtoint16 and pg_strtoint32, so the locations > become rather inconsistent with inconsistent APIs for the manipulation > of int2 and int4 fields, and scanint8 is just a derivative of the same > logic. We have two

Re: refactoring - share str2*int64 functions

2019-07-16 Thread Andres Freund
Hi, On 2019-07-15 07:08:42 +0200, Fabien COELHO wrote: > I do not think that changing the error handling capability is appropriate, > it is really a feature of the function. The function could try to use an > internal pg_strtoint64 which would look like the other unsigned version, but > then it wo

Re: Parallel Append subplan order instability on aye-aye

2019-07-16 Thread Andres Freund
Hi, On 2019-07-15 21:12:32 -0400, Tom Lane wrote: > But I bet that these tables forming > an inheritance hierarchy (with multiple inheritance even) does > have something to do with it somehow, because if this were a > generic VACUUM bug surely we'd be seeing it elsewhere. It's possible that it's

Re: rebased background worker reimplementation prototype

2019-07-16 Thread Tomas Vondra
On Tue, Jul 16, 2019 at 10:53:46AM -0700, Andres Freund wrote: Hi, On 2019-07-12 15:47:02 +0200, Tomas Vondra wrote: I've done a bit of benchmarking / testing on this, so let me report some basic results. I haven't done any significant code review, I've simply ran a bunch of pgbench runs on dif

Re: POC: converting Lists into arrays

2019-07-16 Thread Tom Lane
I wrote: > * Rationalize places that are using combinations of list_copy and > list_concat, probably by inventing an additional list-concatenation > primitive that modifies neither input. I poked around to see what we have in this department. There seem to be several identifiable use-cases: * Co

Re: A little report on informal commit tag usage

2019-07-16 Thread Daniel Gustafsson
> On 16 Jul 2019, at 16:33, Tom Lane wrote: > > Michael Paquier writes: >> As mentioned on different threads, "Discussion" is the only one we had >> a strong agreement with. Could it be possible to consider things like >> Author, Reported-by, Reviewed-by or Backpatch-through for example and >>

Re: Custom table AMs need to include heapam.h because of BulkInsertState

2019-07-16 Thread Andres Freund
Hi, Sorry for not chiming in again earlier, I was a bit exhausted... On 2019-07-03 19:46:06 +1200, David Rowley wrote: > I think the only objection to doing it the way [2] did was, if there > are more than MAX_PARTITION_BUFFERS partitions then we may end up > evicting the CopyMultiInsertBuffer o

Re: SQL/JSON path issues/questions

2019-07-16 Thread Alexander Korotkov
On Tue, Jul 16, 2019 at 9:22 PM Thom Brown wrote: > Now I'm looking at the @? and @@ operators, and getting a bit > confused. This following query returns true, but I can't determine > why: > > # SELECT '{"a":[1,2,3,4,5]}'::jsonb @? '$.b == "hello"'::jsonpath; > ?column? > -- > t > (1 r

Re: SQL/JSON path issues/questions

2019-07-16 Thread Thom Brown
On Thu, 11 Jul 2019 at 16:23, Alexander Korotkov wrote: > > On Thu, Jul 11, 2019 at 5:10 PM Thom Brown wrote: > > On Wed, 10 Jul 2019 at 05:58, Alexander Korotkov > > wrote: > > > > > > On Mon, Jul 8, 2019 at 12:30 AM Alexander Korotkov > > > wrote: > > > > On Thu, Jul 4, 2019 at 4:38 PM Liudmi

Re: rebased background worker reimplementation prototype

2019-07-16 Thread Andres Freund
Hi, On 2019-07-12 15:47:02 +0200, Tomas Vondra wrote: > I've done a bit of benchmarking / testing on this, so let me report some > basic results. I haven't done any significant code review, I've simply > ran a bunch of pgbench runs on different systems with different scales. Thanks! > System #1

Re: Minimal logical decoding on standbys

2019-07-16 Thread Andres Freund
Hi, On 2019-07-12 14:53:21 +0530, tushar wrote: > On 07/10/2019 05:12 PM, Amit Khandekar wrote: > > All right. Will do that in the next patch set. For now, I have quickly > > done the below changes in a single patch again (attached), in order to > > get early comments if any. > Thanks Amit for you

Re: heapam_index_build_range_scan's anyvisible

2019-07-16 Thread Andres Freund
Hi, On 2019-07-11 17:27:46 -0700, Ashwin Agrawal wrote: > Please find attached the patch to remove IndexBuildCallback's dependency on > HeapTuple, as discussed. Changed to have the argument as ItemPointer > instead of HeapTuple. Other larger refactoring if feasible for > index_build_range_scan API

Re: pg_receivewal documentation

2019-07-16 Thread Jesper Pedersen
Hi, On 7/16/19 12:28 PM, Laurenz Albe wrote: This is not true in all cases as since 9.6 it is possible to specify multiple synchronous standbys. So if for example pg_receivewal and another synchronous standby are set in s_s_names and that the number of a FIRST (priority-based) or ANY (quorum se

Re: Index Skip Scan

2019-07-16 Thread Jesper Pedersen
Hi David, On 7/11/19 7:38 AM, David Rowley wrote: The UniqueKeys idea is quite separate from pathkeys. Currently, a Path can have a List of PathKeys which define the order that the tuples will be read from the Plan node that's created from that Path. The idea with UniqueKeys is that a Path can

Re: pg_receivewal documentation

2019-07-16 Thread Laurenz Albe
On Tue, 2019-07-16 at 14:05 +0900, Michael Paquier wrote: > >> How about > >> Note that while WAL will be flushed with this setting, > >> pg_receivewal never applies it, so > >> must not be set to > >> remote_apply if > >> pg_receivewal > >> is the onl

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Robert Haas
On Mon, Jul 1, 2019 at 3:54 AM Thomas Munro wrote: > Here's a new version. Here's a relatively complete review of 0019 and 0020 and a remark or two on the beginning of 0003. Regarding 0020: The documentation claims that undo data exists in a 64-bit address space divided into 2^34 undo logs, eac

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Robert Haas
On Tue, Jul 16, 2019 at 7:13 AM Amit Kapila wrote: > > I also strongly suspect it is altogether wrong to do > > this before CommitSubTransaction sets s->state to TRANS_COMMIT; what > > if a subxact callback throws an error? > > Are you worried that it might lead to the execution of actions twice?

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Robert Haas
On Tue, Jul 16, 2019 at 12:32 AM Amit Kapila wrote: > The idea is that the queues can get full, but not rollback hash table. > In the case where the error queue gets full, we mark the entry as > Invalid in the hash table and later when discard worker again > encounters this request, it adds it to

Re: POC: converting Lists into arrays

2019-07-16 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 16, 2019 at 10:44 AM Tom Lane wrote: >> OK, I'm outvoted, will do it that way. > I cast my vote in the other direction i.e. for sticking with qsort. Didn't see this until after pushing a commit that uses "list_sort". While composing that commit message another

Re: POC: converting Lists into arrays

2019-07-16 Thread Peter Geoghegan
On Tue, Jul 16, 2019 at 9:01 AM Robert Haas wrote: > I cast my vote in the other direction i.e. for sticking with qsort. I do too. -- Peter Geoghegan

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Robert Haas
On Tue, Jul 16, 2019 at 10:02 AM Amit Kapila wrote: > On Tue, Jul 16, 2019 at 4:43 PM Amit Kapila wrote: > > On Tue, Jul 16, 2019 at 2:09 AM Robert Haas wrote: > > > This patch has some problems with naming consistency. There's a > > > function called PushUndoRequest() which calls a function ca

Re: POC: converting Lists into arrays

2019-07-16 Thread Robert Haas
On Tue, Jul 16, 2019 at 10:44 AM Tom Lane wrote: > David Steele writes: > > On 7/15/19 11:07 PM, Tom Lane wrote: > >> David Rowley writes: > >>> The only thoughts I have so far here are that it's a shame that the > >>> function got called list_qsort() and not just list_sort(). > > > I agree with

Re: fix for BUG #3720: wrong results at using ltree

2019-07-16 Thread Nikita Glukhov
On 09.07.2019 17:57, Oleg Bartunov wrote: On Mon, Jul 8, 2019 at 7:22 AM Thomas Munro wrote: On Sun, Apr 7, 2019 at 3:46 AM Tom Lane wrote: =?UTF-8?Q?Filip_Rembia=C5=82kowski?= writes: Here is my attempt to fix a 12-years old ltree bug (which is a todo item). I see it's not backward-compat

Re: POC: converting Lists into arrays

2019-07-16 Thread Tom Lane
David Steele writes: > On 7/15/19 11:07 PM, Tom Lane wrote: >> David Rowley writes: >>> The only thoughts I have so far here are that it's a shame that the >>> function got called list_qsort() and not just list_sort(). > I agree with David -- list_sort() is better. I don't think "sort" is > su

Re: A little report on informal commit tag usage

2019-07-16 Thread Tom Lane
Michael Paquier writes: > As mentioned on different threads, "Discussion" is the only one we had > a strong agreement with. Could it be possible to consider things like > Author, Reported-by, Reviewed-by or Backpatch-through for example and > extend to that? The first three ones are useful for p

Re: [proposal] de-TOAST'ing using a iterator

2019-07-16 Thread Binguo Bao
Hi, John First, I'd like to advocate for caution when using synthetic > benchmarks involving compression. Consider this test: > insert into detoast_c (a) > select > 'abc'|| > repeat( > (SELECT string_agg(md5(chr(i)), '') > FROM generate_series(1,127) i) > , 1) > ||'xyz'

Re: make \d pg_toast.foo show its indices ; and, \d toast show its main table ; and \d relkind=I show its partitions

2019-07-16 Thread Justin Pryzby
I realized that the test added to show-childs patch was listing partitioned tables not indices..fixed. >From 237f0bb2a048aa71726eff2580d01404ae3a98b4 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 30 Apr 2019 19:05:53 -0500 Subject: [PATCH v5] print table associated with given TOAST table

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Amit Kapila
On Tue, Jul 16, 2019 at 4:43 PM Amit Kapila wrote: > > On Tue, Jul 16, 2019 at 2:09 AM Robert Haas wrote: > > > > This patch has some problems with naming consistency. There's a > > function called PushUndoRequest() which calls a function called > > RegisterRollbackReq() to do the heart of the w

Re: getting ERROR "relation 16401 has no triggers" with partition foreign key alter

2019-07-16 Thread Tom Lane
Rajkumar Raghuwanshi writes: > I am getting ERROR: relation 16401 has no triggers error while executing > below query. Yeah, I can reproduce that back to v11. If you try the same scenario with a non-partitioned table you get ERROR: 55006: cannot ALTER TABLE "tbl2" because it has pending trigg

Re: refactoring - share str2*int64 functions

2019-07-16 Thread Tom Lane
Robert Haas writes: > On Jul 16, 2019, at 3:30 AM, Fabien COELHO wrote: >>> Cool. I'm not exactly sure when we should include 'pg_' in identifier >>> names. >> I added the pg_ prefix as a poor man's namespace because the function can be >> used by external tools (eg contribs), so as to avoid p

Re: [HACKERS] proposal: schema variables

2019-07-16 Thread Pavel Stehule
patch >>> >> >> rebase after pgindent >> > > fresh rebase > just rebase again Regards Pavel > Regards > > Pavel > > >> Regards >> >> Pavel >> >>> >>> Regards >>> >>> Pavel >>> >>> >>> schema-variables-20190716.patch.gz Description: application/gzip

Re: refactoring - share str2*int64 functions

2019-07-16 Thread Robert Haas
On Jul 16, 2019, at 3:30 AM, Fabien COELHO wrote: >> Cool. I'm not exactly sure when we should include 'pg_' in identifier >> names. It seems to be used for functions/macros that wrap or replace >> something else with a similar name, like pg_pwrite(), >> pg_attribute_noreturn(), ... In this ca

Re: Add parallelism and glibc dependent only options to reindexdb

2019-07-16 Thread Julien Rouhaud
On Fri, Jul 12, 2019 at 11:47 AM Julien Rouhaud wrote: > > I didn't change the behavior wrt. possible deadlock if user specify > catalog objects using --index or --table and ask for multiple > connection, as I'm afraid that it'll add too much code for a little > benefit. Please shout if you think

Re: Resume vacuum and autovacuum from interruption and cancellation

2019-07-16 Thread Masahiko Sawada
On Wed, Jun 12, 2019 at 1:30 PM Masahiko Sawada wrote: > > Hi all, > > Long-running vacuum could be sometimes cancelled by administrator. And > autovacuums could be cancelled by concurrent processes. Even if it > retries after cancellation, since it always restart from the first > block of table i

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Dilip Kumar
On Mon, Jul 1, 2019 at 1:24 PM Thomas Munro wrote: > > > 1. Renamed UndoPersistence to UndoLogCategory everywhere, and add a > fourth category UNDO_SHARED where transactions can write 'out of band' > data that relates to more than one transaction. > > 2. Introduced a new RMGR callback rm_undo_st

Re: d25ea01275 and partitionwise join

2019-07-16 Thread Etsuro Fujita
On Tue, Jul 2, 2019 at 6:29 PM Amit Langote wrote: > 0001 - fix partitionwise join to work correctly with n-way joins of > which some are full joins (+ cosmetic improvements around the code > that was touched) Here are my comments about the cosmetic improvements: they seem pretty large to me, so

Re: Change ereport level for QueuePartitionConstraintValidation

2019-07-16 Thread Sergei Kornilov
Hello Here is two patches with NOTICE ereport: one for partitions operations and one for "set not null" (for consistency) regards, Sergeidiff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 2a2c161695..4ec61d4833 100644 --- a/src/backend/commands/tablecmds.c +++

Re: POC: Cleaning up orphaned files using undo logs

2019-07-16 Thread Amit Kapila
On Tue, Jul 16, 2019 at 2:09 AM Robert Haas wrote: > > On Mon, Jul 1, 2019 at 3:54 AM Thomas Munro wrote: > > Reviewing Amit's 0016: > > performUndoActions appears to be badly-designed. For starters, it's > sometimes wrong: the only place it gets set to true is in > UndoActionsRequired (which is

Re: SegFault on 9.6.14

2019-07-16 Thread Thomas Munro
On Tue, Jul 16, 2019 at 8:22 PM Tomas Vondra wrote: > On Mon, Jul 15, 2019 at 08:20:00PM -0500, Jerry Sievers wrote: > >We have a reproduceable case of $subject that issues a backtrace such as > >seen below. > >#0 initscan (scan=scan@entry=0x55d7a7daa0b0, key=0x0, > >keep_startb

Re: Custom table AMs need to include heapam.h because of BulkInsertState

2019-07-16 Thread Michael Paquier
Hi David, On Wed, Jul 10, 2019 at 09:40:59PM +1200, David Rowley wrote: > On Wed, 3 Jul 2019 at 19:35, Michael Paquier wrote: >> This has been reverted as of f5db56f, still it seems to me that this >> was moving in the right direction. > > I've pushed this again, this time with the cleanup code

Re: Comment fix of config_default.pl

2019-07-16 Thread Kyotaro Horiguchi
At Sat, 13 Jul 2019 16:53:45 +0900, Michael Paquier wrote in <20190713075345.gc2...@paquier.xyz> > On Fri, Jul 12, 2019 at 05:01:41PM +0900, Michael Paquier wrote: > > I would also patch GetFakeConfigure in Solution.pm (no need to send a > > new patch), and I thought that you'd actually do the ch

Re: Check-out mutable functions in check constraints

2019-07-16 Thread Kyotaro Horiguchi
Hello, Thanks all! At Sat, 13 Jul 2019 11:17:32 -0400, Tom Lane wrote in <18372.1563031...@sss.pgh.pa.us> > Tomas Vondra writes: > > On Fri, Jul 12, 2019 at 07:59:13PM -0400, Tom Lane wrote: > >> I'm pretty sure this change has been proposed before, and rejected before. > >> Has anybody excavat

Re: Detailed questions about pg_xact_commit_timestamp

2019-07-16 Thread Adrien Nayrat
On 7/12/19 2:50 PM, Morris de Oryx wrote: > Adrien, thanks very much for answering my question. Just a couple of follow-up > points, if you don't mind. > > In our answer, you offer an example of pg_xact_commit_timestamp showing > out-of-sequence commit times: > > Session     xid          pg_xact_

  1   2   >