fix for BUG #3720: wrong results at using ltree

2019-03-05 Thread Filip Rembiałkowski
Hi all, Here is my attempt to fix a 12-years old ltree bug (which is a todo item). I see it's not backward-compatible, but in my understanding that's what is documented. Previous behavior was inconsistent with documentation (where single asterisk should match zero or more labels). http://archive

Re: Delay locking partitions during query execution

2019-03-05 Thread David Rowley
On Wed, 6 Mar 2019 at 04:46, Tomas Vondra wrote: > > On 3/5/19 6:55 AM, David Rowley wrote: > > The only way I can think to fix this is to just never lock partitions > > at all, and if a lock is to be obtained on a partition, it must be > > instead obtained on the top-level partitioned table. Tha

Re: [HACKERS] CLUSTER command progress monitor

2019-03-05 Thread Alvaro Herrera
On 2019-Mar-04, Robert Haas wrote: > On Mon, Mar 4, 2019 at 5:38 AM Tatsuro Yamada > wrote: > > === Discussion points === > > > > - Progress counter for "3. sorting tuples" phase > > - Should we add pgstat_progress_update_param() in tuplesort.c like a > >"trace_sort"? > >T

Re: patch to allow disable of WAL recycling

2019-03-05 Thread Alvaro Herrera
Jerry, On 2019-Mar-05, Jerry Jelinek wrote: > Thanks again for your review. I went through your proposed patch diffs and > applied most of them to my original changes. I did a few things slightly > differently since I wanted to keep to to 80 columns for the source code, > but I can revisit that i

Re: few more wait events to add to docs

2019-03-05 Thread Alvaro Herrera
On 2019-Mar-05, Jeremy Schneider wrote: > It's possible I'm misreading this, but I'm thinking that commits > cc5f8136 and ab9e0e71 added a few tranches which we need to add to the docs. > > session_dsa, session_record_table, session_typmod_table, and > shared_tuplestore > > https://www.postgresq

Re: few more wait events to add to docs

2019-03-05 Thread Thomas Munro
On Wed, Mar 6, 2019 at 2:18 PM Alvaro Herrera wrote: > On 2019-Mar-05, Jeremy Schneider wrote: > > It's possible I'm misreading this, but I'm thinking that commits > > cc5f8136 and ab9e0e71 added a few tranches which we need to add to the docs. > > > > session_dsa, session_record_table, session_ty

Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Paul Ramsey
> On Mar 5, 2019, at 3:56 PM, Tom Lane wrote: > > Paul Ramsey writes: >> On Mar 5, 2019, at 3:26 PM, Tom Lane wrote: >>> Hm, I think your addition of this bit is wrong: >>> >>> +/* >>> +* Arguments were swapped to put the index value on the >>> +

Re: Delay locking partitions during query execution

2019-03-05 Thread Robert Haas
On Tue, Mar 5, 2019 at 8:04 PM David Rowley wrote: > Actually, I'm not sure it could work at all. It does not seem very > safe to lookup a partition's parent without actually holding a lock on > the partition and we can't lock the partition and then lock each > parent in turn as that's the exact

Re: Patch to document base64 encoding

2019-03-05 Thread Karl O. Pinc
Hi Fabien, On Tue, 5 Mar 2019 23:02:26 +0100 (CET) Fabien COELHO wrote: > > Attached: doc_base64_v3.patch > > I'm ok with referencing the historical MIME RFC. For the record, RFC 2045 is updated but not yet obsolete. The updates don't invalidate section 6.8. > "RFC2045 section 6.8" -> "RFC

Re: speeding up planning with partitions

2019-03-05 Thread Amit Langote
On 2019/03/06 0:57, Jesper Pedersen wrote: > On 3/5/19 5:24 AM, Amit Langote wrote: >> Attached an updated version.  This incorporates fixes for both Jesper's >> and Imai-san's review.  I haven't been able to pin down the bug (or >> whatever) that makes throughput go down as the partition count inc

Re: Fix memleaks and error handling in jsonb_plpython

2019-03-05 Thread Michael Paquier
On Tue, Mar 05, 2019 at 02:10:01PM +0300, Nikita Glukhov wrote: > I known about this volatility issues, but maybe I incorrectly understand what > should be marked as volatile for pointer variables: the pointer itself and/or > the memory referenced by it. I thought that only pointer needs to be mar

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread David Rowley
On Tue, 5 Mar 2019 at 03:01, Derek Hans wrote: > Based on a reply to reporting this as a bug, moving rows out of foreign > partitions is not yet implemented so this is behaving as expected. There's a > mention of this limitation in the Notes section of the Update docs. (Moving this discussion t

RE: speeding up planning with partitions

2019-03-05 Thread Imai, Yoshikazu
Amit-san, On Tue, Mar 5, 2019 at 10:24 AM, Amit Langote wrote: > On 2019/03/05 9:50, Amit Langote wrote: > > I'll post the updated patches after diagnosing what I'm suspecting a > > memory over-allocation bug in one of the patches. If you configure > > build with --enable-cassert, you'll see that

RE: speeding up planning with partitions

2019-03-05 Thread Imai, Yoshikazu
On Wed, Mar 6, 2019 at 2:10 AM, Imai, Yoshikazu wrote: > > and Imai-san's review. I haven't been able to pin down the bug (or > > whatever) that makes throughput go down as the partition count > > increases, when tested with a --enable-cassert build. > > I didn't investigate that problem, but the

Re: Prevent extension creation in temporary schemas

2019-03-05 Thread Michael Paquier
On Tue, Mar 05, 2019 at 12:47:54PM +, Chris Travers wrote: > I tried installing a test extension into a temp schema. I found > this was remarkably difficult to do because pg_temp did not work (I > had to create a temporary table and then locate the actual table it > was created in). While tha

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
Hi David, On 2019/03/06 11:06, David Rowley wrote: > On Tue, 5 Mar 2019 at 03:01, Derek Hans wrote: >> Based on a reply to reporting this as a bug, moving rows out of foreign >> partitions is not yet implemented so this is behaving as expected. There's a >> mention of this limitation in the Not

Re: Patch to document base64 encoding

2019-03-05 Thread Michael Paquier
On Tue, Mar 05, 2019 at 07:55:22PM -0600, Karl O. Pinc wrote: > Attached: doc_base64_v4.patch Details about the "escape" mode are already available within the description of function "encode". Wouldn't we want to consolidate a description for all the modes at the same place, including some words

Re: Pluggable Storage - Andres's take

2019-03-05 Thread Ashwin Agrawal
Hi, While playing with the tableam, usage of which starts with commit v12-0023-tableam-Introduce-and-use-begin-endscan-and-do-i.patch, should we check for NULL function pointer before actually calling the same and ERROR out instead as NOT_SUPPORTED or something on those lines. Understand its kind

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread David Rowley
On Wed, 6 Mar 2019 at 15:26, Amit Langote wrote: > > > I've tried to put all this right again in the attached. However, I was > > a bit unsure of what "but they can be moved into a foreign table if > > the foreign data wrapper supports it." is referring to. Copying Robert > > and Etsuro as this wa

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
On 2019/03/06 11:29, David Rowley wrote: > On Wed, 6 Mar 2019 at 15:26, Amit Langote > wrote: >> >>> I've tried to put all this right again in the attached. However, I was >>> a bit unsure of what "but they can be moved into a foreign table if >>> the foreign data wrapper supports it." is referri

Re: Pluggable Storage - Andres's take

2019-03-05 Thread Andres Freund
Hi, Thanks for looking! On 2019-03-05 18:27:45 -0800, Ashwin Agrawal wrote: > While playing with the tableam, usage of which starts with commit > v12-0023-tableam-Introduce-and-use-begin-endscan-and-do-i.patch, should we > check for NULL function pointer before actually calling the same and ERROR

Re: Pluggable Storage - Andres's take

2019-03-05 Thread Andres Freund
Hi, Thanks for looking! On 2019-03-05 18:27:45 -0800, Ashwin Agrawal wrote: > While playing with the tableam, usage of which starts with commit > v12-0023-tableam-Introduce-and-use-begin-endscan-and-do-i.patch, should we > check for NULL function pointer before actually calling the same and ERROR

Re: Online verification of checksums

2019-03-05 Thread Michael Paquier
On Tue, Mar 05, 2019 at 02:08:03PM +0100, Tomas Vondra wrote: > Based on quickly skimming that thread the main issue seems to be > deciding which files in the data directory are expected to have > checksums. Which is a valid issue, of course, but I was expecting > something about partial read/write

Re: Online verification of checksums

2019-03-05 Thread Stephen Frost
Greetings, On Tue, Mar 5, 2019 at 18:36 Michael Paquier wrote: > On Tue, Mar 05, 2019 at 02:08:03PM +0100, Tomas Vondra wrote: > > Based on quickly skimming that thread the main issue seems to be > > deciding which files in the data directory are expected to have > > checksums. Which is a valid

Tab completion for SKIP_LOCKED option

2019-03-05 Thread Masahiko Sawada
Hi, I realized that the tab completions for SKIP_LOCKED option of both VACUUM and ANALYZE are missing. Attached patch adds them. Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center skip_locked.patch Description: Binary data

Re: few more wait events to add to docs

2019-03-05 Thread Michael Paquier
On Wed, Mar 06, 2019 at 02:21:15PM +1300, Thomas Munro wrote: > Here's some missing documentation. The addition looks fine to me. > Hmm, yeah. I wish these were alphabetised, I wish there was an > automated warning about this, I wish these tranches were declared a > better way that by adding cod

Re: Rare SSL failures on eelpout

2019-03-05 Thread Thomas Munro
On Wed, Mar 6, 2019 at 4:07 PM Shawn Debnath wrote: > On Wed, Mar 06, 2019 at 11:13:31AM +1300, Thomas Munro wrote: > > So... can anyone tell us what happens on Windows? > C:\Users\Shawn Debnath\Desktop>c:\Python27\python.exe tmunro-ssl-test.py > --client > Sending A... > 2 > Sending B... > [Err

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Etsuro Fujita
(2019/03/06 11:06), David Rowley wrote: On Tue, 5 Mar 2019 at 03:01, Derek Hans wrote: Based on a reply to reporting this as a bug, moving rows out of foreign partitions is not yet implemented so this is behaving as expected. There's a mention of this limitation in the Notes section of the Up

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread David Rowley
On Wed, 6 Mar 2019 at 16:29, Etsuro Fujita wrote: > > (2019/03/06 11:06), David Rowley wrote: > > I don't quite understand what a "foreign table to some other > > partition" is meant to mean. Partitions don't have foreign tables, > > they can only be one themselves. > > I think "foreign table" is

Re: [PATCH v20] GSSAPI encryption support

2019-03-05 Thread Robbie Harwood
Stephen Frost writes: > * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > >> Or maybe we avoid that, and you rename be-secure-gssapi.c to just >> be-gssapi.c and also combine that with the contents of >> be-gssapi-common.c. > > I don't know why we would need to, or want to, combine >

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Etsuro Fujita
(2019/03/06 11:34), Amit Langote wrote: Ah, indeed. In the documentation fix patch I'd posted, I also made changes to release-11.sgml to link to the limitations section. (I'm attaching it here for your reference.) I'm not sure it's a good idea to make changes to the release notes like that,

Re: Protect syscache from bloating with negative cache entries

2019-03-05 Thread Kyotaro HORIGUCHI
Hello. At Mon, 4 Mar 2019 03:03:51 +, "Ideriha, Takeshi" wrote in <4E72940DA2BF16479384A86D54D0988A6F44564E@G01JPEXMBKW04> > Does this result show that hard-limit size option with memory accounting > doesn't harm to usual users who disable hard limit size option? Not sure, but 4% seems be

Re: Converting NOT IN to anti-joins during planning

2019-03-05 Thread David Rowley
Hi Jim, Thanks for replying here. On Wed, 6 Mar 2019 at 16:37, Jim Finnerty wrote: > > Actually, we're working hard to integrate the two approaches. I haven't had > time since I returned to review your patch, but I understand that you were > checking for strict predicates as part of the nullnes

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
Fujita-san, On 2019/03/06 13:04, Etsuro Fujita wrote: > (2019/03/06 11:34), Amit Langote wrote: >> Ah, indeed.  In the documentation fix patch I'd posted, I also made >> changes to release-11.sgml to link to the limitations section.  (I'm >> attaching it here for your reference.) > > I'm not sure

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
On 2019/03/06 12:47, David Rowley wrote: > On Wed, 6 Mar 2019 at 16:29, Etsuro Fujita > wrote: >> That means that rows can be moved from a local partition to a foreign >> partition if the FDW supports it. > > It seems a bit light on detail to me. If I was a user I'd want to know > what exactly t

Re: New vacuum option to do only freezing

2019-03-05 Thread Masahiko Sawada
On Tue, Mar 5, 2019 at 8:01 PM Kyotaro HORIGUCHI wrote: > > Hello, I have some other comments. > Thank you for the comment! On Tue, Mar 5, 2019 at 8:01 PM Kyotaro HORIGUCHI wrote: > > > + nleft;/* item pointers we left */ > > The name seems to be something other, a

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread David Rowley
On Wed, 6 Mar 2019 at 17:20, Amit Langote wrote: > > On 2019/03/06 12:47, David Rowley wrote: > > It seems a bit light on detail to me. If I was a user I'd want to know > > what exactly the FDW needed to support this. Does it need a special > > partition move function? Looking at ExecFindPartitio

Re: Inheriting table AMs for partitioned tables

2019-03-05 Thread David Rowley
On Tue, 5 Mar 2019 at 19:08, Andres Freund wrote: > > On 2019-03-05 16:01:50 +1300, David Rowley wrote: > > I'd suggest it's made to work the same way as ca4103025dfe26 made > > tablespaces work. > > Hm, is that actually correct? Because as far as I can tell that doesn't > have the necessary pg_d

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
On 2019/03/06 13:30, David Rowley wrote: > On Wed, 6 Mar 2019 at 17:20, Amit Langote > wrote: >> >> On 2019/03/06 12:47, David Rowley wrote: >>> It seems a bit light on detail to me. If I was a user I'd want to know >>> what exactly the FDW needed to support this. Does it need a special >>> parti

Re: Patch to document base64 encoding

2019-03-05 Thread Karl O. Pinc
On Wed, 6 Mar 2019 11:27:38 +0900 Michael Paquier wrote: > On Tue, Mar 05, 2019 at 07:55:22PM -0600, Karl O. Pinc wrote: > > Attached: doc_base64_v4.patch > > Details about the "escape" mode are already available within the > description of function "encode". Wouldn't we want to consolidate a

Re: speeding up planning with partitions

2019-03-05 Thread Amit Langote
Imai-san, Thanks for the review. On 2019/03/06 11:09, Imai, Yoshikazu wrote: > Here is the code review for previous v26 patches. > > [0002] > In expand_inherited_rtentry(): > > expand_inherited_rtentry() > { > ... > + RelOptInfo *rel = NULL; > > can be declared at more later: > > if (ol

Re: Tab completion for SKIP_LOCKED option

2019-03-05 Thread Michael Paquier
On Wed, Mar 06, 2019 at 11:45:01AM +0900, Masahiko Sawada wrote: > I realized that the tab completions for SKIP_LOCKED option of both > VACUUM and ANALYZE are missing. Attached patch adds them. Thanks Sawada-san, committed. -- Michael signature.asc Description: PGP signature

Re: Allowing extensions to supply operator-/function-specific info

2019-03-05 Thread Tom Lane
Paul Ramsey writes: >> On Mar 5, 2019, at 3:56 PM, Tom Lane wrote: >> Then you're at least missing adequate tests for the 3-arg functions... >> 3 args with the index column second will not work as this stands. > Some of the operators are indifferent to order (&&, overlaps) and others are > not

Re: Tab completion for SKIP_LOCKED option

2019-03-05 Thread Masahiko Sawada
On Wed, Mar 6, 2019 at 2:46 PM Michael Paquier wrote: > > On Wed, Mar 06, 2019 at 11:45:01AM +0900, Masahiko Sawada wrote: > > I realized that the tab completions for SKIP_LOCKED option of both > > VACUUM and ANALYZE are missing. Attached patch adds them. > > Thanks Sawada-san, committed. Thank y

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Etsuro Fujita
(2019/03/06 13:18), Amit Langote wrote: The main problem here is indeed that the limitation is not listed under the partitioning limitations in ddl.sgml, where it's easier to notice than in the UPDATE's page. Agreed. I've updated my patch to remove the release-11.sgml changes. Thanks for th

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Etsuro Fujita
(2019/03/06 13:53), Amit Langote wrote: On 2019/03/06 13:30, David Rowley wrote: I think you missed my point. If there's no special support for "tuple moving", as you say, then what help is it to tell the user "if the FDW supports tuple routing"? The answer is, it's not any help. How would

Re: [HACKERS] CLUSTER command progress monitor

2019-03-05 Thread Tatsuro Yamada
On 2019/03/06 1:13, Robert Haas wrote: On Tue, Mar 5, 2019 at 3:56 AM Tatsuro Yamada wrote: === Discussion points === - Progress counter for "3. sorting tuples" phase - Should we add pgstat_progress_update_param() in tuplesort.c like a "trace_sort"? Thanks to Peter

Re: [HACKERS] Block level parallel vacuum

2019-03-05 Thread Masahiko Sawada
On Mon, Mar 4, 2019 at 10:27 AM Masahiko Sawada wrote: > > On Sat, Mar 2, 2019 at 3:54 AM Robert Haas wrote: > > > > On Fri, Mar 1, 2019 at 12:19 AM Masahiko Sawada > > wrote: > > > > I wonder if we really want this behavior. Should a setting that > > > > controls the degree of parallelism whe

Re: Update does not move row across foreign partitions in v11

2019-03-05 Thread Amit Langote
Fujita-san, On 2019/03/06 15:10, Etsuro Fujita wrote: > --- a/doc/src/sgml/ddl.sgml > +++ b/doc/src/sgml/ddl.sgml > @@ -3376,6 +3376,13 @@ ALTER TABLE measurement ATTACH PARTITION > measurement_y2008m02 >    >   > > + > +  > +   UPDATE row movement is not supported in the

Re: [HACKERS] CLUSTER command progress monitor

2019-03-05 Thread Tatsuro Yamada
On 2019/03/05 17:56, Tatsuro Yamada wrote: Hi Robert! On 2019/03/05 11:35, Robert Haas wrote: On Mon, Mar 4, 2019 at 5:38 AM Tatsuro Yamada wrote: === Current design === CLUSTER command uses Index Scan or Seq Scan when scanning the heap. Depending on which one is chosen, the command will pro

pg_dump is broken for partition tablespaces

2019-03-05 Thread David Rowley
Over on [1] Andres pointed out that the pg_dump support for the new to PG12 tablespace inheritance feature is broken. This is the feature added in ca4103025dfe26 to allow a partitioned table to have a tablespace that acts as the default tablespace for newly attached partitions. The idea being that

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-05 Thread Michael Paquier
On Tue, Mar 05, 2019 at 11:04:17PM +0900, Amit Langote wrote: > Maybe we should error out as follows in > transformPartitionRangeBounds(), although that means we'll get > different error message than when using list partitioning syntax: Hm. I don't think that this is a good idea as you could lose

Re: BUG #15668: Server crash in transformPartitionRangeBounds

2019-03-05 Thread Amit Langote
Hi, On 2019/03/06 15:48, Michael Paquier wrote: > On Tue, Mar 05, 2019 at 11:04:17PM +0900, Amit Langote wrote: >> Maybe we should error out as follows in >> transformPartitionRangeBounds(), although that means we'll get >> different error message than when using list partitioning syntax: > > Hm.

ECPG regression with DECLARE STATEMENT support

2019-03-05 Thread Rushabh Lathia
Hi, Commit bd7c95f0c1a38becffceb3ea7234d57167f6d4bf add DECLARE STATEMENT support to ECPG. This introduced the new rule for EXEC SQL CLOSE cur and with that it gets transformed into ECPGclose(). Now prior to the above commit, someone can declare the cursor in the SQL statement and "CLOSE cur_nam

Re: pg_dump is broken for partition tablespaces

2019-03-05 Thread Michael Paquier
On Wed, Mar 06, 2019 at 07:45:06PM +1300, David Rowley wrote: > Partitioned indexes have this similar inherit tablespace from parent > feature, so ca4103025dfe26 was intended to align the behaviour of the > two. Partitioned indexes happen not to suffer from the same issue as > the indexes are attac

<    1   2