Re: [HACKERS] Parallel Append implementation

2017-11-23 Thread Rajkumar Raghuwanshi
On Thu, Nov 23, 2017 at 9:45 AM, amul sul wrote: > Attaching updated version of "ParallelAppend_v19_rebased" includes this > fix. > Hi, I have applied attached patch and got a crash with below query. please take a look. CREATE TABLE tbl (a int, b int, c text, d int) PARTITION BY LIST(c); CREAT

Re: [HACKERS] Parallel Append implementation

2017-11-23 Thread amul sul
Look like it is the same crash what v20 claim to be fixed, indeed I missed to add fix[1] in v20 patch, sorry about that. Attached updated patch includes aforementioned fix. 1] http://postgr.es/m/CAAJ_b97kLNW8Z9nvc_JUUG5wVQUXvG=f37WsX8ALF0A=kah...@mail.gmail.com Regards, Amul On Thu, Nov 23, 2

Re: [HACKERS] [POC] Faster processing at Gather node

2017-11-23 Thread Rafia Sabih
On Thu, Nov 16, 2017 at 12:24 AM, Andres Freund wrote: > Hi, > > On 2017-11-15 13:48:18 -0500, Robert Haas wrote: >> I think that we need a little bit deeper analysis here to draw any >> firm conclusions. > > Indeed. > > >> I suspect that one factor is that many of the queries actually send >> ver

How is the PostgreSQL debuginfo file generated

2017-11-23 Thread Rui Hai Jiang
Hello hackers, I’m wondering how to build the debuginfo package from the PostgreSQL source. For example to generate something like this one : postgresql91-debuginfo.x86_64. Is there existing support in the current PostgreSQL Makefiles to generate such debuginfo? I have searched in the sourc

Re: With commit 4e5fe9ad19, range partition missing handling for the NULL partition key

2017-11-23 Thread Rushabh Lathia
On Wed, Nov 22, 2017 at 2:36 PM, Amit Langote wrote: > On 2017/11/22 17:42, amul sul wrote: > > On Wed, Nov 22, 2017 at 11:38 AM, Amit Langote wrote: > >> On 2017/11/22 13:45, Rushabh Lathia wrote: > >>> Attaching patch to fix as well as regression test. > >> > >> Thanks for the patch. About the

PostgreSLQ v10.1 and xlC compiler on AIX

2017-11-23 Thread REIX, Tony
Hi, We are porting PostgreSQL v10.1 on AIX (7.2 for now). And we have several tests failures, in 32bit and 64bit. We are using xlc 13.01.0003.0003 with -O2. Tests were 100% OK with version 9.6.2 . About 32 bit failures within v10.1, we have 3 failures including: create_aggregate ...

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2017-11-23 Thread amul sul
On Sat, Nov 11, 2017 at 1:05 AM, Robert Haas wrote: > On Wed, Sep 27, 2017 at 7:07 AM, amul sul wrote: >> Attaching POC patch that throws an error in the case of a concurrent update >> to an already deleted tuple due to UPDATE of partition key[1]. >> >> In a normal update new tuple is linked to t

Re: [HACKERS] logical decoding of two-phase transactions

2017-11-23 Thread Nikhil Sontakke
Hi all, > > I think Nikhils has done some significant work on this patch. > Hopefully he'll be able to share it. > PFA, latest patch. This builds on top of the last patch submitted by Sokolov Yura and adds the actual logical replication interfaces to allow PREPARE or COMMIT/ROLLBACK PREPARED on a

Re: [HACKERS] UPDATE of partition key

2017-11-23 Thread Amit Khandekar
On 7 November 2017 at 00:33, Robert Haas wrote: > + /* The caller must have already locked all the partitioned tables. */ > + root_rel = heap_open(root_relid, NoLock); > + *all_part_cols = NULL; > + foreach(lc, partitioned_rels) > + { > + Index

Re: PostgreSLQ v10.1 and xlC compiler on AIX

2017-11-23 Thread Michael Paquier
On Thu, Nov 23, 2017 at 7:57 PM, REIX, Tony wrote: > We are porting PostgreSQL v10.1 on AIX (7.2 for now). > And we have several tests failures, in 32bit and 64bit. > We are using xlc 13.01.0003.0003 with -O2. > Tests were 100% OK with version 9.6.2 . > > About 32 bit failures within v10.1, we hav

Re: [HACKERS] Partition-wise aggregation/grouping

2017-11-23 Thread Jeevan Chalke
On Fri, Nov 17, 2017 at 5:54 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Wed, Nov 15, 2017 at 5:31 PM, Jeevan Chalke > wrote: > > > > OK. Done in the attached patch set. > > > > I have rebased all my patches on latest HEAD which is at > > 7518049980be1d90264addab003476ae105f

Re: [HACKERS] Challenges preventing us moving to 64 bit transaction id (XID)?

2017-11-23 Thread Amit Kapila
On Thu, Jun 22, 2017 at 9:06 PM, Alexander Korotkov wrote: > On Wed, Jun 7, 2017 at 11:33 AM, Alexander Korotkov > wrote: >> >> On Tue, Jun 6, 2017 at 4:05 PM, Peter Eisentraut >> wrote: >>> >>> On 6/6/17 08:29, Bruce Momjian wrote: >>> > On Tue, Jun 6, 2017 at 06:00:54PM +0800, Craig Ringer wr

Re: PostgreSLQ v10.1 and xlC compiler on AIX

2017-11-23 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Nov 23, 2017 at 7:57 PM, REIX, Tony wrote: > > I invite anyone involved in porting/using PostgreSQL 10.1 on AIX to take > > part of a discussion about how to make v10.1 work perfectly on AIX. > > Note that xlc 12.1 is tested with AIX machines on the buildfarms, b

RE:PostgreSLQ v10.1 and xlC compiler on AIX

2017-11-23 Thread REIX, Tony
Hi Michael, Thanks for the information ! I'm not a PostgreSQL expert. I've found the file: ./32bit/src/test/regress/regression.diffs Since I'm rebuilding right now, I do not have just now the file for v10.1 with the issue. However, I have it for 10.0 and 9.6.6 which show the same issues. I've

Re: Query became very slow after 9.6 -> 10 upgrade

2017-11-23 Thread Dmitry Shalashov
We tried to apply the patch on 10.1 source, but something is wrong it seems: patch -p1 < ../1.patch (Stripping trailing CRs from patch; use --binary to disable.) patching file src/backend/optimizer/plan/analyzejoins.c (Stripping trailing CRs from patch; use --binary to disable.) patching file src/

documentation is now XML

2017-11-23 Thread Peter Eisentraut
The documentation sources are now DocBook XML, not SGML. (The files are still named *.sgml. That's something to think about separately.) Besides the recent change to require full end tags, which is compatible between SGML and XML, being XML now requires empty-element tags to use the XML syntax l

Re: [HACKERS] SQL procedures

2017-11-23 Thread Peter Eisentraut
On 11/23/17 00:59, Craig Ringer wrote: > Exactly. If we want to handle OUT params this way they really need to be > the first resultset for just this reason. That could possibly be done by > the glue code reserving a spot in the resultset list and filling it in > at the end of the p

Re: has_sequence_privilege() never got the memo

2017-11-23 Thread Peter Eisentraut
On 11/22/17 22:58, Tom Lane wrote: > Joe Conway writes: >> I just noticed that has_sequence_privilege() never got the memo about >> "WITH GRANT OPTION". Any objections to the attached going back to all >> supported versions? > > That looks odd. Patch certainly makes this case consistent with the

Re: documentation is now XML

2017-11-23 Thread Oleg Bartunov
On Thu, Nov 23, 2017 at 6:01 PM, Peter Eisentraut wrote: > The documentation sources are now DocBook XML, not SGML. (The files are > still named *.sgml. That's something to think about separately.) Congratulations to you and Alexander ! That is what I waited for a long time. Now we could think

Re: How is the PostgreSQL debuginfo file generated

2017-11-23 Thread Tom Lane
Rui Hai Jiang writes: > I’m wondering how to build the debuginfo package from the PostgreSQL source. > For example to generate something like this one : > postgresql91-debuginfo.x86_64. On platforms where such things exist, that's handled by the packaging system, not by PG proper. You should

Re: documentation is now XML

2017-11-23 Thread Tom Lane
Peter Eisentraut writes: > The documentation sources are now DocBook XML, not SGML. (The files are > still named *.sgml. That's something to think about separately.) I think we should have a discussion about whether it'd be smart to convert the back branches' documentation to XML as well. The

Re: [HACKERS] Custom compression methods

2017-11-23 Thread Tomas Vondra
Hi, On 11/23/2017 10:38 AM, Ildus Kurbangaliev wrote: > On Tue, 21 Nov 2017 18:47:49 +0100 > Tomas Vondra wrote: > >>> >> >> Hmmm, it still doesn't work for me. See this: >> >> test=# create extension pg_lz4 ; >> CREATE EXTENSION >> test=# create table t_lz4 (v text compressed lz4

Re: documentation is now XML

2017-11-23 Thread Michael Paquier
On Fri, Nov 24, 2017 at 12:21 AM, Oleg Bartunov wrote: > On Thu, Nov 23, 2017 at 6:01 PM, Peter Eisentraut > wrote: >> The documentation sources are now DocBook XML, not SGML. (The files are >> still named *.sgml. That's something to think about separately.) > > Congratulations to you and Alexa

Re: [HACKERS] Supporting huge pages on Windows

2017-11-23 Thread Thomas Munro
On Thu, Sep 14, 2017 at 12:32 AM, Magnus Hagander wrote: > It's my plan to get to this patch during this commitfest. I've been > travelling for open and some 24/7 work so far, but hope to get CFing soon. I hope Tsunakawa-san doesn't mind me posting another rebased version of his patch. The last

RE: [HACKERS] Supporting huge pages on Windows

2017-11-23 Thread Tsunakawa, Takayuki
From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] > I hope Tsunakawa-san doesn't mind me posting another rebased version of > his patch. The last version conflicted with the change from SGML to XML > that just landed in commit 3c49c6fa. Thank you very much for keeping it fresh. I hope th

Re: Combine function returning NULL unhandled?

2017-11-23 Thread Andres Freund
On 2017-11-21 15:51:59 -0500, Robert Haas wrote: > On Mon, Nov 20, 2017 at 10:36 PM, Andres Freund wrote: > > The plain transition case contains: > > if (pergroupstate->transValueIsNull) > > { > > /* > > * Don't call

Re: Failed to delete old ReorderBuffer spilled files

2017-11-23 Thread atorikoshi
On 2017/11/22 16:49, Masahiko Sawada wrote: On Wed, Nov 22, 2017 at 2:56 PM, Craig Ringer wrote: On 22 November 2017 at 12:15, Kyotaro HORIGUCHI wrote: At Wed, 22 Nov 2017 12:57:34 +0900, Michael Paquier wrote in On Wed, Nov 22, 2017 at 11:49 AM, Craig Ringer wrote: On 20 November 2017

Re: [HACKERS] HASH_CHUNK_SIZE vs malloc rounding

2017-11-23 Thread Thomas Munro
On Tue, Nov 29, 2016 at 6:27 AM, Tom Lane wrote: > Thomas Munro writes: >> I bet other allocators also do badly with "32KB plus a smidgen". To >> minimise overhead we'd probably need to try to arrange for exactly >> 32KB (or some other power of 2 or at least factor of common page/chunk >> size?)

Re: Failed to delete old ReorderBuffer spilled files

2017-11-23 Thread Craig Ringer
On 24 November 2017 at 09:20, atorikoshi wrote: > > Thanks for letting me know. > I think I only tested running "make check" at top directory, sorry > for my insufficient test. > > The test failure happened at the beginning of replication(creating > slot), so there are no changes yet and getting

Re: How is the PostgreSQL debuginfo file generated

2017-11-23 Thread Craig Ringer
On 23 November 2017 at 18:38, Rui Hai Jiang wrote: > Hello hackers, > > > > I’m wondering how to build the debuginfo package from the PostgreSQL > source. > > > > For example to generate something like this one : > postgresql91-debuginfo.x86_64. > > > > Is there existing support in the current P

Re: [HACKERS] INSERT ON CONFLICT and partitioned tables

2017-11-23 Thread Amit Langote
Thank you Simon for the review. On 2017/11/20 7:33, Simon Riggs wrote: > On 2 August 2017 at 00:56, Amit Langote wrote: > >> The patch's job is simple: > > Patch no longer applies and has some strange behaviours. > >> - Remove the check in the parser that causes an error the moment the >> ON

Re: [HACKERS] INSERT ON CONFLICT and partitioned tables

2017-11-23 Thread Amit Langote
On 2017/11/24 11:45, Amit Langote wrote: > Meanwhile, rebased patch is attached. Oops, forgot to attach in the last email. Attached now. Thanks, Amit From ee7ef9d35f810c8c38c0ec40205f7b8c5d1f696d Mon Sep 17 00:00:00 2001 From: amit Date: Mon, 3 Apr 2017 19:13:38 +0900 Subject: [PATCH] Allow ON

Re: [HACKERS] logical decoding of two-phase transactions

2017-11-23 Thread Craig Ringer
On 23 November 2017 at 20:27, Nikhil Sontakke wrote: > > Is there any other locking scenario that we need to consider? > Otherwise, are we all ok on this point being a non-issue for 2PC > logical decoding? > Yeah. I didn't find any sort of sensible situation where locking would pose issues. Un

Re: [HACKERS] HASH_CHUNK_SIZE vs malloc rounding

2017-11-23 Thread Tom Lane
Thomas Munro writes: > On Tue, Nov 29, 2016 at 6:27 AM, Tom Lane wrote: >> We could imagine providing an mmgr API function along the lines of "adjust >> this request size to the nearest thing that can be allocated efficiently". >> That would avoid the need for callers to know about aset.c overhea

Re: [HACKERS] SERIALIZABLE with parallel query

2017-11-23 Thread Haribabu Kommi
On Tue, Sep 26, 2017 at 4:41 PM, Haribabu Kommi wrote: > > > On Mon, Sep 25, 2017 at 6:57 PM, Thomas Munro < > thomas.mu...@enterprisedb.com> wrote: > >> On Mon, Sep 25, 2017 at 8:37 PM, Haribabu Kommi >> wrote: >> > After I tune the GUC to go with sequence scan, still I am not getting >> the >>

Re: Failed to delete old ReorderBuffer spilled files

2017-11-23 Thread atorikoshi
On 2017/11/24 10:57, Craig Ringer wrote: > On 24 November 2017 at 09:20, atorikoshi >> wrote: > >> >> Thanks for letting me know. >> I think I only tested running "make check" at top directory, sorry >> for my insufficient test. >> >> The test failure happened at the beginning of replication(c

Re: [HACKERS] HASH_CHUNK_SIZE vs malloc rounding

2017-11-23 Thread Thomas Munro
On Fri, Nov 24, 2017 at 4:54 PM, Tom Lane wrote: >> It doesn't seem that crazy to expose aset.c's overhead size to client >> code does it? Most client code wouldn't care, but things that are >> doing something closer to memory allocator work themselves like >> dense_alloc could care. It could de

Re: [HACKERS] Partition-wise aggregation/grouping

2017-11-23 Thread Ashutosh Bapat
On Thu, Nov 23, 2017 at 6:38 PM, Jeevan Chalke wrote: > > > Agree. However, there was ab existing comment in create_append_path() saying > "We don't bother with inventing a cost_append(), but just do it here", which > implies at sometime in future we may need it; so why not now where we are > expl

Re: [HACKERS] path toward faster partition pruning

2017-11-23 Thread Amit Langote
Thanks Jesper. On 2017/11/23 3:56, Jesper Pedersen wrote: > Hi Amit, > > On 11/22/2017 03:59 AM, Amit Langote wrote: >> Fixed in the attached.  No other changes beside that. >> > > I have been using the following script to look at the patch > > -- test.sql -- [ ... ] > EXPLAIN (ANALYZE) SELEC

Re: [HACKERS] UPDATE of partition key

2017-11-23 Thread Amit Langote
On 2017/11/23 21:57, Amit Khandekar wrote: > If we collect the partition keys in expand_partitioned_rtentry(), we > need to pass the root relation also, so that we can convert the > partition key attributes to root rel descriptor. And the other thing > is, may be, we can check beforehand (in expand

Re: Failed to delete old ReorderBuffer spilled files

2017-11-23 Thread Craig Ringer
On 24 November 2017 at 12:38, atorikoshi wrote: > > > On 2017/11/24 10:57, Craig Ringer wrote: > > On 24 November 2017 at 09:20, atorikoshi co.jp > >> wrote: > > > >> > >> Thanks for letting me know. > >> I think I only tested running "make check" at top directory, sorry > >> for my insufficient

Re: [HACKERS] logical decoding of two-phase transactions

2017-11-23 Thread Nikhil Sontakke
Hi Craig, > I didn't find any sort of sensible situation where locking would pose > issues. Unless you're taking explicit LOCKs on catalog tables, you should be > fine. > > There may be issues with CLUSTER or VACUUM FULL of non-relmapped catalog > relations I guess. Personally I put that in the "d

Re: [HACKERS] logical decoding of two-phase transactions

2017-11-23 Thread Craig Ringer
On 24 November 2017 at 13:44, Nikhil Sontakke wrote: > > > This could create an inconsistent view of the catalogs if our prepared > txn > > did any DDL. For example, we might've updated a pg_class row, so we > created > > a new row and set xmax on the old one. Vacuum may merrily remove our new >

Re: How is the PostgreSQL debuginfo file generated

2017-11-23 Thread 高增琦
A little trick: 1. configure with —enable-debug, run “make install”, use this build result as debuginfo 2. then run “make install-strip”, use this result as release However the it is not the regular debuginfo, you still can call gdb with it. Additionally, you can dump the real debuginfo from it