Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Noah Misch
On Sat, Aug 08, 2015 at 02:30:47AM +0200, Andres Freund wrote: > On 2015-08-07 20:16:20 -0400, Noah Misch wrote: > > I agree that lock.h offers little to frontend code. Headers that the > > lockdefs.h patch made usable in the frontend, particularly genam.h and > > hash.h, > > are no better. > >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-08-07 Thread Amit Kapila
On Tue, Aug 4, 2015 at 8:45 AM, Amit Kapila wrote: > > On Mon, Aug 3, 2015 at 7:44 PM, Fujii Masao wrote: > > > BTW, while reading the code related to tablespace_map, I found that > > CancelBackup() emits the WARNING message "online backup mode was not canceled" > > when rename() fails. Isn't thi

Re: [HACKERS] [DESIGN] ParallelAppend

2015-08-07 Thread Amit Kapila
On Fri, Aug 7, 2015 at 2:15 PM, Kouhei Kaigai wrote: > > > On Sat, Aug 1, 2015 at 6:39 PM, Kouhei Kaigai wrote: > > > > > > > > > If we pull Funnel here, I think the plan shall be as follows: > > > > > Funnel > > > > >--> SeqScan on rel1 > > > > >--> PartialSeqScan on rel2 > > > > >

Re: [HACKERS] Bug? Small samples in TABLESAMPLE SYSTEM returns zero rows

2015-08-07 Thread Josh Berkus
Petr, Just user-tested SYSTEM_ROWS and SYSTEM_TIME. They work as expected. Useful! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Tom Lane
Andres Freund writes: > On 2015-08-07 20:16:20 -0400, Noah Misch wrote: >> On another note, I'm perplexed by the high speed commits from this thread. >> Commit de6fd1c landed just 191 minutes after you posted the first version of >> its patch. Now lockdefs.h is committed, right in the middle of d

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 20:16:20 -0400, Noah Misch wrote: > I agree that lock.h offers little to frontend code. Headers that the > lockdefs.h patch made usable in the frontend, particularly genam.h and hash.h, > are no better. It's not that simple. Those two, and tuptoaster.h, are actually somewhat validly

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Noah Misch
On Fri, Aug 07, 2015 at 03:18:06PM +0200, Andres Freund wrote: > On 2015-08-06 23:23:43 -0400, Noah Misch wrote: > > On Thu, Aug 06, 2015 at 05:34:50PM +0200, Andres Freund wrote: > > > On 2015-08-06 12:29:15 -0300, Alvaro Herrera wrote: > > > > Andres Freund wrote: > > > > > @@ -0,0 +1,56 @@ > > >

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Andres Freund
On 2015-08-07 14:20:55 -0400, Jesper Pedersen wrote: > On 08/07/2015 02:03 PM, Andres Freund wrote: > Confirmed. > > Running w/o -P x and the problem goes away. Pushed the fix. Thanks for pointing the problem out! - Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] tap tests remove working directories

2015-08-07 Thread Andrew Dunstan
On 08/07/2015 05:11 PM, Tom Lane wrote: Andrew Dunstan writes: One of the things that makes the TAP tests very difficult and annoying to debug is their insistence on removing their data directories. I'm not sure why they are doing that. We don't do that with pg_regress. Instead we have clean t

Re: [HACKERS] WIP: SCRAM authentication

2015-08-07 Thread Michael Paquier
On Sat, Aug 8, 2015 at 3:45 AM, Heikki Linnakangas wrote: > On 08/07/2015 09:26 PM, Robert Haas wrote: >> >> Maybe I'm chiming in too late here but I am sorta unimpressed by this. >> If the user's password is stored both MD5-hashed and hashed some other >> way in the system catalogs, that's less s

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Gavin Flower
On 08/08/15 06:43, Bruce Momjian wrote: On Fri, Aug 7, 2015 at 11:53:30AM -0400, Robert Haas wrote: [...] Well, we could just throw a "Postgres 9.5 is faster" release note item in there and call it a day. ;-) Nah! Just say it is Much Shinier, I'll buy it. Unfortunately, we have to have much

Re: [HACKERS] tap tests remove working directories

2015-08-07 Thread Tom Lane
Andrew Dunstan writes: > One of the things that makes the TAP tests very difficult and annoying > to debug is their insistence on removing their data directories. I'm not > sure why they are doing that. We don't do that with pg_regress. Instead > we have clean targets to remove them if necessar

[HACKERS] tap tests remove working directories

2015-08-07 Thread Andrew Dunstan
One of the things that makes the TAP tests very difficult and annoying to debug is their insistence on removing their data directories. I'm not sure why they are doing that. We don't do that with pg_regress. Instead we have clean targets to remove them if necessary. I suggest that we either dis

Re: [HACKERS] ALTER SYSTEM and GUC_LIST_QUOTE

2015-08-07 Thread Tom Lane
Adam Brightwell writes: > While testing some behaviors with ALTER SYSTEM I discovered that GUC > parameters with the GUC_LIST_QUOTE flag have a potential issue. > As an example, > ALTER SYSTEM SET shared_preload_libraries = ''; > Results in the following output in postgresql.auto.conf: > shared_p

[HACKERS] ALTER SYSTEM and GUC_LIST_QUOTE

2015-08-07 Thread Adam Brightwell
All, While testing some behaviors with ALTER SYSTEM I discovered that GUC parameters with the GUC_LIST_QUOTE flag have a potential issue. As an example, ALTER SYSTEM SET shared_preload_libraries = ''; Results in the following output in postgresql.auto.conf: shared_preload_libraries = '""'; Th

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Peter Geoghegan
On Fri, Aug 7, 2015 at 12:02 PM, Andres Freund wrote: > Based on my experience one of the prime reason people move to a new > version of postgres is performance. And it's not like 'faster!!!' is > really helpful for them to evaluate the benefits appropriately. A > scalability improvement isn't goi

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Alvaro Herrera
Andres Freund wrote: > You re-added the #ifndef FRONTEND there in a9baeb361d635 referencing a > buildfarm failure. It seems to originally have been added in > 7507b193bc54 referencing buildfarm member warthog which unfortunately > doesn't exist anymore... FWIW we make a point of not reusing build

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 14:48:43 -0400, Tom Lane wrote: > Indeed, but the buildfarm results say that the set of such platforms is > nearly empty. It's very likely that a lot of third-party authors won't > ever care if their code doesn't build on such platforms; certainly not > nearly as much as they'll care

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Andres Freund
On 2015-08-07 14:43:11 -0400, Bruce Momjian wrote: > Well, we could just throw a "Postgres 9.5 is faster" release note item > in there and call it a day. ;-) Based on my experience one of the prime reason people move to a new version of postgres is performance. And it's not like 'faster!!!' is re

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Tom Lane
Andres Freund writes: > On 2015-08-07 14:32:35 -0400, Tom Lane wrote: >> Eventually I think we're going to have to spend some effort on making a >> clearer separation between "front end safe" and "not front end safe" >> header files. Until we do that, though, adding these #error directives >> may

Re: [HACKERS] WIP: SCRAM authentication

2015-08-07 Thread Heikki Linnakangas
On 08/07/2015 09:26 PM, Robert Haas wrote: Maybe I'm chiming in too late here but I am sorta unimpressed by this. If the user's password is stored both MD5-hashed and hashed some other way in the system catalogs, that's less secure than storing it in the least secure of those ways. And I'm afrai

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Bruce Momjian
On Fri, Aug 7, 2015 at 11:53:30AM -0400, Robert Haas wrote: > On Thu, Aug 6, 2015 at 10:24 PM, Bruce Momjian wrote: > >> * 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping > >> partition.. > >> should we mention this? This has been patched by a number of > >> downstream

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 14:32:35 -0400, Tom Lane wrote: > Robert Haas writes: > > Well, I just work here, but it seems silly to me to reorgnize the > > headers so that you can include fewer definitions where necessary, but > > then not revise the existing headers to use the slimmed-down versions > > where p

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Tom Lane
Robert Haas writes: > Well, I just work here, but it seems silly to me to reorgnize the > headers so that you can include fewer definitions where necessary, but > then not revise the existing headers to use the slimmed-down versions > where possible. Yeah, somebody might have to adjust their #inc

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Robert Haas
On Fri, Aug 7, 2015 at 2:27 PM, Andres Freund wrote: > On 2015-08-07 14:15:58 -0400, Robert Haas wrote: >> On Fri, Aug 7, 2015 at 1:11 PM, Andres Freund wrote: >> > On 2015-08-07 12:30:04 -0400, Robert Haas wrote: >> >> It may not be included from any IN CORE frontend code, but that is not >> >>

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 14:15:58 -0400, Robert Haas wrote: > On Fri, Aug 7, 2015 at 1:11 PM, Andres Freund wrote: > > On 2015-08-07 12:30:04 -0400, Robert Haas wrote: > >> It may not be included from any IN CORE frontend code, but that is not > >> the same thing as saying it's not included from any frontend

Re: [HACKERS] WIP: SCRAM authentication

2015-08-07 Thread Robert Haas
On Fri, Aug 7, 2015 at 3:22 AM, Michael Paquier wrote: > On Tue, Aug 4, 2015 at 4:20 PM, Michael Paquier wrote: >> I have been looking more in depths at this one, which adds essential >> infrastructure to support multiple authentication hashes for more protocols. >> Here are some comments: >> [spe

Re: [HACKERS] All-zero page in GIN index causes assertion failure

2015-08-07 Thread Alvaro Herrera
-- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c index 0b257d9..909e4c6 100644 --- a/src/backend/access/brin/brin_pageops

Re: [HACKERS] All-zero page in GIN index causes assertion failure

2015-08-07 Thread Alvaro Herrera
Heikki Linnakangas wrote: > BRIN update is not quite right, however. brin_getinsertbuffer() can > initialize a page, but the caller might bail out without using the page and > WAL-logging the change. If that happens, the next update that uses the same > page will WAL-log the change but it will not

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Jesper Pedersen
On 08/07/2015 02:03 PM, Andres Freund wrote: but you will have to use a 9.5 pgbench to see it, especially with higher client counts. Hm, you were using -P X, is that right? This bisects down to 1bc90f7a7b7441a88e2c6d4a0e9b6f9c1499ad30 - "Remove thread-emulation support from pgbench." And th

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Robert Haas
On Fri, Aug 7, 2015 at 1:11 PM, Andres Freund wrote: > On 2015-08-07 12:30:04 -0400, Robert Haas wrote: >> It may not be included from any IN CORE frontend code, but that is not >> the same thing as saying it's not included from any frontend code at >> all. For example, EDB has code that includes

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Andres Freund
Hi, On 2015-08-07 19:30:46 +0200, Andres Freund wrote: > On 2015-08-07 12:49:20 -0400, Jesper Pedersen wrote: > > No, this patch helps on performance - there is an improvement in numbers > > between > > > > http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=253de7e1eb9abbcf57e6c229a8a3

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Andres Freund
On 2015-08-07 12:49:20 -0400, Jesper Pedersen wrote: > On 08/07/2015 11:40 AM, Robert Haas wrote: > >On Fri, Aug 7, 2015 at 10:30 AM, Jesper Pedersen > > wrote: > >>Just thought I would post it in this thread, because this change does help > >>on the performance numbers compared to 9.5 :) > > > >So

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 19:11:52 +0200, Andres Freund wrote: > I think the likelihood of actually breaking correct working extension > code that uses namespace.h that'd be broken if we removed lock.h from > namespace.h is an order of magnitude bigger than the possible impact on > frontend code. Same with dr

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 12:30:04 -0400, Robert Haas wrote: > It may not be included from any IN CORE frontend code, but that is not > the same thing as saying it's not included from any frontend code at > all. For example, EDB has code that includes namespace.h in frontend > code. That compiled before this

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Jesper Pedersen
On 08/07/2015 11:40 AM, Robert Haas wrote: On Fri, Aug 7, 2015 at 10:30 AM, Jesper Pedersen wrote: Just thought I would post it in this thread, because this change does help on the performance numbers compared to 9.5 :) So are you saying that the performance was already worse before this patc

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Robert Haas
On Thu, Aug 6, 2015 at 11:34 AM, Andres Freund wrote: > On 2015-08-06 12:29:15 -0300, Alvaro Herrera wrote: >> Andres Freund wrote: >> >> > I had to split of three things: LOCKMASK, the individual lock levels and >> > xl_standby_lock to be able to prohibit lock.h to be included by frontend >> > co

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Andres Freund
On 2015-08-07 11:53:30 -0400, Robert Haas wrote: > On Thu, Aug 6, 2015 at 10:24 PM, Bruce Momjian wrote: > >> * 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping > >> partition.. > >> should we mention this? This has been patched by a number of > >> downstream vendors and

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Robert Haas
On Thu, Aug 6, 2015 at 10:24 PM, Bruce Momjian wrote: >> * 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping >> partition.. >> should we mention this? This has been patched by a number of >> downstream vendors and users, so it's probably worth calling out? > > Uh, I am not

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Noah Misch
On Fri, Aug 07, 2015 at 03:20:00PM +0200, Andres Freund wrote: > Unless you have a better idea I'll now move the detection of that case > to aix.h. Nope, that seemed right to me. > I rather liked being able to emit a warning about disabling inlines > *once* during configuration, but it's also pro

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Robert Haas
On Fri, Aug 7, 2015 at 10:30 AM, Jesper Pedersen wrote: > Just thought I would post it in this thread, because this change does help > on the performance numbers compared to 9.5 :) So are you saying that the performance was already worse before this patch landed, and then this patch made it somew

Re: [HACKERS] Autonomous Transaction is back

2015-08-07 Thread Robert Haas
On Thu, Aug 6, 2015 at 11:04 PM, Merlin Moncure wrote: > I don't necessarily disagree with what you're saying, but it's not > clear to me what the proposed behavior is. Since the AT can commit > before the outer, ISTM *any* ungranted lock requested by the AT but > held by the outer leads to eithe

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Andres Freund
On 2015-08-07 20:17:28 +0530, Amit Kapila wrote: > On Fri, Aug 7, 2015 at 8:00 PM, Jesper Pedersen > wrote: > > > On 08/07/2015 12:41 AM, Amit Kapila wrote: > > > >> On Thu, Aug 6, 2015 at 9:36 PM, Robert Haas > >> wrote: > >> > >>> > >>> > >>> OK, committed. > >>> > >>> > >> Thank you. > >> > >

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Jesper Pedersen
On 08/07/2015 10:47 AM, Amit Kapila wrote: Fyi, there is something in pgbench that has caused a testing regression - havn't tracked down what yet. Against 9.6 server (846f8c9483a8f31e45bf949db1721706a2765771) 9.6 pgbench: progress: 10.0 s, 53525.0 tps, lat 1.485 ms stddev 0.523 pro

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Amit Kapila
On Fri, Aug 7, 2015 at 8:00 PM, Jesper Pedersen wrote: > On 08/07/2015 12:41 AM, Amit Kapila wrote: > >> On Thu, Aug 6, 2015 at 9:36 PM, Robert Haas >> wrote: >> >>> >>> >>> OK, committed. >>> >>> >> Thank you. >> >> > Fyi, there is something in pgbench that has caused a testing regression - > h

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Jesper Pedersen
On 08/07/2015 12:41 AM, Amit Kapila wrote: On Thu, Aug 6, 2015 at 9:36 PM, Robert Haas wrote: OK, committed. Thank you. Fyi, there is something in pgbench that has caused a testing regression - havn't tracked down what yet. Against 9.6 server (846f8c9483a8f31e45bf949db1721706a2765771

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Alvaro Herrera
Andres Freund wrote: > > lock.h includes lwlock.h only for the benefit of the three > > LockHashPartition* > > macros. Those macros are pieces of the lock.c implementation that cross > > into > > proc.c, not pieces of the lock.c public API. > > I argued that way as well upthread. But I do thin

Re: [HACKERS] Race conditions in shm_mq.c

2015-08-07 Thread Robert Haas
On Thu, Aug 6, 2015 at 5:59 PM, Antonin Houska wrote: > Robert Haas wrote: >> On Thu, Aug 6, 2015 at 2:38 PM, Robert Haas wrote: >> > On Thu, Aug 6, 2015 at 10:10 AM, Antonin Houska wrote: >> >> During my experiments with parallel workers I sometimes saw the "master" >> >> and >> >> worker pro

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-05 21:39:26 -0400, Noah Misch wrote: > On Wed, Aug 05, 2015 at 10:32:48AM -0400, Tom Lane wrote: > > Andres Freund writes: > > > Wasn't the point that 32 bit AIX as a whole hasn't been supported for a > > > couple years now? My willingness to expend effort for that is rather > > > limit

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-06 23:23:43 -0400, Noah Misch wrote: > On Thu, Aug 06, 2015 at 05:34:50PM +0200, Andres Freund wrote: > > On 2015-08-06 12:29:15 -0300, Alvaro Herrera wrote: > > > Andres Freund wrote: > > > > @@ -0,0 +1,56 @@ > > > > +/*---

Re: [HACKERS] [DESIGN] ParallelAppend

2015-08-07 Thread Kouhei Kaigai
> On Sat, Aug 1, 2015 at 6:39 PM, Kouhei Kaigai wrote: > > > > > On Tue, Jul 28, 2015 at 6:08 PM, Kouhei Kaigai > > > wrote: > > > > > > I am not sure, but what problem do you see in putting Funnel node > > > for one of the relation scans and not for the others. > > > > > At this moment, I'm not

[HACKERS] [sqlsmith] ERROR: too late to create a new PlaceHolderInfo

2015-08-07 Thread Andreas Seltenreich
Hi, on master at 36d9345, the attached queries raised "too late to create a new PlaceHolderInfo". regards, Andreas select subq_218206.c0 as c0 from (select rel_1375794.sl_name as c0, coalesce(rel_1375793.f2, rel_1375793.f2) as c1 from public.subselect_tbl

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-08-07 Thread Kouhei Kaigai
> I could have a discussion with Fujita-san about this topic. > Also, let me share with the discussion towards entire solution. The primitive reason of this problem is, Scan node with scanrelid==0 represents a relation join that can involve multiple relations, thus, its TupleDesc of the records wi