Re: [HACKERS] [WIP] showing index maintenance on EXPLAIN

2014-05-08 Thread Craig Ringer
On 05/08/2014 08:04 AM, Jaime Casanova wrote: > Maybe is not the right term... i mean: the time that take to update > indexes on an INSERT/UPDATE operation That'd be a big plus IMO, especially for expensive GiST or GIN index updates. -- Craig Ringer http://www.2ndQuadrant.com/

Re: [HACKERS] default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-05-08 Thread Peter Geoghegan
On Thu, May 8, 2014 at 3:39 PM, Tom Lane wrote: > Barring objections I'll commit this tomorrow Looks good to me. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-05-08 Thread Gavin Flower
On 09/05/14 15:34, Bruce Momjian wrote: On Thu, May 8, 2014 at 06:39:11PM -0400, Tom Lane wrote: I wrote: I think the idea of hashing only keys/values that are "too long" is a reasonable compromise. I've not finished coding it (because I keep getting distracted by other problems in the code :

Re: [HACKERS] postgresql.auto.conf read from wrong directory

2014-05-08 Thread Amit Kapila
On Thu, May 8, 2014 at 9:47 PM, Christoph Berg wrote: > Re: Andres Freund 2014-05-08 <20140508145901.gb1...@awork2.anarazel.de> >> > Maybe this is nitpicking, but what happens when postgresql.auto.conf also >> > includes the setting of data_directory? This is possible because we can >> > set data_

Re: [HACKERS] [WIP] showing index maintenance on EXPLAIN

2014-05-08 Thread David G Johnston
Robert Haas wrote > On Thu, May 8, 2014 at 2:31 AM, Jaime Casanova < > jaime@ > > wrote: >> On Wed, May 7, 2014 at 10:52 PM, Amit Kapila < > amit.kapila16@ > > wrote: >>> On Thu, May 8, 2014 at 5:30 AM, Jaime Casanova < > jaime@ > > wrote: >> >>QUERY PLAN >> --

Re: [HACKERS] [WIP] showing index maintenance on EXPLAIN

2014-05-08 Thread Amit Kapila
On Thu, May 8, 2014 at 12:01 PM, Jaime Casanova wrote: > On Wed, May 7, 2014 at 10:52 PM, Amit Kapila wrote: >> >> Why to capture only for Index Insert/Update and not for Read; is it >> because Read will be always fast ot implementation complexity? >> > > EXPLAIN ANALYZE already shows that on any

Re: [HACKERS] default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-05-08 Thread Bruce Momjian
On Thu, May 8, 2014 at 06:39:11PM -0400, Tom Lane wrote: > I wrote: > > I think the idea of hashing only keys/values that are "too long" is a > > reasonable compromise. I've not finished coding it (because I keep > > getting distracted by other problems in the code :-() but it does not > > look t

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > I am not sure what your point is here. Here's mine: if we can strip > this down to the executor support plus the most minimal planner > support possible, we might be able to get *something* committed. Then > we can extend it in subsequent commits. I

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Robert Haas
On Thu, May 8, 2014 at 10:16 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> Well, I consider that somewhat good news, because I think it would be >> rather nice if we could get by with solving one problem at a time, and >> if the executor part is close to being well-sol

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote: > > So the extension will need to be aware of all custom data types and then > > installed *after* all other extensions are installed? That doesn't strike > > me as workable... > > > I'm not certain why do you think an extension will need to support a

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Peter Geoghegan
On Thu, May 8, 2014 at 7:13 PM, Stephen Frost wrote: > Of course, things will change when we are able to parallelize joins > across multiple CPUs ourselves.. In a way, the PGStrom approach gets to > "cheat" us today, since it can parallelize the work where core can't and > that ends up not being

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Kouhei Kaigai
> * Kouhei Kaigai (kai...@ak.jp.nec.com) wrote: > > > I didn't ask this before but it's been on my mind for a while- how > > > will this work for custom data types, ala the 'geometry' type from > PostGIS? > > > There's user-provided code that we have to execute to check equality > > > for those, bu

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote: > > I didn't ask this before but it's been on my mind for a while- how will > > this work for custom data types, ala the 'geometry' type from PostGIS? > > There's user-provided code that we have to execute to check equality for > > those, but they're no

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote: > GPU has workloads likes and dislikes. It is a reasonable idea to list up > operators (or something else) that have advantage to run on custom-path. > For example, numeric calculation on fixed-length variables has greate > advantage on GPU, but locale

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Kouhei Kaigai
> * Kouhei Kaigai (kai...@ak.jp.nec.com) wrote: > > I initially intended to allow extensions to add their custom-path > > based on their arbitrary decision, because the core backend cannot > > have expectation towards the behavior of custom-plan. > > However, of course, the custom-path that replace

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Kouhei Kaigai (kai...@ak.jp.nec.com) wrote: > I initially intended to allow extensions to add their custom-path based > on their arbitrary decision, because the core backend cannot have > expectation towards the behavior of custom-plan. > However, of course, the custom-path that replaces built-in

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > Well, I consider that somewhat good news, because I think it would be > rather nice if we could get by with solving one problem at a time, and > if the executor part is close to being well-solved, excellent. Sadly, I'm afraid the news really isn't all

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: > On Thu, May 8, 2014 at 6:34 AM, Kouhei Kaigai wrote: > > Umm... I'm now missing the direction towards my goal. > > What approach is the best way to glue PostgreSQL and PGStrom? > > I haven't really paid any attention to PGStrom. Perhaps it's just that

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Kouhei Kaigai
> > So it seems reasonable to have a way to define/declare what is > > possible and what is not. But my take is that adding a new column to > > pg_operator for every CustomJoin node is probably out of the question, > > hence my suggestion to list the operators we know it can work with. > > It does

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: > On 8 May 2014 20:40, Robert Haas wrote: > > For my money, we'd be better off > > getting some kind of basic custom scan node functionality committed > > first, even if the cases where you can actually inject them into real > > plans are highly restric

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Kouhei Kaigai
> On Thu, May 8, 2014 at 6:34 AM, Kouhei Kaigai wrote: > > Umm... I'm now missing the direction towards my goal. > > What approach is the best way to glue PostgreSQL and PGStrom? > > I haven't really paid any attention to PGStrom. Perhaps it's just that I > missed it, but I would find it useful i

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Kouhei Kaigai
> On Thu, May 8, 2014 at 4:43 PM, Tom Lane wrote: > > I thought that the executor side of his patch wasn't in bad shape. > > The real problems were in the planner, and indeed largely in the "backend" > > part of the planner where there's a lot of hard-wired logic for fixing > > up low-level detail

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Kouhei Kaigai
> > I also think that there are really two separate problems here: getting > > the executor to call a custom scan node when it shows up in the plan > > tree; and figuring out how to get it into the plan tree in the first > > place. I'm not sure we've properly separated those problems, and I'm > >

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Robert Haas
On Thu, May 8, 2014 at 4:43 PM, Tom Lane wrote: > I thought that the executor side of his patch wasn't in bad shape. The > real problems were in the planner, and indeed largely in the "backend" > part of the planner where there's a lot of hard-wired logic for fixing up > low-level details of the

Re: [HACKERS] 9.4 release notes

2014-05-08 Thread Michael Paquier
On Fri, May 9, 2014 at 2:50 AM, MauMau wrote: > From: "Bruce Momjian" >> >> I have completed the initial version of the 9.4 release notes. You can >> view them here: >> >> http://www.postgresql.org/docs/devel/static/release-9-4.html >> >> Feedback expected and welcomed. I expect to be modifying

[HACKERS] Re: popen and pclose redefinitions causing many warning in Windows build

2014-05-08 Thread Noah Misch
On Thu, May 08, 2014 at 12:14:44PM -0400, Tom Lane wrote: > Andrew Dunstan writes: > > I'm pretty sure we need this on Mingw - this SYSTEMQUOTE stuff dates > > back well before 8.3, IIRC, which is when we first got full MSVC support. > > I tried googling for some info on this, and got a number o

Re: [HACKERS] Compilation errors with mingw build caused by undefined optreset

2014-05-08 Thread Michael Paquier
On Fri, May 9, 2014 at 12:30 AM, Tom Lane wrote: > Michael Paquier writes: >> Since commit 60ff2fd introducing the centralizated getopt-related >> things in a global header file, build on Windows with mingw is failing > > Hm, buildfarm member narwhal doesn't seem to be having any such problem. >

Re: [HACKERS] Compilation errors with mingw build caused by undefined optreset

2014-05-08 Thread Michael Paquier
On Fri, May 9, 2014 at 12:55 AM, Andrew Dunstan wrote: > On 05/08/2014 11:30 AM, Tom Lane wrote: >> Michael Paquier writes: >>> >>> Since commit 60ff2fd introducing the centralizated getopt-related >>> things in a global header file, build on Windows with mingw is failing >> >> Hm, buildfarm memb

Re: [HACKERS] Issue with GRANT/COMMENT ON FUNCTION with default

2014-05-08 Thread Jim Nasby
On 5/5/14, 3:22 PM, Alvaro Herrera wrote: > Jim Nasby wrote: >> Prior to default parameters on functions, GRANT and COMMENT accepted full >> parameter syntax. IE: >> >> GRANT EXECUTE ON test(t text) TO public >> >> as opposed to regprocedure, which only accepts the data types ( test(text),

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-08 Thread Tomas Vondra
On 8.5.2014 23:48, Andrew Dunstan wrote: > > On 05/08/2014 05:21 PM, Alvaro Herrera wrote: >> Andrew Dunstan wrote: >> >>> I really don't get what your objection to the setup is. And no, I >>> don't want them to run concurrently, I'd rather spread out the >>> cycles. >> I wasn't objecting, merely

Re: [HACKERS] PQputCopyEnd doesn't adhere to its API contract

2014-05-08 Thread David G Johnston
Tom Lane-2 wrote > Robert Haas < > robertmhaas@ > > writes: >> On Thu, May 8, 2014 at 1:51 PM, Tom Lane < > tgl@.pa > > wrote: >>> Robert Haas < > robertmhaas@ > > writes: OK. It still seems to me that there's a doc fix needed here, if nothing else. The documentation for PQputCopy

Re: [HACKERS] default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-05-08 Thread Tom Lane
I wrote: > I think the idea of hashing only keys/values that are "too long" is a > reasonable compromise. I've not finished coding it (because I keep > getting distracted by other problems in the code :-() but it does not > look to be very difficult. I'm envisioning the cutoff as being something

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-08 Thread Andrew Dunstan
On 05/08/2014 05:21 PM, Alvaro Herrera wrote: Andrew Dunstan wrote: I really don't get what your objection to the setup is. And no, I don't want them to run concurrently, I'd rather spread out the cycles. I wasn't objecting, merely an observation. Note that Tomas mentioned he's okay with run

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Peter Geoghegan
On Thu, May 8, 2014 at 6:34 AM, Kouhei Kaigai wrote: > Umm... I'm now missing the direction towards my goal. > What approach is the best way to glue PostgreSQL and PGStrom? I haven't really paid any attention to PGStrom. Perhaps it's just that I missed it, but I would find it useful if you could

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-08 Thread Alvaro Herrera
Andrew Dunstan wrote: > I really don't get what your objection to the setup is. And no, I > don't want them to run concurrently, I'd rather spread out the > cycles. I wasn't objecting, merely an observation. Note that Tomas mentioned he's okay with running 4 builds at once. My main point here,

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Simon Riggs
On 8 May 2014 21:55, Tom Lane wrote: > So I'm not real sure how we move forward. Maybe something to brainstorm > about in Ottawa. I'm just about to go on away for a week, so that's probably the best place to leave (me out of) the discussion until Ottawa. I've requested some evidence this hardw

Re: [HACKERS] PQputCopyEnd doesn't adhere to its API contract

2014-05-08 Thread Tom Lane
Robert Haas writes: > On Thu, May 8, 2014 at 1:51 PM, Tom Lane wrote: >> Robert Haas writes: >>> OK. It still seems to me that there's a doc fix needed here, if >>> nothing else. The documentation for PQputCopyEnd() clearly implies >>> that if you get a return value of 1, the message is sent,

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-08 Thread Andrew Dunstan
On 05/08/2014 04:54 PM, Andrew Dunstan wrote: Why? This was actually discussed when I set this up and Tom opined that a once a day run with CLOBBER_CACHE_ALWAYS was plenty. It takes about 4 /12 hours. The rest of the time nightjar runs. friarbird runs a bit after midnight US East Coast tim

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Tom Lane
Simon Riggs writes: > We only consider merge joins if the join uses operators with oprcanmerge=true. > We only consider hash joins if the join uses operators with oprcanhash=true > So it seems reasonable to have a way to define/declare what is > possible and what is not. But my take is that addin

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Tom Lane
Simon Riggs writes: > On 8 May 2014 20:40, Robert Haas wrote: >> For my money, we'd be better off >> getting some kind of basic custom scan node functionality committed >> first, even if the cases where you can actually inject them into real >> plans are highly restricted. Then, we could later w

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-08 Thread Andrew Dunstan
On 05/08/2014 04:09 PM, Alvaro Herrera wrote: Andrew Dunstan wrote: Here is what I do on my FreeBSD VM. I have 2 animals, nightjar and friarbird. They have the same buildroot. friarbird is set up to build with CLOBBER_CACHE_ALWAYS, building just HEAD and just testing C locale; nightjar builds

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Tom Lane
Robert Haas writes: > I'm pretty skeptical about this whole line of inquiry. We've only got > three kinds of joins, and each one of them has quite a bit of bespoke > logic, and all of this code is pretty performance-sensitive on large > join nests. If there's a way to make this work for KaiGai's

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Simon Riggs
On 8 May 2014 20:10, Stephen Frost wrote: >> * A USERSET mechanism to allow users to turn it off for testing or >> otherwise, at user, database level > > If we re-implement our existing components through this ("eat our own > dogfood" as it were), I'm not sure that we'd be able to have a way to >

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-08 Thread Alvaro Herrera
Andrew Dunstan wrote: > Here is what I do on my FreeBSD VM. I have 2 animals, nightjar and > friarbird. They have the same buildroot. friarbird is set up to > build with CLOBBER_CACHE_ALWAYS, building just HEAD and just testing > C locale; nightjar builds all branches we are interested in and > te

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Simon Riggs
On 8 May 2014 20:40, Robert Haas wrote: > For my money, we'd be better off > getting some kind of basic custom scan node functionality committed > first, even if the cases where you can actually inject them into real > plans are highly restricted. Then, we could later work on adding more > ways

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-08 Thread Andrew Dunstan
On 05/08/2014 12:21 PM, Tomas Vondra wrote: On 6.5.2014 23:01, Tomas Vondra wrote: On 6.5.2014 22:24, Tom Lane wrote: Tomas Vondra writes: I recall there was a call for more animals with CLOBBER_CACHE_ALWAYS some time ago, so I went and enabled that on all three animals. Let's see how long

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Robert Haas
On Thu, May 8, 2014 at 3:10 PM, Stephen Frost wrote: > * Simon Riggs (si...@2ndquadrant.com) wrote: >> It would seem normal and natural to have >> >> * pg_joinam catalog table for "join methods" with a join method API >> Which would include some way of defining which operators/datatypes we >> cons

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: > It would seem normal and natural to have > > * pg_joinam catalog table for "join methods" with a join method API > Which would include some way of defining which operators/datatypes we > consider this for, so if PostGIS people come up with some fancy

Re: [HACKERS] PQputCopyEnd doesn't adhere to its API contract

2014-05-08 Thread Robert Haas
On Thu, May 8, 2014 at 1:51 PM, Tom Lane wrote: > Robert Haas writes: >> OK. It still seems to me that there's a doc fix needed here, if >> nothing else. The documentation for PQputCopyEnd() clearly implies >> that if you get a return value of 1, the message is sent, and that's >> just not true

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-08 Thread Alvaro Herrera
Tomas Vondra wrote: > H, with CLOBBER_CACHE_ALWAYS + CLOBBER_FREED_MEMORY the tests take > ~20h on a single branch/animal. With a single locale (e.g. "C") it would > take ~4h, but we're testing a bunch of additional czech/slovak locales. > > The tests are running in sequence (magpie->treepie-

Re: [HACKERS] 9.4 release notes

2014-05-08 Thread Peter Geoghegan
On Thu, May 8, 2014 at 10:50 AM, MauMau wrote: > Could you add the following item, client-only installation on Windows, if > it's appropriate for release note? I think that it is appropriate. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] [WIP] showing index maintenance on EXPLAIN

2014-05-08 Thread Jaime Casanova
On Thu, May 8, 2014 at 10:41 AM, Robert Haas wrote: > On Thu, May 8, 2014 at 2:31 AM, Jaime Casanova wrote: >> On Wed, May 7, 2014 at 10:52 PM, Amit Kapila wrote: >>> On Thu, May 8, 2014 at 5:30 AM, Jaime Casanova >>> wrote: Hi, This patch implements $subject only when ANALYZE a

Re: [HACKERS] PQputCopyEnd doesn't adhere to its API contract

2014-05-08 Thread Tom Lane
Robert Haas writes: > OK. It still seems to me that there's a doc fix needed here, if > nothing else. The documentation for PQputCopyEnd() clearly implies > that if you get a return value of 1, the message is sent, and that's > just not true. That's fair. regards, tom l

Re: [HACKERS] 9.4 release notes

2014-05-08 Thread MauMau
From: "Bruce Momjian" I have completed the initial version of the 9.4 release notes. You can view them here: http://www.postgresql.org/docs/devel/static/release-9-4.html Feedback expected and welcomed. I expect to be modifying this until we release 9.4 final. I have marked items where I nee

Re: [HACKERS] PQputCopyEnd doesn't adhere to its API contract

2014-05-08 Thread Robert Haas
On Thu, May 8, 2014 at 12:53 PM, Tom Lane wrote: > Robert Haas writes: >> What I'm now thinking I need to do is something like this: > >> 1. If PQputCopyEnd returns -1, error. >> 2. while ((rc = PQflush(conn)) != 0) { if (rc < 0) { error; } else { >> wait for socket to become read-ready or write-

Re: [HACKERS] popen and pclose redefinitions causing many warning in Windows build

2014-05-08 Thread Andrew Dunstan
On 05/08/2014 12:14 PM, Tom Lane wrote: Andrew Dunstan writes: I'm pretty sure we need this on Mingw - this SYSTEMQUOTE stuff dates back well before 8.3, IIRC, which is when we first got full MSVC support. I tried googling for some info on this, and got a number of hits suggesting that mingw

Re: [HACKERS] PQputCopyEnd doesn't adhere to its API contract

2014-05-08 Thread Tom Lane
Robert Haas writes: > What I'm now thinking I need to do is something like this: > 1. If PQputCopyEnd returns -1, error. > 2. while ((rc = PQflush(conn)) != 0) { if (rc < 0) { error; } else { > wait for socket to become read-ready or write-ready; } } > 3. while (PQisBusy(conn)) { wait for the soc

Re: [HACKERS] Recursive ReceiveSharedInvalidMessages not safe

2014-05-08 Thread Andres Freund
On 2014-05-05 18:50:59 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-05-05 15:41:22 -0400, Tom Lane wrote: > >> Looks all right to me. Yeah, the right shift might have undefined > >> high-order bits, but we don't care because we're storing the result > >> into an int16. > > > Doesn'

Re: [HACKERS] Recursive ReceiveSharedInvalidMessages not safe

2014-05-08 Thread Andres Freund
On 2014-05-05 15:41:22 -0400, Tom Lane wrote: > > After far, far too much confused head scratching, code reading, random > > elog()s et al I found out that this is just because of a deficiency in > > valgrind's undefinedness tracking. [...] > > Unfortunately this cannot precisely be caught by valgr

[HACKERS] [PATCH] Fix harmless access to uninitialized memory in ri_triggers.c.

2014-05-08 Thread andres
From: Andres Freund When cache invalidations arrive while ri_LoadConstraintInfo() is busy filling a new cache entry, InvalidateConstraintCacheCallBack() compares the - not yet initialized - oidHashValue field with the to-be-invalidated hash value. To fix check whether the entry is already marked

[HACKERS] A couple logical decoding fixes/patches

2014-05-08 Thread Andres Freund
Hi, Patch 01: Fix a couple of embarassing typos. Most of them harmless, but one isn't and can lead to crashing or decoding wrong data. Patch 02: Don't crash with an Assert() failure if wal_level=logical but max_replication_slots=0. Patch 03: Add valgrind suppression for writing out padding bytes

Re: [HACKERS] PQputCopyEnd doesn't adhere to its API contract

2014-05-08 Thread Robert Haas
On Thu, May 8, 2014 at 12:09 PM, Tom Lane wrote: > Robert Haas writes: >> According to the documentation for PQputCopyEnd: >>> The result is 1 if the termination data was sent, zero if it was not sent >>> because the attempt would block (this case is only possible if the >>> connection is in >>

Re: [HACKERS] Sending out a request for more buildfarm animals?

2014-05-08 Thread Tomas Vondra
On 6.5.2014 23:01, Tomas Vondra wrote: > On 6.5.2014 22:24, Tom Lane wrote: >> Tomas Vondra writes: >>> I recall there was a call for more animals with CLOBBER_CACHE_ALWAYS >>> some time ago, so I went and enabled that on all three animals. Let's >>> see how long that will take. >> >>> I see there

Re: [HACKERS] postgresql.auto.conf read from wrong directory

2014-05-08 Thread Christoph Berg
Re: Andres Freund 2014-05-08 <20140508145901.gb1...@awork2.anarazel.de> > > Maybe this is nitpicking, but what happens when postgresql.auto.conf also > > includes the setting of data_directory? This is possible because we can > > set data_directory via ALTER SYSTEM now. Should we just ignore such >

Re: [HACKERS] popen and pclose redefinitions causing many warning in Windows build

2014-05-08 Thread Tom Lane
Andrew Dunstan writes: > I'm pretty sure we need this on Mingw - this SYSTEMQUOTE stuff dates > back well before 8.3, IIRC, which is when we first got full MSVC support. I tried googling for some info on this, and got a number of hits suggesting that mingw didn't emulate popen at all till pretty

Re: [HACKERS] PQputCopyEnd doesn't adhere to its API contract

2014-05-08 Thread Tom Lane
Robert Haas writes: > According to the documentation for PQputCopyEnd: >> The result is 1 if the termination data was sent, zero if it was not sent >> because the attempt would block (this case is only possible if the >> connection is in >> nonblocking mode), or -1 if an error occurred. (Use PQe

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Simon Riggs
On 7 May 2014 02:05, Kouhei Kaigai wrote: > (1) DDL support and system catalog > > Simon suggested that DDL command should be supported to track custom- > plan providers being installed, and to avoid nonsense hook calls > if it is an obvious case that custom-plan provider can help. It also > make

Re: [HACKERS] Compilation errors with mingw build caused by undefined optreset

2014-05-08 Thread Andrew Dunstan
On 05/08/2014 11:30 AM, Tom Lane wrote: Michael Paquier writes: Since commit 60ff2fd introducing the centralizated getopt-related things in a global header file, build on Windows with mingw is failing Hm, buildfarm member narwhal doesn't seem to be having any such problem. (It's got its own i

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Simon Riggs
On 8 May 2014 15:25, Stephen Frost wrote: > * Simon Riggs (si...@2ndquadrant.com) wrote: >> On 8 May 2014 14:40, Stephen Frost wrote: >> > Allow me to re-state your suggestion here: >> > >> > An FDW is loaded which provides hook for join push-down (whatever those >> > end up being). >> > >> > A q

Re: [HACKERS] [WIP] showing index maintenance on EXPLAIN

2014-05-08 Thread Robert Haas
On Thu, May 8, 2014 at 2:31 AM, Jaime Casanova wrote: > On Wed, May 7, 2014 at 10:52 PM, Amit Kapila wrote: >> On Thu, May 8, 2014 at 5:30 AM, Jaime Casanova wrote: >>> Hi, >>> >>> This patch implements $subject only when ANALYZE and VERBOSE are on. >>> I made it that way because for years nobod

Re: [HACKERS] popen and pclose redefinitions causing many warning in Windows build

2014-05-08 Thread Andrew Dunstan
On 05/08/2014 11:19 AM, Tom Lane wrote: Heikki Linnakangas writes: On 05/08/2014 08:01 AM, Michael Paquier wrote: Since commit a692ee5, code compilation on windows is full of warnings caused by the re-definitions of popen and pclose: Hmm. Does the MinGW version of popen() and system() do the

Re: [HACKERS] Compilation errors with mingw build caused by undefined optreset

2014-05-08 Thread Tom Lane
Michael Paquier writes: > Since commit 60ff2fd introducing the centralizated getopt-related > things in a global header file, build on Windows with mingw is failing Hm, buildfarm member narwhal doesn't seem to be having any such problem. (It's got its own issues, but not this.) Do you think you

Re: [HACKERS] popen and pclose redefinitions causing many warning in Windows build

2014-05-08 Thread Tom Lane
Heikki Linnakangas writes: > On 05/08/2014 08:01 AM, Michael Paquier wrote: >> Since commit a692ee5, code compilation on windows is full of warnings >> caused by the re-definitions of popen and pclose: > Hmm. Does the MinGW version of popen() and system() do the quoting for > you? If we just #if

[HACKERS] PQputCopyEnd doesn't adhere to its API contract

2014-05-08 Thread Robert Haas
According to the documentation for PQputCopyEnd: > The result is 1 if the termination data was sent, zero if it was not sent > because the attempt would block (this case is only possible if the connection > is in > nonblocking mode), or -1 if an error occurred. (Use PQerrorMessage to > retrieve

Re: [HACKERS] postgresql.auto.conf read from wrong directory

2014-05-08 Thread Andres Freund
On 2014-05-08 22:21:43 +0900, Fujii Masao wrote: > On Wed, May 7, 2014 at 4:57 PM, Amit Kapila wrote: > > On Tue, May 6, 2014 at 7:04 PM, Christoph Berg wrote: > >> Hi, > >> > >> if you split configuration and data by setting data_directory, > >> postgresql.auto.conf is writen to the data directo

Re: [HACKERS] postgresql.auto.conf read from wrong directory

2014-05-08 Thread Stephen Frost
* Amit Kapila (amit.kapil...@gmail.com) wrote: > On Thu, May 8, 2014 at 6:51 PM, Fujii Masao wrote: > > Should we just ignore such > > problematic setting in postgresql.auto.conf with warning message? > > Another way could be that we detect the same during server start > (while parsing postgresql

Re: [HACKERS] postgresql.auto.conf read from wrong directory

2014-05-08 Thread Amit Kapila
On Thu, May 8, 2014 at 6:51 PM, Fujii Masao wrote: > On Wed, May 7, 2014 at 4:57 PM, Amit Kapila wrote: >> This problem occurs because we don't have the value of data_directory >> set in postgresql.conf by the time we want to parse .auto.conf file >> during server start. The value of data_direct

Re: [HACKERS] default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-05-08 Thread Bruce Momjian
On Thu, May 8, 2014 at 04:37:05PM +0200, Andres Freund wrote: > On 2014-05-08 10:34:04 -0400, Bruce Momjian wrote: > > On Thu, May 8, 2014 at 10:16:54AM -0400, Tom Lane wrote: > > > > Can we hash just the values, not the keys, in jsonb_ops, and hash the > > > > combo in jsonb_hash_ops. That woul

Re: [HACKERS] default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-05-08 Thread Andres Freund
On 2014-05-08 10:34:04 -0400, Bruce Momjian wrote: > On Thu, May 8, 2014 at 10:16:54AM -0400, Tom Lane wrote: > > > Can we hash just the values, not the keys, in jsonb_ops, and hash the > > > combo in jsonb_hash_ops. That would give us key-only lookups without a > > > recheck. > > > > No, becaus

Re: [HACKERS] default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-05-08 Thread Bruce Momjian
On Thu, May 8, 2014 at 10:16:54AM -0400, Tom Lane wrote: > > Can we hash just the values, not the keys, in jsonb_ops, and hash the > > combo in jsonb_hash_ops. That would give us key-only lookups without a > > recheck. > > No, because there's nothing in JSON limiting the length of keys, any more

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: > On 8 May 2014 14:40, Stephen Frost wrote: > > Allow me to re-state your suggestion here: > > > > An FDW is loaded which provides hook for join push-down (whatever those > > end up being). > > > > A query is run which joins *local* table A to *local* t

Re: [HACKERS] default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-05-08 Thread Tom Lane
Bruce Momjian writes: > On Tue, May 6, 2014 at 06:20:53PM -0400, Tom Lane wrote: >> I wonder why there's not an option to store keys and values separately, >> but as hashes not as the original strings, so that indexability of >> everything could be guaranteed. Or a variant of that might be to ha

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Simon Riggs
On 8 May 2014 14:49, Stephen Frost wrote: > Your downthread comments on 'CREATE MATERIALIZED VIEW' are in the same > vein, though there I agree that we need it per-relation as there are > other trade-offs to consider (storage costs of the matview, cost to > maintain the matview, etc, similar to i

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Simon Riggs
On 8 May 2014 14:40, Stephen Frost wrote: > Allow me to re-state your suggestion here: > > An FDW is loaded which provides hook for join push-down (whatever those > end up being). > > A query is run which joins *local* table A to *local* table B. Standard > heaps, standard indexes, all local to

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
Simon, Perhaps you've changed your proposal wrt LOOKASIDES's and I've missed it somewhere in the thread, but this is what I was referring to with my concerns regarding per-relation definition of 'LOOKASIDES': * Simon Riggs (si...@2ndquadrant.com) wrote: > Roughly, I'm thinking of this... > > CRE

Re: [HACKERS] New pg_lsn type doesn't have hash/btree opclasses

2014-05-08 Thread Andres Freund
Hi, On 2014-05-06 23:55:04 -0300, Fabrízio de Royes Mello wrote: > If helps, I added some regression tests to the lastest patch. I think we should apply this patch now. It's much more sensible with the opclasses present and we don't win anything by waiting for 9.5. Greetings, Andres Freund --

Re: [HACKERS] default opclass for jsonb (was Re: Call for GIST/GIN/SP-GIST opclass documentation)

2014-05-08 Thread Bruce Momjian
On Tue, May 6, 2014 at 06:20:53PM -0400, Tom Lane wrote: > "David E. Wheeler" writes: > > On May 6, 2014, at 2:20 PM, Bruce Momjian wrote: > >> Well, then, we only have a few days to come up with a name. > > > What are the options? > > We have no proposals as yet. > > I've been looking at the

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Simon Riggs
On 8 May 2014 14:32, Stephen Frost wrote: > The API you've outlined requires users to specify on a per-relation > basis what join types are valid. No, it doesn't. I've not said or implied that at any point. If you keep telling me what I mean, rather than asking, we won't get anywhere. I think

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: > >From here, it looks exactly like pushing a join into an FDW. If we had > that, we wouldn't need Custom Scan at all. > > I may be mistaken and there is a critical difference. Local sub-plans > doesn't sound like a big difference. Erm. I'm not sure t

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Kouhei Kaigai
> On Wed, May 7, 2014 at 4:01 AM, Simon Riggs wrote: > > Agreed. My proposal is that if the planner allows the lookaside to an > > FDW then we pass the query for full execution on the FDW. That means > > that the scan, aggregate and join could take place via the FDW. i.e. > > "Custom Plan" == look

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: > On 8 May 2014 13:48, Stephen Frost wrote: > > I don't view on-GPU memory as being an alternate *permanent* data store. > > As I've said, others have expressed an interest in placing specific > data on specific external resources that we would like to

Re: [HACKERS] pg_shmem_allocations view

2014-05-08 Thread Andres Freund
On 2014-05-08 07:58:34 -0400, Robert Haas wrote: > On Wed, May 7, 2014 at 5:54 PM, Andres Freund wrote: > > Hm. Not sure what you're ACKing here ;). > > The idea of giving the unallocated memory a NULL key. Ok. A new version of the patches implementing that are attached. Including a couple of sm

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Simon Riggs
On 8 May 2014 04:33, Kouhei Kaigai wrote: > In case when it replaced join relations by ForeignScan, it will be almost > same as expected ForeignScan with join-pushed down. Unlike usual table scan, > it does not have actual relation definition on catalog, and its result > tuple-slot is determined

Re: [HACKERS] postgresql.auto.conf read from wrong directory

2014-05-08 Thread Fujii Masao
On Wed, May 7, 2014 at 4:57 PM, Amit Kapila wrote: > On Tue, May 6, 2014 at 7:04 PM, Christoph Berg wrote: >> Hi, >> >> if you split configuration and data by setting data_directory, >> postgresql.auto.conf is writen to the data directory >> (/var/lib/postgresql/9.4/main in Debian), but tried to

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Simon Riggs
On 8 May 2014 13:48, Stephen Frost wrote: >> We have multiple use cases where an alternate data structure could be >> used to speed up queries. > > I don't view on-GPU memory as being an alternate *permanent* data store. As I've said, others have expressed an interest in placing specific data on

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > As Stephen notes downthread, Tom has already expressed opposition to > this idea on other threads, and I tend to agree with him, at least to > some degree. I think the drive to use foreign data wrappers for > PGStrom, CitusDB, and other things that ar

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: > On 8 May 2014 03:36, Stephen Frost wrote: > > I'm all for making use of MatViews and GPUs, but there's more than one > > way to get there and look-asides feels like pushing the decision, > > unnecessarily, on to the user. > > I'm not sure I understan

Re: [HACKERS] [v9.5] Custom Plan API

2014-05-08 Thread Robert Haas
On Wed, May 7, 2014 at 4:01 AM, Simon Riggs wrote: > Agreed. My proposal is that if the planner allows the lookaside to an > FDW then we pass the query for full execution on the FDW. That means > that the scan, aggregate and join could take place via the FDW. i.e. > "Custom Plan" == lookaside + FD

Re: [HACKERS] PGDLLEXPORTing all GUCs?

2014-05-08 Thread Andres Freund
On 2014-05-08 07:56:46 -0400, Robert Haas wrote: > On Thu, May 8, 2014 at 12:19 AM, Craig Ringer wrote: > > In terms of ugliness, would you be happier using a pg_extern instead of > > extern, where we: > > > > #ifdef WIN32 > > #define pg_extern extern PGDLLIMPORT > > #else > > #define pg_extern ex

Re: [HACKERS] pg_shmem_allocations view

2014-05-08 Thread Robert Haas
On Wed, May 7, 2014 at 5:54 PM, Andres Freund wrote: > On 2014-05-07 17:48:15 -0400, Robert Haas wrote: >> On Tue, May 6, 2014 at 6:09 PM, Andres Freund wrote: >> >> I guess I'd vote for >> >> ditching the allocated column completely and outputting the memory >> >> allocated without ShmemIndex us

  1   2   >