Re: [HACKERS] proposal: PL/Pythonu - function ereport

2015-10-22 Thread Catalin Iacob
On Sat, Oct 17, 2015 at 8:18 PM, Pavel Stehule wrote: > here is new patch > > cleaned, all unwanted artefacts removed. I am not sure if used way for > method registration is 100% valid, but I didn't find any related > documentation. Pavel, some notes about the patch, not a full review (yet?). In

Re: [HACKERS] Parallel Seq Scan

2015-10-22 Thread Robert Haas
On Fri, Oct 23, 2015 at 12:31 AM, Tom Lane wrote: > BTW, my Salesforce colleagues have been bit^H^H^Hgriping for quite some > time about the performance costs associated with translating between > plpgsql's internal PLpgSQL_datum-array format and the ParamListInfo > representation. Maybe it's tim

Re: [HACKERS] Parallel Seq Scan

2015-10-22 Thread Robert Haas
On Tue, Oct 20, 2015 at 3:04 AM, Amit Kapila wrote: > I have rebased the partial seq scan patch based on the above committed > patches. Now for rescanning it reuses the dsm and to achieve that we > need to ensure that workers have been completely shutdown and then > reinitializes the dsm. To ens

Re: [HACKERS] Parallel Seq Scan

2015-10-22 Thread Tom Lane
Robert Haas writes: > On Thu, Oct 15, 2015 at 8:23 PM, Noah Misch wrote: >>> Would it work to define this as "if non-NULL, >>> params lacking a 1-bit may be safely ignored"? Or some other tweak >>> that basically says that you don't need to care about this, but you >>> can if you want to. >> ..

Re: [HACKERS] Parallel Seq Scan

2015-10-22 Thread Robert Haas
On Thu, Oct 15, 2015 at 8:23 PM, Noah Misch wrote: > Agreed. More specifically, I had in mind for copyParamList() to check the > mask while e.g. ExecEvalParamExtern() would either check nothing or merely > assert that any mask included the requested parameter. It would be tricky to > verify that

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen Frost wrote: > > > The issue for Debian, at least, isn't really about libreadline or > > libedit, it's about the GPL and the OpenSSL license. From the Debian > > perspective, if we were able to build with GNUTLS or another SSL library >

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread Alvaro Herrera
Stephen Frost wrote: > The issue for Debian, at least, isn't really about libreadline or > libedit, it's about the GPL and the OpenSSL license. From the Debian > perspective, if we were able to build with GNUTLS or another SSL library > other than OpenSSL (which I know we've made some progress on

Re: [HACKERS] Re: [BUGS] BUG #13611: test_postmaster_connection failed (Windows, listen_addresses = '0.0.0.0' or '::')

2015-10-22 Thread Tom Lane
Noah Misch writes: > On Thu, Oct 22, 2015 at 04:15:10PM -0700, Tom Lane wrote: >> I continue to think >> that the OP's complaint is somehow founded on a bad address obtained by >> looking up "localhost", because where else would it've come from? > pg_ctl reads the address from postmaster.pid, whi

Re: [HACKERS] shm_mq fix for non-blocking mode

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 4:45 PM, Robert Haas wrote: > On Fri, Oct 16, 2015 at 5:08 PM, Robert Haas wrote: >> The shm_mq code handles blocking mode and non-blocking mode >> asymmetrically in a couple of places, with the unfortunate result that >> if you are using non-blocking mode, and your counte

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > David Fetter writes: > > On Thu, Oct 22, 2015 at 04:05:06PM -0700, Tom Lane wrote: > >> Given the license issues around GNU readline, requiring it seems like > >> probably a non-starter. > > > I should have made this more clear. I am not an IP attorney an

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Alvaro Herrera
Andres Freund wrote: > On 2015-10-22 20:27:20 -0300, Alvaro Herrera wrote: > > You can run the new one in old pg_xlog ... > > You can? The xlog format between 9.4 and 9.5 changed, so I can't see how > that'd work? Oh, crap. Must have been some other cross-version trial run I did, then. I would

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 8:09 PM, Andres Freund wrote: > On 2015-10-22 16:26:10 -0700, David Fetter wrote: >> To be affective negatively by libreadline's viral license, an entity >> would need to fork the psql client in proprietary ways that they did >> not wish not to make available to end users,

Re: [HACKERS] COPY FREEZE and PD_ALL_VISIBLE

2015-10-22 Thread Simon Riggs
On 18 October 2015 at 17:23, Jeff Janes wrote: > I'm planning on adding a todo item to have COPY FREEZE set > PD_ALL_VISIBLE. Or is there some reason this can't be done? > > Since the whole point of COPY FREEZE is to avoid needing to rewrite the > entire table, it seems rather perverse that the

Re: [HACKERS] COPY FREEZE and PD_ALL_VISIBLE

2015-10-22 Thread Simon Riggs
On 21 October 2015 at 13:31, Jeff Janes wrote: > On Tue, Oct 20, 2015 at 7:02 AM, Robert Haas > wrote: > >> On Sun, Oct 18, 2015 at 5:23 PM, Jeff Janes wrote: >> > I'm planning on adding a todo item to have COPY FREEZE set >> PD_ALL_VISIBLE. >> > Or is there some reason this can't be done? >> >

Re: [HACKERS] Re: [BUGS] BUG #13611: test_postmaster_connection failed (Windows, listen_addresses = '0.0.0.0' or '::')

2015-10-22 Thread Noah Misch
On Thu, Oct 22, 2015 at 04:15:10PM -0700, Tom Lane wrote: > Tatsuo Ishii writes: > >> The original post used only "0.0.0.0" and "::", not "localhost" or anything > >> else entailing name resolution. As I wrote above, Kondo proposed for > >> pg_ctl > >> to use PQping("host='127.0.0.1'") in place

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread Andres Freund
On 2015-10-22 16:26:10 -0700, David Fetter wrote: > To be affective negatively by libreadline's viral license, an entity > would need to fork the psql client in proprietary ways that they did > not wish not to make available to end users, at the same time linking > in libreadline. > Maybe I'm miss

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Michael Paquier
On Fri, Oct 23, 2015 at 8:42 AM, Andres Freund wrote: > On 2015-10-22 20:27:20 -0300, Alvaro Herrera wrote: >> You can run the new one in old pg_xlog ... > > You can? The xlog format between 9.4 and 9.5 changed, so I can't see how > that'd work? That's not going to work. -- Michael -- Sent vi

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Michael Paquier
On Fri, Oct 23, 2015 at 7:37 AM, Jim Nasby wrote: > What I'm wondering is how compressible a 'normal' FPI is. Certainly if the > hole is zero'd out and the page is mostly empty you'll get great > compression. What about other workloads? For reference, if a 'FPI > placeholder' WAL record is 16 bytes

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread Tom Lane
David Fetter writes: > On Thu, Oct 22, 2015 at 04:05:06PM -0700, Tom Lane wrote: >> Given the license issues around GNU readline, requiring it seems like >> probably a non-starter. > I should have made this more clear. I am not an IP attorney and don't > play one on TV, but this is what I've got

Re: [HACKERS] Re: [BUGS] BUG #13611: test_postmaster_connection failed (Windows, listen_addresses = '0.0.0.0' or '::')

2015-10-22 Thread Tom Lane
Andres Freund writes: > Perhaps we should start to emit a notice at startup if localhost doesn't > resolve to either v4 or v6 definitions. The few environments where > that's indeed intentionally not the case, should be fine with such a > message at pgstat startup. I think there's already a compl

Re: [HACKERS] Parallel Seq Scan

2015-10-22 Thread Robert Haas
On Tue, Oct 13, 2015 at 5:59 PM, Robert Haas wrote: > - Although the changes in parallelpaths.c are in a good direction, I'm > pretty sure this is not yet up to scratch. I am less sure exactly > what needs to be fixed, so I'll have to give some more thought to > that. Please find attached a prop

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Andres Freund
On 2015-10-22 20:27:20 -0300, Alvaro Herrera wrote: > You can run the new one in old pg_xlog ... You can? The xlog format between 9.4 and 9.5 changed, so I can't see how that'd work? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Freezing without cleanup lock

2015-10-22 Thread Alvaro Herrera
Jim Nasby wrote: > That would be the minimal-impact version, yes. But I suspect if we went > through the trouble to do that, it would be just as easy to attempt the > freeze regardless of what scan_all is set to. You mean if !scan_all we conditional-get the cleanup lock, if we get it then prune,

Re: [HACKERS] Freezing without cleanup lock

2015-10-22 Thread Jim Nasby
On 10/21/15 3:14 PM, Alvaro Herrera wrote: Jim Nasby wrote: While warning a client that just did a Slony-based version upgrade to make sure to freeze the new database, it occurred to me that it should be safe to freeze without the cleanup lock. This is interesting because it would allow a scan_a

Re: [HACKERS] ATT_FOREIGN_TABLE and ATWrongRelkindError()

2015-10-22 Thread Amit Langote
On 2015/10/23 6:06, Robert Haas wrote: > On Wed, Oct 21, 2015 at 1:51 AM, Amit Langote > wrote: >> Attached adds those. > > Good catch. Committed and back-patched to 9.5. > > (Yes, one of these problems goes back to 9.3, but it's a minor issue > so I didn't bother. If someone really feels the

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Alvaro Herrera
Jim Nasby wrote: > On 10/22/15 5:53 PM, Alvaro Herrera wrote: > >Jim Nasby wrote: > > > >>But yes, this is all very hand-wavy without any actual data on what > >>percentage of the WAL stream is FPIs. Looks like pageinspect doesn't work > >>for WAL... does anyone have a script/tool that breaks out w

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread David Fetter
On Thu, Oct 22, 2015 at 04:05:06PM -0700, Tom Lane wrote: > David Fetter writes: > > On Thu, Oct 22, 2015 at 02:36:53PM -0700, Tom Lane wrote: > >> I have no very good idea how to do that, though. Bison does have a > >> notion of which symbols are possible as the next symbol at any given > >> par

Re: [HACKERS] Change behavior of (m)xid_age

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 6:57 PM, Jim Nasby wrote: > Is there no case where it can be a permanent XID for a table or toast table? I don't think so. > The other issue is relminmxid, which if you're looking at relfrozenxid you'd > want to look at as well. So you can't do a simple WHERE here. I don

Re: [HACKERS] Re: [BUGS] BUG #13611: test_postmaster_connection failed (Windows, listen_addresses = '0.0.0.0' or '::')

2015-10-22 Thread Andres Freund
On 2015-10-22 16:15:10 -0700, Tom Lane wrote: > AFAICS, the only hard-wired hostname reference in pg_ctl is "localhost", > not "127.0.0.1" (much less "0.0.0.0"). So what you're proposing doesn't > seem to me to have anything to do with what's there. I continue to think > that the OP's complaint i

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
On 10/22/15 5:53 PM, Alvaro Herrera wrote: Jim Nasby wrote: But yes, this is all very hand-wavy without any actual data on what percentage of the WAL stream is FPIs. Looks like pageinspect doesn't work for WAL... does anyone have a script/tool that breaks out what percentage of a WAL file is FP

Re: [HACKERS] Re: [BUGS] BUG #13611: test_postmaster_connection failed (Windows, listen_addresses = '0.0.0.0' or '::')

2015-10-22 Thread Tom Lane
Tatsuo Ishii writes: >> The original post used only "0.0.0.0" and "::", not "localhost" or anything >> else entailing name resolution. As I wrote above, Kondo proposed for pg_ctl >> to use PQping("host='127.0.0.1'") in place of PQping("host='0.0.0.0'"). >> That's all. pg_ctl would continue to us

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread David G. Johnston
On Thu, Oct 22, 2015 at 7:05 PM, Tom Lane wrote: > I think it's already true that the previous lines of the query buffer > are stashed somewhere that psql knows about, ​Just skimming but:​ ​""" ​\p or \print Print the current query buffer to the standard output. """ ​http://www.postgresql.org/

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread Tom Lane
David Fetter writes: > On Thu, Oct 22, 2015 at 02:36:53PM -0700, Tom Lane wrote: >> I have no very good idea how to do that, though. Bison does have a >> notion of which symbols are possible as the next symbol at any given >> parse point, but it doesn't really make that accessible. There's a lac

Re: [HACKERS] Re: [BUGS] BUG #13611: test_postmaster_connection failed (Windows, listen_addresses = '0.0.0.0' or '::')

2015-10-22 Thread Tatsuo Ishii
> The original post used only "0.0.0.0" and "::", not "localhost" or anything > else entailing name resolution. As I wrote above, Kondo proposed for pg_ctl > to use PQping("host='127.0.0.1'") in place of PQping("host='0.0.0.0'"). > That's all. pg_ctl would continue to use PQping("host='localhost'

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Andres Freund
On 2015-10-22 17:59:06 -0500, Jim Nasby wrote: > The WAL would *not* differ. It would. Hint bits and all. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
On 10/22/15 5:52 PM, Andres Freund wrote: If the receiver didn't write the WAL before processing it then it can just >stick the page image into the WAL it's writing for itself. Probably not good >for syncrep, but I don't think you'd want this on for syncrep anyway. To me this sounds like a recip

Re: [HACKERS] Change behavior of (m)xid_age

2015-10-22 Thread Jim Nasby
On 10/22/15 5:07 PM, Robert Haas wrote: On Thu, Oct 22, 2015 at 5:51 PM, Jim Nasby wrote: >It's also a permanent ID when the relation is first created. No it isn't. Is there no case where it can be a permanent XID for a table or toast table? The other issue is relminmxid, which if you're lo

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Alvaro Herrera
Jim Nasby wrote: > But yes, this is all very hand-wavy without any actual data on what > percentage of the WAL stream is FPIs. Looks like pageinspect doesn't work > for WAL... does anyone have a script/tool that breaks out what percentage of > a WAL file is FPIs? pg_xlogdump -z -- Álvaro Herrer

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Andres Freund
On 2015-10-22 17:47:01 -0500, Jim Nasby wrote: > On 10/22/15 5:11 PM, Robert Haas wrote: > >It's true that if the standby didn't have the master's FPIs, it could > >generate its own in some side location that behaves like a separate > >WAL stream or a double-write buffer. But that would be a heck

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
On 10/22/15 5:11 PM, Robert Haas wrote: It's true that if the standby didn't have the master's FPIs, it could generate its own in some side location that behaves like a separate WAL stream or a double-write buffer. But that would be a heck of a lot of work to implement for an uncertain benefit.

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
On 10/22/15 5:03 PM, Andres Freund wrote: On 2015-10-22 16:34:38 -0500, Jim Nasby wrote: ISTM it should be possible to avoid sending full page writes to a streaming replica once the replica has reached a consistent state. I assume that the replica would still need to write full pages to it's dis

Re: [HACKERS] Change behavior of (m)xid_age

2015-10-22 Thread Alvaro Herrera
Andres Freund wrote: > FWIW, adding an <> operator for xid seems like a perfectly good idea. +1 (two of them actually -- another one for <>(xid,int) which mirrors the =(xid,int) we already have). -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Pavel Stehule
2015-10-23 0:07 GMT+02:00 Jim Nasby : > On 10/22/15 4:59 PM, Pavel Stehule wrote: > >> It prevents everyone from reinventing the 'create a function wrapper >> around RAISE' wheel that several people on this list alone have >> admitted to. I think there's plenty of value in that. >> >>

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread David Fetter
On Thu, Oct 22, 2015 at 02:36:53PM -0700, Tom Lane wrote: > Robert Haas writes: > > On Wed, Oct 21, 2015 at 8:54 PM, Thomas Munro > > wrote: > >> (Apologies for sending so many versions. tab-complete.c keeps moving > >> and I want to keep a version that applies on top of master out there, > >> f

Re: [HACKERS] Change behavior of (m)xid_age

2015-10-22 Thread Andres Freund
On 2015-10-22 18:07:06 -0400, Robert Haas wrote: > > BTW, ignoring relfrozenxid = 0 also isn't as easy as you'd think: > > > > select count(*) from pg_class where relfrozenxid <> 0; > > ERROR: operator does not exist: xid <> integer at character 50 > > It takes a few more characters than that, bu

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 5:57 PM, Jim Nasby wrote: >> We could in theory send a "this would be been a fpi but it's skipped" >> record which would only exist in streaming and just make the standby >> write a noop of some kind? It would still be on the standby but it would >> at least not consume the

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Jim Nasby
On 10/22/15 4:59 PM, Pavel Stehule wrote: It prevents everyone from reinventing the 'create a function wrapper around RAISE' wheel that several people on this list alone have admitted to. I think there's plenty of value in that. I have different opinion, I am sorry. The RAISE statem

Re: [HACKERS] Change behavior of (m)xid_age

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 5:51 PM, Jim Nasby wrote: > It's also a permanent ID when the relation is first created. No it isn't. If it were, the first insert into the table would have to update the pg_class tuple, which it certainly doesn't. Before we had MVCC catalog scans, that wouldn't have bee

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Andres Freund
On 2015-10-22 16:34:38 -0500, Jim Nasby wrote: > ISTM it should be possible to avoid sending full page writes to a streaming > replica once the replica has reached a consistent state. I assume that the > replica would still need to write full pages to it's disk in case of a > crash, but the sender

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Pavel Stehule
2015-10-22 23:54 GMT+02:00 Jim Nasby : > On 10/22/15 4:42 PM, Pavel Stehule wrote: > >> the behave of pg_report_log should not be exactly same as RAISE >> statement in PLpgSQL. >> > > That makes no sense to me. Having different behaviors is just going to > lead to confusion. > > If this function w

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
On 10/22/15 4:42 PM, Magnus Hagander wrote: > How are you going to make that work without LSNs in the WAL received by > the replica diverging from those in the master's WAL? > We could in theory send a "this would be been a fpi but it's skipped" record which would only exist in streaming and

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Pavel Stehule
2015-10-22 20:15 GMT+02:00 Jim Nasby : > On 10/22/15 2:20 AM, dinesh kumar wrote: > >> >> 2. Using this function, if we provide any "NULL" argument to the function, >> we should either skip it or report it. I see this is what the function >> is doing. >> >> postgres=# SELECT pg_report_log('INFO'

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Jim Nasby
On 10/22/15 4:42 PM, Pavel Stehule wrote: the behave of pg_report_log should not be exactly same as RAISE statement in PLpgSQL. That makes no sense to me. Having different behaviors is just going to lead to confusion. If this function will be exactly same, then it lost a sense and anybody c

Re: [HACKERS] Change behavior of (m)xid_age

2015-10-22 Thread Jim Nasby
On 10/22/15 4:18 PM, Robert Haas wrote: On Wed, Oct 21, 2015 at 1:33 PM, Jim Nasby wrote: Currently, xid_age() returns INT_MAX for a permanent xid. The comment in the function that 'Permanent XIDs are always infinitely old' may be technically correct, but returning INT_MAX is a useless behavior

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread Alvaro Herrera
Tom Lane wrote: > What I would like is to find a way to auto-generate basically this entire > file from gram.y. That would imply going over to something at least > somewhat parser-based, instead of the current way that is more or less > totally ad-hoc. That would be a very good thing though, bec

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Magnus Hagander
On Oct 22, 2015 23:38, "Tom Lane" wrote: > > Jim Nasby writes: > > ISTM it should be possible to avoid sending full page writes to a > > streaming replica once the replica has reached a consistent state. I > > assume that the replica would still need to write full pages to it's > > disk in case o

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Pavel Stehule
Hi 2015-10-22 22:03 GMT+02:00 Peter Eisentraut : > On 10/22/15 3:20 AM, dinesh kumar wrote: > > postgres=# SELECT pg_report_log('INFO', 'NULL', false, NULL, NULL); > > INFO: NULL > > > > postgres=# SELECT pg_report_log('INFO', 'NULL', false, 'NULL', 'NULL'); > > INFO: NULL > > DETAIL: NULL /-

Re: [HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Tom Lane
Jim Nasby writes: > ISTM it should be possible to avoid sending full page writes to a > streaming replica once the replica has reached a consistent state. I > assume that the replica would still need to write full pages to it's > disk in case of a crash, but the sender could insert special WAL

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread Tom Lane
Robert Haas writes: > On Wed, Oct 21, 2015 at 8:54 PM, Thomas Munro > wrote: >> (Apologies for sending so many versions. tab-complete.c keeps moving >> and I want to keep a version that applies on top of master out there, >> for anyone interested in looking at this. As long as no one objects >>

[HACKERS] Avoid full page images in streaming replication?

2015-10-22 Thread Jim Nasby
ISTM it should be possible to avoid sending full page writes to a streaming replica once the replica has reached a consistent state. I assume that the replica would still need to write full pages to it's disk in case of a crash, but the sender could insert special WAL records to tell it when to

Re: [HACKERS] Making tab-complete.c easier to maintain

2015-10-22 Thread Robert Haas
On Wed, Oct 21, 2015 at 8:54 PM, Thomas Munro wrote: > Here is a new version merging the recent CREATE EXTENSION ... VERSION > patch from master. > > (Apologies for sending so many versions. tab-complete.c keeps moving > and I want to keep a version that applies on top of master out there, > for

Re: [HACKERS] Change behavior of (m)xid_age

2015-10-22 Thread Robert Haas
On Wed, Oct 21, 2015 at 1:33 PM, Jim Nasby wrote: > Currently, xid_age() returns INT_MAX for a permanent xid. The comment in the > function that 'Permanent XIDs are always infinitely old' may be technically > correct, but returning INT_MAX is a useless behavior because it actually > makes it look

Re: [HACKERS] COPY FREEZE and PD_ALL_VISIBLE

2015-10-22 Thread Robert Haas
On Wed, Oct 21, 2015 at 1:31 PM, Jeff Janes wrote: > It turns out it was pretty easy to set PD_ALL_VISIBLE on the new pages, > since the code in hio that requests the relation to be extended already has > info on the tuple's intended freeze status. > > Then you just need to refrain from clearing P

Re: [HACKERS] Getting sorted data from foreign server

2015-10-22 Thread Robert Haas
On Wed, Oct 21, 2015 at 5:23 AM, Ashutosh Bapat wrote: > Increasing the sorting cost factor (when use_remote_estimates = false) from > 1.1 to 1.2 makes the difference disappear. > > Since the startup costs for postgres_fdw are large portion of total cost, > extra 10% of rest of the cost is compara

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 5:09 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Oct 22, 2015 at 4:32 PM, Tom Lane wrote: >>> I'm not sure that you could get to a point where you were >>> generating this stuff from anything that wasn't in essence an arcane >>> representation of the .c files. I

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread Tom Lane
Robert Haas writes: > On Thu, Oct 22, 2015 at 4:32 PM, Tom Lane wrote: >> I'm not sure that you could get to a point where you were >> generating this stuff from anything that wasn't in essence an arcane >> representation of the .c files. It might be slightly harder to make >> errors of omission

Re: [HACKERS] ATT_FOREIGN_TABLE and ATWrongRelkindError()

2015-10-22 Thread Robert Haas
On Wed, Oct 21, 2015 at 1:51 AM, Amit Langote wrote: > This may just be nitpicking but I noticed that ATWrongRelkindError() could > emit a better message in case of such errors during ALTER COLUMN DEFAULT > and ALTER COLUMN SET STORAGE than "%s is of the wrong type" which is what > it would emit

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 4:32 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Oct 22, 2015 at 1:40 PM, YUriy Zhuravlev >> wrote: >>> I can gen xml/json from actual struct. I offered XML/JSON as the analysis >>> of C >>> code much more difficult. But my current generator just use the structu

Re: [HACKERS] shm_mq fix for non-blocking mode

2015-10-22 Thread Robert Haas
On Fri, Oct 16, 2015 at 5:08 PM, Robert Haas wrote: > The shm_mq code handles blocking mode and non-blocking mode > asymmetrically in a couple of places, with the unfortunate result that > if you are using non-blocking mode, and your counterparty dies before > attaching the queue, operations on th

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread Tom Lane
Robert Haas writes: > On Thu, Oct 22, 2015 at 1:40 PM, YUriy Zhuravlev > wrote: >> I can gen xml/json from actual struct. I offered XML/JSON as the analysis of >> C >> code much more difficult. But my current generator just use the structure >> from >> the header files (by pycparser). > Anythi

Re: [HACKERS] Dangling Client Backend Process

2015-10-22 Thread Robert Haas
On Tue, Oct 20, 2015 at 11:42 PM, Rajeev rastogi wrote: > Agreed. Attached is the patch with changes. Well, I'm not buying this extra PostmasterIsAlive() call on every pass through the main loop. That seems more expensive than we can really justify. Checking this when we're already calling WaitL

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Peter Eisentraut
On 10/22/15 3:20 AM, dinesh kumar wrote: > postgres=# SELECT pg_report_log('INFO', 'NULL', false, NULL, NULL); > INFO: NULL > > postgres=# SELECT pg_report_log('INFO', 'NULL', false, 'NULL', 'NULL'); > INFO: NULL > DETAIL: NULL /-- Are you suggesting to change this behaviour/ > HINT: NULL Th

Re: [HACKERS] make Gather node projection-capable

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 3:18 PM, Alvaro Herrera wrote: >> You probably would, but sometimes that might not be possible; for >> example, the tlist might contain a parallel-restricted function (which >> therefore has to run in the leader). > > I don't understand your reply. Failing to parallelize t

Re: [HACKERS] [PROPOSAL] DIAGNOSTICS = SKIPPED_ROW_COUNT

2015-10-22 Thread dinesh kumar
On Wed, Oct 14, 2015 at 4:06 PM, Robert Haas wrote: > On Wed, Oct 14, 2015 at 6:28 PM, dinesh kumar > wrote: > > I see this feature as an add on to do the parallel DML operations. > > There won't be any problem, if operations are mutually exclusive. > > I mean, each session operates on unique se

Re: [HACKERS] make Gather node projection-capable

2015-10-22 Thread Alvaro Herrera
Robert Haas wrote: > On Thu, Oct 22, 2015 at 1:38 PM, Tom Lane wrote: > > Robert Haas writes: > >> The Gather node, as currently committed, is neither projection-capable > >> nor listed as an exception in is_projection_capable_plan. Amit > >> discovered this in testing, and I hit it in my testin

Re: [HACKERS] [PATCH] Typos in comments

2015-10-22 Thread Robert Haas
On Tue, Oct 20, 2015 at 6:29 PM, Jim Nasby wrote: > On 10/20/15 11:08 AM, CharSyam wrote: >> >> I fixed some typos in posgres. >> They are all in comments. :) > > These all look good to me. RFC. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Compan

Re: [HACKERS] make Gather node projection-capable

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 1:38 PM, Tom Lane wrote: > Robert Haas writes: >> The Gather node, as currently committed, is neither projection-capable >> nor listed as an exception in is_projection_capable_plan. Amit >> discovered this in testing, and I hit it in my testing as well. We >> could just

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 1:40 PM, YUriy Zhuravlev wrote: > On Thursday 22 October 2015 13:25:52 you wrote: >> It would be more useful, if we're going to autogenerate code, > Are we going to use autogenerate code? I thought that's what you were proposing. Process the struct definitions and emit .c

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread dinesh kumar
On Thu, Oct 22, 2015 at 11:15 AM, Jim Nasby wrote: > On 10/22/15 2:20 AM, dinesh kumar wrote: > >> >> 2. Using this function, if we provide any "NULL" argument to the function, >> we should either skip it or report it. I see this is what the function >> is doing. >> >> postgres=# SELECT pg_repo

Re: [HACKERS] pg_recvlogical fixes

2015-10-22 Thread Euler Taveira
On 21-10-2015 18:36, Andres Freund wrote: On 2015-10-21 11:52:31 -0300, Euler Taveira wrote: While testing wal2json, I faced some problems with pg_recvlogical. Attached is a serie of patches that can improve pg_recvlogical. Patches #2 and #3 are bugfixes (and should be applied to 9.5 too). Patch

Re: [HACKERS] [PATCH] SQL function to report log message

2015-10-22 Thread Jim Nasby
On 10/22/15 2:20 AM, dinesh kumar wrote: 2. Using this function, if we provide any "NULL" argument to the function, we should either skip it or report it. I see this is what the function is doing. postgres=# SELECT pg_report_log('INFO', 'NULL', false, NULL, NULL); INFO: NULL postgres=# SELE

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread YUriy Zhuravlev
On Thursday 22 October 2015 13:25:52 you wrote: > It would be more useful, if we're going to autogenerate code, Are we going to use autogenerate code? > to do it from the actual struct definitions. I can gen xml/json from actual struct. I offered XML/JSON as the analysis of C code much more diffi

Re: [HACKERS] make Gather node projection-capable

2015-10-22 Thread Tom Lane
Robert Haas writes: > The Gather node, as currently committed, is neither projection-capable > nor listed as an exception in is_projection_capable_plan. Amit > discovered this in testing, and I hit it in my testing as well. We > could just mark it as being not projection-capable, but I think it

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 1:19 PM, YUriy Zhuravlev wrote: > And then another question: > What do you think if the generated equalfuncs.c, copyfuncs.c, readfuncs.c, > outfuncs.c from XML or JSON? > In order not to change the code in four places when changing nodes. It would be more useful, if we're

[HACKERS] make Gather node projection-capable

2015-10-22 Thread Robert Haas
The Gather node, as currently committed, is neither projection-capable nor listed as an exception in is_projection_capable_plan. Amit discovered this in testing, and I hit it in my testing as well. We could just mark it as being not projection-capable, but I think it might be better to go the oth

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread YUriy Zhuravlev
On Thursday 22 October 2015 12:53:49 you wrote: > On Thu, Oct 22, 2015 at 12:15 PM, YUriy Zhuravlev > > wrote: > > Hello. > > Currently using nodeToString and stringToNode you can not pass a full > > plan. In this regard, what is the plan to fix it? Or in the under task > > parallel query does no

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-22 Thread Masahiko Sawada
On Thu, Oct 22, 2015 at 4:11 PM, Torsten Zühlsdorff wrote: > On 21.10.2015 02:05, Masahiko Sawada wrote: >> >> On Sat, Oct 10, 2015 at 4:20 AM, Robert Haas >> wrote: >>> >>> On Thu, Oct 8, 2015 at 1:52 PM, Andres Freund wrote: I don't see the problem? I mean catversion will reliably te

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread Robert Haas
On Thu, Oct 22, 2015 at 12:15 PM, YUriy Zhuravlev wrote: > Hello. > Currently using nodeToString and stringToNode you can not pass a full plan. In > this regard, what is the plan to fix it? Or in the under task parallel query > does not have such a problem? It's already fixed. See commits a0d9f6

Re: [HACKERS] Checkpoint throttling issues

2015-10-22 Thread Andres Freund
On 2015-10-22 09:52:25 -0400, Robert Haas wrote: > On Mon, Oct 19, 2015 at 6:10 AM, Andres Freund wrote: > > 1) The progress passed to CheckpointWriteDelay() will often be wrong - > >it's calculated as num_written / num_to_write, but num_written is only > >incremented if the buffer hasn't

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-10-22 Thread Alvaro Herrera
Syed, Rahila wrote: > @@ -355,6 +356,7 @@ vacuum(int options, RangeVar *relation, Oid relid, > VacuumParams *params, > vac_update_datfrozenxid(); > } > > + pgstat_reset_activityflag; > /* >* Clean up working storage --- note we must do this after >

Re: [HACKERS] Re: [BUGS] BUG #13694: Row Level Security by-passed with CREATEUSER permission

2015-10-22 Thread Tom Lane
Stephen Frost writes: > * Joe Conway (m...@joeconway.com) wrote: >> On 10/21/2015 12:46 PM, Tom Lane wrote: >>> Attached patch rips out CREATEUSER and NOCREATEUSER options lock, stock, >>> and barrel. >> Looks good to me. >>> Another possibility is to change them to actually mean CREATEROLE and

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread David Fetter
On Thu, Oct 22, 2015 at 07:15:35PM +0300, YUriy Zhuravlev wrote: > Hello. > Currently using nodeToString and stringToNode you can not pass a > full plan. In this regard, what is the plan to fix it? Or in the > under task parallel query does not have such a problem? > > > This turns out not to be s

Re: [HACKERS] clearing opfuncid vs. parallel query

2015-10-22 Thread YUriy Zhuravlev
Hello. Currently using nodeToString and stringToNode you can not pass a full plan. In this regard, what is the plan to fix it? Or in the under task parallel query does not have such a problem? > This turns out not to be straightforward to code, because we > don't have a generic plan tree walker,

Re: [HACKERS] checkpointer continuous flushing

2015-10-22 Thread Fabien COELHO
Hello Andres, there would be as much as was written since the last sleep, about 100 ms ago, which is not likely to be gigabytes? In many cases we don't sleep all that frequently - after one 100ms sleep we're already behind a lot. I think that "being behind" is not a problem as such, it is r

Re: [HACKERS] Re: [BUGS] BUG #13694: Row Level Security by-passed with CREATEUSER permission

2015-10-22 Thread Stephen Frost
* Joe Conway (m...@joeconway.com) wrote: > On 10/21/2015 12:46 PM, Tom Lane wrote: > > Attached patch rips out CREATEUSER and NOCREATEUSER options lock, stock, > > and barrel. > > Looks good to me. > > > Another possibility is to change them to actually mean CREATEROLE and > > NOCREATEROLE. I th

Re: [HACKERS] pgbench throttling latency limit

2015-10-22 Thread Fabien COELHO
Argh. It's just because I used -P5. It's a bit confusing that the other options are per second, and this is per interval... I agree, but I'm unsure of a fix, beyond what is already done which is to show units next to the figures... ISTM that people expect "tps" for performance, even on severa

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-22 Thread Robert Haas
On Wed, Oct 21, 2015 at 9:04 AM, Amit Langote wrote: > ... node *need* not be parallel aware? Yes, thanks. Committed that way. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-22 Thread Robert Haas
On Tue, Oct 20, 2015 at 6:12 PM, Simon Riggs wrote: > Not on your case in a big way, just noting the need for change there. Yes, I appreciate your attitude. I think we are on the same wavelength. > I'll help as well, but if you could start with enough basics to allow me to > ask questions that

Re: [HACKERS] [PATCH v3] GSSAPI encryption support

2015-10-22 Thread Robbie Harwood
Andres Freund writes: > On 2015-10-22 16:47:09 +0900, Michael Paquier wrote: >> Hm, and that's why you chose this way of going. My main concern about >> this patch is that it adds on top of the existing Postgres protocol a >> layer to encrypt and decrypt the messages between server and client >>

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-10-22 Thread Robert Haas
On Tue, Oct 20, 2015 at 4:58 AM, Syed, Rahila wrote: >>I think that you should add the flag or something which indicates whether >>this backend is running VACUUM or not, into PgBackendStatus. >>pg_stat_vacuum_progress should display the entries of only backends with that >>flag set true. This de

  1   2   >