Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-10 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Haribabu Kommi > Sent: Tuesday, November 11, 2014 1:13 PM > To: Amit Kapila > Cc: Andres Freund; Robert Haas; Simon Riggs; Tom Lane; > pgsql-hackers@postgresql.org >

Re: [HACKERS] WAL format and API changes (9.5)

2014-11-10 Thread Michael Paquier
On Tue, Nov 11, 2014 at 4:29 AM, Heikki Linnakangas wrote: > Attached is a new version. It's rebased on current git master, including > BRIN. I've also fixed the laundry list of small things you reported, as > well as a bunch of bugs I uncovered during my own testing. This patch needs a small re

Re: [HACKERS] Missing line for postgresql.auto.conf?

2014-11-10 Thread Josh Berkus
On 11/10/2014 10:48 PM, Josh Berkus wrote: > Hackers, > > I thought 9.4's postgresql.conf.sample was supposed to have a > commented-out line for postgresql.auto.conf? In the "includes" section? > > It's not there. If we don't give folks a commented-out line to > uncomment, it'll be pretty hard

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-10 Thread Peter Geoghegan
On Mon, Nov 10, 2014 at 10:29 PM, Peter Geoghegan wrote: > Why not? > > You've already said that you're happy to defer to whatever committer > picks this up with regard to whether or not more than a single > suggestion can come from an RTE. I agreed with this (i.e. I said I'd > defer to their opin

[HACKERS] Missing line for postgresql.auto.conf?

2014-11-10 Thread Josh Berkus
Hackers, I thought 9.4's postgresql.conf.sample was supposed to have a commented-out line for postgresql.auto.conf? In the "includes" section? It's not there. If we don't give folks a commented-out line to uncomment, it'll be pretty hard for them to figure out auto.conf ... -- Josh Berkus Pos

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-10 Thread Peter Geoghegan
On Mon, Nov 10, 2014 at 8:13 PM, Michael Paquier wrote: > Hm. The last version of this patch has not really changed since since my > first review, and I have no more feedback to provide about it except what I > already mentioned. I honestly don't think that this patch is ready for > committer as-i

Re: [HACKERS] [GSoC2014] Patch ALTER TABLE ... SET LOGGED

2014-11-10 Thread Michael Paquier
Thanks for the updated patch, Fabrizio. On Tue, Nov 11, 2014 at 7:44 AM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > A couple of minor comments about this patch: > > 1) Reading it, I am wondering if it would not be finally time to > > switch to a macro to get a relation's persis

Re: [HACKERS] [v9.5] Custom Plan API

2014-11-10 Thread Amit Kapila
On Mon, Nov 10, 2014 at 6:33 PM, Robert Haas wrote: > > On Mon, Nov 10, 2014 at 6:55 AM, Amit Kapila wrote: > > I thought that in general if user has the API to register the custom path > > methods, it should have some way to unregister them and also user might > > need to register some different

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-10 Thread Amit Kapila
On Tue, Nov 11, 2014 at 9:42 AM, Haribabu Kommi wrote: > > On Tue, Nov 11, 2014 at 2:35 PM, Amit Kapila wrote: > > On Tue, Nov 11, 2014 at 5:30 AM, Haribabu Kommi < kommi.harib...@gmail.com> > > wrote: > >> > >> On Tue, Nov 11, 2014 at 10:21 AM, Andres Freund > >> wrote: > >> > On 2014-11-10 10:

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-10 Thread Michael Paquier
On Tue, Nov 11, 2014 at 3:24 AM, Jeff Janes wrote: > > On Wed, Nov 5, 2014 at 8:49 PM, Michael Paquier > wrote: >> >> On Thu, Oct 30, 2014 at 5:19 PM, Michael Paquier >> wrote: >> > Updated patch is attached. >> Please find attached an updated patch with the following things changed: >> - Addit

Re: [HACKERS] Race in "tablespace" test on Windows

2014-11-10 Thread Amit Kapila
On Sat, Nov 8, 2014 at 10:34 AM, Noah Misch wrote: > > In my Windows development environment, the tablespace regression test fails > approximately half the time. Buildfarm member frogmouth failed in the same > manner at least once: > http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogmout

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-10 Thread Haribabu Kommi
On Tue, Nov 11, 2014 at 2:35 PM, Amit Kapila wrote: > On Tue, Nov 11, 2014 at 5:30 AM, Haribabu Kommi > wrote: >> >> On Tue, Nov 11, 2014 at 10:21 AM, Andres Freund >> wrote: >> > On 2014-11-10 10:57:16 -0500, Robert Haas wrote: >> >> Does parallelism help at all? >> > >> > I'm pretty damn sure.

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-11-10 Thread Michael Paquier
On Mon, Nov 10, 2014 at 1:48 PM, Peter Geoghegan wrote: > Reminder: I maintain a slight preference for only offering one > suggestion per relation RTE, which is what this revision does (so no > change there). If a committer who picks this up wants me to alter > that, I don't mind doing so; since

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-10 Thread Amit Kapila
On Tue, Nov 11, 2014 at 5:30 AM, Haribabu Kommi wrote: > > On Tue, Nov 11, 2014 at 10:21 AM, Andres Freund wrote: > > On 2014-11-10 10:57:16 -0500, Robert Haas wrote: > >> Does parallelism help at all? > > > > I'm pretty damn sure. We can't even make a mildly powerfull storage > > fully busy righ

Re: [HACKERS] Convert query plan to sql query

2014-11-10 Thread mariem
>If what you're wishing for is that you could also capture the effects >of planner steps that are in the nature of source-to-source >transformations, I think that's going to be harder because no great >effort has been made to keep those at arm's length from steps that >don't have results descri

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Alvaro Herrera
Greg Stark wrote: > There's another API question I have. To implement Consistent I need to > call the hash function which in the case of functions like hashtext > could be fairly expensive and I even need to generate multiple hash > values(though currently I'm slicing them all from the integer has

Re: [HACKERS] Add CREATE support to event triggers

2014-11-10 Thread Jim Nasby
On 11/10/14, 4:58 PM, Andres Freund wrote: But the main reason all this is interesting isn't so much CREATE itself. But that it can be (and Alvaro has mostly done it!) for ALTER as well. And there it imo becomes really interesting. Because you can quite easily check whether the affected relation

Re: [HACKERS] On partitioning

2014-11-10 Thread Amit Langote
Hi, > From: Robert Haas [mailto:robertmh...@gmail.com] > Sent: Saturday, November 08, 2014 5:41 AM > > I'd be in favor of that. Thanks! > I am not sure whether the code is close > enough to what we need to be really useful, but that's for you to > decide. Hmm, I'm not entirely convinced about

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-10 Thread Tom Lane
Jim Nasby writes: > On 11/10/14, 12:56 PM, Andres Freund wrote: >> If you want to do this - and I sure don't want to stop you from it - you >> should look at it from a general perspective, not from the perspective >> of how skipped cleanup locks are logged. > Honestly, my desire at this point is

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-10 Thread Jim Nasby
On 11/10/14, 12:56 PM, Andres Freund wrote: On 2014-11-10 12:37:29 -0600, Jim Nasby wrote: On 11/10/14, 12:15 PM, Andres Freund wrote: If what we want is to quantify the extent of the issue, would it be more convenient to save counters to pgstat? Vacuum already sends pgstat messages, so there'

Re: [HACKERS] SSL information view

2014-11-10 Thread Michael Paquier
On Tue, Nov 11, 2014 at 1:43 AM, Magnus Hagander wrote: > Right now it just truncates the dn at NAMEDATALEN - so treating it the > same as we do with hostnames. My guess is this is not a big problem > because in the case of long DNs, most of the time the important stuff > is at the beginning anyw

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-10 Thread Haribabu Kommi
On Tue, Nov 11, 2014 at 10:21 AM, Andres Freund wrote: > On 2014-11-10 10:57:16 -0500, Robert Haas wrote: >> Does parallelism help at all? > > I'm pretty damn sure. We can't even make a mildly powerfull storage > fully busy right now. Heck, I can't make my workstation's storage with a > raid 10 ou

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-10 Thread Andres Freund
Hi Robert, All, On 2014-11-10 10:57:16 -0500, Robert Haas wrote: > On Wed, Oct 15, 2014 at 2:55 PM, Simon Riggs wrote: > > Something usable, with severe restrictions, is actually better than we > > have now. I understand the journey this work represents, so don't be > > embarrassed by submitting

Re: [HACKERS] Add CREATE support to event triggers

2014-11-10 Thread Andres Freund
On 2014-11-10 17:37:40 -0500, Christopher Browne wrote: > On 8 November 2014 17:49, Robert Haas wrote: > > > We could just integrate those parts, and be done with it. But would that > > > actually be a good thing for the community? Then slony needs to do it > > > and potentially others as well? Th

Re: [HACKERS] Add CREATE support to event triggers

2014-11-10 Thread Petr Jelinek
On 10/11/14 23:37, Christopher Browne wrote: On 8 November 2014 17:49, Robert Haas > My impression, based on something Christopher Brown said a few years > ago, is that Slony's DDL trigger needs are largely satisfied by the > existing event trigger stuff. It would be helpful to get confirmat

Re: [HACKERS] [GSoC2014] Patch ALTER TABLE ... SET LOGGED

2014-11-10 Thread Fabrízio de Royes Mello
On Thu, Nov 6, 2014 at 3:42 AM, Michael Paquier wrote: > > On Sat, Sep 13, 2014 at 11:02 PM, Fabrízio de Royes Mello > wrote: > > Patch rebased and added to commitfest [1]. > It looks like a good thing to remove ATChangeIndexesPersistence, this > puts the persistence switch directly into reindex

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Greg Stark
On Mon, Nov 10, 2014 at 9:31 PM, Alvaro Herrera wrote: > Every time the index is accessed, yeah. I'm not sure about figuring the > initial creation details. Do you think we need another support > procedure to help with that? We can add it if needed; minmax would just > define it to InvalidOid.

Re: [HACKERS] Add CREATE support to event triggers

2014-11-10 Thread Christopher Browne
On 8 November 2014 17:49, Robert Haas wrote: > > We could just integrate those parts, and be done with it. But would that > > actually be a good thing for the community? Then slony needs to do it > > and potentially others as well? Then auditing can't use it? Then > > potential schema tracking sol

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-10 Thread Andres Freund
On 2014-11-10 14:54:15 -0500, Robert Haas wrote: > On Mon, Nov 10, 2014 at 1:29 PM, Andres Freund wrote: > >> That's an issue to which we may need to engineer some solution, but > >> not in this patch. Overall, the patch's architecture is modeled > >> closely on the way we synchronize GUCs to new

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Alvaro Herrera
Greg Stark wrote: > On Sun, Nov 9, 2014 at 5:57 PM, Greg Stark wrote: > > 2) The mention about additional opclass operators and to number them > > from 11 up is fine -- but there's no explanation of how to decide what > > operators need to be explicitly added like that. Specifically I gather > > f

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Alvaro Herrera
Greg Stark wrote: > 1) The manual describes the exensibility API including the BrinOpcInfo > struct -- but it doesn't define the BrinDesc struct that every API > method takes. It's not clear what exactly that argument is for or how > to make use of it. Hm, I guess this could use some expansion.

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Alvaro Herrera
Alvaro Herrera wrote: > Hm, I think I see what's happening. The xl_brin_update record > references two buffers, one which is target for the updated tuple and > another which is the revmap buffer. When the update target buffer is > being first used we set the INIT bit which removes the buffer ref

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-10 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen Frost wrote: > > at least for 9.5, which is coming up awful fast at this point, if > > we're going to stick with the 'normal' schedule and freeze in the > > spring. > > https://wiki.postgresql.org/wiki/PgCon_2014_Developer_Meeting#9.5_Sc

Re: [HACKERS] BRIN indexes - TRAP: BadArgument

2014-11-10 Thread Alvaro Herrera
Fujii Masao wrote: > I got the following PANIC error in the standby server when I set up > the replication servers and ran "make installcheck". Note that I was > repeating the manual CHECKPOINT every second while "installcheck" > was running. Without the checkpoints, I could not reproduce the > pr

Re: [HACKERS] row_to_json bug with index only scans: empty keys!

2014-11-10 Thread Tom Lane
Robert Haas writes: > On Mon, Nov 10, 2014 at 10:11 AM, Tom Lane wrote: >> BTW, has anyone got an opinion about whether to stick the full fix into >> 9.4? > I think we should put the full fix in 9.4. Since nobody spoke against that, I've put the full fix into HEAD and 9.4, and the restricted ve

Re: [HACKERS] WIP: Access method extendability

2014-11-10 Thread Alexander Korotkov
Hi! Thanks everybody for discussion. Sorry for delay. I'll try to address most of questions arised in this discussion. In general, I'm agree with concept of marking index as invalid in certain cases. I see following possible consequences of buggy WAL-logged custom AM: 1) Server crash during inser

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-10 Thread Robert Haas
On Mon, Nov 10, 2014 at 1:29 PM, Andres Freund wrote: >> That's an issue to which we may need to engineer some solution, but >> not in this patch. Overall, the patch's architecture is modeled >> closely on the way we synchronize GUCs to new backends when using >> EXEC_BACKEND, and I don't think w

Re: [HACKERS] WAL format and API changes (9.5)

2014-11-10 Thread Heikki Linnakangas
Attached is a new version. It's rebased on current git master, including BRIN. I've also fixed the laundry list of small things you reported, as well as a bunch of bugs I uncovered during my own testing. Alvaro: you still have the BRIN WAL-logging code fresh in your memory, so could you take a

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-10 Thread Andres Freund
On 2014-11-10 12:37:29 -0600, Jim Nasby wrote: > On 11/10/14, 12:15 PM, Andres Freund wrote: > >>>If what we want is to quantify the extent of the issue, would it be more > >>>convenient to save counters to pgstat? Vacuum already sends pgstat > >>>messages, so there's no additional traffic there.

Re: [HACKERS] remove pg_standby?

2014-11-10 Thread Magnus Hagander
On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas wrote: > On 11/10/2014 07:50 PM, Joshua D. Drake wrote: >> >> >> On 11/04/2014 01:36 PM, Peter Eisentraut wrote: >>> >>> >>> While we're talking about removing old things, is there any use left for >>> pg_standby? >> >> >> -1. >> >> A lot of peop

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-10 Thread Andres Freund
On 2014-11-10 15:36:55 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2014-11-10 14:28:30 -0300, Alvaro Herrera wrote: > > > > If what we want is to quantify the extent of the issue, would it be more > > > convenient to save counters to pgstat? Vacuum already sends pgstat > > > messag

Re: [HACKERS] remove pg_standby?

2014-11-10 Thread Heikki Linnakangas
On 11/10/2014 07:50 PM, Joshua D. Drake wrote: On 11/04/2014 01:36 PM, Peter Eisentraut wrote: While we're talking about removing old things, is there any use left for pg_standby? -1. A lot of people, a lot of customers use log shipping for various creative and business requirement setups.

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-10 Thread Alvaro Herrera
Stephen Frost wrote: > at least for 9.5, which is coming up awful fast at this point, if > we're going to stick with the 'normal' schedule and freeze in the > spring. https://wiki.postgresql.org/wiki/PgCon_2014_Developer_Meeting#9.5_Schedule Doesn't look all that "normal" to me, with that commit

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-10 Thread Jim Nasby
On 11/10/14, 12:15 PM, Andres Freund wrote: >If what we want is to quantify the extent of the issue, would it be more >convenient to save counters to pgstat? Vacuum already sends pgstat >messages, so there's no additional traffic there. I'm pretty strongly against that one in isolation. They'd

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-10 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > > 1. Any other opinions for or against pg_background as a concept? I > > thought the ability to kick of vacuums (or other stuff with > > PreventTransactionChain) asynchronously was pretty cool, as we as the > > ability to use it as an authenticatio

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-10 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-11-10 14:28:30 -0300, Alvaro Herrera wrote: > > If what we want is to quantify the extent of the issue, would it be more > > convenient to save counters to pgstat? Vacuum already sends pgstat > > messages, so there's no additional traffic there. > > I'm pretty stro

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-10 Thread Andres Freund
On 2014-11-10 12:10:49 -0500, Robert Haas wrote: > - Patch 4 has had some review and really, as far as I can tell, the > only really major issue that has cropped up is the possibility that > the GUC settings that are legal in backend A aren't legal in backend B > for some reason; in that case, rest

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2014-11-10 Thread Jeff Janes
On Wed, Nov 5, 2014 at 8:49 PM, Michael Paquier wrote: > On Thu, Oct 30, 2014 at 5:19 PM, Michael Paquier > wrote: > > Updated patch is attached. > Please find attached an updated patch with the following things changed: > - Addition of tab completion in psql for all new commands > - Addition of

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-10 Thread Andres Freund
On 2014-11-10 14:28:30 -0300, Alvaro Herrera wrote: > Jim Nasby wrote: > > On 11/7/14, 8:21 PM, Robert Haas wrote: > > >On Thu, Nov 6, 2014 at 8:03 PM, Jim Nasby wrote: > > >>The problem right now is there's no way to actually obtain evidence that > > >>this is (or isn't) something to worry about,

Re: [HACKERS] Allow signal handlers to optionally use SA_SIGINFO information?

2014-11-10 Thread Andres Freund
On 2014-11-10 09:50:17 -0500, Tom Lane wrote: > Andres Freund writes: > > Posix provides information about the source of the signal when using > > SA_SIGINFO style handlers via si_code/si_pid. That information has been > > available for a *long* while > > (c.f. http://pubs.opengroup.org/onlinepubs

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-10 Thread Jim Nasby
On 11/10/14, 11:28 AM, Alvaro Herrera wrote: Jim Nasby wrote: On 11/7/14, 8:21 PM, Robert Haas wrote: On Thu, Nov 6, 2014 at 8:03 PM, Jim Nasby wrote: The problem right now is there's no way to actually obtain evidence that this is (or isn't) something to worry about, because we just silently

Re: [HACKERS] Teaching pg_dump to use NOT VALID constraints

2014-11-10 Thread Simon Riggs
On 10 November 2014 17:33, Alvaro Herrera wrote: >> pg_dump --no-revalidaton >> >> will add "NOT VALID" onto the recreation SQL for any FKs, but only for >> ones that were already known to be valid. > > Well. Constraints that haven't been validated already have a NOT VALID > emitted by ruleutil

Re: [HACKERS] remove pg_standby?

2014-11-10 Thread Joshua D. Drake
On 11/04/2014 01:36 PM, Peter Eisentraut wrote: While we're talking about removing old things, is there any use left for pg_standby? -1. A lot of people, a lot of customers use log shipping for various creative and business requirement setups. JD -- Command Prompt, Inc. - http://www.com

Re: [HACKERS] remove pg_standby?

2014-11-10 Thread Fujii Masao
On Tue, Nov 11, 2014 at 2:19 AM, Magnus Hagander wrote: > > On Nov 10, 2014 6:16 PM, "Fujii Masao" wrote: >> >> On Wed, Nov 5, 2014 at 6:36 AM, Peter Eisentraut wrote: >> > While we're talking about removing old things, is there any use left for >> > pg_standby? >> >> -1 for removing it. There i

Re: [HACKERS] Teaching pg_dump to use NOT VALID constraints

2014-11-10 Thread Alvaro Herrera
Simon Riggs wrote: > Magnus and I discussed the need for pg_dump to offer the use of NOT > VALID constraints. > Here's the patch. > > > pg_dump --no-revalidaton > > will add "NOT VALID" onto the recreation SQL for any FKs, but only for > ones that were already known to be valid. Well. Constra

Re: PENDING_LIST_CLEANUP_SIZE - maximum size of GIN pending list Re: [HACKERS] HEAD seems to generate larger WAL regarding GIN index

2014-11-10 Thread Fujii Masao
On Mon, Nov 10, 2014 at 4:15 PM, Etsuro Fujita wrote: > (2014/11/06 23:38), Fujii Masao wrote: >> >> On Tue, Nov 4, 2014 at 12:04 PM, Etsuro Fujita >> wrote: >>> >>> IIUC, I think that min = 0 disables fast update, so ISTM that it'd be >>> appropriate to set min to some positive value. And ISTM

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-10 Thread Alvaro Herrera
Jim Nasby wrote: > On 11/7/14, 8:21 PM, Robert Haas wrote: > >On Thu, Nov 6, 2014 at 8:03 PM, Jim Nasby wrote: > >>The problem right now is there's no way to actually obtain evidence that > >>this is (or isn't) something to worry about, because we just silently skip > >>pages. If we had any kind o

Re: [HACKERS] remove pg_standby?

2014-11-10 Thread Magnus Hagander
On Nov 10, 2014 6:16 PM, "Fujii Masao" wrote: > > On Wed, Nov 5, 2014 at 6:36 AM, Peter Eisentraut wrote: > > While we're talking about removing old things, is there any use left for > > pg_standby? > > -1 for removing it. There is still the case where I'd like to use log-shipping > rather than r

Re: [HACKERS] remove pg_standby?

2014-11-10 Thread Fujii Masao
On Wed, Nov 5, 2014 at 6:36 AM, Peter Eisentraut wrote: > While we're talking about removing old things, is there any use left for > pg_standby? -1 for removing it. There is still the case where I'd like to use log-shipping rather than replication. For example, it's the case where I need to compr

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-11-10 Thread Robert Haas
So, summarizing the state of this patch set: - Patches 1, 2, 3, and 5 are committed. - Patch 4 has had some review and really, as far as I can tell, the only really major issue that has cropped up is the possibility that the GUC settings that are legal in backend A aren't legal in backend B for s

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2014-11-10 Thread Jim Nasby
On 11/7/14, 8:21 PM, Robert Haas wrote: On Thu, Nov 6, 2014 at 8:03 PM, Jim Nasby wrote: The problem right now is there's no way to actually obtain evidence that this is (or isn't) something to worry about, because we just silently skip pages. If we had any kind of tracking on this we could sto

Re: [HACKERS] tracking commit timestamps

2014-11-10 Thread Steve Singer
On 11/10/2014 08:39 AM, Petr Jelinek wrote: On 09/11/14 17:57, Steve Singer wrote: On 11/07/2014 07:07 PM, Petr Jelinek wrote: The list of what is useful might be long, but we can't have everything there as there are space constraints, and LSN is another 8 bytes and I still want to have some by

Re: [HACKERS] SSL information view

2014-11-10 Thread Magnus Hagander
On Mon, Jul 21, 2014 at 5:24 PM, Bernd Helmle wrote: > > > --On 12. Juli 2014 15:08:01 +0200 Magnus Hagander > wrote: > >> Before doing that, however, I'd like to ask for opinions :) The hack >> currently exposes a separate view that you can join to >> pg_stat_activity (or pg_stat_replication) on

[HACKERS] Teaching pg_dump to use NOT VALID constraints

2014-11-10 Thread Simon Riggs
Magnus and I discussed the need for pg_dump to offer the use of NOT VALID constraints. Here's the patch. pg_dump --no-revalidaton will add "NOT VALID" onto the recreation SQL for any FKs, but only for ones that were already known to be valid. -- Simon Riggs http://www.2ndQu

[HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-11-10 Thread Robert Haas
On Wed, Oct 15, 2014 at 2:55 PM, Simon Riggs wrote: > Something usable, with severe restrictions, is actually better than we > have now. I understand the journey this work represents, so don't be > embarrassed by submitting things with heuristics and good-enoughs in > it. Our mentor, Mr.Lane, achi

Re: [HACKERS] Compiler warning in master branch

2014-11-10 Thread Kevin Grittner
David Rowley wrote:> On Mon, Nov 10, 2014 at 4:31 PM, Peter Geoghegan wrote: >> I see this when I build at -O2 from master's tip: >> brin_revmap.c: In function ‘brinRevmapExtend’: >> brin_revmap.c:113:14: error: variable ‘mapBlk’ set but not used >> [-Werror=unused-but-set-variable] >> Block

Re: [HACKERS] row_to_json bug with index only scans: empty keys!

2014-11-10 Thread Andrew Dunstan
On 11/10/2014 10:19 AM, Robert Haas wrote: On Mon, Nov 10, 2014 at 10:11 AM, Tom Lane wrote: I wrote: Attached are patches meant for HEAD and 9.2-9.4 respectively. BTW, has anyone got an opinion about whether to stick the full fix into 9.4? The argument for, of course, is that we'd get the

Re: [HACKERS] row_to_json bug with index only scans: empty keys!

2014-11-10 Thread Ross Reedstrom
Yes, it's late beta, but especially if we're pushing json/jsonb usage as a major feature of this release, I'd say remove surprising cases now. It's late beta, but that's what beta is for, the last chance for bug fixes, before we live w/ it for the support life of the release. The affected class ar

Re: [HACKERS] row_to_json bug with index only scans: empty keys!

2014-11-10 Thread David G Johnston
Tom Lane-2 wrote > I wrote: >> Attached are patches meant for HEAD and 9.2-9.4 respectively. > > BTW, has anyone got an opinion about whether to stick the full fix into > 9.4? The argument for, of course, is that we'd get the full fix out to > the public a year sooner. The argument against is th

Re: [HACKERS] Fwd: How parser works and to add keyword to keywords.h

2014-11-10 Thread Tom Lane
Pankaj Bagul writes: > Can anyone tell about how parser works and if we want to add a new keyword > to the keywords.h, > and how to use that keyword ? How the all process works ? The minimum requirements are as stated in gram.y: add the keyword to kwlist.h, to the "%token " list near the head of

Re: [HACKERS] remove pg_standby?

2014-11-10 Thread Magnus Hagander
On Tue, Nov 4, 2014 at 10:36 PM, Peter Eisentraut wrote: > While we're talking about removing old things, is there any use left for > pg_standby? Was the original reason to keep it around anything other than backwards compatibility? If not, then it was backwards compatibility with a version that'

Re: [HACKERS] row_to_json bug with index only scans: empty keys!

2014-11-10 Thread Robert Haas
On Mon, Nov 10, 2014 at 10:11 AM, Tom Lane wrote: > I wrote: >> Attached are patches meant for HEAD and 9.2-9.4 respectively. > > BTW, has anyone got an opinion about whether to stick the full fix into > 9.4? The argument for, of course, is that we'd get the full fix out to > the public a year so

Re: [HACKERS] row_to_json bug with index only scans: empty keys!

2014-11-10 Thread Tom Lane
I wrote: > Attached are patches meant for HEAD and 9.2-9.4 respectively. BTW, has anyone got an opinion about whether to stick the full fix into 9.4? The argument for, of course, is that we'd get the full fix out to the public a year sooner. The argument against is that someone might have alread

[HACKERS] Fwd: How parser works and to add keyword to keywords.h

2014-11-10 Thread Pankaj Bagul
Hello, Can anyone tell about how parser works and if we want to add a new keyword to the keywords.h, and how to use that keyword ? How the all process works ? -- Pankaj B.

Re: [HACKERS] Allow signal handlers to optionally use SA_SIGINFO information?

2014-11-10 Thread Tom Lane
Andres Freund writes: > Posix provides information about the source of the signal when using > SA_SIGINFO style handlers via si_code/si_pid. That information has been > available for a *long* while > (c.f. http://pubs.opengroup.org/onlinepubs/7908799/xsh/signal.h.html). That does not mean that it

Re: [HACKERS] tracking commit timestamps

2014-11-10 Thread Robert Haas
On Mon, Nov 10, 2014 at 8:40 AM, Alvaro Herrera wrote: > Ah, right. So AFAIK we don't need to keep anything older than > RecentXmin or something like that -- which is not too old. If I recall > correctly Josh Berkus was saying in a thread about pg_multixact that it > used about 128kB or so in <=

Re: [HACKERS] tracking commit timestamps

2014-11-10 Thread Robert Haas
On Mon, Nov 10, 2014 at 8:39 AM, Petr Jelinek wrote: > I did the calculation above wrong btw, it's actually 20 bytes not 24 bytes > per record, I am inclined to just say we can live with that. If you do it as 20 bytes, you'll have to do some work to squeeze out the alignment padding. I'm incline

Re: [HACKERS] pg_multixact not getting truncated

2014-11-10 Thread Robert Haas
On Sat, Nov 8, 2014 at 3:10 PM, Josh Berkus wrote: > I also think our > defaults for multixact freezing should be tied to the ones for xid > freezing, and should not by default be completely independent numbers; > I'm still not convinced that it makes sense to have a separate multixact > threshold

Re: [HACKERS] psql tab completion: \c [ dbname [ username ] ]

2014-11-10 Thread Ian Barwick
On 10/11/14 22:20, Robert Haas wrote: On Sun, Nov 9, 2014 at 6:13 PM, Ian Barwick wrote: Attached is a mighty trivial patch to extend psql tab completion for \c / \connect to generate a list of role names, as lack thereof was annoying me recently and I can't see any downside to doing this. Co

Re: [HACKERS] tracking commit timestamps

2014-11-10 Thread Alvaro Herrera
Robert Haas wrote: > On Sun, Nov 9, 2014 at 8:41 PM, Alvaro Herrera > wrote: > > Robert Haas wrote: > >> I think the key question here is the time for which the data needs to > >> be retained. 2^32 of anything is a lot, but why keep around that > >> number of records rather than more (after all,

Re: [HACKERS] tracking commit timestamps

2014-11-10 Thread Petr Jelinek
On 09/11/14 17:57, Steve Singer wrote: On 11/07/2014 07:07 PM, Petr Jelinek wrote: The list of what is useful might be long, but we can't have everything there as there are space constraints, and LSN is another 8 bytes and I still want to have some bytes for storing the "origin" or whatever you

Re: [HACKERS] tracking commit timestamps

2014-11-10 Thread Robert Haas
On Mon, Nov 10, 2014 at 2:01 AM, Anssi Kääriäinen wrote: > There is no guarantee that a commit with later LSN has a later > timestamp. There are cases where the clock could move significantly > backwards. Good point. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgr

Re: [HACKERS] tracking commit timestamps

2014-11-10 Thread Petr Jelinek
On 10/11/14 08:01, Anssi Kääriäinen wrote: On Sun, 2014-11-09 at 11:57 -0500, Steve Singer wrote: The reason why Jim and myself are asking for the LSN and not just the timestamp is that I want to be able to order the transactions. Jim pointed out earlier in the thread that just ordering on time

Re: [HACKERS] tracking commit timestamps

2014-11-10 Thread Robert Haas
On Sun, Nov 9, 2014 at 8:41 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> I think the key question here is the time for which the data needs to >> be retained. 2^32 of anything is a lot, but why keep around that >> number of records rather than more (after all, we have epochs to >> distinguis

Re: [HACKERS] psql tab completion: \c [ dbname [ username ] ]

2014-11-10 Thread Robert Haas
On Sun, Nov 9, 2014 at 6:13 PM, Ian Barwick wrote: > Attached is a mighty trivial patch to extend psql tab completion > for \c / \connect to generate a list of role names, as lack thereof > was annoying me recently and I can't see any downside to doing > this. Committed, thanks. -- Robert Haas

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-10 Thread Robert Haas
On Mon, Nov 10, 2014 at 6:33 AM, Amit Kapila wrote: > I think this logic can sometimes block processes from acquiring a lock > which no one is holding. Assume Group-1 (G-1) is waiting on two locks > (Lock L1 on Table T1 and Lock L2 on Table T2) which are held by > unrelated processes P-2 and P-3

Re: [HACKERS] Column/type dependency recording inconsistencies

2014-11-10 Thread Petr Jelinek
On 09/11/14 23:36, Petr Jelinek wrote: On 09/11/14 23:04, Tom Lane wrote: I suspect this is actually a bug in the dependency search logic in DROP. Don't have the energy to chase it right now though. Yes that's where I started searching also and yes it is. The actual situation is that the col

Re: [HACKERS] [v9.5] Custom Plan API

2014-11-10 Thread Robert Haas
On Mon, Nov 10, 2014 at 6:55 AM, Amit Kapila wrote: > I thought that in general if user has the API to register the custom path > methods, it should have some way to unregister them and also user might > need to register some different custom path methods after unregistering > the previous one's.

Re: [HACKERS] [v9.5] Custom Plan API

2014-11-10 Thread Amit Kapila
On Mon, Nov 10, 2014 at 4:18 PM, Kouhei Kaigai wrote: > > > > Few observations/questions related to this commit: > > > > 1. > > @@ -5546,6 +5568,29 @@ get_variable(Var *var, int levelsup, bool istoplevel, > > deparse_context *context) > > colinfo = deparse_columns_fetch(var->varno, dpns); > >

Re: [HACKERS] Allow signal handlers to optionally use SA_SIGINFO information?

2014-11-10 Thread Heikki Linnakangas
On 11/10/2014 01:32 PM, Andres Freund wrote: During benchmarking/debugging I just had the problem that autovacuum was signalled at an insane rate - leading to more than one hundred autovac workers being started per second. Leading to a overall slowdown of more than 90% and the anti-wraparound vac

Re: [HACKERS] group locking: incomplete patch, just for discussion

2014-11-10 Thread Amit Kapila
On Fri, Nov 7, 2014 at 3:55 AM, Robert Haas wrote: > > On Fri, Oct 31, 2014 at 9:07 AM, Andres Freund wrote: > > Maybe we can, as a first step, make those edges in the lock graph > > visible to the deadlock detector? It's pretty clear that undetected > > deadlocks aren't ok, but detectable deadlo

[HACKERS] Allow signal handlers to optionally use SA_SIGINFO information?

2014-11-10 Thread Andres Freund
Hi, During benchmarking/debugging I just had the problem that autovacuum was signalled at an insane rate - leading to more than one hundred autovac workers being started per second. Leading to a overall slowdown of more than 90% and the anti-wraparound vacuum not finishing. The problem is that I

Re: [HACKERS] postgres_fdw behaves oddly

2014-11-10 Thread Ashutosh Bapat
Since two separate issues 1. using reltargetlist instead of attr_needed and 2. system columns usage in FDW are being tackled here, we should separate the patch into two one for each of the issues. While I agree that the system columns shouldn't be sent to the remote node, it doesn't look clear to

Re: [HACKERS] [v9.5] Custom Plan API

2014-11-10 Thread Kouhei Kaigai
> On Sat, Nov 8, 2014 at 4:16 AM, Robert Haas wrote: > > > > On Mon, Oct 27, 2014 at 2:35 AM, Kouhei Kaigai > wrote: > > >> FYI, patch v12 part 2 no longer applies cleanly. > > >> > > > Thanks. I rebased the patch set according to the latest master branch. > > > The attached v13 can be applied to

Re: [HACKERS] alter user/role CURRENT_USER

2014-11-10 Thread Kyotaro HORIGUCHI
Hello, This is the new version. (WIP v2) The first attachment is the patch and the second is test sql script. - Behavior changing Almost all syntax taking role accepts CURRENT_USER and SESSION_USER and they are distinguished from "current_user" and "session_user". The exceptions are follows. -

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-11-10 Thread furuyao
> On Fri, Oct 31, 2014 at 5:46 PM, wrote: > >> > We seem to be going in circles. You suggested having two options, > >> > --feedback, and --fsync, which is almost exactly what Furuya posted > >> > originally. I objected to that, because I think that user interface > >> is > >> > too complicated.

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-11-10 Thread Michael Paquier
On Sun, Nov 9, 2014 at 10:32 PM, Fujii Masao wrote: > On Sun, Nov 9, 2014 at 6:41 AM, Rahila Syed wrote: >> Hello, >> >>>The patch was not applied to the master cleanly. Could you update the >>> patch? >> Please find attached updated and rebased patch to compress FPW. Review >> comments given abo