Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-25 Thread Amit Kapila
On Wed, Sep 24, 2014 at 3:18 PM, Dilip kumar wrote: > On 24 August 2014 11:33, Amit Kapila Wrote > > >7. I think in new mechanism cancel handler will not work. > > >In single connection vacuum it was always set/reset > > >in function executeMaintenanceCommand(). You might need > > >to set/reset it

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Alexander Korotkov
On Fri, Sep 26, 2014 at 5:18 AM, Bruce Momjian wrote: > On Sun, Sep 14, 2014 at 11:34:26PM +0400, Alexander Korotkov wrote: > > > Cost estimation of GiST is a big problem anyway. It doesn't care > (and > > > can't) about amount of recheck for regular operators. In this > patch, same > >

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 08:10 PM, Tom Lane wrote: > I wrote: >> The "offsets-and-lengths" patch seems like the approach we ought to >> compare to my patch, but it looks pretty unfinished to me: AFAICS it >> includes logic to understand offsets sprinkled into a mostly-lengths >> array, but no logic that would

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Tom Lane
I wrote: > The "offsets-and-lengths" patch seems like the approach we ought to > compare to my patch, but it looks pretty unfinished to me: AFAICS it > includes logic to understand offsets sprinkled into a mostly-lengths > array, but no logic that would actually *store* any such offsets, > which me

Re: [HACKERS] Replication identifiers, take 3

2014-09-25 Thread Robert Haas
Thanks for this write-up. On Tue, Sep 23, 2014 at 2:24 PM, Andres Freund wrote: > 1) The ability Monitor how for replication has progressed in a >crashsafe manner to allow it to restart at the right point after >errors/crashes. > 2) Efficiently identify the origin of individual changes an

Re: [HACKERS] Add CREATE support to event triggers

2014-09-25 Thread Michael Paquier
On Fri, Sep 26, 2014 at 6:59 AM, Alvaro Herrera wrote: > Actually here's a different split of these patches, which I hope makes > more sense. My intention here is that patches 0001 to 0004 are simple > changes that can be pushed right away; they are not directly related to > the return-creation-c

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Bruce Momjian
On Sun, Sep 14, 2014 at 11:34:26PM +0400, Alexander Korotkov wrote: > > Cost estimation of GiST is a big problem anyway. It doesn't care (and > > can't) about amount of recheck for regular operators. In this patch, > same > > would be for knn recheck. The problem is that touching heap

Re: [HACKERS] better atomics - v0.6

2014-09-25 Thread Andres Freund
On 2014-09-25 21:02:28 -0400, Robert Haas wrote: > I feel like this could really use a developer README: what primitives > do we have, which ones are likely to be efficient or inefficient on > which platforms, how do atomics interact with barriers, etc. atomics.h has most of that. Including docume

Re: [HACKERS] better atomics - v0.6

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 6:03 PM, Andres Freund wrote: > On 2014-09-24 20:27:39 +0200, Andres Freund wrote: >> On 2014-09-24 21:19:06 +0300, Heikki Linnakangas wrote: >> I won't repost a version with it removed, as removing a function as the >> only doesn't seem to warrant reposting it. > > I've fi

Re: [HACKERS] delta relations in AFTER triggers

2014-09-25 Thread Craig Ringer
On 09/25/2014 11:54 PM, Kevin Grittner wrote: > I have fixed the bug reported by Heikki; be sure to grab that. Will do. > I have been merging in changes to master as I go, so that bit rot > doesn't accumulate, but I don't squash or rebase; hopefully that > style works for you. IMO it only really

Re: [HACKERS] Add CREATE support to event triggers

2014-09-25 Thread Andres Freund
On 2014-09-25 18:59:31 -0300, Alvaro Herrera wrote: > /* tid.c */ > diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h > new file mode 100644 > index 000..520b066 > --- /dev/null > +++ b/src/include/utils/ruleutils.h > @@ -0,0 +1,34 @@ > +/*-

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Gavin Flower
On 26/09/14 08:21, Simon Riggs wrote: On 25 September 2014 20:11, Robert Haas wrote: My approach would be to insert an index tuple for that value into the index, but with the leaf ituple marked with an xid rather than a ctid. If someone tries to insert into the index they would see this and wa

Re: [HACKERS] better atomics - v0.6

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 3:23 PM, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: >> Heikki has marked the patch as 'ready for commiter' in the commitfest >> (conditional on his remarks being addressed) and I agree. There seems to >> be little benefit in waiting further. There *d

Re: [HACKERS] better atomics - v0.6

2014-09-25 Thread Stephen Frost
Andres, * Andres Freund (and...@anarazel.de) wrote: > Heikki has marked the patch as 'ready for commiter' in the commitfest > (conditional on his remarks being addressed) and I agree. There seems to > be little benefit in waiting further. There *definitely* will be some > platform dependant issues

Re: [HACKERS] RLS feature has been committed

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 3:07 PM, Stephen Frost wrote: > A revert wasn't requested by the individual who raised the concern (or, > indeed, explicitly by *anyone*.. it was hinted at, but I felt the > individuals who were hinting at it were leaving it up to that individual > who had a concern), and

Re: [HACKERS] RLS feature has been committed

2014-09-25 Thread Stephen Frost
Simon, * Simon Riggs (si...@2ndquadrant.com) wrote: > On 23 September 2014 07:45, Heikki Linnakangas > wrote: > > OTOH, if the patch is actually OK as it was committed, there's no point > > reverting it only to commit it again later. At the end of the day, the > > important thing is that the pat

Re: [HACKERS] better atomics - v0.6

2014-09-25 Thread Andres Freund
On 2014-09-24 20:27:39 +0200, Andres Freund wrote: > On 2014-09-24 21:19:06 +0300, Heikki Linnakangas wrote: > I won't repost a version with it removed, as removing a function as the > only doesn't seem to warrant reposting it. I've fixed (thanks Alvaro!) some minor additional issues besides the r

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 2:37 PM, Simon Riggs wrote: >> I'd hoped that the commit messages, and my discussion of the feature >> were adequate. > > I'd hoped that my discussion was sufficient to persuade you too, but it > wasn't. I'll write user-visible docs soon, then. -- Peter Geoghegan --

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 22:13, Peter Geoghegan wrote: >> All of the above, as a separate committable patch. I hate the fact >> that you have written no user facing documentation for this feature. >> How can anyone tell whether the tests you've written are correct or >> even consistent to a particula

Re: [HACKERS] RLS feature has been committed

2014-09-25 Thread Simon Riggs
On 23 September 2014 07:45, Heikki Linnakangas wrote: > On 09/23/2014 09:15 AM, Peter Geoghegan wrote: >> >> On Mon, Sep 22, 2014 at 8:25 PM, Robert Haas >> wrote: Should RLS be reverted, and revisited in a future CF? >>> >>> >>> IMHO, that would be entirely appropriate. >> >> >> That s

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 1:48 PM, Simon Riggs wrote: > A. UPDATE/INSERT privilege infrastructure. > Add tests to it, make it separately committable, so we can get that done. > Submit to Oct CF; get that done early. Makes sense. As long as we assume that we want a unified syntax like this - that is

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Andres Freund
On 2014-09-26 02:34:06 +0530, Abhijit Menon-Sen wrote: > At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: > > Unless I miss something this isn't sufficient. We need to fsync the > > files in the data directory, not just the toplevel directory? > > No, of course you're right. So a separa

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Abhijit Menon-Sen
At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: > > * Also recovery shouldn't be regarded successful if the reset fails - > * e.g. because of ENOSPC. OK. > * Doing this in a separate pass is advantageous for performance reasons > * because it allows the kernel to perform all the flu

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Simon Riggs
On 25 September 2014 18:30, Andres Freund wrote: > On 2014-09-25 18:18:09 +0100, Simon Riggs wrote: >> On 25 September 2014 16:29, Andres Freund wrote: >> > I think that's not really related. Such a promotion doesn't cause data >> > loss in the sense of loosing data a *clueful* operator wanted to

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 1:21 PM, Simon Riggs wrote: > The test index is unique, so our to-be-inserted value exists on only > one page, hence page locking applies while we insert it. The next > person to insert waits for the page lock and then sees the test tuple. > > The page lock lasts only for t

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 20:38, Peter Geoghegan wrote: > On Thu, Sep 25, 2014 at 7:12 AM, Simon Riggs wrote: >> The way forwards, in my view, is to define precisely the behaviour we >> wish to have. That definition will include the best current mechanism >> for running an UPSERT using INSERT/UPDATE/l

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Stephen Frost
* Gregory Smith (gregsmithpg...@gmail.com) wrote: > On 9/25/14, 2:02 PM, Peter Eisentraut wrote: > >But having the same parameter setting mean different things in > >different versions is the path to complete madness. > > Could we go so far as to remove support for unitless time settings > eventua

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Andres Freund
Hi, On 2014-09-24 17:06:05 +0530, Abhijit Menon-Sen wrote: > Hi Andres, Robert. > > I've attached four patches here. Cool. Will review. > 1. Move the call to ResetUnloggedRelations(UNLOGGED_RELATION_INIT) to >earlier in StartupXLOG. > > 2. Inside that function, issue fsync()s for the main

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 3:17 PM, Peter Geoghegan wrote: >> To find out how much that optimization buys, you >> should use tuples with many variable-length columns (say, 50) >> preceding the text column you're sorting on. I won't be surprised if >> that turns out to be expensive enough to be worth

Re: [HACKERS] Review of GetUserId() Usage

2014-09-25 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: > On 9/24/14 4:58 PM, Stephen Frost wrote: > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > >> I think the case for pgstat_get_backend_current_activity() and > >> pg_stat_get_activity and the other pgstatfuncs.c callers is easy to make > >> and se

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 19:59, Peter Geoghegan wrote: > On Thu, Sep 25, 2014 at 9:20 AM, Robert Haas wrote: >> I've never been a fan of putting the index name in there. > > Me neither. Although I do understand Kevin's concern about the user's > intent surrounding which unique index to merge on. The

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 20:11, Robert Haas wrote: >> My approach would be to insert an index tuple for that value into the >> index, but with the leaf ituple marked with an xid rather than a ctid. >> If someone tries to insert into the index they would see this and wait >> for the inserting transact

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Gregory Smith
On 9/25/14, 2:02 PM, Peter Eisentraut wrote: But having the same parameter setting mean different things in different versions is the path to complete madness. Could we go so far as to remove support for unitless time settings eventually? The fact that people are setting raw numbers in the

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-25 Thread Jeff Janes
On Thu, Sep 25, 2014 at 10:00 AM, Jeff Janes wrote: > On Wed, Sep 24, 2014 at 2:48 AM, Dilip kumar > wrote: > >> On 24 August 2014 11:33, Amit Kapila Wrote >> >> >> >> Thanks for you comments, i have worked on both the review comment lists, >> sent on 19 August, and 24 August. >> >> >> >> Lates

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 7:12 AM, Simon Riggs wrote: > The way forwards, in my view, is to define precisely the behaviour we > wish to have. That definition will include the best current mechanism > for running an UPSERT using INSERT/UPDATE/loops and comparing that > against what is being provided

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 12:11 PM, Robert Haas wrote: > I think that something like this might work, but the devil is in the > details. Suppose two people try to upsert into the same table at the > same time. There's one index. If the transactions search that index > for conflicts first, neither

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Alvaro Herrera
Bruce Momjian wrote: > 3. 9.3 multi-xact bugs spooked us into being more careful Uh. Multixact changes in 9.3 were infinitely more invasive than the jsonb changes will ever be. a) they touched basic visibility design and routines, which are complex, understood by very few people, and ha

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Bruce Momjian
On Thu, Sep 25, 2014 at 09:00:07PM +0200, Andres Freund wrote: > On 2014-09-25 14:46:18 -0400, Bruce Momjian wrote: > > On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: > > > BTW, it seems like there is consensus that we ought to reorder the items > > > in a jsonb object to have keys first

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 11:53 AM, Robert Haas wrote: > I haven't looked at that part of the patch in detail yet, so... not > really. But I don't see why you'd ever need to restart heap tuple > copying. At most you'd need to re-extract datum1 from the tuples you > have already copied. Well, okay

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 2:17 PM, Simon Riggs wrote: >> 1. You don't accept that value locks must be easily released in the >> event of a conflict. Is anyone in this camp? It's far from obvious to >> me what side of this question Andres is on at this stage, for example. >> Robert might have somethi

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 14:46:18 -0400, Bruce Momjian wrote: > On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: > > BTW, it seems like there is consensus that we ought to reorder the items > > in a jsonb object to have keys first and then values, independently of the > > other issues under discussio

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 9:20 AM, Robert Haas wrote: > I've never been a fan of putting the index name in there. Me neither. Although I do understand Kevin's concern about the user's intent surrounding which unique index to merge on. > I agree > that's stuff that a DML statement shouldn't need to

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 2:05 PM, Peter Geoghegan wrote: > On Thu, Sep 25, 2014 at 9:21 AM, Robert Haas wrote: >> The top issue on my agenda is figuring out a way to get rid of the >> extra SortSupport object. > > Really? I'm surprised. Clearly the need to restart heap tuple copying > from scratch

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Bruce Momjian
On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: > BTW, it seems like there is consensus that we ought to reorder the items > in a jsonb object to have keys first and then values, independently of the > other issues under discussion. This means we *will* be breaking on-disk > compatibilit

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 7:35 AM, Robert Haas wrote: > On Thu, Sep 25, 2014 at 10:12 AM, Simon Riggs wrote: >> IMHO it is impossible to know if any of the other code is correct >> until we have a clear and stable vision of what the command is >> supposed to perform. > > +1. > >> The inner workings

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Tom Lane
BTW, it seems like there is consensus that we ought to reorder the items in a jsonb object to have keys first and then values, independently of the other issues under discussion. This means we *will* be breaking on-disk compatibility with 9.4beta2, which means pg_upgrade will need to be taught to

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 11:17 AM, Simon Riggs wrote: > Basically, I have absolutely no idea whether I object to or agree with > 1) and don't know where to look to find out. We need a clear > exposition of design and the alternatives. > > My approach would be to insert an index tuple for that value

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 11:22 AM, Tom Lane wrote: > In the interests of pushing this forward, I will work today on > trying to finish and review Heikki's offsets-and-lengths patch > so that we have something we can do performance testing on. > I doubt that the performance testing will tell us anything we > d

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Tom Lane
Josh Berkus writes: > On 09/25/2014 10:26 AM, Andres Freund wrote: >> On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: >>> If Heikki says it's ready, I'll test. So far he's said that it wasn't >>> done yet. >> http://www.postgresql.org/message-id/541c242e.3030...@vmware.com > Yeah, and that did

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 28 August 2014 03:43, Peter Geoghegan wrote: > "Value locking" > === > > To date, on-list discussion around UPSERT has almost exclusively > concerned what I've called "value locking"; the idea of locking values > in unique indexes in the abstract (to establish the right to insert > ahe

Re: [HACKERS] Review of GetUserId() Usage

2014-09-25 Thread Peter Eisentraut
On 9/24/14 4:58 PM, Stephen Frost wrote: > Alvaro, > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: >> I think the case for pgstat_get_backend_current_activity() and >> pg_stat_get_activity and the other pgstatfuncs.c callers is easy to make >> and seems acceptable to me; but I would leave p

Re: [HACKERS] GIN improvements part2: fast scan

2014-09-25 Thread Thom Brown
On 12 March 2014 16:29, Heikki Linnakangas wrote: > Good point. We have done two major changes to GIN in this release cycle: > changed the data page format and made it possible to skip items without > fetching all the keys ("fast scan"). gincostestimate doesn't know about > either change. Did thi

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 9:21 AM, Robert Haas wrote: > The top issue on my agenda is figuring out a way to get rid of the > extra SortSupport object. Really? I'm surprised. Clearly the need to restart heap tuple copying from scratch, in order to make the datum1 representation consistent, rather th

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Peter Eisentraut
On 9/25/14 11:03 AM, Robert Haas wrote: > I couldn't agree more. There's something to be said for just leaving > this alone. I agree. > potentitally draw complaints. But I also agree with his last one - of > those three possible complaints, I certainly prefer "I had to fix my > configuration fi

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 1:30 PM, Andres Freund wrote: >> Yes it does cause data loss. The clueful operator has no idea where >> they are so there is no "used rightly" in that case. > > What? There definitely are cases where you don't need to know that to > the T. Just think of the - quite frequent

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: > If Heikki says it's ready, I'll test. So far he's said that it wasn't > done yet. http://www.postgresql.org/message-id/541c242e.3030...@vmware.com Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ Pos

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 10:29:51 -0700, Josh Berkus wrote: > On 09/25/2014 10:26 AM, Andres Freund wrote: > > On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: > >> If Heikki says it's ready, I'll test. So far he's said that it wasn't > >> done yet. > > > > http://www.postgresql.org/message-id/541c242e.3030

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Andres Freund
On 2014-09-25 18:18:09 +0100, Simon Riggs wrote: > On 25 September 2014 16:29, Andres Freund wrote: > > I think that's not really related. Such a promotion doesn't cause data > > loss in the sense of loosing data a *clueful* operator wanted to > > keep. Yes, it can be used wrongly, but it's far fr

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 10:26 AM, Andres Freund wrote: > On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: >> If Heikki says it's ready, I'll test. So far he's said that it wasn't >> done yet. > > http://www.postgresql.org/message-id/541c242e.3030...@vmware.com Yeah, and that didn't include some of Tom's b

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 10:20 AM, Andres Freund wrote: > On 2014-09-25 10:18:24 -0700, Josh Berkus wrote: >> On 09/25/2014 10:14 AM, Bruce Momjian wrote: >>> On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: But independent of which version is chosen, we *REALLY* need to make the decis

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Alexander Korotkov
On Thu, Sep 25, 2014 at 9:00 PM, Emre Hasegeli wrote: > > Fixed, thanks. > > Here are my questions and comments about the code. > > doc/src/sgml/gist.sgml:812: > >be rechecked from heap tuple before tuple is returned. If > >recheck flag isn't set then it's true by default for > >

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 10:18:24 -0700, Josh Berkus wrote: > On 09/25/2014 10:14 AM, Bruce Momjian wrote: > > On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: > >> But independent of which version is chosen, we *REALLY* need to make the > >> decision soon. This issue has held up the next beta (

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 10:14 AM, Bruce Momjian wrote: > On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: >> But independent of which version is chosen, we *REALLY* need to make the >> decision soon. This issue has held up the next beta (like jsonb has >> blocked previous beta) for *weeks*. >>

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Simon Riggs
On 25 September 2014 16:29, Andres Freund wrote: >> > To me, being able to say "pg_ctl promote_right_now -m yes_i_mean_it" >> > seems like a friendlier interface than making somebody shut down the >> > server, run pg_resetxlog, and start it up again. >> >> It makes sense to go from paused --> pro

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Bruce Momjian
On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: > But independent of which version is chosen, we *REALLY* need to make the > decision soon. This issue has held up the next beta (like jsonb has > blocked previous beta) for *weeks*. > > Personally it doesn't make me very happy that He

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-25 Thread Jeff Janes
On Wed, Sep 24, 2014 at 2:48 AM, Dilip kumar wrote: > On 24 August 2014 11:33, Amit Kapila Wrote > > > > Thanks for you comments, i have worked on both the review comment lists, > sent on 19 August, and 24 August. > > > > Latest patch is attached with the mail.. > Hi Dilip, I think you have an

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Emre Hasegeli
> Fixed, thanks. Here are my questions and comments about the code. doc/src/sgml/gist.sgml:812: >be rechecked from heap tuple before tuple is returned. If >recheck flag isn't set then it's true by default for >compatibility reasons. The recheck flag can be used only Rec

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 09:01 AM, Andres Freund wrote: > But independent of which version is chosen, we *REALLY* need to make the > decision soon. This issue has held up the next beta (like jsonb has > blocked previous beta) for *weeks*. Yes, please! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Robert Haas
On Wed, Sep 24, 2014 at 7:04 PM, Peter Geoghegan wrote: > On Fri, Sep 19, 2014 at 2:54 PM, Peter Geoghegan wrote: >> Probably not - it appears to make very little difference to >> unoptimized pass-by-reference types whether or not datum1 can be used >> (see my simulation of Kevin's worst case, fo

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 11:21 AM, Simon Riggs wrote: > On 25 September 2014 15:35, Robert Haas wrote: >>> The only problem I see is if the newly inserted row matches one row on >>> one unique value and a different row on a different unique index. >>> Turning the INSERT into an UPDATE will still f

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 11:34 AM, Andres Freund wrote: > On 2014-09-25 11:13:50 -0400, Robert Haas wrote: >> On Wed, Sep 24, 2014 at 4:36 PM, Simon Riggs wrote: >> >> To me, being able to say "pg_ctl promote_right_now -m yes_i_mean_it" >> >> seems like a friendlier interface than making somebody

Re: [HACKERS] RLS Design

2014-09-25 Thread Thom Brown
On 25 September 2014 15:26, Stephen Frost wrote: >> I expected this to still trigger an error due to the first policy. Am >> I to infer from this that the policy model is permissive rather than >> restrictive? > > That's correct and I believe pretty clear in the documentation- policies > are OR'd

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-19 15:40:14 +0300, Heikki Linnakangas wrote: > On 09/18/2014 09:27 PM, Heikki Linnakangas wrote: > >I'll try to write a more polished patch tomorrow. We'll then see what it > >looks like, and can decide if we want it. > > Ok, here are two patches. One is a refined version of my earlier

Re: [HACKERS] delta relations in AFTER triggers

2014-09-25 Thread Kevin Grittner
Heikki Linnakangas wrote: > You cast the TuplestoreRelation to Plan, and pass it to CopyPlanFields. > That will crash, because TuplestoreRelation is nothing like a Plan: Oops. That's a copy/paste error I should have noticed. Fixed, even though the node type might be going away. Since all of t

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Andres Freund
On 2014-09-25 11:13:50 -0400, Robert Haas wrote: > On Wed, Sep 24, 2014 at 4:36 PM, Simon Riggs wrote: > >> To me, being able to say "pg_ctl promote_right_now -m yes_i_mean_it" > >> seems like a friendlier interface than making somebody shut down the > >> server, run pg_resetxlog, and start it up

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Andres Freund
On 2014-09-24 21:36:50 +0100, Simon Riggs wrote: > On 18 September 2014 01:22, Robert Haas wrote: > > >> "fast" promotion was actually a supported option in r8 of Postgres but > >> this option was removed when we implemented streaming replication in > >> r9.0 > >> > >> The *rough* requirement is

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Sep 25, 2014 at 12:46 AM, Tom Lane wrote: > > TBH I've also been wondering whether any of these proposed cures are > > better than the disease. > > I couldn't agree more. There's something to be said for just leaving > this alone. I've been

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 15:35, Robert Haas wrote: >> The only problem I see is if the newly inserted row matches one row on >> one unique value and a different row on a different unique index. >> Turning the INSERT into an UPDATE will still fail on one or other, no >> matter which index you pick. If

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Robert Haas
On Wed, Sep 24, 2014 at 4:36 PM, Simon Riggs wrote: >> To me, being able to say "pg_ctl promote_right_now -m yes_i_mean_it" >> seems like a friendlier interface than making somebody shut down the >> server, run pg_resetxlog, and start it up again. > > It makes sense to go from paused --> promoted.

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 12:46 AM, Tom Lane wrote: > TBH I've also been wondering whether any of these proposed cures are > better than the disease. I couldn't agree more. There's something to be said for just leaving this alone. > The changes that can be argued to make the > behavior more sane

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 09:34:57 -0500, Merlin Moncure wrote: > On Thu, Sep 25, 2014 at 9:14 AM, Andres Freund wrote: > >> Why stop at 128 mapping locks? Theoretical downsides to having more > >> mapping locks have been mentioned a few times but has this ever been > >> measured? I'm starting to wonder if

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:24 AM, Andres Freund wrote: > On 2014-09-25 10:22:47 -0400, Robert Haas wrote: >> On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund >> wrote: >> > That leads me to wonder: Have you measured different, lower, number of >> > buffer mapping locks? 128 locks is, if we'd as w

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Merlin Moncure
On Thu, Sep 25, 2014 at 9:14 AM, Andres Freund wrote: >> Why stop at 128 mapping locks? Theoretical downsides to having more >> mapping locks have been mentioned a few times but has this ever been >> measured? I'm starting to wonder if the # mapping locks should be >> dependent on some other va

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 10:09:30 -0400, Robert Haas wrote: > I think the long-term solution here is that we need a lock-free hash > table implementation for our buffer mapping tables, because I'm pretty > sure that just cranking the number of locks up and up is going to > start to have unpleasant side effect

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:12 AM, Simon Riggs wrote: > IMHO it is impossible to know if any of the other code is correct > until we have a clear and stable vision of what the command is > supposed to perform. +1. > The inner workings are less important than what the feature does. +1. > FWIW, t

Re: [HACKERS] Index scan optimization

2014-09-25 Thread Simon Riggs
On 22 September 2014 14:46, Heikki Linnakangas wrote: > On 09/22/2014 04:45 PM, Tom Lane wrote: >> >> Heikki Linnakangas writes: >>> >>> On 09/22/2014 07:47 AM, Rajeev rastogi wrote: So my proposal is to skip the condition check on the first scan key condition for every tuple. >> >

Re: [HACKERS] RLS Design

2014-09-25 Thread Stephen Frost
Thom, * Thom Brown (t...@linux.com) wrote: > I find it a bit of a limitation that I can't specify both INSERT and > UPDATE for a policy. I'd want to be able to specify something like > this: > > CREATE POLICY no_greys_allowed > ON colours > FOR INSERT, UPDATE > WITH CHECK (name NOT IN ('gr

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 10:22:47 -0400, Robert Haas wrote: > On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund > wrote: > > That leads me to wonder: Have you measured different, lower, number of > > buffer mapping locks? 128 locks is, if we'd as we should align them > > properly, 8KB of memory. Common L1 cac

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund wrote: > That leads me to wonder: Have you measured different, lower, number of > buffer mapping locks? 128 locks is, if we'd as we should align them > properly, 8KB of memory. Common L1 cache sizes are around 32k... Amit has some results upthread s

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 09:02:25 -0500, Merlin Moncure wrote: > On Thu, Sep 25, 2014 at 8:51 AM, Robert Haas wrote: > > 1. To see the effect of reduce-replacement-locking.patch, compare the > > first TPS number in each line to the third, or the second to the > > fourth. At scale factor 1000, the patch wins

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 09:51:17 -0400, Robert Haas wrote: > On Tue, Sep 23, 2014 at 5:50 PM, Robert Haas wrote: > > The patch I attached the first time was just the last commit in the > > git repository where I wrote the patch, rather than the changes that I > > made on top of that commit. So, yes, the re

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 9:34 AM, Andres Freund wrote: > Binaries compiled on solaris using sun studio cc currently don't have > compiler and memory barriers implemented. That means we fall back to > relatively slow generic implementations for those. Especially compiler, > read, write barriers will

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 28 August 2014 03:43, Peter Geoghegan wrote: > The patch currently lacks a way of referencing datums rejected for > insertion when updating. The way MySQL handles the issue seems > questionable. They allow you to do something like this: > > INSERT INTO upsert (key, val) VALUES (1 'val') ON DUP

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-25 Thread Marko Tiikkaja
On 9/25/14 4:08 PM, Heikki Linnakangas wrote: On 09/25/2014 04:56 PM, Marko Tiikkaja wrote: On 9/25/14 3:50 PM, Heikki Linnakangas wrote: On 09/10/2014 04:35 PM, Marko Tiikkaja wrote: It might've been a tad more efficient to return the StringInfo buffer directly from pgp_armor/dearmor, and avoi

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:02 AM, Merlin Moncure wrote: > On Thu, Sep 25, 2014 at 8:51 AM, Robert Haas wrote: >> 1. To see the effect of reduce-replacement-locking.patch, compare the >> first TPS number in each line to the third, or the second to the >> fourth. At scale factor 1000, the patch wi

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-25 Thread Heikki Linnakangas
On 09/25/2014 04:56 PM, Marko Tiikkaja wrote: On 9/25/14 3:50 PM, Heikki Linnakangas wrote: On 09/10/2014 04:35 PM, Marko Tiikkaja wrote: It might've been a tad more efficient to return the StringInfo buffer directly from pgp_armor/dearmor, and avoid the extra palloc and memcpy, but this isn't p

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Merlin Moncure
On Thu, Sep 25, 2014 at 8:51 AM, Robert Haas wrote: > 1. To see the effect of reduce-replacement-locking.patch, compare the > first TPS number in each line to the third, or the second to the > fourth. At scale factor 1000, the patch wins in all of the cases with > 32 or more clients and exactly h

Re: [HACKERS] missing isinf declaration on solaris

2014-09-25 Thread Andres Freund
On 2014-09-25 10:56:56 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > From VS 2013 onwards they're trying hard to be C99 and C11 compatible. > > Sounds great. Is VS2013 released already? Yes. > If so, maybe we can think about moving to C99 in 2016 or so; at least > assuming you can bui

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Andres Freund
On 2014-09-25 14:43:14 +0100, Simon Riggs wrote: > On 25 September 2014 10:41, Andres Freund wrote: > > On 2014-09-25 10:24:39 +0530, Abhijit Menon-Sen wrote: > >> At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: > >> > I think it's completely unacceptable to copy a visibility routine.

Re: [HACKERS] missing isinf declaration on solaris

2014-09-25 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-09-24 17:39:19 -0300, Alvaro Herrera wrote: > > AFAIK we cannot move all the way to C99, because MSVC doesn't support > > it. > > FWIW, msvc has supported a good part of C99 for long while. There's bits > and pieces it doesn't, but it's not things I think we're like

  1   2   >