Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-25 Thread Fabien COELHO
Hello Robert, I think you're making a mountain out of a molehill. Probably. I tend to try the minimum effort first. I implemented this today in about three hours. The patch is attached. Great! Your patch is 544 lines, my size evaluation was quite good:-) Note that I probably spent 10 m

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-25 Thread Fabien COELHO
I think you're making a mountain out of a molehill. I implemented this today in about three hours. I think you're greatly understating your own efficiency at shift/reducing parser mountains down to molehills. Fabien even guessed the LOC size of the resulting patch with less than a 9% error.

Re: [HACKERS] Selectivity estimation for inet operators

2014-09-25 Thread Heikki Linnakangas
On 09/07/2014 07:09 PM, Emre Hasegeli wrote: I updated the patch to cover semi and anti joins with eqjoinsel_semi(). I think it is better than returning a constant. What you did there is utterly unacceptable from a modularity standpoint; and considering that the values will be nowhere near righ

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Andres Freund
On 2014-09-25 10:24:39 +0530, Abhijit Menon-Sen wrote: > At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: > > I think it's completely unacceptable to copy a visibility routine. > > OK. Which visibility routine should I use, and should I try to create a > variant that doesn't set hint bi

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Abhijit Menon-Sen
At 2014-09-25 11:41:29 +0200, and...@2ndquadrant.com wrote: > > I've not yet followed your premise that you actually need one that > doesn't set hint bits... Oh. All right, then I'll post a version that addresses Amit's other points, adds a new file/function to pgstattuple, acquires content locks

Re: [HACKERS] make pg_controldata accept "-D dirname"

2014-09-25 Thread Heikki Linnakangas
On 09/24/2014 05:48 PM, Abhijit Menon-Sen wrote: Updated patches attached. Thanks, applied some version of these. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Gregory Smith
On 9/25/14, 1:41 AM, David Johnston wrote: If the error message is written correctly most people upon seeing the error will simply fix their configuration and move on - regardless of whether they were proactive in doing so having read the release notes. The important part to realize here is th

Re: [HACKERS] Patch to support SEMI and ANTI join removal

2014-09-25 Thread Heikki Linnakangas
On 09/16/2014 01:20 PM, David Rowley wrote: + /* +* We mustn't allow any joins to be removed if there are any pending +* foreign key triggers in the queue. This could happen if we are planning +* a query that has been executed from within a volatile function and th

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 1:27 AM, Gregory Smith wrote: > On 9/24/14, 10:10 PM, Robert Haas wrote: >> I think you're making a mountain out of a molehill. I implemented this >> today in about three hours. > > I think you're greatly understating your own efficiency at shift/reducing > parser mountains

Re: [HACKERS] Sloppy thinking about leakproof properties of opclass co-members

2014-09-25 Thread Robert Haas
On Wed, Sep 24, 2014 at 5:34 PM, Tom Lane wrote: > It strikes me that there's a significant gap in the whole "leakproof > function" business, namely that no consideration has been given to > planner-driven transformations of queries. As an example, if we > have "a = b" and "b = c", the planner ma

Re: [HACKERS] add modulo (%) operator to pgbench

2014-09-25 Thread Gregory Smith
On 9/25/14, 8:38 AM, Robert Haas wrote: That's my whole reason for not wanting to adopt Fabien's approach in the first place: I was cool with exposing C's modulo operator, but any other modulo semantics seem like they should be built up from general-purpose primitives. Maybe; I don't quite u

Re: [HACKERS] Spinlocks and compiler/memory barriers

2014-09-25 Thread Robert Haas
On Wed, Sep 24, 2014 at 2:45 PM, Andres Freund wrote: > On 2014-09-09 17:54:03 -0400, Robert Haas wrote: >> So, that's committed, then. I think we should pick something that uses >> spinlocks and is likely to fail spectacularly if we haven't got this >> totally right yet, and de-volatilize it. An

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread David Johnston
On Thursday, September 25, 2014, Gregory Smith wrote: > On 9/25/14, 1:41 AM, David Johnston wrote: > >> If the error message is written correctly most people upon seeing the >> error will simply fix their configuration and move on - regardless of >> whether they were proactive in doing so having

[HACKERS] Inefficient barriers on solaris with sun cc

2014-09-25 Thread Andres Freund
Hi, Binaries compiled on solaris using sun studio cc currently don't have compiler and memory barriers implemented. That means we fall back to relatively slow generic implementations for those. Especially compiler, read, write barriers will be much slower than necessary (since they all just need t

Re: [HACKERS] RLS Design

2014-09-25 Thread Thom Brown
On 19 September 2014 17:54, Stephen Frost wrote: > > Thom, > > * Thom Brown (t...@linux.com) wrote: > > On 19 September 2014 17:32, Stephen Frost wrote: > > > * Thom Brown (t...@linux.com) wrote: > > > > On 14 September 2014 16:38, Stephen Frost wrote: > > > > # create policy visible_colours on

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Simon Riggs
On 25 September 2014 10:41, Andres Freund wrote: > On 2014-09-25 10:24:39 +0530, Abhijit Menon-Sen wrote: >> At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: >> > I think it's completely unacceptable to copy a visibility routine. >> >> OK. Which visibility routine should I use, and shou

Re: [HACKERS] missing isinf declaration on solaris

2014-09-25 Thread Andres Freund
On 2014-09-24 17:39:19 -0300, Alvaro Herrera wrote: > Tom Lane wrote: > > Peter Eisentraut writes: > > > On 9/24/14 9:21 AM, Tom Lane wrote: > > >> Agreed, but what about non-GCC compilers? > > > > > Stick AC_PROG_CC_C99 into configure.in. > > > > I think that's a bad idea, unless you mean to do

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-25 Thread Heikki Linnakangas
On 09/10/2014 04:35 PM, Marko Tiikkaja wrote: On 9/10/14 1:38 PM, Heikki Linnakangas wrote: On 09/10/2014 02:26 PM, Marko Tiikkaja wrote: So I wonder if I shouldn't try and instead keep the code closer to what it is in HEAD right now; I could call enlargeStringInfo() first, then hand out a poin

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Tue, Sep 23, 2014 at 5:50 PM, Robert Haas wrote: > The patch I attached the first time was just the last commit in the > git repository where I wrote the patch, rather than the changes that I > made on top of that commit. So, yes, the results from the previous > message are with the patch atta

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-25 Thread Marko Tiikkaja
On 9/25/14 3:50 PM, Heikki Linnakangas wrote: On 09/10/2014 04:35 PM, Marko Tiikkaja wrote: OK, I've attemped to do that in the attached. I'm pretty sure I didn't get all of the overflows right, so someone should probably take a really good look at it. (I'm not too confident the original code

Re: [HACKERS] missing isinf declaration on solaris

2014-09-25 Thread Oskari Saarenmaa
24.09.2014, 23:26, Tom Lane kirjoitti: Peter Eisentraut writes: On 9/24/14 9:21 AM, Tom Lane wrote: Agreed, but what about non-GCC compilers? Stick AC_PROG_CC_C99 into configure.in. I think that's a bad idea, unless you mean to do it only on Solaris. If we do that unconditionally, we will

Re: [HACKERS] missing isinf declaration on solaris

2014-09-25 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-09-24 17:39:19 -0300, Alvaro Herrera wrote: > > AFAIK we cannot move all the way to C99, because MSVC doesn't support > > it. > > FWIW, msvc has supported a good part of C99 for long while. There's bits > and pieces it doesn't, but it's not things I think we're like

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Andres Freund
On 2014-09-25 14:43:14 +0100, Simon Riggs wrote: > On 25 September 2014 10:41, Andres Freund wrote: > > On 2014-09-25 10:24:39 +0530, Abhijit Menon-Sen wrote: > >> At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: > >> > I think it's completely unacceptable to copy a visibility routine.

Re: [HACKERS] missing isinf declaration on solaris

2014-09-25 Thread Andres Freund
On 2014-09-25 10:56:56 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > From VS 2013 onwards they're trying hard to be C99 and C11 compatible. > > Sounds great. Is VS2013 released already? Yes. > If so, maybe we can think about moving to C99 in 2016 or so; at least > assuming you can bui

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Merlin Moncure
On Thu, Sep 25, 2014 at 8:51 AM, Robert Haas wrote: > 1. To see the effect of reduce-replacement-locking.patch, compare the > first TPS number in each line to the third, or the second to the > fourth. At scale factor 1000, the patch wins in all of the cases with > 32 or more clients and exactly h

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-25 Thread Heikki Linnakangas
On 09/25/2014 04:56 PM, Marko Tiikkaja wrote: On 9/25/14 3:50 PM, Heikki Linnakangas wrote: On 09/10/2014 04:35 PM, Marko Tiikkaja wrote: It might've been a tad more efficient to return the StringInfo buffer directly from pgp_armor/dearmor, and avoid the extra palloc and memcpy, but this isn't p

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:02 AM, Merlin Moncure wrote: > On Thu, Sep 25, 2014 at 8:51 AM, Robert Haas wrote: >> 1. To see the effect of reduce-replacement-locking.patch, compare the >> first TPS number in each line to the third, or the second to the >> fourth. At scale factor 1000, the patch wi

Re: [HACKERS] pgcrypto: PGP armor headers

2014-09-25 Thread Marko Tiikkaja
On 9/25/14 4:08 PM, Heikki Linnakangas wrote: On 09/25/2014 04:56 PM, Marko Tiikkaja wrote: On 9/25/14 3:50 PM, Heikki Linnakangas wrote: On 09/10/2014 04:35 PM, Marko Tiikkaja wrote: It might've been a tad more efficient to return the StringInfo buffer directly from pgp_armor/dearmor, and avoi

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 28 August 2014 03:43, Peter Geoghegan wrote: > The patch currently lacks a way of referencing datums rejected for > insertion when updating. The way MySQL handles the issue seems > questionable. They allow you to do something like this: > > INSERT INTO upsert (key, val) VALUES (1 'val') ON DUP

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 9:34 AM, Andres Freund wrote: > Binaries compiled on solaris using sun studio cc currently don't have > compiler and memory barriers implemented. That means we fall back to > relatively slow generic implementations for those. Especially compiler, > read, write barriers will

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 09:51:17 -0400, Robert Haas wrote: > On Tue, Sep 23, 2014 at 5:50 PM, Robert Haas wrote: > > The patch I attached the first time was just the last commit in the > > git repository where I wrote the patch, rather than the changes that I > > made on top of that commit. So, yes, the re

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 09:02:25 -0500, Merlin Moncure wrote: > On Thu, Sep 25, 2014 at 8:51 AM, Robert Haas wrote: > > 1. To see the effect of reduce-replacement-locking.patch, compare the > > first TPS number in each line to the third, or the second to the > > fourth. At scale factor 1000, the patch wins

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund wrote: > That leads me to wonder: Have you measured different, lower, number of > buffer mapping locks? 128 locks is, if we'd as we should align them > properly, 8KB of memory. Common L1 cache sizes are around 32k... Amit has some results upthread s

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 10:22:47 -0400, Robert Haas wrote: > On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund > wrote: > > That leads me to wonder: Have you measured different, lower, number of > > buffer mapping locks? 128 locks is, if we'd as we should align them > > properly, 8KB of memory. Common L1 cac

Re: [HACKERS] RLS Design

2014-09-25 Thread Stephen Frost
Thom, * Thom Brown (t...@linux.com) wrote: > I find it a bit of a limitation that I can't specify both INSERT and > UPDATE for a policy. I'd want to be able to specify something like > this: > > CREATE POLICY no_greys_allowed > ON colours > FOR INSERT, UPDATE > WITH CHECK (name NOT IN ('gr

Re: [HACKERS] Index scan optimization

2014-09-25 Thread Simon Riggs
On 22 September 2014 14:46, Heikki Linnakangas wrote: > On 09/22/2014 04:45 PM, Tom Lane wrote: >> >> Heikki Linnakangas writes: >>> >>> On 09/22/2014 07:47 AM, Rajeev rastogi wrote: So my proposal is to skip the condition check on the first scan key condition for every tuple. >> >

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:12 AM, Simon Riggs wrote: > IMHO it is impossible to know if any of the other code is correct > until we have a clear and stable vision of what the command is > supposed to perform. +1. > The inner workings are less important than what the feature does. +1. > FWIW, t

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 10:09:30 -0400, Robert Haas wrote: > I think the long-term solution here is that we need a lock-free hash > table implementation for our buffer mapping tables, because I'm pretty > sure that just cranking the number of locks up and up is going to > start to have unpleasant side effect

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Merlin Moncure
On Thu, Sep 25, 2014 at 9:14 AM, Andres Freund wrote: >> Why stop at 128 mapping locks? Theoretical downsides to having more >> mapping locks have been mentioned a few times but has this ever been >> measured? I'm starting to wonder if the # mapping locks should be >> dependent on some other va

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 10:24 AM, Andres Freund wrote: > On 2014-09-25 10:22:47 -0400, Robert Haas wrote: >> On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund >> wrote: >> > That leads me to wonder: Have you measured different, lower, number of >> > buffer mapping locks? 128 locks is, if we'd as w

Re: [HACKERS] Scaling shared buffer eviction

2014-09-25 Thread Andres Freund
On 2014-09-25 09:34:57 -0500, Merlin Moncure wrote: > On Thu, Sep 25, 2014 at 9:14 AM, Andres Freund wrote: > >> Why stop at 128 mapping locks? Theoretical downsides to having more > >> mapping locks have been mentioned a few times but has this ever been > >> measured? I'm starting to wonder if

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 12:46 AM, Tom Lane wrote: > TBH I've also been wondering whether any of these proposed cures are > better than the disease. I couldn't agree more. There's something to be said for just leaving this alone. > The changes that can be argued to make the > behavior more sane

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Robert Haas
On Wed, Sep 24, 2014 at 4:36 PM, Simon Riggs wrote: >> To me, being able to say "pg_ctl promote_right_now -m yes_i_mean_it" >> seems like a friendlier interface than making somebody shut down the >> server, run pg_resetxlog, and start it up again. > > It makes sense to go from paused --> promoted.

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 15:35, Robert Haas wrote: >> The only problem I see is if the newly inserted row matches one row on >> one unique value and a different row on a different unique index. >> Turning the INSERT into an UPDATE will still fail on one or other, no >> matter which index you pick. If

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Sep 25, 2014 at 12:46 AM, Tom Lane wrote: > > TBH I've also been wondering whether any of these proposed cures are > > better than the disease. > > I couldn't agree more. There's something to be said for just leaving > this alone. I've been

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Andres Freund
On 2014-09-24 21:36:50 +0100, Simon Riggs wrote: > On 18 September 2014 01:22, Robert Haas wrote: > > >> "fast" promotion was actually a supported option in r8 of Postgres but > >> this option was removed when we implemented streaming replication in > >> r9.0 > >> > >> The *rough* requirement is

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Andres Freund
On 2014-09-25 11:13:50 -0400, Robert Haas wrote: > On Wed, Sep 24, 2014 at 4:36 PM, Simon Riggs wrote: > >> To me, being able to say "pg_ctl promote_right_now -m yes_i_mean_it" > >> seems like a friendlier interface than making somebody shut down the > >> server, run pg_resetxlog, and start it up

Re: [HACKERS] delta relations in AFTER triggers

2014-09-25 Thread Kevin Grittner
Heikki Linnakangas wrote: > You cast the TuplestoreRelation to Plan, and pass it to CopyPlanFields. > That will crash, because TuplestoreRelation is nothing like a Plan: Oops. That's a copy/paste error I should have noticed. Fixed, even though the node type might be going away. Since all of t

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-19 15:40:14 +0300, Heikki Linnakangas wrote: > On 09/18/2014 09:27 PM, Heikki Linnakangas wrote: > >I'll try to write a more polished patch tomorrow. We'll then see what it > >looks like, and can decide if we want it. > > Ok, here are two patches. One is a refined version of my earlier

Re: [HACKERS] RLS Design

2014-09-25 Thread Thom Brown
On 25 September 2014 15:26, Stephen Frost wrote: >> I expected this to still trigger an error due to the first policy. Am >> I to infer from this that the policy model is permissive rather than >> restrictive? > > That's correct and I believe pretty clear in the documentation- policies > are OR'd

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 11:34 AM, Andres Freund wrote: > On 2014-09-25 11:13:50 -0400, Robert Haas wrote: >> On Wed, Sep 24, 2014 at 4:36 PM, Simon Riggs wrote: >> >> To me, being able to say "pg_ctl promote_right_now -m yes_i_mean_it" >> >> seems like a friendlier interface than making somebody

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 11:21 AM, Simon Riggs wrote: > On 25 September 2014 15:35, Robert Haas wrote: >>> The only problem I see is if the newly inserted row matches one row on >>> one unique value and a different row on a different unique index. >>> Turning the INSERT into an UPDATE will still f

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Robert Haas
On Wed, Sep 24, 2014 at 7:04 PM, Peter Geoghegan wrote: > On Fri, Sep 19, 2014 at 2:54 PM, Peter Geoghegan wrote: >> Probably not - it appears to make very little difference to >> unoptimized pass-by-reference types whether or not datum1 can be used >> (see my simulation of Kevin's worst case, fo

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 09:01 AM, Andres Freund wrote: > But independent of which version is chosen, we *REALLY* need to make the > decision soon. This issue has held up the next beta (like jsonb has > blocked previous beta) for *weeks*. Yes, please! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Emre Hasegeli
> Fixed, thanks. Here are my questions and comments about the code. doc/src/sgml/gist.sgml:812: >be rechecked from heap tuple before tuple is returned. If >recheck flag isn't set then it's true by default for >compatibility reasons. The recheck flag can be used only Rec

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-25 Thread Jeff Janes
On Wed, Sep 24, 2014 at 2:48 AM, Dilip kumar wrote: > On 24 August 2014 11:33, Amit Kapila Wrote > > > > Thanks for you comments, i have worked on both the review comment lists, > sent on 19 August, and 24 August. > > > > Latest patch is attached with the mail.. > Hi Dilip, I think you have an

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Bruce Momjian
On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: > But independent of which version is chosen, we *REALLY* need to make the > decision soon. This issue has held up the next beta (like jsonb has > blocked previous beta) for *weeks*. > > Personally it doesn't make me very happy that He

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Simon Riggs
On 25 September 2014 16:29, Andres Freund wrote: >> > To me, being able to say "pg_ctl promote_right_now -m yes_i_mean_it" >> > seems like a friendlier interface than making somebody shut down the >> > server, run pg_resetxlog, and start it up again. >> >> It makes sense to go from paused --> pro

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 10:14 AM, Bruce Momjian wrote: > On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: >> But independent of which version is chosen, we *REALLY* need to make the >> decision soon. This issue has held up the next beta (like jsonb has >> blocked previous beta) for *weeks*. >>

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 10:18:24 -0700, Josh Berkus wrote: > On 09/25/2014 10:14 AM, Bruce Momjian wrote: > > On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: > >> But independent of which version is chosen, we *REALLY* need to make the > >> decision soon. This issue has held up the next beta (

Re: [HACKERS] KNN-GiST with recheck

2014-09-25 Thread Alexander Korotkov
On Thu, Sep 25, 2014 at 9:00 PM, Emre Hasegeli wrote: > > Fixed, thanks. > > Here are my questions and comments about the code. > > doc/src/sgml/gist.sgml:812: > >be rechecked from heap tuple before tuple is returned. If > >recheck flag isn't set then it's true by default for > >

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 10:20 AM, Andres Freund wrote: > On 2014-09-25 10:18:24 -0700, Josh Berkus wrote: >> On 09/25/2014 10:14 AM, Bruce Momjian wrote: >>> On Thu, Sep 25, 2014 at 06:01:08PM +0200, Andres Freund wrote: But independent of which version is chosen, we *REALLY* need to make the decis

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Andres Freund
On 2014-09-25 18:18:09 +0100, Simon Riggs wrote: > On 25 September 2014 16:29, Andres Freund wrote: > > I think that's not really related. Such a promotion doesn't cause data > > loss in the sense of loosing data a *clueful* operator wanted to > > keep. Yes, it can be used wrongly, but it's far fr

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 10:26 AM, Andres Freund wrote: > On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: >> If Heikki says it's ready, I'll test. So far he's said that it wasn't >> done yet. > > http://www.postgresql.org/message-id/541c242e.3030...@vmware.com Yeah, and that didn't include some of Tom's b

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 10:29:51 -0700, Josh Berkus wrote: > On 09/25/2014 10:26 AM, Andres Freund wrote: > > On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: > >> If Heikki says it's ready, I'll test. So far he's said that it wasn't > >> done yet. > > > > http://www.postgresql.org/message-id/541c242e.3030

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: > If Heikki says it's ready, I'll test. So far he's said that it wasn't > done yet. http://www.postgresql.org/message-id/541c242e.3030...@vmware.com Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ Pos

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 1:30 PM, Andres Freund wrote: >> Yes it does cause data loss. The clueful operator has no idea where >> they are so there is no "used rightly" in that case. > > What? There definitely are cases where you don't need to know that to > the T. Just think of the - quite frequent

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Peter Eisentraut
On 9/25/14 11:03 AM, Robert Haas wrote: > I couldn't agree more. There's something to be said for just leaving > this alone. I agree. > potentitally draw complaints. But I also agree with his last one - of > those three possible complaints, I certainly prefer "I had to fix my > configuration fi

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 9:21 AM, Robert Haas wrote: > The top issue on my agenda is figuring out a way to get rid of the > extra SortSupport object. Really? I'm surprised. Clearly the need to restart heap tuple copying from scratch, in order to make the datum1 representation consistent, rather th

Re: [HACKERS] GIN improvements part2: fast scan

2014-09-25 Thread Thom Brown
On 12 March 2014 16:29, Heikki Linnakangas wrote: > Good point. We have done two major changes to GIN in this release cycle: > changed the data page format and made it possible to skip items without > fetching all the keys ("fast scan"). gincostestimate doesn't know about > either change. Did thi

Re: [HACKERS] Review of GetUserId() Usage

2014-09-25 Thread Peter Eisentraut
On 9/24/14 4:58 PM, Stephen Frost wrote: > Alvaro, > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: >> I think the case for pgstat_get_backend_current_activity() and >> pg_stat_get_activity and the other pgstatfuncs.c callers is easy to make >> and seems acceptable to me; but I would leave p

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 28 August 2014 03:43, Peter Geoghegan wrote: > "Value locking" > === > > To date, on-list discussion around UPSERT has almost exclusively > concerned what I've called "value locking"; the idea of locking values > in unique indexes in the abstract (to establish the right to insert > ahe

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Tom Lane
Josh Berkus writes: > On 09/25/2014 10:26 AM, Andres Freund wrote: >> On 2014-09-25 10:25:24 -0700, Josh Berkus wrote: >>> If Heikki says it's ready, I'll test. So far he's said that it wasn't >>> done yet. >> http://www.postgresql.org/message-id/541c242e.3030...@vmware.com > Yeah, and that did

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Josh Berkus
On 09/25/2014 11:22 AM, Tom Lane wrote: > In the interests of pushing this forward, I will work today on > trying to finish and review Heikki's offsets-and-lengths patch > so that we have something we can do performance testing on. > I doubt that the performance testing will tell us anything we > d

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 11:17 AM, Simon Riggs wrote: > Basically, I have absolutely no idea whether I object to or agree with > 1) and don't know where to look to find out. We need a clear > exposition of design and the alternatives. > > My approach would be to insert an index tuple for that value

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Tom Lane
BTW, it seems like there is consensus that we ought to reorder the items in a jsonb object to have keys first and then values, independently of the other issues under discussion. This means we *will* be breaking on-disk compatibility with 9.4beta2, which means pg_upgrade will need to be taught to

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 7:35 AM, Robert Haas wrote: > On Thu, Sep 25, 2014 at 10:12 AM, Simon Riggs wrote: >> IMHO it is impossible to know if any of the other code is correct >> until we have a clear and stable vision of what the command is >> supposed to perform. > > +1. > >> The inner workings

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Bruce Momjian
On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: > BTW, it seems like there is consensus that we ought to reorder the items > in a jsonb object to have keys first and then values, independently of the > other issues under discussion. This means we *will* be breaking on-disk > compatibilit

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 2:05 PM, Peter Geoghegan wrote: > On Thu, Sep 25, 2014 at 9:21 AM, Robert Haas wrote: >> The top issue on my agenda is figuring out a way to get rid of the >> extra SortSupport object. > > Really? I'm surprised. Clearly the need to restart heap tuple copying > from scratch

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 9:20 AM, Robert Haas wrote: > I've never been a fan of putting the index name in there. Me neither. Although I do understand Kevin's concern about the user's intent surrounding which unique index to merge on. > I agree > that's stuff that a DML statement shouldn't need to

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Andres Freund
On 2014-09-25 14:46:18 -0400, Bruce Momjian wrote: > On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: > > BTW, it seems like there is consensus that we ought to reorder the items > > in a jsonb object to have keys first and then values, independently of the > > other issues under discussio

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 2:17 PM, Simon Riggs wrote: >> 1. You don't accept that value locks must be easily released in the >> event of a conflict. Is anyone in this camp? It's far from obvious to >> me what side of this question Andres is on at this stage, for example. >> Robert might have somethi

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 11:53 AM, Robert Haas wrote: > I haven't looked at that part of the patch in detail yet, so... not > really. But I don't see why you'd ever need to restart heap tuple > copying. At most you'd need to re-extract datum1 from the tuples you > have already copied. Well, okay

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Bruce Momjian
On Thu, Sep 25, 2014 at 09:00:07PM +0200, Andres Freund wrote: > On 2014-09-25 14:46:18 -0400, Bruce Momjian wrote: > > On Thu, Sep 25, 2014 at 02:39:37PM -0400, Tom Lane wrote: > > > BTW, it seems like there is consensus that we ought to reorder the items > > > in a jsonb object to have keys first

Re: [HACKERS] jsonb format is pessimal for toast compression

2014-09-25 Thread Alvaro Herrera
Bruce Momjian wrote: > 3. 9.3 multi-xact bugs spooked us into being more careful Uh. Multixact changes in 9.3 were infinitely more invasive than the jsonb changes will ever be. a) they touched basic visibility design and routines, which are complex, understood by very few people, and ha

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 12:11 PM, Robert Haas wrote: > I think that something like this might work, but the devil is in the > details. Suppose two people try to upsert into the same table at the > same time. There's one index. If the transactions search that index > for conflicts first, neither

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 7:12 AM, Simon Riggs wrote: > The way forwards, in my view, is to define precisely the behaviour we > wish to have. That definition will include the best current mechanism > for running an UPSERT using INSERT/UPDATE/loops and comparing that > against what is being provided

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-09-25 Thread Jeff Janes
On Thu, Sep 25, 2014 at 10:00 AM, Jeff Janes wrote: > On Wed, Sep 24, 2014 at 2:48 AM, Dilip kumar > wrote: > >> On 24 August 2014 11:33, Amit Kapila Wrote >> >> >> >> Thanks for you comments, i have worked on both the review comment lists, >> sent on 19 August, and 24 August. >> >> >> >> Lates

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Gregory Smith
On 9/25/14, 2:02 PM, Peter Eisentraut wrote: But having the same parameter setting mean different things in different versions is the path to complete madness. Could we go so far as to remove support for unitless time settings eventually? The fact that people are setting raw numbers in the

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 20:11, Robert Haas wrote: >> My approach would be to insert an index tuple for that value into the >> index, but with the leaf ituple marked with an xid rather than a ctid. >> If someone tries to insert into the index they would see this and wait >> for the inserting transact

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 19:59, Peter Geoghegan wrote: > On Thu, Sep 25, 2014 at 9:20 AM, Robert Haas wrote: >> I've never been a fan of putting the index name in there. > > Me neither. Although I do understand Kevin's concern about the user's > intent surrounding which unique index to merge on. The

Re: [HACKERS] Review of GetUserId() Usage

2014-09-25 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: > On 9/24/14 4:58 PM, Stephen Frost wrote: > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > >> I think the case for pgstat_get_backend_current_activity() and > >> pg_stat_get_activity and the other pgstatfuncs.c callers is easy to make > >> and se

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-09-25 Thread Robert Haas
On Thu, Sep 25, 2014 at 3:17 PM, Peter Geoghegan wrote: >> To find out how much that optimization buys, you >> should use tuples with many variable-length columns (say, 50) >> preceding the text column you're sorting on. I won't be surprised if >> that turns out to be expensive enough to be worth

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Andres Freund
Hi, On 2014-09-24 17:06:05 +0530, Abhijit Menon-Sen wrote: > Hi Andres, Robert. > > I've attached four patches here. Cool. Will review. > 1. Move the call to ResetUnloggedRelations(UNLOGGED_RELATION_INIT) to >earlier in StartupXLOG. > > 2. Inside that function, issue fsync()s for the main

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Simon Riggs
On 25 September 2014 20:38, Peter Geoghegan wrote: > On Thu, Sep 25, 2014 at 7:12 AM, Simon Riggs wrote: >> The way forwards, in my view, is to define precisely the behaviour we >> wish to have. That definition will include the best current mechanism >> for running an UPSERT using INSERT/UPDATE/l

Re: [HACKERS] proposal: rounding up time value less than its unit.

2014-09-25 Thread Stephen Frost
* Gregory Smith (gregsmithpg...@gmail.com) wrote: > On 9/25/14, 2:02 PM, Peter Eisentraut wrote: > >But having the same parameter setting mean different things in > >different versions is the path to complete madness. > > Could we go so far as to remove support for unitless time settings > eventua

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-09-25 Thread Peter Geoghegan
On Thu, Sep 25, 2014 at 1:21 PM, Simon Riggs wrote: > The test index is unique, so our to-be-inserted value exists on only > one page, hence page locking applies while we insert it. The next > person to insert waits for the page lock and then sees the test tuple. > > The page lock lasts only for t

Re: [HACKERS] Immediate standby promotion

2014-09-25 Thread Simon Riggs
On 25 September 2014 18:30, Andres Freund wrote: > On 2014-09-25 18:18:09 +0100, Simon Riggs wrote: >> On 25 September 2014 16:29, Andres Freund wrote: >> > I think that's not really related. Such a promotion doesn't cause data >> > loss in the sense of loosing data a *clueful* operator wanted to

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Abhijit Menon-Sen
At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: > > * Also recovery shouldn't be regarded successful if the reset fails - > * e.g. because of ENOSPC. OK. > * Doing this in a separate pass is advantageous for performance reasons > * because it allows the kernel to perform all the flu

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-09-25 Thread Andres Freund
On 2014-09-26 02:34:06 +0530, Abhijit Menon-Sen wrote: > At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: > > Unless I miss something this isn't sufficient. We need to fsync the > > files in the data directory, not just the toplevel directory? > > No, of course you're right. So a separa

  1   2   >