Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread David Rowley
On Sun, Dec 15, 2013 at 3:08 PM, Tom Lane wrote: > I wrote: > > It's not so good with two-row windows though: > > Actually, carrying that example a bit further makes the point even more > forcefully: > > Table correct sum of negative-transition > this + next val

Re: [HACKERS] encoding name "SHIFT_JIS" is absent in chklocale.c

2013-12-14 Thread Tatsuo Ishii
> I got a complain from a user. > > If current locale is SJIS, psql does not set client encoding to SJIS. > > # localedef -f SHIFT_JIS -i ja_JP /usr/lib/locale/ja_JP.SJIS > $ export LANG=ja_JP.SJIS > $ psql > \encoding > SQL_ASCII <-- This should be SJIS > > This is because the encoding map (enc

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread Tom Lane
I wrote: > It's not so good with two-row windows though: Actually, carrying that example a bit further makes the point even more forcefully: Table correct sum of negative-transition this + next value result 1e201e201e20 + 1

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread Tom Lane
David Rowley writes: > On Sun, Dec 15, 2013 at 2:27 PM, Josh Berkus wrote: >> If we're going to disqualify NUMERIC too, we might as well bounce the >> feature. Without a fast FLOAT or NUMERIC, you've lost most of the >> target audience. > I think the feature is worth it alone if we could improv

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread Tom Lane
Josh Berkus writes: > I think even the FLOAT case deserves some consideration. What's the > worst-case drift? Complete loss of all significant digits. The case I was considering earlier of single-row windows could be made safe (I think) if we apply the negative transition function first, before

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread David Rowley
On Sun, Dec 15, 2013 at 2:27 PM, Josh Berkus wrote: > If we're going to disqualify NUMERIC too, we might as well bounce the > feature. Without a fast FLOAT or NUMERIC, you've lost most of the > target audience. > > I don't agree with this. I'm going with the opinion that the more types and aggre

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread Josh Berkus
On 12/14/2013 05:00 PM, Tom Lane wrote: > This consideration also makes me question whether we should apply the > method for NUMERIC. Although in principle numeric addition/subtraction > is exact, such a sequence could leave us with a different dscale than > is returned by the existing code. I'm

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread Tom Lane
Greg Stark writes: > On 14 Dec 2013 15:40, "Tom Lane" wrote: >> I think you *can't* cover them for the float types; roundoff error >> would mean you don't get the same answers as before. > I was going to say the same thing. But then I started to wonder What's > so special about the answers w

Re: [HACKERS] ruleutils vs. empty targetlists

2013-12-14 Thread Tom Lane
Dean Rasheed writes: > On 13 December 2013 15:07, Tom Lane wrote: >> How about as-is in the back branches, and throw the new errors only >> in HEAD? > Seems reasonable. After further experimentation I've concluded that maybe we'd better not back-patch this change. The reason for my change of h

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread David Rowley
On Sun, Dec 15, 2013 at 12:48 PM, Greg Stark wrote: > > On 14 Dec 2013 15:40, "Tom Lane" wrote: > > > > David Rowley writes: > > > The attached patch is not quite finished yet, I've not yet fully > covered > > > SUM and AVG for all types. > > > > I think you *can't* cover them for the float typ

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread Greg Stark
On 14 Dec 2013 15:40, "Tom Lane" wrote: > > David Rowley writes: > > The attached patch is not quite finished yet, I've not yet fully covered > > SUM and AVG for all types. > > I think you *can't* cover them for the float types; roundoff error > would mean you don't get the same answers as before

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread Tom Lane
David Rowley writes: > The attached patch is not quite finished yet, I've not yet fully covered > SUM and AVG for all types. I think you *can't* cover them for the float types; roundoff error would mean you don't get the same answers as before. regards, tom lane -- Sen

[HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2013-12-14 Thread David Rowley
I've been working on speeding up aggregate functions when used in the context of a window's with non fixed frame heads. Currently if the top of the window is not in a fixed position then when the window frame moves down the aggregates must be recalculated by looping over each record in the new sco

Re: [HACKERS] Extension Templates S03E11

2013-12-14 Thread Josh Berkus
All: Can someone summarize the issues with this patch for those of us who haven't been following it closely? I was just chatting with a couple other contributors, and at this point none of just know what it implements, what it doesn't implement, what the plans are for expanding its feature set (i

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-12-14 Thread Peter Geoghegan
On Fri, Dec 13, 2013 at 4:06 PM, Tom Lane wrote: > I spent a little bit of time looking at btreelock_insert_on_dup. AFAICT > it executes FormIndexDatum() for later indexes while holding assorted > buffer locks in earlier indexes. That really ain't gonna do, because in > the case of an expression

Re: [HACKERS] typo: XIDs are actually compared using modulo-2^32 arithmetic

2013-12-14 Thread Andres Freund
On 2013-12-14 20:19:11 +, Greg Stark wrote: > I don't have a source tree handy but iirc we treaty 2^31 values as being in > the past and 2^31 values as being in the future. > > I've been trying to think how to protect better against the recent vacuum > freeze bug. If someone ruins vacuum freez

Re: [HACKERS] Extension Templates S03E11

2013-12-14 Thread Stephen Frost
* Jeff Davis (pg...@j-davis.com) wrote: > To make much more progress, it seems like we either need an ingenious > idea about how to change existing extensions to work for all purposes, > or we need to invent a new concept. I'm still in favor of supporting SQL/trusted-language only 'extensions' and

Re: [HACKERS] typo: XIDs are actually compared using modulo-2^32 arithmetic

2013-12-14 Thread Greg Stark
I don't have a source tree handy but iirc we treaty 2^31 values as being in the past and 2^31 values as being in the future. I've been trying to think how to protect better against the recent vacuum freeze bug. If someone ruins vacuum freeze now and has any wrapped values they'll destroy their pos

Re: [HACKERS] Extension Templates S03E11

2013-12-14 Thread Jeff Davis
On Fri, 2013-12-13 at 13:42 -0500, Stephen Frost wrote: > * Jeff Davis (pg...@j-davis.com) wrote: > > For what it's worth, I think the idea of extension templates has good > > conceptual integrity. Extensions are external blobs. To make them work > > more smoothly in several ways, we move them into

Re: [HACKERS] PoC: Partial sort

2013-12-14 Thread Andreas Karlsson
On 12/14/2013 10:59 AM, Alexander Korotkov wrote: This patch allows to use index for order-by if order-by clause and index has non-empty common prefix. So, index gives right ordering for first n order-by columns. In order to provide right order for rest m columns, sort node is inserted. This sort

Re: [HACKERS] stats for network traffic WIP

2013-12-14 Thread Jim Nasby
On 12/11/13 12:51 PM, Tom Lane wrote: Atri Sharma writes: On Wed, Dec 11, 2013 at 11:12 PM, Peter Eisentraut wrote: Is there a reason why you can't get this directly from the OS? I would say that its more of a convenience to track the usage directly from the database instead of setting up

Re: [HACKERS] Changeset Extraction Interfaces

2013-12-14 Thread Jim Nasby
On 12/12/13 11:13 AM, Robert Haas wrote: I think it sucks (that's the technical term) to have to wait for all currently-running transactions to terminate before being able to begin streaming changes, because that could take a long time. And you might well know that the long-running transaction w

Re: [HACKERS] stats for network traffic WIP

2013-12-14 Thread Greg Stark
I could see this being interesting for FDW plan nodes of the status were visible in explain. Possibly also time spent waiting on network reads and writes. I have a harder time seeing why it's useful to have these stays in aggregate but I suppose if you had lots of FDW connections or lots of steami

Re: [HACKERS] clang's -Wmissing-variable-declarations shows some shoddy programming

2013-12-14 Thread Andres Freund
On 2013-12-14 12:14:25 -0500, Tom Lane wrote: > Andres Freund writes: > > Compiling postgres with said option in CFLAGS really gives an astounding > > number of warnings. Except some bison/flex generated ones, none of them > > looks acceptable to me. > > Given that we're not going to be able to g

Re: [HACKERS] Changeset Extraction Interfaces

2013-12-14 Thread Andres Freund
On 2013-12-14 11:50:00 -0500, Robert Haas wrote: > On Fri, Dec 13, 2013 at 9:14 AM, Andres Freund wrote: > > But can you imagine those users needing an exported snapshot? I can > > think of several short-lived usages, but all of those are unlikely to > > need a consistent view of the overall datab

Re: [HACKERS] clang's -Wmissing-variable-declarations shows some shoddy programming

2013-12-14 Thread Tom Lane
Andres Freund writes: > Compiling postgres with said option in CFLAGS really gives an astounding > number of warnings. Except some bison/flex generated ones, none of them > looks acceptable to me. Given that we're not going to be able to get rid of the bison/flex cases, is this really something t

Re: [HACKERS] Changeset Extraction Interfaces

2013-12-14 Thread Robert Haas
On Fri, Dec 13, 2013 at 9:14 AM, Andres Freund wrote: >> If you imagine a scenario where somebody establishes a replication >> slot and then keeps it forever, not often. But if you're trying to do >> something more ad hoc, where replication slots might be used just for >> short periods of time an

Re: [HACKERS] Useless "Replica Identity: NOTHING" noise from psql \d

2013-12-14 Thread Andres Freund
On 2013-12-14 11:27:53 -0500, Tom Lane wrote: > In HEAD: > > regression=# \d pg_depend >Table "pg_catalog.pg_depend" >Column| Type | Modifiers > -+-+--- > classid | oid | not null > objid | oid | not null > objsubid| integer | no

[HACKERS] Useless "Replica Identity: NOTHING" noise from psql \d

2013-12-14 Thread Tom Lane
In HEAD: regression=# \d pg_depend Table "pg_catalog.pg_depend" Column| Type | Modifiers -+-+--- classid | oid | not null objid | oid | not null objsubid| integer | not null refclassid | oid | not null refobjid| oid

[HACKERS] clang's -Wmissing-variable-declarations shows some shoddy programming

2013-12-14 Thread Andres Freund
Hi, Compiling postgres with said option in CFLAGS really gives an astounding number of warnings. Except some bison/flex generated ones, none of them looks acceptable to me. Most are just file local variables with a missing static and easy to fix. Several other are actually shared variables, where

Re: [HACKERS] PoC: Partial sort

2013-12-14 Thread Andres Freund
Hi, > > > Limit (cost=69214.06..69214.08 rows=10 width=16) (actual > > > time=0.097..0.099 rows=10 loops=1) > > >-> Sort (cost=69214.06..71714.06 rows=100 width=16) (actual > > > time=0.096..0.097 rows=10 loops=1) > > > Sort Key: v1, v2 > > > Sort Method: top-N heapso

Re: [HACKERS] PoC: Partial sort

2013-12-14 Thread Martijn van Oosterhout
On Sat, Dec 14, 2013 at 06:21:18PM +0400, Alexander Korotkov wrote: > > Is that actually all that beneficial when sorting with a bog standard > > qsort() since that doesn't generally benefit from data being pre-sorted? > > I think we might need to switch to a different algorithm to really > > benef

Re: [HACKERS] PoC: Partial sort

2013-12-14 Thread Jeremy Harris
On 14/12/13 12:54, Andres Freund wrote: On 2013-12-14 13:59:02 +0400, Alexander Korotkov wrote: Currently when we need to get ordered result from table we have to choose one of two approaches: get results from index in exact order we need or do sort of tuples. However, it could be useful to mix

Re: [HACKERS] PoC: Partial sort

2013-12-14 Thread Alexander Korotkov
Hi! Thanks for feedback! On Sat, Dec 14, 2013 at 4:54 PM, Andres Freund wrote: > Hi, > > Cool stuff. > > On 2013-12-14 13:59:02 +0400, Alexander Korotkov wrote: > > Currently when we need to get ordered result from table we have to choose > > one of two approaches: get results from index in exac

Re: [HACKERS] Time-Delayed Standbys

2013-12-14 Thread Andres Freund
On 2013-12-13 13:44:30 +, Simon Riggs wrote: > On 13 December 2013 13:22, Andres Freund wrote: > > On 2013-12-13 13:09:13 +, Simon Riggs wrote: > >> On 13 December 2013 11:58, Andres Freund wrote: > >> >> I removed it because it was after the pause. I'll replace it, but > >> >> before the

Re: [HACKERS] PoC: Partial sort

2013-12-14 Thread Andres Freund
Hi, Cool stuff. On 2013-12-14 13:59:02 +0400, Alexander Korotkov wrote: > Currently when we need to get ordered result from table we have to choose > one of two approaches: get results from index in exact order we need or do > sort of tuples. However, it could be useful to mix both methods: get >

Re: [HACKERS] "stuck spinlock"

2013-12-14 Thread Andres Freund
On 2013-12-13 13:39:42 -0500, Robert Haas wrote: > On Fri, Dec 13, 2013 at 1:15 PM, Andres Freund wrote: > > Agreed on not going forward like now, but I don't really see how they > > could usefully use die(). I think we should just mandate that every > > bgworker conneced to shared memory register

Re: [HACKERS] "stuck spinlock"

2013-12-14 Thread Andres Freund
Hi, On 2013-12-13 15:57:14 -0300, Alvaro Herrera wrote: > If there was a way for raising an #error at compile time whenever a > worker relies on the existing signal handler, I would vote for doing > that. (But then I have no idea how to do such a thing.) I don't see a way either given how discon

Re: [HACKERS] "stuck spinlock"

2013-12-14 Thread Andres Freund
On 2013-12-13 15:49:45 -0600, Merlin Moncure wrote: > On Fri, Dec 13, 2013 at 12:32 PM, Robert Haas wrote: > > On Fri, Dec 13, 2013 at 11:26 AM, Tom Lane wrote: > >> And while we're on the subject ... isn't bgworker_die() utterly and > >> completely broken? That unconditional elog(FATAL) means t

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-14 Thread Andres Freund
On 2013-12-13 17:08:46 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Afaics this will cause HEAP_KEYS_UPDATED to be reset, is that > > problematic? I don't really remember what it's needed for TBH... > > So we do reset HEAP_KEYS_UPDATED, and in general that bit seems critical > for sever

Re: [HACKERS] Compression of tables

2013-12-14 Thread Thomas Munro
On 10 December 2013 15:15, Merlin Moncure wrote: > I doubt you'll ever see generally heap compressed data in the way > you're thinking: postgres has a strong informal policy of not > implementing features which are dubious and or excessively complicated > with limited benefit, particularly if the