Re: [HACKERS] Synchronous Standalone Master Redoux

2012-07-16 Thread Daniel Farina
On Mon, Jul 16, 2012 at 10:58 PM, Heikki Linnakangas wrote: > BTW, one little detail that I don't think has been mentioned in this thread > before: Even though the master currently knows whether a standby is > connected or not, and you could write a patch to act based on that, there > are other fa

Re: [HACKERS] Synchronous Standalone Master Redoux

2012-07-16 Thread Heikki Linnakangas
On 16.07.2012 22:01, Robert Haas wrote: On Sat, Jul 14, 2012 at 7:54 PM, Josh Berkus wrote: So, here's the core issue with degraded mode. I'm not mentioning this to block any patch anyone has, but rather out of a desire to see someone address this core problem with some clever idea I've not th

Re: [HACKERS] [PATCH] lock_timeout and common SIGALRM framework

2012-07-16 Thread Alvaro Herrera
Excerpts from Tom Lane's message of vie jul 13 18:23:31 -0400 2012: > > Boszormenyi Zoltan writes: > >>> Try SET deadlock_timeout = 0; > > Actually, when I try that I get > > ERROR: 0 is outside the valid range for parameter "deadlock_timeout" (1 .. > 2147483647) > > So I don't see any bug

Re: [HACKERS] [v9.3] Row-Level Security

2012-07-16 Thread Kohei KaiGai
2012/7/17 Robert Haas : > On Sun, Jul 15, 2012 at 5:52 AM, Kohei KaiGai wrote: >> The attached patch is a revised version of row-level security feature. >> >> I added a feature to invalidate plan cache if user-id was switched >> between planner and optimizer. It enabled to generate more optimized

Re: [HACKERS] [v9.3] Row-Level Security

2012-07-16 Thread Robert Haas
On Sun, Jul 15, 2012 at 5:52 AM, Kohei KaiGai wrote: > The attached patch is a revised version of row-level security feature. > > I added a feature to invalidate plan cache if user-id was switched > between planner and optimizer. It enabled to generate more optimized > plan than the previous appro

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 9:58 PM, Tom Lane wrote: > BTW, I wonder whether the code that checks for relfilenode conflict > when selecting a pg_class or relfilenode OID tries both file naming > conventions? If not, should we make it do so? I don't believe it does, nor do I see what we would gain by

Re: [HACKERS] b-tree index search algorithms

2012-07-16 Thread Tom Lane
Samuel Vogel writes: > I'm currently on a university research project if performance could be > increased by substituting different inter-node search algorithms instead > of the currently used binary search. Hm, what have you got in mind exactly? > But I'm having troubles understanding how the

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Tom Lane
Robert Haas writes: > With respect to this chunk: > + * We do not need to go through this dance for temp relations, though, > because > + * we never make WAL entries for temp rels, and so a temp rel poses no > threat > + * to the health of a regular rel that has taken over its relfilenode >

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 7:17 PM, Tom Lane wrote: > The attached patch covers everything discussed in this thread, except > for the buggy handling of stats, which I think should be fixed in a > separate patch since it's only relevant to 9.2+. With respect to this chunk: + * We do not need to go

Re: [HACKERS] Using pg_upgrade on log-shipping standby servers

2012-07-16 Thread Jeff Davis
On Tue, 2012-07-10 at 11:50 -0400, Bruce Momjian wrote: > I don't think we can assume that because pg_upgrade was run on the > master and standby that they are binary identical, can we? Technically > the user file are identical, but the system catalogs and WAL might be > different, hence my sugges

[HACKERS] b-tree index search algorithms

2012-07-16 Thread Samuel Vogel
Hello, I'm currently on a university research project if performance could be increased by substituting different inter-node search algorithms instead of the currently used binary search. But I'm having troubles understanding how the general b-tree implementation (nbtree.h) is used to repres

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Tom Lane
Robert Haas writes: > On Mon, Jul 16, 2012 at 2:47 PM, Tom Lane wrote: >> So I started to do this, and immediately hit a roadblock: although it's >> clear that we can discard any attempt to fsync a backend-local relation, >> it's not so clear that we don't need to queue UNLINK_RELATION_REQUEST >>

Re: [HACKERS] Closing out the June commitfest

2012-07-16 Thread Josh Berkus
> Trim trailing NULL columns > https://commitfest.postgresql.org/action/patch_view?id=840 Feh, put my name on this one for a performance test/functionality review. Someone else needs to do the code review though. > None of the three above seem to me to be blocking further work, > so I don't hav

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Tom Lane
Robert Haas writes: > Yes, it seems to have done just that. The comment for > ForwardFsyncRequest is a few bricks short of a load too: > ... > Line 2 seems to have been mechanically changed from "background > writer" to "checkpointer", but of course it should still say > "background writer" in th

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane wrote: > Robert Haas writes: >> At any rate, I'm somewhat less convinced that the split was a good >> idea than I was when we did it, mostly because we haven't really gone >> anywhere with it subsequently. > > BTW, while we are on the subject: hasn't this

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Tom Lane
Robert Haas writes: > At any rate, I'm somewhat less convinced that the split was a good > idea than I was when we did it, mostly because we haven't really gone > anywhere with it subsequently. BTW, while we are on the subject: hasn't this split completely broken the statistics about backend-init

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 2:47 PM, Tom Lane wrote: > I wrote: >> Robert Haas writes: >>> On Mon, Jul 16, 2012 at 11:57 AM, Tom Lane wrote: So that brings us back to the question of why this code is supporting fsync requests for local relations in the first place. Couldn't we have i

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Tom Lane
Robert Haas writes: > Unfortunately, there are lots of important operations (like bulk > loading, SELECT * FROM bigtable, and VACUUM notverybigtable) that > inevitably end up writing out their own dirty buffers. And even when > the background writer does write something, it's not always clear tha

Re: [HACKERS] Synchronous Standalone Master Redoux

2012-07-16 Thread Robert Haas
On Sat, Jul 14, 2012 at 7:54 PM, Josh Berkus wrote: > So, here's the core issue with degraded mode. I'm not mentioning this > to block any patch anyone has, but rather out of a desire to see someone > address this core problem with some clever idea I've not thought of. > The problem in a nutshell

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Tom Lane
I wrote: > Robert Haas writes: >> On Mon, Jul 16, 2012 at 11:57 AM, Tom Lane wrote: >>> So that brings us back to the question of why this code is supporting >>> fsync requests for local relations in the first place. Couldn't we have >>> it ignore those, and then only ship RelFileNode to the che

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 12:57 PM, Tom Lane wrote: > Robert Haas writes: >> In my view, the elephant in the room here is that it's dramatically >> inefficient for every backend to send an fsync request on every block >> write. > > Yeah. This was better before the decision was taken to separate >

Re: [HACKERS] Getting rid of pre-assignment of index names in CREATE TABLE LIKE

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 1:10 PM, Tom Lane wrote: > Robert Haas writes: >> The problem isn't confined to CREATE TABLE LIKE; it's a widespread >> design flaw that will likely take years of work to clean up >> completely. I don't think that's a reason not to commit your change >> though; it fixes a

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 12:27 PM, Tom Lane wrote: > Robert Haas writes: >> The documentation on MacOS X isn't quite as explicit, but I'd still be >> astonished if we found any other behavior. TBH, I'd be kind of >> surprised if this is the only place in our code base that relies on >> the initia

Re: [HACKERS] Getting rid of pre-assignment of index names in CREATE TABLE LIKE

2012-07-16 Thread Tom Lane
Robert Haas writes: > The problem isn't confined to CREATE TABLE LIKE; it's a widespread > design flaw that will likely take years of work to clean up > completely. I don't think that's a reason not to commit your change > though; it fixes a bug and is an incremental improvement, even if a > smal

Re: [HACKERS] Getting rid of pre-assignment of index names in CREATE TABLE LIKE

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 12:43 PM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Jul 14, 2012 at 4:02 PM, Tom Lane wrote: >>> I suggested that we could dodge the problem by allowing IndexStmt to >>> carry a comment to be attached to the new index, and thereby avoid >>> needing an explicit COMM

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Tom Lane
Robert Haas writes: > In my view, the elephant in the room here is that it's dramatically > inefficient for every backend to send an fsync request on every block > write. Yeah. This was better before the decision was taken to separate bgwriter from checkpointer; before that, only local communica

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 12:36 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Jul 16, 2012 at 12:08 PM, Tom Lane wrote: >>> Uh, that's exactly what's under discussion. Not sending useless fsync >>> requests when fsync is off is just one part of it; a part that happens >>> to be quite usefu

Re: [HACKERS] Getting rid of pre-assignment of index names in CREATE TABLE LIKE

2012-07-16 Thread Tom Lane
Robert Haas writes: > On Sat, Jul 14, 2012 at 4:02 PM, Tom Lane wrote: >> I suggested that we could dodge the problem by allowing IndexStmt to >> carry a comment to be attached to the new index, and thereby avoid >> needing an explicit COMMENT command. Attached is a patch that fixes it >> that w

Re: [HACKERS] Getting rid of pre-assignment of index names in CREATE TABLE LIKE

2012-07-16 Thread Robert Haas
On Sat, Jul 14, 2012 at 4:02 PM, Tom Lane wrote: > In bug #6734 we have a complaint about a longstanding misfeature of > CREATE TABLE LIKE. Ordinarily, this command doesn't select names for > copied indexes, but leaves that to be done at runtime by DefineIndex. > But if it's copying comments, and

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Tom Lane
Robert Haas writes: > On Mon, Jul 16, 2012 at 12:08 PM, Tom Lane wrote: >> Uh, that's exactly what's under discussion. Not sending useless fsync >> requests when fsync is off is just one part of it; a part that happens >> to be quite useful for some test scenarios, even if not so much for >> pro

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Tom Lane
Robert Haas writes: > The documentation on MacOS X isn't quite as explicit, but I'd still be > astonished if we found any other behavior. TBH, I'd be kind of > surprised if this is the only place in our code base that relies on > the initial contents of shared memory being all-zeros. Maybe so, b

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 12:08 PM, Tom Lane wrote: > Robert Haas writes: >> On Sun, Jul 15, 2012 at 2:29 PM, Tom Lane wrote: >>> Yeah, you have a point there. It's not real clear that switching fsync >>> from off to on is an operation that we can make any guarantees about, >>> short of executing

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 11:44 AM, Heikki Linnakangas wrote: > I wouldn't rely on that, though. I wouldn't be surprised if there was some > debugging flag or similar that initialized all pages to random values or > 0xdeadbeef or something, before handing them out to the application. We > could easi

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Tom Lane
Robert Haas writes: > On Mon, Jul 16, 2012 at 11:57 AM, Tom Lane wrote: >> BTW, I'd be a lot happier about assuming that bare RelFileNode contains >> no padding, because that's at least got all the fields the same type. >> So that brings us back to the question of why this code is supporting >> f

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Tom Lane
Robert Haas writes: > On Sun, Jul 15, 2012 at 2:29 PM, Tom Lane wrote: >> Yeah, you have a point there. It's not real clear that switching fsync >> from off to on is an operation that we can make any guarantees about, >> short of executing something like the code recently added to initdb >> to f

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Robert Haas
On Mon, Jul 16, 2012 at 11:57 AM, Tom Lane wrote: > I wrote: >> Robert Haas writes: >>> So I'm having a hard time understanding under what imaginable set of >>> circumstances this might break. > >> Padding inside RelFileNodeBackend would break it, because >> ForwardFsyncRequest copies the rnode a

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-16 Thread Marko Kreen
On Mon, Jul 16, 2012 at 6:47 PM, Tom Lane wrote: > Marko Kreen writes: >> On Mon, Jul 16, 2012 at 4:33 PM, Tom Lane wrote: >>> Mm. I still think we should drop it, because it's still a dangerous API >>> that's not necessary for the principal benefit of this feature. > >> Yes, it is a secondary

Re: [HACKERS] [PERFORM] DELETE vs TRUNCATE explanation

2012-07-16 Thread Robert Haas
On Sun, Jul 15, 2012 at 2:29 PM, Tom Lane wrote: > I think what we ought to do is bite the bullet and refactor the > representation of the pendingOps table. What I'm thinking about > is reducing the hash key to just RelFileNodeBackend + ForkNumber, > so that there's one hashtable entry per fork,

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Tom Lane
I wrote: > Robert Haas writes: >> So I'm having a hard time understanding under what imaginable set of >> circumstances this might break. > Padding inside RelFileNodeBackend would break it, because > ForwardFsyncRequest copies the rnode as a struct. So that's why I'm > asking whether we want to

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-16 Thread Tom Lane
Marko Kreen writes: > On Mon, Jul 16, 2012 at 4:33 PM, Tom Lane wrote: >> Mm. I still think we should drop it, because it's still a dangerous API >> that's not necessary for the principal benefit of this feature. > Yes, it is a secondary feature, but it fits the needs of the actual target > aud

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Heikki Linnakangas
On 16.07.2012 18:24, Robert Haas wrote: On Sun, Jul 15, 2012 at 5:36 PM, Tom Lane wrote: We could fairly cheaply dodge the problem with padding after ForkNumber if we were to zero out the whole request array at shmem initialization, so that any such pad bytes are guaranteed zero. However, padd

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Tom Lane
Robert Haas writes: > On Sun, Jul 15, 2012 at 5:36 PM, Tom Lane wrote: >> We could fairly cheaply dodge the problem with padding after ForkNumber >> if we were to zero out the whole request array at shmem initialization, >> so that any such pad bytes are guaranteed zero. However, padding in >> R

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-16 Thread Marko Kreen
On Mon, Jul 16, 2012 at 4:33 PM, Tom Lane wrote: > Marko Kreen writes: >> On Mon, Jul 16, 2012 at 4:11 AM, Tom Lane wrote: >>> I'm starting to look at this patch now. I think we could drop the >>> PQgetRowData() API: it complicates matters for little gain that I can >>> see. The argument for i

Re: [HACKERS] CompactCheckpointerRequestQueue versus pad bytes

2012-07-16 Thread Robert Haas
On Sun, Jul 15, 2012 at 5:36 PM, Tom Lane wrote: > We could fairly cheaply dodge the problem with padding after ForkNumber > if we were to zero out the whole request array at shmem initialization, > so that any such pad bytes are guaranteed zero. However, padding in > RelFileNodeBackend would be

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-16 Thread Tom Lane
Marko Kreen writes: > On Mon, Jul 16, 2012 at 4:11 AM, Tom Lane wrote: >> I'm starting to look at this patch now. I think we could drop the >> PQgetRowData() API: it complicates matters for little gain that I can >> see. The argument for it was to avoid the cost of creating a PGresult >> per ro

Re: [HACKERS] [patch] libpq one-row-at-a-time API

2012-07-16 Thread Marko Kreen
On Mon, Jul 16, 2012 at 4:11 AM, Tom Lane wrote: > Marko Kreen writes: >> Now, looking at the problem with some perspective, the solution >> is obvious: when in single-row mode, the PQgetResult() must return >> proper PGresult for that single row. And everything else follows that. > >> Such API