Re: [HACKERS] [Bug] Inconsistent result for inheritance and FOR UPDATE.

2014-12-12 Thread Etsuro Fujita
(2014/12/12 11:33), Etsuro Fujita wrote: > (2014/12/12 11:19), Tom Lane wrote: >> Etsuro Fujita writes: >>> (2014/12/12 10:37), Tom Lane wrote: Yeah, this is clearly a thinko: really, nothing in the planner should be using get_parse_rowmark(). I looked around for other errors of the >>>

Re: [HACKERS] logical column ordering

2014-12-12 Thread Andres Freund
On 2014-12-10 19:06:28 -0800, Josh Berkus wrote: > On 12/10/2014 05:14 PM, Stephen Frost wrote: > > * Andres Freund (and...@2ndquadrant.com) wrote: > >> > But the scheduling of commits with regard to the 9.5 schedule actually > >> > opens a relevant question: When are we planning to release 9.5? Be

Re: [HACKERS] Review of Refactoring code for sync node detection

2014-12-12 Thread Heikki Linnakangas
On 12/12/2014 04:29 AM, Michael Paquier wrote: On Thu, Dec 11, 2014 at 10:07 PM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: I propose the attached (I admit I haven't tested it). Actually if you do it this way I think that it would be worth adding the small optimization Fujii-san men

Re: [HACKERS] Review of Refactoring code for sync node detection

2014-12-12 Thread Michael Paquier
On Fri, Dec 12, 2014 at 9:38 PM, Heikki Linnakangas wrote: > On 12/12/2014 04:29 AM, Michael Paquier wrote: >> >> On Thu, Dec 11, 2014 at 10:07 PM, Heikki Linnakangas < >> hlinnakan...@vmware.com> wrote: >> >>> I propose the attached (I admit I haven't tested it). >>> >> Actually if you do it this

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Peter Eisentraut
On 12/9/14 1:18 PM, Josh Berkus wrote: > The one exception I might make above is pg_standby. What do we need > this for today, exactly? This was discussed recently and people wanted to keep it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subsc

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Peter Eisentraut
On 12/9/14 4:10 PM, Alvaro Herrera wrote: > Maybe it makes sense to have a distinction between client programs and > server programs. Can we have src/sbin/ and move stuff that involves the > server side in there? I think that'd be pg_xlogdump, pg_archivecleanup, > pg_upgrade, pg_test_timing, pg_t

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Peter Eisentraut
On 12/9/14 4:32 PM, Bruce Momjian wrote: > On Tue, Dec 9, 2014 at 06:10:02PM -0300, Alvaro Herrera wrote: >> (For pg_upgrade you also need to do something about pg_upgrade_support, >> which is good because that is one very ugly crock.) > > FYI, pg_upgrade_support was segregated from pg_upgrade on

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Heikki Linnakangas
On 12/12/2014 03:07 PM, Peter Eisentraut wrote: On 12/9/14 4:10 PM, Alvaro Herrera wrote: Maybe it makes sense to have a distinction between client programs and server programs. Can we have src/sbin/ and move stuff that involves the server side in there? I think that'd be pg_xlogdump, pg_archi

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Andres Freund
On 2014-12-12 15:11:01 +0200, Heikki Linnakangas wrote: > On 12/12/2014 03:07 PM, Peter Eisentraut wrote: > >On 12/9/14 4:10 PM, Alvaro Herrera wrote: > >>Maybe it makes sense to have a distinction between client programs and > >>server programs. Can we have src/sbin/ and move stuff that involves

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Heikki Linnakangas
On 12/12/2014 03:11 PM, Heikki Linnakangas wrote: On 12/12/2014 03:07 PM, Peter Eisentraut wrote: On 12/9/14 4:10 PM, Alvaro Herrera wrote: Maybe it makes sense to have a distinction between client programs and server programs. Can we have src/sbin/ and move stuff that involves the server side

Re: [HACKERS] PATCH: hashjoin - gracefully increasing NTUP_PER_BUCKET instead of batching

2014-12-12 Thread Robert Haas
On Thu, Dec 11, 2014 at 5:46 PM, Tomas Vondra wrote: >>> The idea was that if we could increase the load a bit (e.g. using 2 >>> tuples per bucket instead of 1), we will still use a single batch in >>> some cases (when we miss the work_mem threshold by just a bit). The >>> lookups will be slower,

Re: [HACKERS] Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Thu, Dec 11, 2014 at 10:33 PM, Michael Paquier wrote: > On Tue, Dec 9, 2014 at 4:09 AM, Robert Haas wrote: >> On Sun, Dec 7, 2014 at 9:30 PM, Simon Riggs wrote: >> > * parameter should be SUSET - it doesn't *need* to be set only at >> > server start since all records are independent of each o

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Thu, Dec 11, 2014 at 11:34 AM, Bruce Momjian wrote: >> compression = 'on' : 1838 secs >> = 'off' : 1701 secs >> >> Different is around 140 secs. > > OK, so the compression took 2x the cpu and was 8% slower. The only > benefit is WAL files are 35% smaller? Compression didn't take

Re: [HACKERS] pg_regress writes into source tree

2014-12-12 Thread Alvaro Herrera
Peter Eisentraut wrote: > When using a vpath build pg_regress writes the processed input/*.source > files into the *source* tree, which isn't supposed to happen. > > This appears to be a thinko introduced in this patch: > e3fc4a97bc8ee82a78605b5ffe79bd4cf3c6213b Oh, I noticed this while doing the

[HACKERS] Streaming replication and WAL archive interactions

2014-12-12 Thread Heikki Linnakangas
There have been a few threads on the behavior of WAL archiving, after a standby server is promoted [1] [2]. In short, it doesn't work as you might expect. The standby will start archiving after it's promoted, but it will not archive files that were replicated from the old master via streaming r

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Robert Haas
On Thu, Dec 11, 2014 at 5:55 PM, Tom Lane wrote: > Josh Berkus writes: >> How about *you* run the next one, Tom? > > I think the limited amount of time I can put into a commitfest is better > spent on reviewing patches than on managing the process. That's not really the point. The point is that

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-12 Thread Heikki Linnakangas
On 12/10/2014 04:32 PM, Dennis Kögel wrote: Hi, Am 04.09.2014 um 17:50 schrieb Jehan-Guillaume de Rorthais : Since few months, we occasionally see .ready files appearing on some slave instances from various context. The two I have in mind are under 9.2.x. […] So it seems for some reasons, these

Re: [HACKERS] On partitioning

2014-12-12 Thread Robert Haas
On Thu, Dec 11, 2014 at 11:43 PM, Amit Langote wrote: > In case of what we would have called a 'LIST' partition, this could look like > > ... FOR VALUES (val1, val2, val3, ...) > > Assuming we only support partition key to contain only one column in such a > case. > > In case of what we would hav

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 03:13:44PM +0200, Heikki Linnakangas wrote: > On 12/12/2014 03:11 PM, Heikki Linnakangas wrote: > >On 12/12/2014 03:07 PM, Peter Eisentraut wrote: > >>On 12/9/14 4:10 PM, Alvaro Herrera wrote: > >>>Maybe it makes sense to have a distinction between client programs and > >>>s

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 08:27:59 -0500, Robert Haas wrote: > On Thu, Dec 11, 2014 at 11:34 AM, Bruce Momjian wrote: > >> compression = 'on' : 1838 secs > >> = 'off' : 1701 secs > >> > >> Different is around 140 secs. > > > > OK, so the compression took 2x the cpu and was 8% slower. The only >

[HACKERS] pg_rewind in contrib

2014-12-12 Thread Heikki Linnakangas
Hi, I'd like to include pg_rewind in contrib. I originally wrote it as an external project so that I could quickly get it working with the existing versions, and because I didn't feel it was quite ready for production use yet. Now, with the WAL format changes in master, it is a lot more maint

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 08:11:31AM -0500, Peter Eisentraut wrote: > On 12/9/14 4:32 PM, Bruce Momjian wrote: > > On Tue, Dec 9, 2014 at 06:10:02PM -0300, Alvaro Herrera wrote: > >> (For pg_upgrade you also need to do something about pg_upgrade_support, > >> which is good because that is one very u

Re: [HACKERS] Compression of full-page-writes

2014-12-12 Thread Michael Paquier
On Fri, Dec 12, 2014 at 10:23 PM, Robert Haas wrote: > On Thu, Dec 11, 2014 at 10:33 PM, Michael Paquier > wrote: >> On Tue, Dec 9, 2014 at 4:09 AM, Robert Haas wrote: >>> On Sun, Dec 7, 2014 at 9:30 PM, Simon Riggs wrote: >>> > * parameter should be SUSET - it doesn't *need* to be set only at

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Alvaro Herrera
Robert Haas wrote: > (I note that the proposal to have the CFM review everything is merely > one way of meeting the need to have senior people spend more time > reviewing. But I assure all of you that I spend as much time > reviewing as I can find time for. If someone wants to pay me the same >

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 08:27:59AM -0500, Robert Haas wrote: > On Thu, Dec 11, 2014 at 11:34 AM, Bruce Momjian wrote: > >> compression = 'on' : 1838 secs > >> = 'off' : 1701 secs > >> > >> Different is around 140 secs. > > > > OK, so the compression took 2x the cpu and was 8% slower.

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Peter Eisentraut
On 12/8/14 10:50 PM, Tom Lane wrote: > oid2name and vacuumlo, besides being of very dubious general utility, > are fails from a namespacing standpoint. If we were to promote them > into standard install components I think a minimum requirement should be > to rename them to pg_something. (oid2name

Re: [HACKERS] pg_rewind in contrib

2014-12-12 Thread Andres Freund
Hi, On 2014-12-12 16:13:13 +0200, Heikki Linnakangas wrote: > I'd like to include pg_rewind in contrib. I originally wrote it as an > external project so that I could quickly get it working with the existing > versions, and because I didn't feel it was quite ready for production use > yet. Now, wi

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 09:18:01 -0500, Bruce Momjian wrote: > On Fri, Dec 12, 2014 at 08:27:59AM -0500, Robert Haas wrote: > > On Thu, Dec 11, 2014 at 11:34 AM, Bruce Momjian wrote: > > >> compression = 'on' : 1838 secs > > >> = 'off' : 1701 secs > > >> > > >> Different is around 140 secs. > >

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 03:22:24PM +0100, Andres Freund wrote: > On 2014-12-12 09:18:01 -0500, Bruce Momjian wrote: > > On Fri, Dec 12, 2014 at 08:27:59AM -0500, Robert Haas wrote: > > > On Thu, Dec 11, 2014 at 11:34 AM, Bruce Momjian wrote: > > > >> compression = 'on' : 1838 secs > > > >>

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Peter Eisentraut
On 12/12/14 8:13 AM, Andres Freund wrote: > Wouldn't a make install-server/client targets or something similar > actually achieve the same thing? Seems simpler to maintain to me. Adding non-standard makefile targets comes with its own set of maintenance issues. Restructuring the source tree and h

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Alvaro Herrera
Bruce Momjian wrote: > On Fri, Dec 12, 2014 at 03:13:44PM +0200, Heikki Linnakangas wrote: > > On 12/12/2014 03:11 PM, Heikki Linnakangas wrote: > > >Sounds good. We already separate server and client programs in the docs, > > >and packagers put them in different packages too. This should make > >

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 09:24:27 -0500, Bruce Momjian wrote: > On Fri, Dec 12, 2014 at 03:22:24PM +0100, Andres Freund wrote: > > > Well, the larger question is why wouldn't we just have the user compress > > > the entire WAL file before archiving --- why have each backend do it? > > > Is it the write volum

Re: [HACKERS] pg_rewind in contrib

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 03:20:47PM +0100, Andres Freund wrote: > Hi, > > On 2014-12-12 16:13:13 +0200, Heikki Linnakangas wrote: > > I'd like to include pg_rewind in contrib. I originally wrote it as an > > external project so that I could quickly get it working with the existing > > versions, and

Re: [HACKERS] Compression of full-page-writes

2014-12-12 Thread Michael Paquier
On Fri, Dec 12, 2014 at 11:32 PM, Robert Haas wrote: > On Fri, Dec 12, 2014 at 9:15 AM, Michael Paquier > wrote: >> I just meant that the scratch buffers used to store temporarily the >> compressed and uncompressed data should be palloc'd all the time, even >> if the switch is off. > > If they're

Re: [HACKERS] Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 9:15 AM, Michael Paquier wrote: > I just meant that the scratch buffers used to store temporarily the > compressed and uncompressed data should be palloc'd all the time, even > if the switch is off. If they're fixed size, you can just put them on the heap as static globals

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 9:15 AM, Alvaro Herrera wrote: > Robert Haas wrote: >> (I note that the proposal to have the CFM review everything is merely >> one way of meeting the need to have senior people spend more time >> reviewing. But I assure all of you that I spend as much time >> reviewing as

Re: [HACKERS] Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 9:34 AM, Michael Paquier wrote: >> I don't think that's a cost worth caring about. > OK, I thought it was. Space on the heap that never gets used is basically free. The OS won't actually allocate physical memory unless the pages are actually accessed. -- Robert Haas Ent

Re: [HACKERS] pg_rewind in contrib

2014-12-12 Thread Michael Paquier
On Fri, Dec 12, 2014 at 11:13 PM, Heikki Linnakangas wrote: > I'd like to include pg_rewind in contrib. I originally wrote it as an > external project so that I could quickly get it working with the existing > versions, and because I didn't feel it was quite ready for production use > yet. Now, wi

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Rahila Syed
Hello, >Well, the larger question is why wouldn't we just have the user compress >the entire WAL file before archiving --- why have each backend do it? >Is it the write volume we are saving? IIUC, the idea here is to not only save the on disk size of WAL but to reduce the overhead of flushing WA

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 03:27:33PM +0100, Andres Freund wrote: > On 2014-12-12 09:24:27 -0500, Bruce Momjian wrote: > > On Fri, Dec 12, 2014 at 03:22:24PM +0100, Andres Freund wrote: > > > > Well, the larger question is why wouldn't we just have the user compress > > > > the entire WAL file before

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Michael Paquier
On Wed, Dec 10, 2014 at 11:25 PM, Bruce Momjian wrote: > On Wed, Dec 10, 2014 at 07:40:46PM +0530, Rahila Syed wrote: > > The tests ran for around 30 mins.Manual checkpoint was run before each > test. > > > > Compression WAL generated%compressionLatency-avg CPU usage > > (seconds)

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 09:46:13 -0500, Bruce Momjian wrote: > On Fri, Dec 12, 2014 at 03:27:33PM +0100, Andres Freund wrote: > > On 2014-12-12 09:24:27 -0500, Bruce Momjian wrote: > > > On Fri, Dec 12, 2014 at 03:22:24PM +0100, Andres Freund wrote: > > > > > Well, the larger question is why wouldn't we just

Re: [HACKERS] pg_rewind in contrib

2014-12-12 Thread Heikki Linnakangas
On 12/12/2014 04:20 PM, Andres Freund wrote: Not sure if the copyright notices in the current form are actually ok? Hmm. We do have such copyright notices in the source tree, but I know that we're trying to avoid it in new code. They had to be there when the code lived as a separate project,

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 23:50:43 +0900, Michael Paquier wrote: > I got curious to see how the compression of an entire record would perform > and how it compares for small WAL records, and here are some numbers based > on the patch attached, this patch compresses the whole record including the > block header

Re: [HACKERS] pg_rewind in contrib

2014-12-12 Thread Tom Lane
Heikki Linnakangas writes: > I'd like to include pg_rewind in contrib. I don't object to adding the tool as such, but let's wait to see what happens with Peter's proposal to move contrib command-line tools into src/bin/. If it should be there it'd be less code churn if it went into there in the

Re: [HACKERS] Commitfest problems

2014-12-12 Thread David Fetter
On Thu, Dec 11, 2014 at 05:55:56PM -0500, Tom Lane wrote: > Josh Berkus writes: > > How about *you* run the next one, Tom? > > I think the limited amount of time I can put into a commitfest is > better spent on reviewing patches than on managing the process. With utmost respect, Tom, you seem to

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Andres Freund
On 2014-12-12 11:27:01 -0300, Alvaro Herrera wrote: > We already have src/bin/; the mixture of "src/" and "bin/" predates us. > Of course, the stuff we keep in there is not binaries but source code > that produces binaries. > > As for src/sbin/, we wouldn't install anything to the system's > /usr/

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Heikki Linnakangas writes: > On 12/12/2014 03:07 PM, Peter Eisentraut wrote: >> On 12/9/14 4:10 PM, Alvaro Herrera wrote: >>> Maybe it makes sense to have a distinction between client programs and >>> server programs. Can we have src/sbin/ and move stuff that involves the >>> server side in there

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Peter Eisentraut writes: > On 12/9/14 4:32 PM, Bruce Momjian wrote: >> On Tue, Dec 9, 2014 at 06:10:02PM -0300, Alvaro Herrera wrote: >>> (For pg_upgrade you also need to do something about pg_upgrade_support, >>> which is good because that is one very ugly crock.) >> FYI, pg_upgrade_support was

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Peter Eisentraut writes: > On 12/12/14 8:13 AM, Andres Freund wrote: >> Wouldn't a make install-server/client targets or something similar >> actually achieve the same thing? Seems simpler to maintain to me. > Adding non-standard makefile targets comes with its own set of > maintenance issues. I

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Andres Freund
On 2014-12-12 07:10:40 -0800, David Fetter wrote: > On Thu, Dec 11, 2014 at 05:55:56PM -0500, Tom Lane wrote: > > Josh Berkus writes: > > > How about *you* run the next one, Tom? > > > > I think the limited amount of time I can put into a commitfest is > > better spent on reviewing patches than o

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Andres Freund
On 2014-12-12 10:20:58 -0500, Tom Lane wrote: > Peter Eisentraut writes: > > On 12/12/14 8:13 AM, Andres Freund wrote: > >> Wouldn't a make install-server/client targets or something similar > >> actually achieve the same thing? Seems simpler to maintain to me. > > > Adding non-standard makefile

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-12 Thread Alex Shulgin
Alex Shulgin writes: > Alex Shulgin writes: >> >> Here's an attempt to revive this patch. > > Here's the patch rebased against current HEAD, that is including the > recently committed action_at_recovery_target option. > > The default for the new GUC is 'pause', as in HEAD, and > pause_at_recover

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Tom Lane
Robert Haas writes: > On Fri, Dec 12, 2014 at 9:15 AM, Alvaro Herrera > wrote: >> Robert Haas wrote: >>> (I note that the proposal to have the CFM review everything is merely >>> one way of meeting the need to have senior people spend more time >>> reviewing. But I assure all of you that I spend

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Alvaro Herrera
Tom Lane wrote: > Heikki Linnakangas writes: > > On 12/12/2014 03:07 PM, Peter Eisentraut wrote: > >> On 12/9/14 4:10 PM, Alvaro Herrera wrote: > >>> Maybe it makes sense to have a distinction between client programs and > >>> server programs. Can we have src/sbin/ and move stuff that involves th

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 10:04 AM, Andres Freund wrote: >> Note that autovacuum and fsync are off. >> =# select phase, user_diff, system_diff, >> pg_size_pretty(pre_update - pre_insert), >> pg_size_pretty(post_update - pre_update) from results; >>phase| user_diff | system_diff | pg_

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 11:08:52 -0500, Robert Haas wrote: > Unless I'm missing something, this test is showing that FPW > compression saves 298MB of WAL for 17.3 seconds of CPU time, as > against master. And compressing the whole record saves a further 1MB > of WAL for a further 13.39 seconds of CPU time.

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 11:00 AM, Alvaro Herrera wrote: >> I'm pretty much -1 on relocating anything that's under src/bin already. I agree. I can't see packagers putting it anywhere except for $SOMETHING/bin in the final install, so what do we get out of dividing it up in some weird way in our t

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 11:12 AM, Andres Freund wrote: > On 2014-12-12 11:08:52 -0500, Robert Haas wrote: >> Unless I'm missing something, this test is showing that FPW >> compression saves 298MB of WAL for 17.3 seconds of CPU time, as >> against master. And compressing the whole record saves a f

Re: [HACKERS] [Bug] Inconsistent result for inheritance and FOR UPDATE.

2014-12-12 Thread Tom Lane
Etsuro Fujita writes: >> (2014/12/12 10:37), Tom Lane wrote: >>> Yeah, this is clearly a thinko: really, nothing in the planner should >>> be using get_parse_rowmark(). I looked around for other errors of the >>> same type and found that postgresGetForeignPlan() is also using >>> get_parse_rowmar

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 11:15:46 -0500, Robert Haas wrote: > On Fri, Dec 12, 2014 at 11:12 AM, Andres Freund wrote: > > On 2014-12-12 11:08:52 -0500, Robert Haas wrote: > >> Unless I'm missing something, this test is showing that FPW > >> compression saves 298MB of WAL for 17.3 seconds of CPU time, as > >>

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Joshua D. Drake
On 12/08/2014 07:50 PM, Tom Lane wrote: Peter Eisentraut writes: Last time this was attempted, the discussion got lost in exactly which extensions are worthy enough to be considered official or something like that. I want to dodge that here by starting at the opposite end: 1. move programs to

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Robert Haas writes: > I'm not really convinced this is a very good idea. What do we get out > of moving everything, or even anything, from contrib? It will make > back-patching harder, but more importantly, it will possibly create > the false impression that everything we distribute is on equal

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Andres Freund
On 2014-12-12 11:14:56 -0500, Robert Haas wrote: > I'm not really convinced this is a very good idea. What do we get out > of moving everything, or even anything, from contrib? The benefit of moving relevant stuff is that it'll actually be installed by default when installing postgres on many pla

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 11:26 AM, Tom Lane wrote: > Robert Haas writes: >> I'm not really convinced this is a very good idea. What do we get out >> of moving everything, or even anything, from contrib? It will make >> back-patching harder, but more importantly, it will possibly create >> the fa

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 11:40 AM, Andres Freund wrote: > The benefit of moving relevant stuff is that it'll actually be installed > by default when installing postgres on many platforms. That's currently > often not the case. The contrib umbrella, as used by many other > projects, actually justifi

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Andres Freund
On 2014-12-12 11:42:57 -0500, Robert Haas wrote: > > I think the amount of effort a simple renamed directory which wholly > > contains a binary creates is acceptable. Just use patch -p4 instead of > > patch -p1... > > That is fine if you are manually applying a patch that touches only > that direc

Re: [HACKERS] PATCH: hashjoin - gracefully increasing NTUP_PER_BUCKET instead of batching

2014-12-12 Thread Tomas Vondra
On 12.12.2014 14:19, Robert Haas wrote: > On Thu, Dec 11, 2014 at 5:46 PM, Tomas Vondra wrote: > >> Regarding the "sufficiently small" - considering today's hardware, we're >> probably talking about gigabytes. On machines with significant memory >> pressure (forcing the temporary files to disk), i

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Andres Freund writes: > On 2014-12-12 11:42:57 -0500, Robert Haas wrote: >> And I don't know how well git cherry-pick will follow >> the moves. > Not well if the patch is done in master first. FWIW, I always patch master first, and have zero intention of changing that workflow. (I have given re

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 10:16:05AM -0500, Tom Lane wrote: > Peter Eisentraut writes: > > On 12/9/14 4:32 PM, Bruce Momjian wrote: > >> On Tue, Dec 9, 2014 at 06:10:02PM -0300, Alvaro Herrera wrote: > >>> (For pg_upgrade you also need to do something about pg_upgrade_support, > >>> which is good b

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Alvaro Herrera
Robert Haas wrote: > On Fri, Dec 12, 2014 at 11:00 AM, Alvaro Herrera > wrote: > > So let's put the whole bunch under src/bin/ and be done with it. > > I'm not really convinced this is a very good idea. What do we get out > of moving everything, or even anything, from contrib? We show that it'

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 05:19:42PM +0100, Andres Freund wrote: > On 2014-12-12 11:15:46 -0500, Robert Haas wrote: > > On Fri, Dec 12, 2014 at 11:12 AM, Andres Freund wrote: > > > On 2014-12-12 11:08:52 -0500, Robert Haas wrote: > > >> Unless I'm missing something, this test is showing that FPW > >

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 10:50:56AM -0500, Tom Lane wrote: > Also, one part of the point of the review mechanism is that it's supposed > to provide an opportunity for less-senior reviewers to look at parts of > the code that they maybe don't know so well, and thereby help grow them > into senior peo

Re: [HACKERS] jsonb generator functions

2014-12-12 Thread Andrew Dunstan
On 12/08/2014 01:00 PM, Andrew Dunstan wrote: On 12/08/2014 04:21 AM, Alvaro Herrera wrote: Andrew Dunstan wrote: OK, here is a new patch version that * uses find_coercion_path() to find the cast function if any, as discussed elsewhere * removes calls to getTypeOutputInfo() except w

Re: [HACKERS] jsonb generator functions

2014-12-12 Thread Andrew Dunstan
On 12/12/2014 01:10 PM, Andrew Dunstan wrote: On 12/08/2014 01:00 PM, Andrew Dunstan wrote: On 12/08/2014 04:21 AM, Alvaro Herrera wrote: Andrew Dunstan wrote: OK, here is a new patch version that * uses find_coercion_path() to find the cast function if any, as discussed elsewhere

Re: [HACKERS] Commitfest problems

2014-12-12 Thread David Fetter
On Fri, Dec 12, 2014 at 04:21:43PM +0100, Andres Freund wrote: > On 2014-12-12 07:10:40 -0800, David Fetter wrote: > > On Thu, Dec 11, 2014 at 05:55:56PM -0500, Tom Lane wrote: > > > Josh Berkus writes: > > > > How about *you* run the next one, Tom? > > > > > > I think the limited amount of time

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
On 12/12/2014 06:30 AM, Robert Haas wrote: > Yeah, that would be great, and even better if we could get 2 or 3 > positions funded so that the success or failure isn't too much tied to > a single individual. But even getting 1 position funded in a > stable-enough fashion that someone would be willi

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Simon Riggs
On 12 December 2014 at 18:04, Bruce Momjian wrote: > Well, it seems we need to see some actual cases where compression does > help before moving forward. I thought Amit had some amazing numbers for > WAL compression --- has that changed? For background processes, like VACUUM, then WAL compressi

Re: [HACKERS] jsonb generator functions

2014-12-12 Thread Tom Lane
Andrew Dunstan writes: >> Also here is a patch factored out which applies the >> find_coercion_pathway change to json.c. I'm inclined to say we should >> backpatch this to 9.4 (and with a small change 9.3). Thoughts? Meh. Maybe I'm just feeling gunshy because I broke something within the past

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Tomas Vondra
On 12.12.2014 19:07, Bruce Momjian wrote: > On Fri, Dec 12, 2014 at 10:50:56AM -0500, Tom Lane wrote: >> Also, one part of the point of the review mechanism is that it's >> supposed to provide an opportunity for less-senior reviewers to >> look at parts of the code that they maybe don't know so wel

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Simon Riggs
On 12 December 2014 at 15:10, David Fetter wrote: > On Thu, Dec 11, 2014 at 05:55:56PM -0500, Tom Lane wrote: >> Josh Berkus writes: >> > How about *you* run the next one, Tom? >> >> I think the limited amount of time I can put into a commitfest is >> better spent on reviewing patches than on man

Re: [HACKERS] jsonb generator functions

2014-12-12 Thread Andrew Dunstan
On 12/12/2014 01:55 PM, Tom Lane wrote: Andrew Dunstan writes: Also here is a patch factored out which applies the find_coercion_pathway change to json.c. I'm inclined to say we should backpatch this to 9.4 (and with a small change 9.3). Thoughts? Meh. Maybe I'm just feeling gunshy because I

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Joshua D. Drake
On 12/11/2014 02:55 PM, Tom Lane wrote: Josh Berkus writes: How about *you* run the next one, Tom? I think the limited amount of time I can put into a commitfest is better spent on reviewing patches than on managing the process. Agreed but That means committers/hackers have to suck it up

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Joshua D. Drake
On 12/12/2014 06:30 AM, Robert Haas wrote: Yeah, that would be great, and even better if we could get 2 or 3 positions funded so that the success or failure isn't too much tied to a single individual. But even getting 1 position funded in a stable-enough fashion that someone would be willing t

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Joshua D. Drake
On 12/12/2014 10:59 AM, Simon Riggs wrote: On 12 December 2014 at 15:10, David Fetter wrote: On Thu, Dec 11, 2014 at 05:55:56PM -0500, Tom Lane wrote: Josh Berkus writes: How about *you* run the next one, Tom? I think the limited amount of time I can put into a commitfest is better spent

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Alvaro Herrera
Joshua D. Drake wrote: > > On 12/12/2014 06:30 AM, Robert Haas wrote: > > >Yeah, that would be great, and even better if we could get 2 or 3 > >positions funded so that the success or failure isn't too much tied to > >a single individual. But even getting 1 position funded in a > >stable-enough

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Tomas Vondra
On 11.12.2014 16:06, Bruce Momjian wrote: > On Wed, Dec 10, 2014 at 11:00:21PM -0800, Josh Berkus wrote: >> >> I will add: >> >> 4. commitfest managers have burned out and refuse to do it again > > Agreed. The "fun", if it was ever there, has left the commitfest > process. I've never been a CFM,

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
On 12/12/2014 11:35 AM, Alvaro Herrera wrote: > Uh, really? Last I looked at the numbers from SPI treasurer reports, > they are not impressive enough to hire a full-time engineer, let alone a > senior one. > > The Linux Foundation has managed to pay for Linus Torvalds somehow, so > it does sound

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Joshua D. Drake
On 12/12/2014 11:35 AM, Alvaro Herrera wrote: We (not CMD, the community) with proper incentive could fund this. It really wouldn't be that hard. That said, there would have to be a clear understanding of expectations, results, and authority. Uh, really? Yeah I think so. Money can be easy t

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Magnus Hagander
On Fri, Dec 12, 2014 at 8:43 PM, Tomas Vondra wrote: > On 11.12.2014 16:06, Bruce Momjian wrote: >> On Wed, Dec 10, 2014 at 11:00:21PM -0800, Josh Berkus wrote: > 3) manual processing that could be automated > >I think the CF site was a huge step forward, but maybe we could >improve it, to

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
On 12/12/2014 11:52 AM, Magnus Hagander wrote: > On Fri, Dec 12, 2014 at 8:43 PM, Tomas Vondra wrote: >> On 11.12.2014 16:06, Bruce Momjian wrote: >>> On Wed, Dec 10, 2014 at 11:00:21PM -0800, Josh Berkus wrote: >> 3) manual processing that could be automated >> >>I think the CF site was a hug

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Christoph Berg
Re: Andres Freund 2014-12-12 <20141212152723.go31...@awork2.anarazel.de> > On 2014-12-12 10:20:58 -0500, Tom Lane wrote: > > Peter Eisentraut writes: > > > On 12/12/14 8:13 AM, Andres Freund wrote: > > >> Wouldn't a make install-server/client targets or something similar > > >> actually achieve th

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Christoph Berg writes: > However, for PostgreSQL this means lengthy debian/*.install files > (the equivalent of %files in rpm spec speak): Right ... > If there were separate "install-client", "install-server", and > "install-contrib" targets, that would probably shorten those files > quite a bit

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Magnus Hagander
On Fri, Dec 12, 2014 at 9:05 PM, Josh Berkus wrote: > On 12/12/2014 11:52 AM, Magnus Hagander wrote: >> On Fri, Dec 12, 2014 at 8:43 PM, Tomas Vondra wrote: >>> On 11.12.2014 16:06, Bruce Momjian wrote: On Wed, Dec 10, 2014 at 11:00:21PM -0800, Josh Berkus wrote: >>> 3) manual processing tha

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Alvaro Herrera
Tom Lane wrote: > Christoph Berg writes: > > However, for PostgreSQL this means lengthy debian/*.install files > > (the equivalent of %files in rpm spec speak): > > Right ... > > > If there were separate "install-client", "install-server", and > > "install-contrib" targets, that would probably s

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Peter Geoghegan
On Fri, Dec 12, 2014 at 12:30 PM, Magnus Hagander wrote: >> It also automates a bunch of the emailing no? > > Yes. Please let me know the details (privately or otherwise). I'd like to try it out again. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
>>> Just as a note abot this one part along (I'll read the rest later). I >>> do have the new version of the CF app more or less ready to deploy, >>> but I got bogged down by thinking "I'll do it between two commitfests >>> to not be disruptive". But there has been no "between two >>> commitfests"

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Jim Nasby
On 12/12/14, 2:38 PM, Josh Berkus wrote: Just as a note abot this one part along (I'll read the rest later). I do have the new version of the CF app more or less ready to deploy, but I got bogged down by thinking "I'll do it between two commitfests to not be disruptive". But there has been no "

Re: [HACKERS] PATCH: hashjoin - gracefully increasing NTUP_PER_BUCKET instead of batching

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 11:50 AM, Tomas Vondra wrote: > On 12.12.2014 14:19, Robert Haas wrote: >> On Thu, Dec 11, 2014 at 5:46 PM, Tomas Vondra wrote: >> >>> Regarding the "sufficiently small" - considering today's hardware, we're >>> probably talking about gigabytes. On machines with significan

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Jim Nasby
On 12/12/14, 1:44 PM, Joshua D. Drake wrote: 1. We don't need a full-time engineer to manage a commitfest. We need a manager or PM. I don't think that's actually true. The major points on this thread are that 1) we don't have enough capacity for doing reviews and 2) the CFM has no authority

  1   2   >