Re: [HACKERS] O_DSYNC broken on MacOS X?

Darren Duncan wrote: This matter reminds me of a discussion on the SQLite list years ago about whether pragma synchronous=normal or synchronous=full should be the default, and thankfully 'full' won. Right now, when I see deployments in the field, serious database servers setup by professional

Re: [HACKERS] O_DSYNC broken on MacOS X?

Greg Smith wrote: You didn't quote the next part of that, which says "fsync() is not sufficient to guarantee that your data is on stable storage and on MacOS X we provide a fcntl(), called F_FULLFSYNC, to ask the drive to flush all buffered data to stable storage." That's exactly what turning

Re: [HACKERS] patch: SQL/MED(FDW) DDL

On Sep 29, 2010, at 10:09 AM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of mar sep 28 10:26:54 -0400 2010: > >> Then: >> >> - Begin a sequential scan with the following set of quals. >> - Begin an index scan using the index called X with the following set of >> quals. >> - Fet

Re: [HACKERS] security hook on table creation

On Sep 30, 2010, at 9:01 PM, KaiGai Kohei wrote: > (2010/10/01 3:09), Robert Haas wrote: >> 2010/9/29 KaiGai Kohei: >>> In addition, I want to give these entrypoints its name which >>> represents an appropriate purpose of the hook, rather than >>> a uniformed one. >> >> It sounds like you're prop

Re: I: [HACKERS] About "Our CLUSTER implementation is pessimal" patch

On Sep 30, 2010, at 9:07 PM, Itagaki Takahiro wrote: > Hi, Leonardo-san, > > On Fri, Oct 1, 2010 at 4:04 AM, Tom Lane wrote: >> The wording should be something like "CLUSTER requires transient disk >> space equal to about twice the size of the table plus its indexes". > > Could you merge those

Re: [HACKERS] Adding getrusage profiling data to EXPLAIN output

On Fri, Oct 1, 2010 at 9:16 AM, Greg Stark wrote: > Attached is a patch to display getrusage output to EXPLAIN output. > This is the patch I mentioned previously in > http://archives.postgresql.org/pgsql-hackers/2010-02/msg00684.php and > it raises the same issues we were talking about then. How

Re: [HACKERS] Postgres vs. intel ccNUMA on Linux

James Robinson wrote: Any tips / conventional wisdom regarding running postgres on large-ish memory ccNUMA intel machines, such as a 32G dual-quad-core, showing two NUMA nodes of 16G each? I expect each postgres backend's non-shared memory usage to remain nice and reasonably sized, hopefull

Re: I: [HACKERS] About "Our CLUSTER implementation is pessimal" patch

Hi, Leonardo-san, On Fri, Oct 1, 2010 at 4:04 AM, Tom Lane wrote: > The wording should be something like "CLUSTER requires transient disk > space equal to about twice the size of the table plus its indexes". Could you merge those discussions into the final patch? Also, please check whether my mo

Re: [HACKERS] O_DSYNC broken on MacOS X?

A.M. wrote: That is not correct. fsync and friends on Darwin synchronizes I/O and flushes dirty kernel caches to the disk which meets the specification and is distinctly different from doing nothing... "On MacOS X, fsync() always has and always will flush all file data from host memory to the d

Re: [HACKERS] security hook on table creation

(2010/10/01 3:09), Robert Haas wrote: > 2010/9/29 KaiGai Kohei: >> In addition, I want to give these entrypoints its name which >> represents an appropriate purpose of the hook, rather than >> a uniformed one. > > It sounds like you're proposing to create a vast number of hooks > rather than just

Re: [HACKERS] O_DSYNC broken on MacOS X?

Tom Lane wrote: I'm not sure whether we should select fsync_writethrough as the default on OSX. We don't make an equivalent attempt to prevent OS or storage malfeasance on other Unixoid platforms --- in fact, I'd say OSX is a bit ahead of the game in that you *can* force writethrough without res

Re: [HACKERS] recovery.conf location

On 9/29/10 7:54 PM, Tom Lane wrote: > Robert Haas writes: >> But that's not what Tom is talking about, I don't think: you might >> also want a way to explicitly whack the flag in pg_control around. >> That would probably be along the lines of pg_resetxlog. I'm not sure >> how much use case there

[HACKERS] Adding getrusage profiling data to EXPLAIN output

Attached is a patch to display getrusage output to EXPLAIN output. This is the patch I mentioned previously in http://archives.postgresql.org/pgsql-hackers/2010-02/msg00684.php and it raises the same issues we were talking about then. Should the resource usage stats displayed be per-iteration total

Re: [HACKERS] O_DSYNC broken on MacOS X?

On Thu, Sep 30, 2010 at 2:22 PM, A.M. wrote: > That is not correct. fsync and friends on Darwin synchronizes I/O and flushes > dirty kernel caches to the disk which meets the specification and is > distinctly different from doing nothing. How exactly is it different from doing nothing? That is,

Re: [HACKERS] Patch author name on commitfest page

On Thu, Sep 30, 2010 at 5:51 PM, Gurjeet Singh wrote: > On Thu, Sep 30, 2010 at 11:45 PM, Robert Haas wrote: >> >> On Thu, Sep 30, 2010 at 5:38 PM, Gurjeet Singh >> wrote: >> > Can we please change the comment lines below the patch heading to have >> > the >> > real name instead of the postgresq

Re: [HACKERS] Patch author name on commitfest page

On 09/30/2010 05:51 PM, Gurjeet Singh wrote: On Thu, Sep 30, 2010 at 11:45 PM, Robert Haas > wrote: On Thu, Sep 30, 2010 at 5:38 PM, Gurjeet Singh mailto:singh.gurj...@gmail.com>> wrote: > Can we please change the comment lines below the patch heading

Re: [HACKERS] Patch author name on commitfest page

On Thu, Sep 30, 2010 at 11:45 PM, Robert Haas wrote: > On Thu, Sep 30, 2010 at 5:38 PM, Gurjeet Singh > wrote: > > Can we please change the comment lines below the patch heading to have > the > > real name instead of the postgresql.org ID? > > > > Patch by Pavel Stehule > > Patch by Gurjeet Sing

Re: [HACKERS] Patch author name on commitfest page

On Thu, Sep 30, 2010 at 5:38 PM, Gurjeet Singh wrote: > Can we please change the comment lines below the patch heading to have the > real name instead of the postgresql.org ID? > > Patch by Pavel Stehule > Patch by Gurjeet Singh > > instead of > > Patch by okbobcz > Patch by singh.gurjeet > > http

[HACKERS] Patch author name on commitfest page

Can we please change the comment lines below the patch heading to have the real name instead of the postgresql.org ID? Patch by Pavel Stehule Patch by Gurjeet Singh instead of Patch by okbobcz Patch by singh.gurjeet https://commitfest.postgresql.org/action/commitfest_view?id=8 Thanks, -- gurj

Re: [HACKERS] O_DSYNC broken on MacOS X?

On Thu, Sep 30, 2010 at 5:02 PM, Tom Lane wrote: > Robert Haas writes: >> Oh, I missed that.  Actually, I wasn't really so concerned with >> whether his benchmark is correct.  I *am* concerned about being broken >> out of the box on MacOS X. > > Actually, the problem with OSX is that OSX is broke

Re: [HACKERS] O_DSYNC broken on MacOS X?

On Sep 30, 2010, at 5:02 PM, Tom Lane wrote: > Robert Haas writes: >> Oh, I missed that. Actually, I wasn't really so concerned with >> whether his benchmark is correct. I *am* concerned about being broken >> out of the box on MacOS X. > > Actually, the problem with OSX is that OSX is broken

Re: [HACKERS] O_DSYNC broken on MacOS X?

Robert Haas writes: > Oh, I missed that. Actually, I wasn't really so concerned with > whether his benchmark is correct. I *am* concerned about being broken > out of the box on MacOS X. Actually, the problem with OSX is that OSX is broken out of the box, at least by that standard. The system's

Re: [HACKERS] Postgres vs. intel ccNUMA on Linux

On Wed, Sep 29, 2010 at 2:45 PM, James Robinson wrote: > Hackers, > >        Any tips / conventional wisdom regarding running postgres on > large-ish memory ccNUMA intel machines, such as a 32G dual-quad-core, > showing two NUMA nodes of 16G each? I expect each postgres backend's > non-shared memo

Re: [HACKERS] git diff --patience

Gurjeet Singh wrote: > The with-patience version has only two hunks, removal of a big > block of comment and addition of a big block of code. > > The without-patience patience is riddled with the mix of two > hunks, spread until line 120. > > --patience is a clear winner here. When I read th

Re: [HACKERS] Patch to reindex primary keys

On Wed, Sep 29, 2010 at 8:42 PM, Gurjeet Singh wrote: >     Attached is a patch that implements replacing a primary key with another > index. This would help overcome the limitation that primary keys cannot be > reindexed > without taking exclusive locks. Sounds useful. You should add this patch

Re: [HACKERS] O_DSYNC broken on MacOS X?

On Thu, Sep 30, 2010 at 4:09 PM, Dave Page wrote: > On Thu, Sep 30, 2010 at 8:26 PM, Robert Haas wrote: >> Jayant Kumar did some benchmarking of InnoDB vs. PostgreSQL and PG >> came out 5 times faster.  The benchmark isn't very thoroughly >> described, but it turns out not to matter. >> >> http:/

Re: [HACKERS] git diff --patience

Attached are two versions of the same patch, with and without --patience. The with-patience version has only two hunks, removal of a big block of comment and addition of a big block of code. The without-patience patience is riddled with the mix of two hunks, spread until line 120. --patience is

Re: [HACKERS] O_DSYNC broken on MacOS X?

On Thu, Sep 30, 2010 at 8:26 PM, Robert Haas wrote: > Jayant Kumar did some benchmarking of InnoDB vs. PostgreSQL and PG > came out 5 times faster.  The benchmark isn't very thoroughly > described, but it turns out not to matter. > > http://jayant7k.blogspot.com/2010/09/database-speed-tests-mysql-

Re: [HACKERS] O_DSYNC broken on MacOS X?

On Thu, Sep 30, 2010 at 8:26 PM, Robert Haas wrote: > Jayant Kumar did some benchmarking of InnoDB vs. PostgreSQL and PG > came out 5 times faster.  The benchmark isn't very thoroughly > described, but it turns out not to matter. > > http://jayant7k.blogspot.com/2010/09/database-speed-tests-mysql-

[HACKERS] patch: subscripts function

Hello The attached patch contains a implementation of "subscripts" function. The functionality of this function is same like generate_subscripts function, but it's based for iteration from plpgsql's for-in-array. Regards Pavel Stehule *** ./doc/src/sgml/func.sgml.orig 2010-09-29 10:40:40.000

[HACKERS] O_DSYNC broken on MacOS X?

Jayant Kumar did some benchmarking of InnoDB vs. PostgreSQL and PG came out 5 times faster. The benchmark isn't very thoroughly described, but it turns out not to matter. http://jayant7k.blogspot.com/2010/09/database-speed-tests-mysql-and.html Apparently, the reason we're faster is that wal_sync

Re: I: [HACKERS] About "Our CLUSTER implementation is pessimal" patch

Simon Riggs writes: > On Wed, 2010-09-29 at 08:44 -0400, Alvaro Herrera wrote: > So, I think "twice disk space of the sum of table and indexes" would be > the simplest explanation for safe margin. >> >> Agreed. > Surely the peak space is x3? > Old space + sort space + new space. The wording sho

Re: [HACKERS] security hook on table creation

2010/9/29 KaiGai Kohei : > In addition, I want to give these entrypoints its name which > represents an appropriate purpose of the hook, rather than > a uniformed one. It sounds like you're proposing to create a vast number of hooks rather than just one. If we have ~20 object types in the system,

Re: [HACKERS] is sync rep stalled?

On Thu, Sep 30, 2010 at 12:52 PM, Simon Riggs wrote: > On Thu, 2010-09-30 at 07:06 -0700, David Fetter wrote: >> On Thu, Sep 30, 2010 at 09:52:46AM -0400, Tom Lane wrote: >> > David Fetter writes: >> > > On Thu, Sep 30, 2010 at 09:14:42AM +0100, Simon Riggs wrote: >> > >> I don't see anything has

Re: I: [HACKERS] About "Our CLUSTER implementation is pessimal" patch

On Wed, 2010-09-29 at 08:44 -0400, Alvaro Herrera wrote: > > So, I think "twice disk space of the sum of table and indexes" would be > > the simplest explanation for safe margin. > > Agreed. Surely the peak space is x3? Old space + sort space + new space. -- Simon Riggs www.2ndQuad

Re: [HACKERS] is sync rep stalled?

On Thu, 2010-09-30 at 07:06 -0700, David Fetter wrote: > On Thu, Sep 30, 2010 at 09:52:46AM -0400, Tom Lane wrote: > > David Fetter writes: > > > On Thu, Sep 30, 2010 at 09:14:42AM +0100, Simon Riggs wrote: > > >> I don't see anything has stalled. > > > > > I do. We're half way through this comm

Re: [HACKERS] Using streaming replication as log archiving

Aidan Van Dyk wrote: > When the "being written to" segmnt copmletes moves to the final > location, he'll get an extra whole "copy" of the file. But of the > "move" can be an exec of his scritpt, the compressed/gzipped final > result shouldn't be that bad. Certainly no worse then what he's > cu

Re: [HACKERS] Using streaming replication as log archiving

Magnus Hagander wrote: > We'd need a second script/command to call to figure out where to > restart from in that case, no? I see your point; I guess we would need that. > It should be safe to just rsync the archive directory as it's > being written by pg_streamrecv. Doesn't that give you the

Re: [HACKERS] Using streaming replication as log archiving

On Thu, Sep 30, 2010 at 16:39, Aidan Van Dyk wrote: > On Thu, Sep 30, 2010 at 10:24 AM, Magnus Hagander wrote: > >>> That would allow some nice options.  I've been thinking what would >>> be the ideal use of this with our backup scheme, and the best I've >>> thought up would be that each WAL segm

Re: [HACKERS] is sync rep stalled?

Heikki Linnakangas wrote: > You do realize that to be able to guarantee zero data loss, the > master will have to stop committing new transactions if the > streaming stops for any reason, like a network glitch. Maybe > that's a tradeoff you want, but I'm asking because that point > isn't clear t

Re: [HACKERS] is sync rep stalled?

Heikki Linnakangas wrote: On 30.09.2010 17:09, Kevin Grittner wrote: Aidan Van Dyk wrote: Heikki Linnakangas wrote: I'm sure there's several things you can accomplish with synchronous replication, perhaps you could describe what the important use case for you is? I'm looking for "data dur

Re: [HACKERS] Using streaming replication as log archiving

On Thu, Sep 30, 2010 at 10:24 AM, Magnus Hagander wrote: >> That would allow some nice options.  I've been thinking what would >> be the ideal use of this with our backup scheme, and the best I've >> thought up would be that each WAL segment file would be a single >> output stream, with the optio

Re: [HACKERS] Standby registration

On 29.09.2010 11:46, Fujii Masao wrote: Aside from standby registration itself, I have another thought for C). Keeping many WAL files in pg_xlog of the master is not good design in the first place. I cannot believe that pg_xlog in most systems has enough capacity to store many WAL files for the s

Re: [HACKERS] Using streaming replication as log archiving

On Thu, Sep 30, 2010 at 15:45, Kevin Grittner wrote: > Magnus Hagander wrote: > >>> If you could keep the development "friendly" to such features, I >>> may get around to adding them to support our needs >> >> Would it be enough to have kind of an "archive_command" switch >> that says "whenev

Re: [HACKERS] is sync rep stalled?

On 30.09.2010 17:09, Kevin Grittner wrote: Aidan Van Dyk wrote: Heikki Linnakangas wrote: I'm sure there's several things you can accomplish with synchronous replication, perhaps you could describe what the important use case for you is? I'm looking for "data durability", not "server query

Re: [HACKERS] is sync rep stalled?

Aidan Van Dyk wrote: Heikki Linnakangas wrote: >> I'm sure there's several things you can accomplish with >> synchronous replication, perhaps you could describe what the >> important use case for you is? > I'm looking for "data durability", not "server query-ability" Same here. If we used

Re: [HACKERS] is sync rep stalled?

On Thu, Sep 30, 2010 at 09:52:46AM -0400, Tom Lane wrote: > David Fetter writes: > > On Thu, Sep 30, 2010 at 09:14:42AM +0100, Simon Riggs wrote: > >> I don't see anything has stalled. > > > I do. We're half way through this commitfest, so if no one's > > actually ready to commit one of the patc

Re: [HACKERS] is sync rep stalled?

On Thu, Sep 30, 2010 at 2:09 AM, Heikki Linnakangas wrote: > Agreed. Actually, given the lack of people jumping in and telling us what > they'd like to do with the feature, maybe it's not that important after all. >> The basic features that I mean is for most basic use case, that is, one >> mast

Re: [HACKERS] is sync rep stalled?

David Fetter writes: > On Thu, Sep 30, 2010 at 09:14:42AM +0100, Simon Riggs wrote: >> I don't see anything has stalled. > I do. We're half way through this commitfest, so if no one's actually > ready to commit one of the patches, I kinda have to bounce them both, > at least to the next CF. [ r

Re: [HACKERS] Using streaming replication as log archiving

Magnus Hagander wrote: >> If you could keep the development "friendly" to such features, I >> may get around to adding them to support our needs > > Would it be enough to have kind of an "archive_command" switch > that says "whenever you've finished a complete wal segment, run > this comman

Re: [HACKERS] is sync rep stalled?

On Thu, Sep 30, 2010 at 09:14:42AM +0100, Simon Riggs wrote: > On Thu, 2010-09-30 at 09:09 +0300, Heikki Linnakangas wrote: > > On 29.09.2010 10:56, Fujii Masao wrote: > > > On Wed, Sep 29, 2010 at 11:47 AM, Robert Haas > > > wrote: > > > >> This feature is important, and we need to get it done.

[HACKERS] final patch - plpgsql: for-in-array

Hello this patch implement a new iteration construct - iteration over an array. The sense of this new iteration is: * a simple and cleaner syntax * a faster execution - this bring down a number of detoast operations create or replace function subscripts(anyarray, int) returns int[] as $$ sele

[HACKERS] Re: starting to review the Extend NOT NULL representation to pg_constraint patch

Ok -- I've updated the commitfest page linking in this review and changing the status to waiting on a new patch from you. Thanks Andrew On Thu, Sep 30, 2010 at 4:12 AM, Bernd Helmle wrote: > > > --On 29. September 2010 23:05:11 -0400 Andrew Geery > wrote: > >> Reference: https://commitfest.post

Re: [HACKERS] proposal: tsearch dictionary initialization hook

Hello 2010/9/28 Robert Haas : > On Tue, Sep 7, 2010 at 12:55 PM, Pavel Stehule > wrote: >> 2010/9/7 Teodor Sigaev : >>> Hm, what is aim of this hook? It looks like a wrapper of dictionary init >>> method. >> >> If I use a mmap for shared dictionary, then I have to prealloc and >> maybe preread d

Re: [HACKERS] Using streaming replication as log archiving

On Wed, Sep 29, 2010 at 23:45, Kevin Grittner wrote: > Magnus Hagander wrote: > >> Comments and contributions are most welcome. > > This is probably too esoteric to be worked on yet, but for this to > be useful for us we would need to pass the resulting files through > pg_clearxlogtail and gzip i

Re: [HACKERS] wip: for-in-array patch

sorry fixed patch Pavel 2010/9/30 Pavel Stehule : > Hello > > This patch add iteration over array to plpgsql > > now is supported only iteration over scalar array > > Regards > > Pavel Stehule > *** ./gram.y.orig 2010-09-29 10:53:44.663270537 +0200 --- ./gram.y 2010-09-30 10:46:41.976025132 +020

[HACKERS] Re: starting to review the Extend NOT NULL representation to pg_constraint patch

--On 29. September 2010 23:05:11 -0400 Andrew Geery wrote: Reference: https://commitfest.postgresql.org/action/patch_view?id=312 The patch from http://archives.postgresql.org/message-id/ca2e4c4762eae28d68404...@amenop his does not apply cleanly to the current git master: Yeah, there wher

Re: [HACKERS] is sync rep stalled?

On Thu, 2010-09-30 at 09:09 +0300, Heikki Linnakangas wrote: > On 29.09.2010 10:56, Fujii Masao wrote: > > On Wed, Sep 29, 2010 at 11:47 AM, Robert Haas wrote: > >> This feature is important, and we need to get it done. How do we get > >> the ball rolling again? > > Agreed. Actually, given the

Re: [HACKERS] Fw: patch for pg_ctl.c to add windows service start-type

On Thu, Sep 30, 2010 at 04:40, Itagaki Takahiro wrote: > Hi, I have a question about the latest patch. > > On Sun, Aug 22, 2010 at 11:03 PM, Quan Zongliang > wrote: >> New patch attached. How about this? >>> I don't see us ever using anything other than auto or demand. The >>> others aren't for "

[HACKERS] wip: for-in-array patch

Hello This patch add iteration over array to plpgsql now is supported only iteration over scalar array Regards Pavel Stehule *** ./gram.y.orig 2010-09-29 10:53:44.663270537 +0200 --- ./gram.y 2010-09-30 09:04:04.809900052 +0200 *** *** 239,244 --- 239,245 %token K_ABSOL

Re: [HACKERS] patch: SQL/MED(FDW) DDL

On Thu, 30 Sep 2010 09:26:54 +0300 Heikki Linnakangas wrote: > > FYI, HiRDB, that implements FDW routines, has CREATE FOREIGN INDEX. > > I think it is a little ugly and won't work in some cases -- for example, > > index organized tables -- but evidently it's a realistic solution. > > A long time

Re: [HACKERS] Stalled post to pgsql-committers

On Wed, 29 Sep 2010, Alvaro Herrera wrote: Excerpts from Peter Eisentraut's message of mi? sep 29 04:08:35 -0400 2010: On s?n, 2010-09-26 at 17:11 +0200, Magnus Hagander wrote: Yeah, that's what you need to do. I would guess you were previously subscribed as pe...@postgresql.org, but the git

Re: [HACKERS] Stalled post to pgsql-committers

Excerpts from Peter Eisentraut's message of mié sep 29 04:08:35 -0400 2010: > On sön, 2010-09-26 at 17:11 +0200, Magnus Hagander wrote: > > Yeah, that's what you need to do. I would guess you were previously > > subscribed as pe...@postgresql.org, but the git commit scrpit sends > > the email from