Re: [HACKERS] auto-sizing wal_buffers

2011-01-15 Thread Fujii Masao
On Sat, Jan 15, 2011 at 3:51 PM, Greg Smith wrote: > Agreed on both fronts.  Attached patch does the magic.  Also available in > branch "walbuffers" from git://github.com/greg2ndQuadrant/postgres.git +intXLOGbuffersMin = 8; XLOGbuffersMin is a fixed value. I think that defini

Re: [HACKERS] We need to log aborted autovacuums

2011-01-15 Thread Greg Smith
Josh Berkus wrote: The lack of vacuum could be occurring for any of 4 reasons: 1) Locking 2) You have a lot of tables and not enough autovac_workers / too much sleep time 3) You need to autovac this particular table more frequently, since it gets dirtied really fast 4) The table has been set wit

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Greg Smith
Robert Haas wrote: On Tue, Nov 30, 2010 at 3:29 PM, Greg Smith wrote: One of the ideas Simon and I had been considering at one point was adding some better de-duplication logic to the fsync absorb code, which I'm reminded by the pattern here might be helpful independently of other improvemen

Re: [HACKERS] Recovery control functions

2011-01-15 Thread Fujii Masao
On Fri, Jan 14, 2011 at 8:15 PM, Simon Riggs wrote: > On Fri, 2011-01-14 at 11:09 +, Simon Riggs wrote: >> Functions to control recovery, to aid PITR and Hot Standby. >> pg_is_xlog_replay_paused() >> pg_xlog_replay_pause() >> pg_xlog_replay_resume() >> >> recovery.conf parameter: pause_at_reco

Re: [HACKERS] Recovery control functions

2011-01-15 Thread Fujii Masao
On Fri, Jan 14, 2011 at 9:00 PM, Simon Riggs wrote: >> How hard would it be to have a pg_xlog_replay_until(> timestamp>), to have it resume recovery up to that point and then >> pause again? > > You can already do that for timestamps. You mean using recovery_target_time and pause_at_recovery_targ

Re: [HACKERS] Recovery control functions

2011-01-15 Thread Simon Riggs
On Sat, 2011-01-15 at 20:11 +0900, Fujii Masao wrote: > On Fri, Jan 14, 2011 at 9:00 PM, Simon Riggs wrote: > >> How hard would it be to have a pg_xlog_replay_until( >> timestamp>), to have it resume recovery up to that point and then > >> pause again? > > > > You can already do that for timestamp

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Florian Pflug
On Jan15, 2011, at 02:03 , Tom Lane wrote: > Robert Haas writes: >> Me, too. But I don't agree with your particular choice of small >> syntax adjustment. Maybe we should just let the issue drop for now. >> Nobody's actually complained about this that I can recall; it's just a >> comment that's b

Re: [HACKERS] Recovery control functions

2011-01-15 Thread Simon Riggs
On Sat, 2011-01-15 at 20:00 +0900, Fujii Masao wrote: > On Fri, Jan 14, 2011 at 8:15 PM, Simon Riggs wrote: > > On Fri, 2011-01-14 at 11:09 +, Simon Riggs wrote: > >> Functions to control recovery, to aid PITR and Hot Standby. > >> pg_is_xlog_replay_paused() > >> pg_xlog_replay_pause() > >> pg

Re: [HACKERS] kill -KILL: What happens?

2011-01-15 Thread Florian Pflug
On Jan14, 2011, at 17:45 , Robert Haas wrote: > On Fri, Jan 14, 2011 at 11:28 AM, Florian Pflug wrote: >> I gather that the behaviour we want is for normal backends to exit >> once the postmaster is gone, and for utility processes (bgwriter, ...) >> to exit once all the backends are gone. >> >> T

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Simon Riggs
On Sat, 2011-01-15 at 12:19 +0100, Florian Pflug wrote: > On Jan15, 2011, at 02:03 , Tom Lane wrote: > > Robert Haas writes: > >> Me, too. But I don't agree with your particular choice of small > >> syntax adjustment. Maybe we should just let the issue drop for now. > >> Nobody's actually compla

Re: [HACKERS] [RRR] reviewers needed!

2011-01-15 Thread Dimitri Fontaine
Robert Haas writes: > lot more people pitch in. I just assigned myself the review of those two patches: https://commitfest.postgresql.org/action/patch_view?id=475 Streaming base backups https://commitfest.postgresql.org/action/patch_view?id=502 FOR KEY LOCK foreign keys I think I'l

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Magnus Hagander
On Jan 15, 2011 12:30 PM, "Simon Riggs" wrote: > > On Sat, 2011-01-15 at 12:19 +0100, Florian Pflug wrote: > > On Jan15, 2011, at 02:03 , Tom Lane wrote: > > > Robert Haas writes: > > >> Me, too. But I don't agree with your particular choice of small > > >> syntax adjustment. Maybe we should ju

Re: [HACKERS] pov 1.0 is released, testers with huge schemas needed

2011-01-15 Thread Joel Jacobson
2011/1/14 Joel Jacobson : > To install: > git clone g...@github.com:gluefinance/pov.git Ops, typo, to do this without my ssh key, you need to do: git clone git://github.com/gluefinance/pov.git -- Best regards, Joel Jacobson Glue Finance -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 5:47 AM, Greg Smith wrote: > No toe damage, this is great, I hadn't gotten to coding for this angle yet > at all.  Suffering from an overload of ideas and (mostly wasted) test data, > so thanks for exploring this concept and proving it works. Yeah - obviously I want to mak

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 6:29 AM, Simon Riggs wrote: > On Sat, 2011-01-15 at 12:19 +0100, Florian Pflug wrote: >> On Jan15, 2011, at 02:03 , Tom Lane wrote: >> > Robert Haas writes: >> >> Me, too.  But I don't agree with your particular choice of small >> >> syntax adjustment.  Maybe we should jus

Re: [HACKERS] kill -KILL: What happens?

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 6:27 AM, Florian Pflug wrote: > On Jan14, 2011, at 17:45 , Robert Haas wrote: >> On Fri, Jan 14, 2011 at 11:28 AM, Florian Pflug wrote: >>> I gather that the behaviour we want is for normal backends to exit >>> once the postmaster is gone, and for utility processes (bgwrit

Re: [HACKERS] Add support for logging the current role

2011-01-15 Thread Robert Haas
On Fri, Jan 14, 2011 at 9:24 PM, Andrew Dunstan wrote: > On 01/14/2011 08:41 PM, Robert Haas wrote: >> I think we're in the process of designing a manned mission to Mars to >> solve the problem that our shoelaces are untied. > What's your suggestion, then? If there's a practical way to add the re

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Simon Riggs
On Sat, 2011-01-15 at 05:47 -0500, Greg Smith wrote: > Robert Haas wrote: > > On Tue, Nov 30, 2010 at 3:29 PM, Greg Smith wrote: > > > > > One of the ideas Simon and I had been considering at one point was adding > > > some better de-duplication logic to the fsync absorb code, which I'm > > >

Re: [HACKERS] ALTER TYPE 0: Introduction; test cases

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 1:30 AM, Noah Misch wrote: > Here's v2 based on your feedback. > > I pruned test coverage down to just the highlights.  By the end of patch > series, > the net change becomes +67 to alter_table.sql and +111 to alter_table.out.   > The > alter_table.out delta is larger in t

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 8:55 AM, Simon Riggs wrote: > On Sat, 2011-01-15 at 05:47 -0500, Greg Smith wrote: >> Robert Haas wrote: >> > On Tue, Nov 30, 2010 at 3:29 PM, Greg Smith wrote: >> > >> > > One of the ideas Simon and I had been considering at one point was adding >> > > some better de-dupl

Re: [HACKERS] Bug in pg_describe_object, patch v2

2011-01-15 Thread Robert Haas
On Thu, Jan 13, 2011 at 1:24 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Jan 12, 2011 at 7:47 PM, Tom Lane wrote: >>> IMO, what this patch needs is to not output the types unless they are >>> actually different from the default (which can be inferred from the AM >>> type and the functio

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Greg Smith
Robert Haas wrote: Idea #2: At the beginning of a checkpoint when we scan all the buffers, count the number of buffers that need to be synced for each relation. Use the same hashtable that we use for tracking pending fsync requests. Then, interleave the writes and the fsyncs... Idea #3: Stick

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 9:25 AM, Greg Smith wrote: > Once upon a time we got a patch from Itagaki Takahiro whose purpose was to > sort writes before sending them out: > > http://archives.postgresql.org/pgsql-hackers/2007-06/msg00541.php Ah, a fine idea! > Which has very low odds of the sync on "

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Heikki Linnakangas
On 14.01.2011 13:38, Magnus Hagander wrote: On Fri, Jan 14, 2011 at 11:19, Heikki Linnakangas wrote: On 14.01.2011 08:45, Fujii Masao wrote: 1. Smart shutdown is requested while walsender is sending a backup. 2. Shutdown causes archiver to end. (Though shutdown sends SIGUSR2 to walsende

[HACKERS] LAST CALL FOR 9.1

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 9:25 AM, Greg Smith wrote: > I'm going to share it before someone announces a deadline has passed or > something.  (whistling) Speaking of that, it is now Saturday, January 15, 2011 and CommitFest 2011-01 is now in progress. If you have any patches that you haven't submit

Re: [HACKERS] limiting hint bit I/O

2011-01-15 Thread Martijn van Oosterhout
On Fri, Jan 14, 2011 at 05:24:31PM -0800, Josh Berkus wrote: > On 1/14/11 11:51 AM, Tom Lane wrote: > > The people whose tables are mostly insert-only complain about it, but > > that's not the majority of our userbase IMO. We just happen to have a > > couple of particularly vocal ones, like Berkus

Re: [HACKERS] ALTER TYPE 0: Introduction; test cases

2011-01-15 Thread Noah Misch
On Sat, Jan 15, 2011 at 08:57:30AM -0500, Robert Haas wrote: > On Sat, Jan 15, 2011 at 1:30 AM, Noah Misch wrote: > > Here's v2 based on your feedback. > > > > I pruned test coverage down to just the highlights. ?By the end of patch > > series, > > the net change becomes +67 to alter_table.sql an

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Tom Lane
Robert Haas writes: > On Sat, Jan 15, 2011 at 6:29 AM, Simon Riggs wrote: >> I think we should have a section in the release notes on Deprecated >> Features, noting that certain things will be removed later and should be >> changed now and not relied upon in the future. A pending >> incompatibili

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Tom Lane
Heikki Linnakangas writes: > BTW, I just spotted a small race condition between creating a new table > space and base backup. We take a snapshot of all the tablespaces in > pg_tblspc before calling pg_start_backup(). If someone creates a new > tablespace and puts some data in it in the window b

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Greg Smith
Robert Haas wrote: I'll believe it when I see it. How about this: a 1 a 2 sync a b 1 b 2 sync b c 1 c 2 sync c Or maybe some variant, where we become willing to fsync a file a certain number of seconds after writing the last block, or when all the writes are done, whichever comes first. That

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Heikki Linnakangas
On 15.01.2011 17:30, Tom Lane wrote: Heikki Linnakangas writes: BTW, I just spotted a small race condition between creating a new table space and base backup. We take a snapshot of all the tablespaces in pg_tblspc before calling pg_start_backup(). If someone creates a new tablespace and puts so

Re: [HACKERS] Bug in pg_describe_object, patch v2

2011-01-15 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 13, 2011 at 1:24 PM, Tom Lane wrote: >> Read the CREATE OPERATOR CLASS source code.  (It likely would be best to >> refactor that a bit so it would expose some way to obtain the implied >> defaults --- I don't think that's done explicitly now, and it's >> certain

Re: [HACKERS] kill -KILL: What happens?

2011-01-15 Thread Florian Pflug
On Jan14, 2011, at 17:45 , Robert Haas wrote: > On Fri, Jan 14, 2011 at 11:28 AM, Florian Pflug wrote: >> I gather that the behaviour we want is for normal backends to exit >> once the postmaster is gone, and for utility processes (bgwriter, ...) >> to exit once all the backends are gone. >> >> T

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Tom Lane
Heikki Linnakangas writes: > On 15.01.2011 17:30, Tom Lane wrote: >> So what? The needed actions will be covered by WAL replay. > No, they won't, if pg_base_backup() is called *after* getting the list > of tablespaces. Ah. Then the fix is to change the order in which those things are done.

Re: [HACKERS] Add support for logging the current role

2011-01-15 Thread Tom Lane
Robert Haas writes: > On Fri, Jan 14, 2011 at 9:24 PM, Andrew Dunstan wrote: >> What's your suggestion, then? > If there's a practical way to add the requested escape, add it to the > text format and leave reengineering the CSV format for another day. > Yeah, I know that's not the most beautiful

Re: [HACKERS] We need to log aborted autovacuums

2011-01-15 Thread Tom Lane
Greg Smith writes: > Does try_relation_open need to have a lock acquisition timeout when AV > is calling it? Hmm. I think when looking at the AV code, I've always subconsciously assumed that try_relation_open would fail immediately if it couldn't get the lock. That certainly seems like it woul

Re: [HACKERS] log_hostname and pg_stat_activity

2011-01-15 Thread Peter Eisentraut
On tor, 2010-12-23 at 22:21 +0100, Magnus Hagander wrote: > On Thu, Dec 23, 2010 at 22:09, Peter Eisentraut wrote: > > Somehow I fantasized that log_hostname would also turn > > pg_stat_activity.client_addr into names instead of IP addresses. It > > doesn't, but perhaps it should? It would be ni

Re: [HACKERS] Add support for logging the current role

2011-01-15 Thread Andrew Dunstan
On 01/15/2011 11:08 AM, Tom Lane wrote: Robert Haas writes: On Fri, Jan 14, 2011 at 9:24 PM, Andrew Dunstan wrote: What's your suggestion, then? If there's a practical way to add the requested escape, add it to the text format and leave reengineering the CSV format for another day. Yeah, I

Re: [HACKERS] auto-sizing wal_buffers

2011-01-15 Thread Greg Smith
Fujii Masao wrote: +intXLOGbuffersMin = 8; XLOGbuffersMin is a fixed value. I think that defining it as a macro rather than a variable seems better. + if (XLOGbuffers > 2048) + XLOGbuffers = 2048; Using "XLOG_SEG_SIZE/XLOG_BLCKSZ" rather

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2011-01-15 Thread Bruce Momjian
Josh Berkus wrote: > On 12/6/10 6:13 PM, Tom Lane wrote: > > Josh Berkus writes: > >> OK, patch coming then. Right now test_fsync aborts when O_DIRECT fails. > >> What should I have it do instead? > > > > Report that it fails, and keep testing the other methods. > > Patch attached. Includes a

Re: [HACKERS] Bug in pg_describe_object, patch v2

2011-01-15 Thread Joel Jacobson
2011/1/15 Tom Lane : > But I can read the handwriting on the wall: if I want this done right, > I'm going to have to do it myself. > >                        regards, tom lane Excellently put! I will with pride steal that phrase and use it whenever I run into the same situation myself. Quite often

[HACKERS] textarray option for file FDW

2011-01-15 Thread Andrew Dunstan
I've been waiting for the latest FDW patches as patiently as I can, and I've been reviewing them this morning, in particular the file_fdw patch and how it interacts with the newly exposed COPY API. Overall it seems to be a whole lot cleaner, and the wholesale duplication of the copy code is g

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Tom Lane
Andrew Dunstan writes: > ... So now I'd like to add a > new option to it: "textarray". This option would require that the > foreign table have exactly one field, of type text[], and would compose > all the field strings read from the file for each record into the array > (however many there ar

[HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-15 Thread Noah Misch
Hello Pavel, I'm reviewing this patch for CommitFest 2011-01. The patch seems fully desirable. It appropriately contains no documentation updates. It contains no new tests, and that's probably fine, too; I can't think of any corner cases where this would do something other than work correctly o

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Andrew Dunstan
On 01/15/2011 12:44 PM, Tom Lane wrote: Andrew Dunstan writes: ... So now I'd like to add a new option to it: "textarray". This option would require that the foreign table have exactly one field, of type text[], and would compose all the field strings read from the file for each record into t

[HACKERS] pg_basebackup for streaming base backups

2011-01-15 Thread Magnus Hagander
This patch creates pg_basebackup in bin/, being a client program for the streaming base backup feature. I think it's more or less done now. I've again split it out of pg_streamrecv, because it had very little shared code with that (basically just the PQconnectdb() wrapper). One thing I'm thinking

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 16:54, Tom Lane wrote: > Heikki Linnakangas writes: >> On 15.01.2011 17:30, Tom Lane wrote: >>> So what?  The needed actions will be covered by WAL replay. > >> No, they won't, if pg_base_backup() is called *after* getting the list >> of tablespaces. > > Ah.  Then the fix

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Itagaki Takahiro
On Sun, Jan 16, 2011 at 02:29, Andrew Dunstan wrote: > "textarray". This option would require that the foreign table have exactly > one field, of type text[], and would compose all the field strings read from > the file for each record into the array (however many there are). > >   CREATE FOREIGN

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Tom Lane
Magnus Hagander writes: > Something like this to fix? or is this going to put those "warnings by > stupid versions of gcc" back? Possibly. If so, I'll fix it --- I have an old gcc to test against here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 19:27, Tom Lane wrote: > Magnus Hagander writes: >> Something like this to fix? or is this going to put those "warnings by >> stupid versions of gcc" back? > > Possibly.  If so, I'll fix it --- I have an old gcc to test against > here. Ok, thanks, I'll commit tihs one the

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Andrew Dunstan
On 01/15/2011 01:24 PM, Itagaki Takahiro wrote: On Sun, Jan 16, 2011 at 02:29, Andrew Dunstan wrote: "textarray". This option would require that the foreign table have exactly one field, of type text[], and would compose all the field strings read from the file for each record into the array

Re: [HACKERS] Visual Studio 2010/Windows SDK 7.1 support

2011-01-15 Thread Magnus Hagander
On Mon, Jan 3, 2011 at 16:45, Brar Piening wrote: > On Mon, 3 Jan 2011 10:44:19 +0100, Magnus Hagander > wrote: >> >> Yeah, it looks that way - it's missing the ordering of the contrib >> I'll run it once for that now, and then please rebase your >> patch on top of that - makes it easier to revie

Re: [HACKERS] LAST CALL FOR 9.1

2011-01-15 Thread David E. Wheeler
On Jan 15, 2011, at 6:50 AM, Robert Haas wrote: > It is really already too late for us to be seriously considering > integrating sync rep into 9.1. It will lead to another enormous beta > period during which the tree will be closed to new patches and > everyone will complain, or else we'll open t

Re: [HACKERS] LAST CALL FOR 9.1

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 1:59 PM, David E. Wheeler wrote: > On Jan 15, 2011, at 6:50 AM, Robert Haas wrote: >>> It is really already too late for us to be seriously considering >> integrating sync rep into 9.1.  It will lead to another enormous beta >> period during which the tree will be closed to

Re: [HACKERS] estimating # of distinct values

2011-01-15 Thread Tomas Vondra
Hi, a short update regarding the ndistinct stream estimators - I've implemented the estimators described in the papers I've mentioned in my previous posts. If you want try it, the sources are available at github, at http://tvondra.github.com/pg_estimator (ignore the page, I have to update it, just

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 10:25 AM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Jan 15, 2011 at 6:29 AM, Simon Riggs wrote: >>> I think we should have a section in the release notes on Deprecated >>> Features, noting that certain things will be removed later and should be >>> changed now and

Re: [HACKERS] We need to log aborted autovacuums

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 11:14 AM, Tom Lane wrote: > Greg Smith writes: >> Does try_relation_open need to have a lock acquisition timeout when AV >> is calling it? > > Hmm.  I think when looking at the AV code, I've always subconsciously > assumed that try_relation_open would fail immediately if i

Re: [HACKERS] ALTER TYPE 0: Introduction; test cases

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 10:25 AM, Noah Misch wrote: > Do you value test coverage so little? If you're asking whether I think real-world usability is more important than test coverage, then yes. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent

[HACKERS] test_fsync made modular

2011-01-15 Thread Bruce Momjian
I have modified test_fsync to use modular C so there is less duplicate code and it can be enhanced easier. Applied patch attached, though the diff is larger than the C file so you might want to just look at the C file in git. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [HACKERS] Visual Studio 2010/Windows SDK 7.1 support

2011-01-15 Thread Andrew Chernow
On 1/15/2011 1:42 PM, Magnus Hagander wrote: On Mon, Jan 3, 2011 at 16:45, Brar Piening wrote: On Mon, 3 Jan 2011 10:44:19 +0100, Magnus Hagander wrote: Yeah, it looks that way - it's missing the ordering of the contrib I'll run it once for that now, and then please rebase your patch on top o

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Dimitri Fontaine
Tom Lane writes: > Why is this a good thing? It seems like it would accomplish little > except to defeat the SQL type system entirely. It simply allows to have the classical ETL problem solved directly in the database server, in SQL. That's huge. Regards, -- Dimitri Fontaine http://2ndQuadran

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-15 Thread Dimitri Fontaine
Hi, I have an unexpected 5 mins window to do a first reading of the patch, so here goes the quick doc and comments proof reading of it. :) Magnus Hagander writes: > This patch creates pg_basebackup in bin/, being a client program for > the streaming base backup feature. Great! We have pg_ctl i

Re: [HACKERS] Review: Fix snapshot taking inconsistencies

2011-01-15 Thread Marko Tiikkaja
On 2010-10-21 3:32 PM +0200, Marko Tiikkaja wrote: Here's the patch rebased against the master. No code changes since the last patch I sent. Regards, Marko Tiikkaja *** a/src/backend/catalog/pg_proc.c --- b/src/backend/catalog/pg_proc.c *** *** 759,765 fmgr_sql_validator(

Re: [HACKERS] EXPLAIN and nfiltered

2011-01-15 Thread Marko Tiikkaja
On 2010-11-18 5:45 PM +0200, Marko Tiikkaja wrote: Here's a patch for showing in EXPLAIN ANALYZE the number of rows a plan qual filtered from a node's input. Rebased against master. Regards, Marko Tiikkaja *** a/src/backend/commands/explain.c --- b/src/backend/commands/explain.c *

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Tom Lane
Robert Haas writes: > On Sat, Jan 15, 2011 at 10:25 AM, Tom Lane wrote: >> I suggest also marking each item with a release in which we intend to do >> it, so we don't have to try to remember whether a reasonable amount of >> time has elapsed. > You mean like the way the 9.1devel documentation sa

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Simon Riggs
On Sat, 2011-01-15 at 09:15 -0500, Robert Haas wrote: > On Sat, Jan 15, 2011 at 8:55 AM, Simon Riggs wrote: > > On Sat, 2011-01-15 at 05:47 -0500, Greg Smith wrote: > >> Robert Haas wrote: > >> > On Tue, Nov 30, 2010 at 3:29 PM, Greg Smith wrote: > >> > > >> > > One of the ideas Simon and I had b

[HACKERS] Include WAL in base backup

2011-01-15 Thread Magnus Hagander
Here's a cutdown version of the idea about including WAL in the base backup. What I initially wanted was to introduce a way to guarantee that the required WAL (with some sort of cutoff of course) would be available for the backup, but I ran out of time for that. We can always add that later. For n

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 21:16, Dimitri Fontaine wrote: > Hi, > > I have an unexpected 5 mins window to do a first reading of the patch, > so here goes the quick doc and comments proof reading of it. :) :-) > Magnus Hagander writes: >> This patch creates pg_basebackup in bin/, being a client pr

Re: [HACKERS] LOCK for non-tables

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 3:37 PM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Jan 15, 2011 at 10:25 AM, Tom Lane wrote: >>> I suggest also marking each item with a release in which we intend to do >>> it, so we don't have to try to remember whether a reasonable amount of >>> time has elapsed

Re: [HACKERS] Transaction-scope advisory locks

2011-01-15 Thread Marko Tiikkaja
On 2010-12-14 12:52 AM +0200, Marko Tiikkaja wrote: Here's the latest version of the patch. It now uses the API proposed by Simon, but still lacks documentation changes, which I'm going to send tomorrow. Regards, Marko Tiikkaja *** a/src/backend/storage/lmgr/lock.c --- b/src/backend/stor

[HACKERS] plperlu problem with utf8 [REVIEW]

2011-01-15 Thread Andy Colson
This is a review of "plperl encoding issues" https://commitfest.postgresql.org/action/patch_view?id=452 Purpose: Your database uses one encoding, and passes data to perl in the same encoding, which perl is not prepared for (it assumes UTF-8). This patch makes sure data is encoded i

Re: [HACKERS] [PATCH] Return command tag 'REPLACE X' for CREATE OR REPLACE statements.

2011-01-15 Thread Marti Raudsepp
Here's an updated patch that reports command status back to ProcessUtility via 'bool' return value. I was a bit unsure about using bool return values because it's not immediately obvious what "true" or "false" refer to, but defining a new enum seemed like overkill, so I went with bool anyway. Any

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 10:31 AM, Greg Smith wrote: > That's going to give worse performance than the current code in some cases. OK. >> How does the checkpoint target give you any time to sync them?  Unless >> you squeeze the writes together more tightly, but that seems sketchy. > > Obviously t

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-15 Thread Euler Taveira de Oliveira
Em 15-01-2011 15:10, Magnus Hagander escreveu: One thing I'm thinking about - right now the tool just takes -c to connect to the database. Should it instead be taught to take the connection parameters that for example pg_dump does - one for each of host, port, user, password? (shouldn't be hard

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 22:40, Simon Riggs wrote: > > Here's the latest patch for sync rep. > > >From here, I will be developing the patch further on public git > repository towards commit. My expectation is that commit is at least 2 That's great. Just one tiny detail - which repository and which

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-15 Thread Simon Riggs
On Tue, 2010-12-14 at 19:48 +, Simon Riggs wrote: > On Tue, 2010-12-14 at 11:34 -0800, Josh Berkus wrote: > > As for the utility of this command: there is no question that I would > > use it. I'm not sure I like the syntax (I'd prefer REPLACE TABLE > > WITH _), but that's painting the

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-15 Thread Dimitri Fontaine
Magnus Hagander writes: > Not sure pg_ctl clone would be the proper name, since it's not > actually a clone at this point (it might be with the second patch I > ust posted that includes the WAL files) Let's keep the clone name for the client that makes it all then :) >> That should be -D --pgdat

Re: [HACKERS] ALTER TYPE 0: Introduction; test cases

2011-01-15 Thread Noah Misch
On Sat, Jan 15, 2011 at 02:31:21PM -0500, Robert Haas wrote: > On Sat, Jan 15, 2011 at 10:25 AM, Noah Misch wrote: > > Do you value test coverage so little? > > If you're asking whether I think real-world usability is more > important than test coverage, then yes. No, I wasn't asking that. The

[HACKERS] What happened to open_sync_without_odirect?

2011-01-15 Thread Josh Berkus
Last I remember, we were going to add this as an option. But I don't see a patch in the queue. Am I missing it? Was I supposed to write it? -- -- Josh Berkus PostgreSQL Experts Inc. http

Re: [HACKERS] Include WAL in base backup

2011-01-15 Thread Dimitri Fontaine
Magnus Hagander writes: > Here's a cutdown version of the idea about including WAL in the base > backup. What I initially wanted was to introduce a way to guarantee > that the required WAL (with some sort of cutoff of course) would be > available for the backup, but I ran out of time for that. We

Re: [HACKERS] auto-sizing wal_buffers

2011-01-15 Thread Josh Berkus
On 1/14/11 10:51 PM, Greg Smith wrote: > > ! Since the data is written out to disk at every transaction > commit, > ! the setting many only need to be be large enough to hold the > amount > ! of WAL data generated by one typical transaction. Larger values, > ! typi

Re: [HACKERS] arrays as pl/perl input arguments [PATCH]

2011-01-15 Thread Alex Hunsaker
On Wed, Jan 12, 2011 at 13:04, Alexey Klyukin wrote: > > On Jan 12, 2011, at 8:52 PM, David E. Wheeler wrote: > >> On Jan 12, 2011, at 5:14 AM, Alexey Klyukin wrote: >> >>> You mean packing both a string representation and a reference to a single >>> SV * value? >> >> Dunno, I'm not a guts guy. >

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Greg Smith
Robert Haas wrote: That seems like a bad idea - don't we routinely recommend that people crank this up to 0.9? You'd be effectively bounding the upper range of this setting to a value to the less than the lowest value we recommend anyone use today. I was just giving an example of how I migh

Re: [HACKERS] LAST CALL FOR 9.1

2011-01-15 Thread Josh Berkus
On 1/15/11 6:50 AM, Robert Haas wrote: > It is really already too late for us to be seriously considering > integrating sync rep into 9.1. It will lead to another enormous beta > period during which the tree will be closed to new patches and > everyone will complain, or else we'll open the tree fo

Re: [HACKERS] limiting hint bit I/O

2011-01-15 Thread Josh Berkus
> If the problem is that all the freezing happens at once, then ISTM the > solution is to add a random factor. Say, when a tuple just passes the > lower threshold it has a 1% chance of being frozen. The chance grows > until it is 100% as it reaches the upper threshold. Doesn't have to be random;

Re: [HACKERS] Include WAL in base backup

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 23:32, Dimitri Fontaine wrote: > Magnus Hagander writes: >> Here's a cutdown version of the idea about including WAL in the base >> backup. What I initially wanted was to introduce a way to guarantee >> that the required WAL (with some sort of cutoff of course) would be >>

[HACKERS] .gitignore file needed for new replication parser

2011-01-15 Thread Kevin Grittner
diff --git a/src/backend/replication/.gitignore b/src/backend/replication/.gitignore new file mode 100644 index 000..a0332b2 --- /dev/null +++ b/src/backend/replication/.gitignore @@ -0,0 +1,3 @@ +/repl_gram.c +/repl_gram.h +/repl_scanner.c -- Sent via pgsql-hackers mailing list (pgsql-hack

[HACKERS] New test_fsync messages for direct I/O

2011-01-15 Thread Bruce Momjian
Josh Berkus wrote: > Greg, > > > This is interesting, because test_fsync consistently reported a rate of > > about half this when using open_datasync instead of the equal > > performance I'm getting from the database. I'll see if I can reproduce > > that further, but it's no reason to be concerne

Re: [HACKERS] What happened to open_sync_without_odirect?

2011-01-15 Thread Bruce Momjian
Josh Berkus wrote: > Last I remember, we were going to add this as an option. But I don't > see a patch in the queue. Am I missing it? Was I supposed to write it? I don't know, but let me add that I am confused how this would look to users. In many cases, kernels don't even support O_DIRECT, s

Re: [HACKERS] textarray option for file FDW

2011-01-15 Thread Andrew Dunstan
On 01/15/2011 12:29 PM, Andrew Dunstan wrote: I've been waiting for the latest FDW patches as patiently as I can, and I've been reviewing them this morning, in particular the file_fdw patch and how it interacts with the newly exposed COPY API. Overall it seems to be a whole lot cleaner, and

[HACKERS] review: FDW API

2011-01-15 Thread Jan Urbański
Hi, what follows is a review of the FDW API patch from http://archives.postgresql.org/message-id/20110114212358.82c7.69899...@metrosystems.co.jp All three patches apply cleanly and compile without warnings. Regression tests pass. Let me go patch by patch, starting with the first one that adds th

Re: [HACKERS] psql: Add \dL to show languages

2011-01-15 Thread Andreas Karlsson
Hi Josh, Here is my review of this patch for the commitfest. Review of https://commitfest.postgresql.org/action/patch_view?id=439 Contents and Purpose This patch adds the \dL command in psql to list the procedual languages. To me this seems like a useful addition to the co

Re: [HACKERS] Streaming base backups

2011-01-15 Thread Tatsuo Ishii
> When do the standby launch its walreceiver? It would be extra-nice for > the base backup tool to optionally continue streaming WALs until the > standby starts doing it itself, so that wal_keep_segments is really > deprecated. No idea how feasible that is, though. Good point. I have been always

Re: [HACKERS] Bug in pg_describe_object, patch v2

2011-01-15 Thread Andreas Karlsson
On Sat, 2011-01-15 at 10:36 -0500, Tom Lane wrote: > But I can read the handwriting on the wall: if I want this done right, > I'm going to have to do it myself. > > regards, tom lane Do I understand you correctly if I interpret what you would like to see is the same format u

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Marti Raudsepp
On Sat, Jan 15, 2011 at 14:05, Robert Haas wrote: > Idea #4: For ext3 filesystems that like to dump the entire buffer > cache instead of only the requested file, write a little daemon that > runs alongside of (and completely indepdently of) PostgreSQL.  Every > 30 s, it opens a 1-byte file, change

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Robert Haas
On Sat, Jan 15, 2011 at 5:57 PM, Greg Smith wrote: > I was just giving an example of how I might do an initial split.  There's a > checkpoint happening now at time T; we have a rough idea that it needs to be > finished before some upcoming time T+D.  Currently with default parameters > this become

Re: [HACKERS] Spread checkpoint sync

2011-01-15 Thread Greg Smith
Robert Haas wrote: What is the basis for thinking that the sync should get the same amount of time as the writes? That seems pretty arbitrary. Right now, you're allowing 3 seconds per fsync, which could be a lot more or a lot less than 40% of the total checkpoint time... Just that it's where