Re: [HACKERS] polyphase merge?

2009-02-03 Thread Don Marvick
Dear All, Since nobody replied, I would give it a try. I am going to implement the merge pattern described in Knuth Page 365 (5.4.9), essentially it is as follow: - create initial runs using replacement selection (basically this is as in the current implementation) - add enough dummy runs of size

Re: [HACKERS] Hot standby, recovery infra

2009-02-03 Thread Fujii Masao
Hi, On Fri, Jan 30, 2009 at 11:55 PM, Heikki Linnakangas wrote: > The startup process now catches SIGTERM, and calls proc_exit() at the next > WAL record. That's what will happen in a fast shutdown. Unexpected death of > the startup process is treated the same as a backend/auxiliary process > cra

Re: [HACKERS] Synch Replication

2009-02-03 Thread Fujii Masao
Hi, Ooops, my mail client has sent the previous message, on the way. Sorry. On Wed, Feb 4, 2009 at 10:24 AM, Fujii Masao wrote: > Hi, > > Thanks for your testing and report! > > On Tue, Feb 3, 2009 at 8:18 PM, K, Niranjan (NSN - IN/Bangalore) > wrote: >> Hi, >> >> I tried using the Synchronous

Re: [HACKERS] Synch Replication

2009-02-03 Thread Fujii Masao
Hi, Thanks for your testing and report! On Tue, Feb 3, 2009 at 8:18 PM, K, Niranjan (NSN - IN/Bangalore) wrote: > Hi, > > I tried using the Synchronous replication and I'am unable to replicate > the queries (Ex. create table temp1(int int);). > > I have downloaded the latest postgres dev snapsho

Re: [HACKERS] Column-Level Privileges

2009-02-03 Thread Robert Haas
On Tue, Feb 3, 2009 at 7:04 PM, Tom Lane wrote: > Stephen Frost writes: >> * Tom Lane (t...@sss.pgh.pa.us) wrote: >>> * Some of the information_schema views are specified to respond to >>> per-column privileges; the column_privileges and columns views >>> certainly need work now to meet spec, and

Re: [HACKERS] Column-Level Privileges

2009-02-03 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> * Some of the information_schema views are specified to respond to >> per-column privileges; the column_privileges and columns views >> certainly need work now to meet spec, and there might be others. > Done. I looked through the

Re: [HACKERS] LIMIT NULL

2009-02-03 Thread Andrew Dunstan
David E. Wheeler wrote: On Feb 3, 2009, at 8:40 AM, Andrew Dunstan wrote: We have one page per main SQL verb (e.g. SELECT or CREATE TABLE). I don't think we want to break it up more than that. One page for each clause would be a nightmare to maintain. Then should the LIMIT/OFFSET page go a

Re: [HACKERS] LIMIT NULL

2009-02-03 Thread Rick Vernam
On Tuesday 03 February 2009 3:06:27 pm Tom Lane wrote: > Rick Vernam writes: > > If looking for information about limits, I would go here: > > http://www.postgresql.org/docs/8.3/static/queries-limit.html > > and consider it to be an authoritative source. > > The reference documentation is *always*

Re: [HACKERS] LIMIT NULL

2009-02-03 Thread David E. Wheeler
On Feb 3, 2009, at 1:06 PM, Tom Lane wrote: The reference documentation is *always* intended to be more complete and more authoritative than the narrative description. If you don't think so then you need to readjust your expectations. Yes, but I didn't even know I was looking at a brief ove

Re: [HACKERS] LIMIT NULL

2009-02-03 Thread David E. Wheeler
On Feb 3, 2009, at 1:30 PM, Andrew Dunstan wrote: I was referring to the reference section. I see you were referring to the more descriptive but probably less complete Section II. Yes. I see Section II says at the beginning: "Readers looking for a complete description of a particular comma

Re: [HACKERS] LIMIT NULL

2009-02-03 Thread Tom Lane
Rick Vernam writes: > If looking for information about limits, I would go here: > http://www.postgresql.org/docs/8.3/static/queries-limit.html > and consider it to be an authoritative source. The reference documentation is *always* intended to be more complete and more authoritative than the narr

Re: [HACKERS] Compiler warning in ecpglib/execute.c

2009-02-03 Thread Alvaro Herrera
Michael Meskes wrote: > On Mon, Feb 02, 2009 at 02:56:18PM -0500, Tom Lane wrote: > > CVS HEAD is producing > > ... > > Thanks for the note, I missed this copy&paste error of mine. Fixed in HEAD. > This should alos make the buildfarm green again. Note that spoonbill is still red. This animal is

Re: [HACKERS] LIMIT NULL

2009-02-03 Thread Rick Vernam
On Tuesday 03 February 2009 10:42:30 am David E. Wheeler wrote: > On Feb 3, 2009, at 8:40 AM, Andrew Dunstan wrote: > > We have one page per main SQL verb (e.g. SELECT or CREATE TABLE). I > > don't think we want to break it up more than that. One page for each > > clause would be a nightmare to mai

Re: [HACKERS] reloptions with a "namespace"

2009-02-03 Thread Alvaro Herrera
Euler Taveira de Oliveira wrote: > Alvaro Herrera escreveu: > >> IIRC, my last patch includes a partial validation code for RESET cases. For > >> example, the last SQL will not be atomic (invalid reloption silently > >> ignored). > >> So, why not apply the namespace validation code to RESET case t

Re: [HACKERS] add_path optimization

2009-02-03 Thread Kevin Grittner
>>> Tom Lane wrote: > Can you let it run to completion? Without explain analyze results > it's going to be pretty difficult to isolate the problem. Barring some currently unforseen need to switch it into service to back the web site, yes. -Kevin -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] add_path optimization

2009-02-03 Thread Tom Lane
"Kevin Grittner" writes: > It's now been an hour and 30 minutes; so, while 8.4 does a much better > job of estimating how many rows will be returned, the plan it > generates is much slower for this query. Can you let it run to completion? Without explain analyze results it's going to be pretty d

Re: [HACKERS] add_path optimization

2009-02-03 Thread Kevin Grittner
>>> I wrote: > I tried the same run on 8.4devel and it is still running after > 20 minutes. I will let it cook for a while. It's now been an hour and 30 minutes; so, while 8.4 does a much better job of estimating how many rows will be returned, the plan it generates is much slower for this quer

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Simon Riggs
On Tue, 2009-02-03 at 18:09 +0200, Hannu Krosing wrote: > On Tue, 2009-02-03 at 14:28 +, Simon Riggs wrote: > > On Tue, 2009-02-03 at 08:40 -0500, Andrew Dunstan wrote: > > > > > > Hannu Krosing wrote: > > > > Actually we came up with a solution to this - use filesystem level > > > > snapshot

Re: [HACKERS] LIMIT NULL

2009-02-03 Thread David E. Wheeler
On Feb 3, 2009, at 8:40 AM, Andrew Dunstan wrote: We have one page per main SQL verb (e.g. SELECT or CREATE TABLE). I don't think we want to break it up more than that. One page for each clause would be a nightmare to maintain. Then should the LIMIT/OFFSET page go away? Best, David -- Se

Re: [HACKERS] add_path optimization

2009-02-03 Thread Kevin Grittner
>>> "Kevin Grittner" wrote: > Attached is EXPLAIN > ANALYZE output from the 8.3.5 database I dumped from for my 8.4devel > tests. Actually, that one is from the sibling machine which is in production. Attached is the one on standby where I've been running the rest of this. Apparently, differ

Re: [HACKERS] LIMIT NULL

2009-02-03 Thread Andrew Dunstan
David E. Wheeler wrote: Heh. Okay. Well, should there be a separate LIMIT/OFFSET documentation page? Why have this stuff in two places? I had assumed that the LIMIT/OFFSET page would have all the specifies on these clauses, and that the SELECT page (which is kind of overwhelming as it is) w

Re: [HACKERS] LIMIT NULL

2009-02-03 Thread David E. Wheeler
On Feb 2, 2009, at 1:52 PM, Robert Haas wrote: We don't really have space to document every little niggling detail in two places; if we did that, the main docs would become unreadably dense. What, disk space? What do you mean by "space"? Brain space. Heh. Okay. Well, should there be a s

Re: [HACKERS] add_path optimization

2009-02-03 Thread Kevin Grittner
>>> Robert Haas wrote: > I had a suspicion we were going to see something like this. You're > using several NOT EXISTS clauses and 8.4devel is converting those into > Anti Joins. Aside from the longer planning time, the resulting plan > appears to have a much higher estimated cost, so I'm suspic

Re: [HACKERS] add_path optimization

2009-02-03 Thread Tom Lane
Robert Haas writes: > I had a suspicion we were going to see something like this. You're > using several NOT EXISTS clauses and 8.4devel is converting those into > Anti Joins. Aside from the longer planning time, the resulting plan > appears to have a much higher estimated cost, so I'm suspicious

Re: [HACKERS] add_path optimization

2009-02-03 Thread Tom Lane
"Kevin Grittner" writes: > [ test case ] It looks to me like the reason for the planning time difference is that this query contains four NOT EXISTS subqueries, which 8.3 was not very smart about but 8.4 has converted into antijoins. That gives it more flexibility to consider different join orde

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Hannu Krosing
On Tue, 2009-02-03 at 14:28 +, Simon Riggs wrote: > On Tue, 2009-02-03 at 08:40 -0500, Andrew Dunstan wrote: > > > > Hannu Krosing wrote: > > > Actually we came up with a solution to this - use filesystem level > > > snapshots (like LVM2+XFS or ZFS), and redirect backends with > > > long-runni

Re: [HACKERS] add_path optimization

2009-02-03 Thread Robert Haas
On Tue, Feb 3, 2009 at 10:17 AM, Kevin Grittner wrote: >> We're going to need to see the test case, because I don't see that in >> some simple tests here. > Plans from 8.3.5 and 8.4devel attached. > > If you need something else, let me know. I had a suspicion we were going to see something like t

Re: [HACKERS] add_path optimization

2009-02-03 Thread Kevin Grittner
>>> I wrote: > Ran it with this: > effective_cache_size = 100GB Actually, the timings shown in the previous post were run with the default for this setting. I updated it after yesterday evening's tests when I noticed I'd missed it, but had to leave before I could rerun the tests. I forgot th

Re: [HACKERS] adding stuff to parser, question

2009-02-03 Thread Peter Eisentraut
On Saturday 31 January 2009 19:30:36 Andrew Dunstan wrote: > > > > > > Allow GRANT/REVOKE permissions to be applied to all schema objects > > with one command > > The proposed syntax is: GRANT SELECT ON ALL TABLES IN public TO

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Hannu Krosing
On Tue, 2009-02-03 at 10:19 -0500, Robert Haas wrote: > On Tue, Feb 3, 2009 at 9:40 AM, Simon Riggs wrote: > > On Tue, 2009-02-03 at 09:14 -0500, Robert Haas wrote: > >> I think _the_ solution is to notice when you're about to vacuum a page > >> that is still visible to a running backend on the st

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Simon Riggs
On Tue, 2009-02-03 at 15:55 +0100, Andres Freund wrote: > Hi, > > On 02/03/2009 02:26 PM, Hannu Krosing wrote: > >> I don't see any way around the fact that when a tuple is removed, it's > >> gone and can't be accessed by queries. Either you don't remove it, or > >> you kill the query. > > Actual

Re: [HACKERS] [NOVICE] LATIN2->UTF8 conversation with dblink

2009-02-03 Thread Tom Lane
Joe Conway writes: > Tom Lane wrote: >> Hmm. You can presumably fix this by setting client_encoding in the >> dblink connection to match the encoding in use in the database it's >> called in. But I wonder why dblink doesn't just do that for you >> automatically. > But if you think automatically

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Robert Haas
On Tue, Feb 3, 2009 at 9:40 AM, Simon Riggs wrote: > On Tue, 2009-02-03 at 09:14 -0500, Robert Haas wrote: >> I think _the_ solution is to notice when you're about to vacuum a page >> that is still visible to a running backend on the standby, and save >> that page off to a separate cache of old pa

Re: [HACKERS] pgevent warnings on mingw

2009-02-03 Thread Hiroshi Saito
- Original Message - From: "Magnus Hagander" Hiroshi Saito wrote: Ahh, sorry..like the spam again! I thought over that the existing msvc the did not have uneasines. so, I wish to make it this as correspondence with worried Magnus-san. It is after sufficient test. If I read t

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Andres Freund
Hi, On 02/03/2009 02:26 PM, Hannu Krosing wrote: I don't see any way around the fact that when a tuple is removed, it's gone and can't be accessed by queries. Either you don't remove it, or you kill the query. Actually we came up with a solution to this - use filesystem level snapshots (like LV

Re: [HACKERS] add_path optimization

2009-02-03 Thread Kevin Grittner
>>> Robert Haas wrote: > FYI, I retested my queries on REL8_3_STABLE and the results were not > all that different from CVS HEAD. So the problem is apparently > specific to something your query is doing that mine isn't., rather > than a general slowdown in planning (or else one of us goofed up t

Re: [HACKERS] pgevent warnings on mingw

2009-02-03 Thread Magnus Hagander
Hiroshi Saito wrote: > Ahh, sorry..like the spam > again! > > I thought over that the existing msvc the did not have uneasines. > so, I wish to make it this as correspondence with worried Magnus-san. > It is after sufficient test. If I read this patch right, it does an "export all symbols" wh

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Hannu Krosing
On Tue, 2009-02-03 at 13:50 +, Gregory Stark wrote: > Hannu Krosing writes: > > > Actually we came up with a solution to this - use filesystem level > > snapshots (like LVM2+XFS or ZFS), and redirect backends with > > long-running queries to use fs snapshot mounted to a different > > mountpoi

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Simon Riggs
On Tue, 2009-02-03 at 09:14 -0500, Robert Haas wrote: > I think _the_ solution is to notice when you're about to vacuum a page > that is still visible to a running backend on the standby, and save > that page off to a separate cache of old page versions (perhaps using > the relation fork mechanis

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Hannu Krosing
On Tue, 2009-02-03 at 09:14 -0500, Robert Haas wrote: > >> I don't see any way around the fact that when a tuple is removed, it's > >> gone and can't be accessed by queries. Either you don't remove it, or > >> you kill the query. > > > > Actually we came up with a solution to this - use filesystem

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Hannu Krosing
On Tue, 2009-02-03 at 08:40 -0500, Andrew Dunstan wrote: > > Hannu Krosing wrote: > > Actually we came up with a solution to this - use filesystem level > > snapshots (like LVM2+XFS or ZFS), and redirect backends with > > long-running queries to use fs snapshot mounted to a different > > mountpoin

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Simon Riggs
On Tue, 2009-02-03 at 08:40 -0500, Andrew Dunstan wrote: > > Hannu Krosing wrote: > > Actually we came up with a solution to this - use filesystem level > > snapshots (like LVM2+XFS or ZFS), and redirect backends with > > long-running queries to use fs snapshot mounted to a different > > mountpoi

Re: [HACKERS] add_path optimization

2009-02-03 Thread Kevin Grittner
>>> Tom Lane wrote: > In fact, the only reason to care whether there is any data in the DB > *at all* is that you need some realistic content in pg_statistic. > So it should be possible to set up a planner test DB with very little > data bulk, which would surely make testing a lot less painful.

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Robert Haas
>> I don't see any way around the fact that when a tuple is removed, it's >> gone and can't be accessed by queries. Either you don't remove it, or >> you kill the query. > > Actually we came up with a solution to this - use filesystem level > snapshots (like LVM2+XFS or ZFS), and redirect backends

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Gregory Stark
Hannu Krosing writes: > Actually we came up with a solution to this - use filesystem level > snapshots (like LVM2+XFS or ZFS), and redirect backends with > long-running queries to use fs snapshot mounted to a different > mountpoint. Uhm, how do you determine which snapshot to direct the backend

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Andrew Dunstan
Hannu Krosing wrote: Actually we came up with a solution to this - use filesystem level snapshots (like LVM2+XFS or ZFS), and redirect backends with long-running queries to use fs snapshot mounted to a different mountpoint. I don't think Simon has yet put full support for it in code, but it is

Re: [HACKERS] Hot Standby (v9d)

2009-02-03 Thread Hannu Krosing
On Wed, 2009-01-28 at 22:19 +0200, Heikki Linnakangas wrote: > Tom Lane wrote: ... > > Well, those unexpectedly cancelled queries could have represented > > critical functionality too. I think this argument calls the entire > > approach into question. If there is no safe setting for the parameter

Re: [HACKERS] reloptions with a "namespace"

2009-02-03 Thread Euler Taveira de Oliveira
Alvaro Herrera escreveu: >> IIRC, my last patch includes a partial validation code for RESET cases. For >> example, the last SQL will not be atomic (invalid reloption silently >> ignored). >> So, why not apply the namespace validation code to RESET case too? Patch is >> attached too. > > No, we m

Re: [HACKERS] Compiler warning in ecpglib/execute.c

2009-02-03 Thread Michael Meskes
On Mon, Feb 02, 2009 at 02:56:18PM -0500, Tom Lane wrote: > CVS HEAD is producing > ... Thanks for the note, I missed this copy&paste error of mine. Fixed in HEAD. This should alos make the buildfarm green again. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|C

Re: [HACKERS] why declare arg as a array in FunctionCallInfoData structure

2009-02-03 Thread Tao Ma
Thank you guys... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers