Re: [HACKERS] parallel pg_restore

2008-09-22 Thread Joshua D. Drake
Andrew Dunstan wrote: There are in fact very few letters available, as we've been fairly profligate in our use of option letters in the pg_dump suite. j and m happen to be two of those that are available. --max-workers Max makes sense because the number of workers won't be consistent, a wo

Re: [HACKERS] FSM, now without WAL-logging

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 20:43 +0300, Heikki Linnakangas wrote: > Attached is a revamped version of the FSM rewrite. WAL-logging is now > gone. Any inconsistencies between levels of the FSM is fixed during > vacuum, and by searchers when they run into a dead end because of a > discrepancy. Corrup

Re: [HACKERS] pg_type.h regression?

2008-09-22 Thread Tom Lane
Greg Sabino Mullane <[EMAIL PROTECTED]> writes: > Looks like the box-array semicolon got changed to a comma at some point - > attached patch changes it back (\054 to \073) [ scratches head... ] I seem to have done that in rev 1.198, but I don't recall why. It's late here though ...

Re: [HACKERS] Proposed patch: make SQL interval-literal syntax work per spec

2008-09-22 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom, which Interval TODO items did you complete with this patch? > http://wiki.postgresql.org/wiki/Todo#Dates_and_Times I think we've at least mostly fixed * Support ISO INTERVAL syntax if units cannot be determined from the string, and are suppl

Re: [HACKERS] [patch] fix dblink security hole

2008-09-22 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > Tommy Gildseth wrote: >> I'm not quite sure I fully understand the consequence of this change. >> Does it basically mean that it's not possible to use .pgpass with dblink >> for authentication? > It only applies to 8.4 (which is not yet released) and beyo

[HACKERS] pg_type.h regression?

2008-09-22 Thread Greg Sabino Mullane
Looks like the box-array semicolon got changed to a comma at some point - attached patch changes it back (\054 to \073) -- Greg Sabino Mullane [EMAIL PROTECTED] PGP Key: 0x14964AC8 20080927 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 Index: src/include/catalog/pg_ty

Re: [HACKERS] Proposed patch: make SQL interval-literal syntax work per spec

2008-09-22 Thread Bruce Momjian
Tom Lane wrote: > >> There is still some unfinished business if anyone wants to make it > >> really exactly 100% spec compliant ... > > > I agree. > > I committed the patch as presented, and I think I might go take a quick Tom, which Interval TODO items did you complete with this patch?

Re: [HACKERS] [patch] fix dblink security hole

2008-09-22 Thread Joe Conway
Tommy Gildseth wrote: Tom Lane wrote: Okay. I just committed the patch without that change, but I'll go back and add it. I'm not quite sure I fully understand the consequence of this change. Does it basically mean that it's not possible to use .pgpass with dblink for authentication? It on

Re: [PATCHES] [HACKERS] Infrastructure changes for recovery

2008-09-22 Thread Simon Riggs
On Thu, 2008-09-18 at 10:09 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Thu, 2008-09-18 at 09:06 -0400, Tom Lane wrote: > >> Do we really need a checkpoint there at all? > > > "Timelines only change at shutdown checkpoints". > > Hmm. I *think* that that is just a deb

Re: [HACKERS] get_relation_stats_hook()

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 18:41 +0100, Gregory Stark wrote: > The easiest way Did you have further review comments? If so, I'll wait for those before making further mods. Thanks for ones so far. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent vi

Re: [HACKERS] [patch] fix dblink security hole

2008-09-22 Thread Tommy Gildseth
Tom Lane wrote: Joe Conway <[EMAIL PROTECTED]> writes: Tom Lane wrote: No, the test to see if the server actually *asked* for the password is the important part at that end. Oh, I see that now. So yes, as far as I can tell, password_from_string is not used for anything anymore and should be

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Ron Mayer
Gregory Stark wrote: Ron Mayer <[EMAIL PROTECTED]> writes: I'd rather a parameter that expressed things more in terms of measurable quantities [...] ...What we're dealing with now is an entirely orthogonal property of your system: how many concurrent requests can the system handle. Really?

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Gregory Stark
Ron Mayer <[EMAIL PROTECTED]> writes: > For example, on our sites hosted with Amazon's compute cloud (a great > place to host web sites), I know nothing about spindles, but know > about Amazon Elastic Block Store[2]'s and Instance Store's[1]. I > have some specs and are able to run benchmarks o

[HACKERS] Interval literal rounding bug(?) and patch.

2008-09-22 Thread Ron Mayer
I think it's a bug that these 3 different ways of writing 0.7 seconds produce different results from each other on HEAD. head=# select interval '0:0:0.7', interval '@ 0.70 secs', interval '0.7 seconds'; interval |interval |interval -+-+

Re: [HACKERS] parallel pg_restore

2008-09-22 Thread Andrew Dunstan
Simon Riggs wrote: I will not argue vehemently here but I will say that "jobs" doesn't seem correct. The term "workers" seems more appropriate. Agreed, but most utilities have "j" free but not w, p, t or other letters that might be synonyms. j is at least used for exactly this purpose i

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Greg Smith
On Mon, 22 Sep 2008, Gregory Stark wrote: Hm, I'm disappointed with the 48-drive array here. I wonder why it maxed out at only 10x the bandwidth of one drive. I would expect more like 24x or more. The ZFS RAID-Z implementation doesn't really scale that linearly. It's rather hard to get the f

Re: [HACKERS] FSM patch - performance test

2008-09-22 Thread Heikki Linnakangas
Tom Lane wrote: What this means is that if we start with "next" pointing at a page without enough space (quite likely considering that we now index all pages not only those with free space), then it is highly possible that the search will end on a page *before* where next was. The most trivial c

Re: [HACKERS] pg_regress inputdir

2008-09-22 Thread Peter Eisentraut
Peter Eisentraut wrote: Tom Lane wrote: But I think Alvaro is worried about something at a higher level: the regression test process as a whole has some directory layout assumptions built into it, particularly in regards to where to find .so's. The only information about the location of the .s

Re: [HACKERS] get_relation_stats_hook()

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 18:41 +0100, Gregory Stark wrote: > The easiest way to fix this seems like also the best way, instead of storing a > boolean store the pointer to the release function. OK, I like that better anyhow. Hadn't thought about turning plugin off, but I can see the requirement now

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Ron Mayer
Gregory Stark wrote: Simon Riggs <[EMAIL PROTECTED]> writes: I'm not in favour of introducing the concept of spindles In principle I quite strongly disagree with this Number of blocks to prefetch is an internal implementation detail that the DBA has absolutely no way to know what the c

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 13:06 -0400, Greg Smith wrote: > > prefetch_... is a much better name since its an existing industry term. > > I'm not in favour of introducing the concept of spindles, since I can > > almost hear the questions about ramdisks and memory-based storage. > > It's possible to m

Re: [HACKERS] parallel pg_restore

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 09:30 -0700, Joshua Drake wrote: > On Mon, 22 Sep 2008 17:24:28 +0100 > Simon Riggs <[EMAIL PROTECTED]> wrote: > > > > More importantly, I'm not convinced it's a good idea. It seems more > > > like a footgun that will potentially try to launch thousands of > > > simultaneous

Re: [HACKERS] [PATCH] allow has_table_privilege(..., 'usage') on sequences

2008-09-22 Thread Jaime Casanova
On Sun, Sep 7, 2008 at 10:55 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > Abhijit Menon-Sen <[EMAIL PROTECTED]> writes: >> (I can't help but think that the USAGE privilege is a bit unfortunate. >> If granting SELECT rights allowed currval(), INSERT allowed nextval(), >> and UPDATE allowed nextval() an

Re: [HACKERS] get_relation_stats_hook()

2008-09-22 Thread Gregory Stark
Hm, I assume we want to be able to turn on and off plugins in a running session? I think the "free_using_plugin" flag: ! if (get_relation_stats_hook) ! vardata->statsTuple = (*get_relation_stats_hook) !

Re: [HACKERS] Where to Host Project

2008-09-22 Thread David E. Wheeler
On Sep 22, 2008, at 10:08, Stefan Kaltenbrunner wrote: The machine is ready to go and as far as I know even has a jail. Stefan would know more. OK, cool. Stefan; what's your take on where we're at? yeah there is a box and a jail I set up a while ago but for various reasons the actual migr

Re: [HACKERS] Where to Host Project

2008-09-22 Thread Stefan Kaltenbrunner
Dave Page wrote: On Sat, Sep 20, 2008 at 8:37 AM, Joshua D. Drake <[EMAIL PROTECTED]> wrote: Dave Page wrote: Well that's not strictly true - I persuaded one of the GForge developers to work on the upgrade. As far as I'm aware, we're still waiting for the hardware/OS platform to be sorted out

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Greg Smith
On Mon, 22 Sep 2008, Simon Riggs wrote: I'd prefer to set this as a tablespace level storage parameter. That seems reasonable, but I'm not working at that level yet. There's still a larger open questions about how the buffer manager interaction will work here, and I'd like to have a better

Re: [HACKERS] parallel pg_restore

2008-09-22 Thread Dimitri Fontaine
Le lundi 22 septembre 2008, Joshua Drake a écrit : > I will not argue vehemently here but I will say that "jobs" doesn't > seem correct. The term "workers" seems more appropriate. Mmmm, it sounds like it depends on the implementation (and how all workers will share the same serializable transacti

Re: [HACKERS] parallel pg_restore

2008-09-22 Thread Joshua Drake
On Mon, 22 Sep 2008 17:24:28 +0100 Simon Riggs <[EMAIL PROTECTED]> wrote: > > More importantly, I'm not convinced it's a good idea. It seems more > > like a footgun that will potentially try to launch thousands of > > simultaneous restore connections. I should have thought that > > optimal perform

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 16:46 +0100, Gregory Stark wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > > On Mon, 2008-09-22 at 04:57 -0400, Greg Smith wrote: > > > >> -As Greg Stark suggested, the larger the spindle count the larger the > >> speedup, and the larger the prefetch size that might ma

Re: [HACKERS] parallel pg_restore

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 11:38 -0400, Andrew Dunstan wrote: > > Simon Riggs wrote: > > On Mon, 2008-09-22 at 09:53 +0200, Dimitri Fontaine wrote: > > > > > >>> My intention is to have single-thread restore remain the default, at > >>> least for this go round, and have the user be able to choose >

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Gregory Stark
Simon Riggs <[EMAIL PROTECTED]> writes: > On Mon, 2008-09-22 at 04:57 -0400, Greg Smith wrote: > >> -As Greg Stark suggested, the larger the spindle count the larger the >> speedup, and the larger the prefetch size that might make sense. His >> suggestion to model the user GUC as "effective_sp

Re: [HACKERS] FSM patch - performance test

2008-09-22 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: Zdenek Kotala napsal(a): Heikki Linnakangas napsal(a): Zdenek Kotala wrote: My conclusion is that new implementation is about 8% slower in OLTP workload. Can you do some analysis of why that is? I tested it several times and last test was

Re: [HACKERS] parallel pg_restore

2008-09-22 Thread Andrew Dunstan
Simon Riggs wrote: On Mon, 2008-09-22 at 09:53 +0200, Dimitri Fontaine wrote: My intention is to have single-thread restore remain the default, at least for this go round, and have the user be able to choose --multi-thread=nn to specify the number of concurrent connections to use.

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Heikki Linnakangas
Zdenek Kotala wrote: Heikki Linnakangas napsal(a): For anyone counting, Firebird added support for ICU more than three years ago. ICU is orthogonal to this patch. This patch didn't provide ICU support, and we could start using ICU without the catalog changes. This patch should allow to use

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-22 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > Hmm I wonder if this doesn't have the same problems you're describing with > the toaster. If someone has a cursor WITH HOLD against the table they don't > get a session level lock against the tables which fed the cursor do > they? Hmm, interesting po

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): For anyone counting, Firebird added support for ICU more than three years ago. ICU is orthogonal to this patch. This patch didn't provide ICU support, and we could start using ICU without the catalog changes. This patch should allow to use both system catalog a

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-22 Thread Gregory Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Simon Riggs <[EMAIL PROTECTED]> writes: >> I think the issue is identifying the problem. Reading the title of the >> post, I think Tom says "no" to *deleting* the toast table. He also says >> "no" to cleaning the table as part of DROP COLUMN. That still leav

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Hans-Jürgen Schönig
On Sep 22, 2008, at 12:02 PM, Simon Riggs wrote: On Mon, 2008-09-22 at 04:57 -0400, Greg Smith wrote: -As Greg Stark suggested, the larger the spindle count the larger the speedup, and the larger the prefetch size that might make sense. His suggestion to model the user GUC as "effective_spi

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 07:53 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I think the issue is identifying the problem. Reading the title of the > > post, I think Tom says "no" to *deleting* the toast table. He also says > > "no" to cleaning the table as part of DROP COLUMN.

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Heikki Linnakangas
Tom Lane wrote: What might be sensible to ask is whether it is ever actually reasonable for LC_COLLATE and LC_CTYPE to have different settings. If we were willing to enforce that they be the same, we could reduce this to just the standard syntax COLLATE=something and be done with it. Not being

Re: [HACKERS] [patch] fix dblink security hole

2008-09-22 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> No, the test to see if the server actually *asked* for the password is >> the important part at that end. > Oh, I see that now. So yes, as far as I can tell, password_from_string > is not used for anything anymore and should be removed.

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > The first thing that we see is that the COLLATE keyword means different > things, so it's probably best to change that into: > CREATE DATABASE WITH LC_COLLATE= LC_CTYPE= > in the stripped-down version. Then we need a way to map the > stripped-do

Re: [HACKERS] [patch] fix dblink security hole

2008-09-22 Thread Joe Conway
Tom Lane wrote: Joe Conway <[EMAIL PROTECTED]> writes: Tom Lane wrote: What do you think about getting rid of the password_from_string state variable? It was always a bit of a kluge, and we don't seem to need it anymore with this approach. It is still used in PQconnectionUsedPassword(). That

Re: [HACKERS] pg_dump feature

2008-09-22 Thread Tom Lane
Naz <[EMAIL PROTECTED]> writes: > ... It would be far easier if there were a mechanism in pg_dump > that allowed you do dump the two parts of the schema separately, Feel free to fix up and resubmit the incomplete patch for that that was rejected during the last commitfest.

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-22 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > I think the issue is identifying the problem. Reading the title of the > post, I think Tom says "no" to *deleting* the toast table. He also says > "no" to cleaning the table as part of DROP COLUMN. That still leaves you > an opening for an out-of-line comma

Re: [HACKERS] [patch] fix dblink security hole

2008-09-22 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> What do you think about getting rid of the password_from_string state >> variable? It was always a bit of a kluge, and we don't seem to need >> it anymore with this approach. > It is still used in PQconnectionUsedPassword(). That is stil

[HACKERS] HOWTO: FK: BIGINT[] -> BIGINT(Theoreticaly AnyElem[] -> AnyElem)

2008-09-22 Thread Oleg Serov
(Sorry for my bad english comments) Demo sql: BEGIN; -- Begins the magic CREATE TABLE pk ( -- pk table id BIGINT, CONSTRAINT pk2_id PRIMARY KEY (id) ); CREATE TABLE fk ( -- fk table fk_ids BIGINT[] ); CREATE FUNCTION bia2bi(bigint[]) RETURNS bigint -- temp type cast AS $$ SELECT

Re: [HACKERS] parallel pg_restore

2008-09-22 Thread Simon Riggs
On Sun, 2008-09-21 at 18:15 -0400, Tom Lane wrote: > Andrew Dunstan <[EMAIL PROTECTED]> writes: > > I am working on getting parallel pg_restore working. I'm currently > > getting all the scaffolding working, and hope to have a naive prototype > > posted within about a week. > > > The major ques

Re: [HACKERS] parallel pg_restore

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 09:53 +0200, Dimitri Fontaine wrote: > > My intention is to have single-thread restore remain the default, at > > least for this go round, and have the user be able to choose > > --multi-thread=nn to specify the number of concurrent connections to use. > > What about the ma

Re: [HACKERS] Initial prefetch performance testing

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 04:57 -0400, Greg Smith wrote: > -As Greg Stark suggested, the larger the spindle count the larger the > speedup, and the larger the prefetch size that might make sense. His > suggestion to model the user GUC as "effective_spindle_count" looks like a > good one. The seq

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-22 Thread Simon Riggs
On Mon, 2008-09-22 at 10:59 +0200, Hans-Jürgen Schönig wrote: > On Sep 22, 2008, at 9:46 AM, Simon Riggs wrote: > > I thought Hans meant cleanup, not drop? > we definitely have to do something about this problem. I think the issue is identifying the problem. Reading the title of the post, I th

[HACKERS] Initial prefetch performance testing

2008-09-22 Thread Greg Smith
The complicated patch I've been working with for a while now is labeled "sequential scan posix fadvise" in the CommitFest queue. There are a lot of parts to that, going back to last December, and I've added the many most relevant links to the September CommitFest page. The first message there

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-22 Thread Hans-Jürgen Schönig
On Sep 22, 2008, at 9:46 AM, Simon Riggs wrote: On Sun, 2008-09-21 at 12:05 -0400, Tom Lane wrote: ... and it goes on to point out how to force immediate space reclamation if you need that. These statements apply independently of whether any particular value is toasted or not. The reas

Re: [HACKERS] parallel pg_restore

2008-09-22 Thread Dimitri Fontaine
Le lundi 22 septembre 2008, Andrew Dunstan a écrit : > > You'd really want the latter anyway for some cases, ie, when you don't > > want the restore trying to hog the machine. Maybe the right form for > > the extra option is just a limit on how many connections to use. Set it > > to one to force

Re: [HACKERS] Toasted table not deleted when no out of line columns left

2008-09-22 Thread Simon Riggs
On Sun, 2008-09-21 at 12:05 -0400, Tom Lane wrote: > ... and it goes on to point out how to force immediate space reclamation > if you need that. These statements apply independently of whether any > particular value is toasted or not. > > The reason for this choice is that reclaiming the space

Re: [HACKERS] macport for libpqxx

2008-09-22 Thread Dave Page
On Sat, Sep 20, 2008 at 5:25 PM, Darren Weber <[EMAIL PROTECTED]> wrote: > Hi Dave, > > Thanks for getting back to me. Please find attached a draft Portfile > for libpqxx-2.6.9 (the stable version). It's easy to read the > Portfile to see what is going on. I think it should work fine, but I > wo

Re: [HACKERS] Proposal: move column defaults into pg_attribute along with attacl

2008-09-22 Thread Asko Oja
On Mon, Sep 22, 2008 at 5:41 AM, Stephen Frost <[EMAIL PROTECTED]> wrote: > * Tom Lane ([EMAIL PROTECTED]) wrote: > > Stephen Frost <[EMAIL PROTECTED]> writes: > > > If we were to accept the pg_attrdef approach, why aren't we > > > doing a pg_attracl table instead of adding a column to pg_attribut

Re: [HACKERS] WIP patch: Collation support

2008-09-22 Thread Heikki Linnakangas
Martijn van Oosterhout wrote: On Fri, Sep 19, 2008 at 10:13:43AM +0300, Heikki Linnakangas wrote: It's not like the patch is going to disappear from planet Earth if it doesn't get committed for 8.4. It's still valuable and available when the new catalogs are needed. I just prefer it as it was

Re: [HACKERS] Where to Host Project

2008-09-22 Thread Dave Page
On Sat, Sep 20, 2008 at 8:44 PM, Dimitri Fontaine <[EMAIL PROTECTED]> wrote: > I suppose the plan is to upgrade to a newer GForge. Is it still time to > propose something completely different? I have real good feedbacks about > VHFFS, a perl based clean-room re-implementation of it, if you want to