Re: [HACKERS] Psql command-line completion bug

2008-01-11 Thread Tom Lane
[ catching up on email... ] Gregory Stark <[EMAIL PROTECTED]> writes: > If you hit tab on a table name containing a \ you get spammed with a > series of WARNINGS and HINTS about nonstandard use of \\ in a string > literal: Yeah. This is actually a generic problem for *all* applications using PQe

[HACKERS] A note about SIGTERM illusion and reality

2008-01-11 Thread Tom Lane
If you've paid any attention to our signal handling and shutdown procedures, you know that there is a whole lot of design and logic based on the assumption that during a forced system shutdown, we will see SIGTERM delivered to all PG processes, with a little bit of grace time before we get SIGKILL'

Re: [HACKERS] DECLARE CURSOR code question

2008-01-11 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > In CVS HEAD, why does exec_simple_query() pass an empty cursorOptions to > pg_plan_queries() at postgres.c:903? If we're planning DECLARE CURSOR, > ISTM we ought to be passing down the DECLARE CURSOR's cursorOptions. This code doesn't know (or care) whethe

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-11 Thread August Zajonc
Simon Riggs wrote: Happy New Year, everybody. This proposal follows on from previous thinking about partitioning, where I've taken up Andrew Sullivan's suggestion to re-examine the current partitioning concept of using tables as partitions. So I've come up with an alternative concept to allow us

Re: [HACKERS] Declarative partitioning grammar

2008-01-11 Thread Jeff Cohen
On Jan 11, 2008, at 4:03 PM, Mike wrote: Pardon my ignorance as I've never actually used partitioning before but plan to in the near future, but couldn't the grammar resemble a common WHERE clause more closely? Hi Mike, Thanks for your suggestions. The current syntax we chose is similar

[HACKERS] DECLARE CURSOR code question

2008-01-11 Thread Neil Conway
In CVS HEAD, why does exec_simple_query() pass an empty cursorOptions to pg_plan_queries() at postgres.c:903? If we're planning DECLARE CURSOR, ISTM we ought to be passing down the DECLARE CURSOR's cursorOptions. -Neil ---(end of broadcast)--- TIP

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-11 Thread Tom Lane
Chris Browne <[EMAIL PROTECTED]> writes: > Note that we required that the "provider transaction" have the > attributes IsXactIsoLevelSerializable and XactReadOnly both being > true, so we have the mandates that the resultant backend process: > a) Is in read only mode, and > b) Is in serializable m

Re: [HACKERS] Declarative partitioning grammar

2008-01-11 Thread Gavin Sherry
On Fri, Jan 11, 2008 at 07:46:36PM -0500, Merlin Moncure wrote: > On Jan 11, 2008 6:19 PM, Gavin Sherry <[EMAIL PROTECTED]> wrote: > > Hi all, > > > > Many of you will have read the dynamic partitioning thread here: > > > > http://archives.postgresql.org/pgsql-hackers/2008-01/msg00028.php > > > > I

Re: [HACKERS] Declarative partitioning grammar

2008-01-11 Thread Merlin Moncure
On Jan 11, 2008 6:19 PM, Gavin Sherry <[EMAIL PROTECTED]> wrote: > Hi all, > > Many of you will have read the dynamic partitioning thread here: > > http://archives.postgresql.org/pgsql-hackers/2008-01/msg00028.php > > I've proposed an alternative approach, which we've called declarative > partition

Re: [HACKERS] Declarative partitioning grammar

2008-01-11 Thread Jeff Cohen
On Jan 11, 2008, at 4:16 PM, Warren Turkal wrote: On Jan 11, 2008 3:42 PM, Ron Mayer <[EMAIL PROTECTED]> wrote: What would be the drawbacks of CREATE TABLE tablename(...) PARTITION BY function_taking_row_returning_partition_name instead of the explicit types? Would that still allow the

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-11 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > For (2) there's a very simple way of transferring the data between > sessions: > a) we connect on session 1 as a serializable transaction > b) we ask session 1 for its snapshot > c) we then connect on session 2 as a serializable transaction > d) we then ex

Re: [HACKERS] Declarative partitioning grammar

2008-01-11 Thread Warren Turkal
On Jan 11, 2008 3:42 PM, Ron Mayer <[EMAIL PROTECTED]> wrote: > What would be the drawbacks of > CREATE TABLE tablename(...) > PARTITION BY function_taking_row_returning_partition_name > instead of the explicit types? Would that still allow the optimizer to work as well as it could? It seems t

Re: [HACKERS] Declarative partitioning grammar

2008-01-11 Thread Mike
Pardon my ignorance as I've never actually used partitioning before but plan to in the near future, but couldn't the grammar resemble a common WHERE clause more closely?  > ... PARTITION BY RANGE(order_date) > ( > START (date '2005-12-01') end (date '2007-12-01') >

Re: [HACKERS] Declarative partitioning grammar

2008-01-11 Thread Ron Mayer
Gavin Sherry wrote: > CREATE TABLE is modified to accept a PARTITION BY clause. This clause > contains one or more partition declarations. The syntax is as follows: > PARTITION BY {partition_type} (column_name[, column_name...]) > [PARTITIONS number] > ( >partition_declaration[, partition

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-11 Thread Chris Browne
[EMAIL PROTECTED] (Simon Riggs) writes: > On Fri, 2008-01-11 at 20:39 +, Simon Riggs wrote: >> On Fri, 2008-01-11 at 15:05 -0500, Tom Lane wrote: >> Simon Riggs <[EMAIL PROTECTED]> writes: >> > > If we had a function >> > > replace_serializable_snapshot(master_xid, txid_snapshot) >> > > this

[HACKERS] Declarative partitioning grammar

2008-01-11 Thread Gavin Sherry
Hi all, Many of you will have read the dynamic partitioning thread here: http://archives.postgresql.org/pgsql-hackers/2008-01/msg00028.php I've proposed an alternative approach, which we've called declarative partitioning which is grammar based. This grammar was developed by Jeff Cohen at Greenp

Re: [HACKERS] Some ideas about Vacuum

2008-01-11 Thread Gokulakannan Somasundaram
Sorry Greg , I missed to read this part before. On Jan 9, 2008 8:40 PM, Gregory Stark <[EMAIL PROTECTED]> wrote: > > "Markus Schiltknecht" <[EMAIL PROTECTED]> writes: > > > Hi, > > > > Gokulakannan Somasundaram wrote: > >> If we can ask the Vacuum process to scan the WAL log, it can get all > the

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-11 Thread Simon Riggs
On Fri, 2008-01-11 at 20:39 +, Simon Riggs wrote: > On Fri, 2008-01-11 at 15:05 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > > If we had a function > > > replace_serializable_snapshot(master_xid, txid_snapshot) > > > this would allow us to use the txid_snapshot values

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-11 Thread Simon Riggs
On Fri, 2008-01-11 at 15:05 -0500, Tom Lane wrote: > the whole thing gives me the willies. Me too :-) > > What I'm thinking about is how we might use this to have multiple > > sessions working simultaneously on tasks like unloading data, > > Then what you want is a function that says "clone the

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-11 Thread Gokulakannan Somasundaram
> > The user isn't going to have any legal way to transfer the data between > backends anyway, since no transaction can see results of an uncommitted > other transaction. There *has* to be some backdoor channel involved > there, and you might as well make it carry the data without the user > touch

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-11 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > If we had a function > replace_serializable_snapshot(master_xid, txid_snapshot) > this would allow us to use the txid_snapshot values to replace our > transaction's serializable snapshot. ... whereupon we'd get wrong answers. Certainly you could no

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-11 Thread Simon Riggs
On Fri, 2008-01-11 at 20:03 +0100, Gavin Sherry wrote: > Okay, it's good that you want the planner to look at those. Did you > consider the point I made about the sheer amount of data the planner > would have to consider for large cases? Sorry, thought I had somewhere in all those emails... If y

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-11 Thread Gavin Sherry
On Fri, Jan 11, 2008 at 11:49:50AM +, Simon Riggs wrote: > On Fri, 2008-01-11 at 10:25 +0100, Gavin Sherry wrote: > > > > > > Of course. It's an identical situation for both. Regrettably, none of > > > your comments about dynamic partitioning and planning were accurate as a > > > result. > >

[HACKERS] proposed libpq type system changes 0.9

2008-01-11 Thread Merlin Moncure
We have put up a new version of our extensions to the libpq api, here: http://www.esilo.com/projects/postgresql/libpq/typesys-beta-0.9.tar.gz Four functions are getting dropped and will be replaced with a generic version taking a va_list. Also, in some functions result format is being moved to t

Re: [HACKERS] [BUGS] BUG #3852: Could not create complex aggregate

2008-01-11 Thread Tom Lane
Joe Conway <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I think we could make enforce_generic_type_consistency() clearer by >> adding an additional argument "bool allow_poly" which specifies >> whether polymorphic "actual" argument and result types are allowed. > This sounds like a reasonable

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-11 Thread Simon Riggs
On Fri, 2008-01-11 at 17:31 +0100, Zeugswetter Andreas ADI SD wrote: > > I've kept a list of requests for improvement that I can share with > you; > > I've always been loathe to publish a list of bad points. The "list of bad points" doesn't refer to shortcomings in my proposal, which I would never

Re: [HACKERS] scan.l: check_escape_warning()

2008-01-11 Thread Andrew Dunstan
Michael Meskes wrote: Hi, could anyone please enlighten me whether this function is still needed? AFAICT check_escape_warning() only has significant action if warn_on_first_escape is true. This variable is set to true only on label xqstart, but to false on xestart. However, check_escape_warni

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-11 Thread Zeugswetter Andreas ADI SD
> I've kept a list of requests for improvement that I can share with you; > I've always been loathe to publish a list of bad points. I think it would help understand the proposal if you also present the shortcomings. When you presented the positive and negative points, the negative list did look

[HACKERS] Transaction Snapshot Cloning

2008-01-11 Thread Simon Riggs
Now that we have a txid_snapshot datatype, it seems easy to imagine that we might use that to pass snapshot information around between sessions. If we had a function replace_serializable_snapshot(master_xid, txid_snapshot) this would allow us to use the txid_snapshot values to replace ou

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-11 Thread Simon Riggs
On Mon, 2008-01-07 at 12:53 -0800, Ron Mayer wrote: > Is my understanding right that these Segment Visibility Maps could > help this case as well? Yes, I think so. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)-

Re: [HACKERS] scan.l: check_escape_warning()

2008-01-11 Thread Tom Lane
Michael Meskes <[EMAIL PROTECTED]> writes: > could anyone please enlighten me whether this function is still needed? > AFAICT check_escape_warning() only has significant action if > warn_on_first_escape is true. This variable is set to true only on label > xqstart, but to false on xestart. However,

[HACKERS] could not open relation: Invalid argument

2008-01-11 Thread Roberts, Jon
Version: PostgreSQL 8.2.5 on i686-pc-mingw32 I recently started getting this error message randomly, "could not open relation 42904/42906/42985: Invalid argument". I also got it for a couple of other files. All three files are related to tables that have just a single row each. I googled the er

[HACKERS] scan.l: check_escape_warning()

2008-01-11 Thread Michael Meskes
Hi, could anyone please enlighten me whether this function is still needed? AFAICT check_escape_warning() only has significant action if warn_on_first_escape is true. This variable is set to true only on label xqstart, but to false on xestart. However, check_escape_warning() and check_string_esca

Re: [HACKERS] Storage Model for Partitioning

2008-01-11 Thread Richard Huxton
Simon Riggs wrote: On Fri, 2008-01-11 at 11:34 +, Richard Huxton wrote: Is the following basically the same as option #3 (multiple RelFileNodes)? 1. Make an on-disk "chunk" much smaller (e.g. 64MB). Each chunk is a contigous range of blocks. 2. Make a table-partition (implied or explicit

Re: [HACKERS] Storage Model for Partitioning

2008-01-11 Thread Simon Riggs
On Fri, 2008-01-11 at 11:34 +, Richard Huxton wrote: > Is the following basically the same as option #3 (multiple RelFileNodes)? > > 1. Make an on-disk "chunk" much smaller (e.g. 64MB). Each chunk is a > contigous range of blocks. > 2. Make a table-partition (implied or explicit constraints)

Re: [HACKERS] Storage Model for Partitioning

2008-01-11 Thread Csaba Nagy
> Which is roughly what Simon's original "Dynamic Partitioning" would be > if it became visible at the planner level (unless I've misunderstood). I > was picturing it in my head as a two-dimensional bitmap with > value-ranges along one axis and block-ranges along the other. Of course, > unlike

Re: [HACKERS] Storage Model for Partitioning

2008-01-11 Thread Richard Huxton
Csaba Nagy wrote: On Fri, 2008-01-11 at 11:34 +, Richard Huxton wrote: 1. Make an on-disk "chunk" much smaller (e.g. 64MB). Each chunk is a contigous range of blocks. 2. Make a table-partition (implied or explicit constraints) map to multiple "chunks". That would reduce fragmentation (you'd

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-11 Thread Simon Riggs
On Fri, 2008-01-11 at 10:25 +0100, Gavin Sherry wrote: > > > > Of course. It's an identical situation for both. Regrettably, none of > > your comments about dynamic partitioning and planning were accurate as a > > result. > > That's not true. We will still have planning drive the partition > sel

Re: [HACKERS] Storage Model for Partitioning

2008-01-11 Thread Csaba Nagy
On Fri, 2008-01-11 at 11:34 +, Richard Huxton wrote: > 1. Make an on-disk "chunk" much smaller (e.g. 64MB). Each chunk is a > contigous range of blocks. > 2. Make a table-partition (implied or explicit constraints) map to > multiple "chunks". > That would reduce fragmentation (you'd have on a

Re: [HACKERS] Storage Model for Partitioning

2008-01-11 Thread Richard Huxton
More dumb user questions... Simon Riggs wrote: 1. Partitions are Contiguous Ranges of Blocks Partitions are a simple subset of a table, i.e. a contiguous range of blocks within the main block range of the table. b) Fixed partitioning - we define partitions as static ranges of blocks, which m

Re: [HACKERS] Storage Model for Partitioning

2008-01-11 Thread Pavel Stehule
> > 3. Partitions are RelFileNodes, but not Tables > > We allow a table to have multiple RelFileNodes, when explicitly declared > that way. > > This means we have to change > - DDL changes to allow TABLE level changes to apply to all RelFileNodes, > while PARTITION level changes to apply to only on

[HACKERS] Storage Model for Partitioning

2008-01-11 Thread Simon Riggs
In my striving towards more effective partitioning for Postgres, I see we have one main decision to make and that all other sub-tasks are driven directly by this one issue. The issue is: At what point we store the data within the existing storage model? We discussed this in 2005 when I started to d

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-11 Thread Simon Riggs
Gavin, I've responded to the technical questions you raise here: On Thu, 2008-01-10 at 02:22 +0100, Gavin Sherry wrote: > > After we note that a segment is read-only we can scan the segment and > > record min/max values for all columns. These are then "implicit > > constraints", which can then b

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-11 Thread Gavin Sherry
On Fri, Jan 11, 2008 at 08:07:18AM +, Simon Riggs wrote: > On Fri, 2008-01-11 at 02:28 +0100, Gavin Sherry wrote: > > On Thu, Jan 10, 2008 at 09:30:10PM +, Simon Riggs wrote: > > > > > We cannot perform partition exclusion using this type of WHERE clause > > > > > at > > > > > planning tim

Re: [HACKERS] Pl/Java broken since Postgresql 8.3-rc1

2008-01-11 Thread Dave Page
On 11/01/2008, Kris Jurka <[EMAIL PROTECTED]> wrote: > Turns out it's not just 83RC1, but all of the security releases, which > will require different pljava packages for the patch versions before/after > the security changes. I've committed a fix to CVS for this, and I guess > I'll try to respin

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-11 Thread Simon Riggs
On Fri, 2008-01-11 at 02:28 +0100, Gavin Sherry wrote: > On Thu, Jan 10, 2008 at 09:30:10PM +, Simon Riggs wrote: > > > > We cannot perform partition exclusion using this type of WHERE clause at > > > > planning time because the CURRENT DATE function is STABLE. > > > > > > We can do the exact