Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Greg Stark
On Wed, Feb 20, 2013 at 9:25 PM, Peter Eisentraut wrote: > More generally, I would consider the invalidation of a materialized view > a DDL command, whereas truncating a table is a DML command. That's not entirely true. From the database's point of view, TRUNCATE is in many ways actually DDL. I

Re: [HACKERS] Re: PostgreSql - access modified rows in prepare transaction command

2013-02-20 Thread Amit Kapila
On Wednesday, February 20, 2013 5:10 PM pierpaolo.cincilla wrote: > Thank you Heikki for your reply. As you suggest, I will explain better > what > I'm trying to accomplish. > > What I'm writing a ditributed two-phase-commit termination protocol > that > work in this manner: > > 1) Each site has

Re: [HACKERS] posix_fadvise missing in the walsender

2013-02-20 Thread Joachim Wieland
On Wed, Feb 20, 2013 at 4:54 PM, Robert Haas wrote: > On Tue, Feb 19, 2013 at 5:48 PM, Simon Riggs wrote: >> I agree with Merlin and Joachim - if we have the call in one place, we >> should have it in both. > > We might want to assess whether we even want to have it one place. > I've seen cases w

Re: [HACKERS] [RFC] indirect toast tuple support

2013-02-20 Thread Greg Stark
On Thu, Feb 21, 2013 at 2:32 AM, Greg Stark wrote: > On Tue, Feb 19, 2013 at 2:00 PM, Andres Freund wrote: >> The only reasonable thing I can see us doing is renaming >> varattrib_1b_e.va_len_1be into va_type and redefine VARSIZE_1B_E into a >> switch that maps types into lengths. But I think I w

Re: [HACKERS] [RFC] indirect toast tuple support

2013-02-20 Thread Greg Stark
On Tue, Feb 19, 2013 at 2:00 PM, Andres Freund wrote: > The only reasonable thing I can see us doing is renaming > varattrib_1b_e.va_len_1be into va_type and redefine VARSIZE_1B_E into a > switch that maps types into lengths. But I think I would put this off, > except placing a comment somewhere,

Re: [HACKERS] Unarchived WALs deleted after crash

2013-02-20 Thread Daniel Farina
On Fri, Feb 15, 2013 at 9:29 AM, Simon Riggs wrote: > On 15 February 2013 17:07, Heikki Linnakangas wrote: > >>> Unfortunately in HEAD, xxx.done file is not created when restoring >>> archived >>> file because of absence of the patch. We need to implement that first. >> >> >> Ah yeah, that thing

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Tatsuo Ishii
> I would like to know what operations you plan to support independently > of the command names. I may have missed much earlier in the discussion > but then I suspect things have evolved since then. > > It sounds like you want to support: > > 1) Selecting from materialized viws > 2) Manually refr

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Kevin Grittner
Peter Eisentraut wrote: > On 2/20/13 2:30 PM, Kevin Grittner wrote: >>> Are there TRUNCATE triggers on materialized views? >> No.  Nor SELECT, INSERT, UPDATE, or DELETE triggers.  You can't >> create a trigger of any type on a materialized view.  I don't >> think that would interfere with event tr

Re: PATCH: Split stats file per database WAS: [HACKERS] autovacuum stress-testing our system

2013-02-20 Thread Alvaro Herrera
Jeff Janes escribió: > On Mon, Feb 18, 2013 at 7:50 AM, Alvaro Herrera > wrote: > > > > So here's v11. I intend to commit this shortly. (I wanted to get it > > out before lunch, but I introduced a silly bug that took me a bit to > > fix.) > > On Windows with Mingw I get this: > > pgstat.c:4389

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Peter Eisentraut
On 2/20/13 2:30 PM, Kevin Grittner wrote: >> Are there TRUNCATE triggers on materialized views? > No. Nor SELECT, INSERT, UPDATE, or DELETE triggers. You can't > create a trigger of any type on a materialized view. I don't think > that would interfere with event triggers, though. More generally

Re: [HACKERS] [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

2013-02-20 Thread Phil Sorber
On Wed, Feb 20, 2013 at 2:16 PM, Heikki Linnakangas wrote: > Where does this leave the PQconninfoParseParams/PQconninfodefaultsMerge > patch? I'm not sure. Somehow I thought it would be necessary for this work, > but it wasn't. I didn't remember that we already have PQconninfoParse() > function, w

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Stephen Frost
* Kevin Grittner (kgri...@ymail.com) wrote: > Peter Eisentraut wrote: > > Is TRUNCATE even the right command here?  For regular tables > > TRUNCATE is a fast DELETE, which logically empties the table. > > For materialized views, there is no deleting, so this command (I > > suppose?) just invalida

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Kevin Grittner
Peter Eisentraut wrote: > On 2/19/13 5:47 PM, Kevin Grittner wrote: >> When I went to do this, I hit a shift/reduce conflict, because >> with TABLE being optional it couldn't tell whether: >> >> TRUNCATE MATERIALIZED VIEW x, y, z; >> >> ... was looking for five relations or three.  That goes away

Re: [HACKERS] [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

2013-02-20 Thread Heikki Linnakangas
On 20.02.2013 11:42, Amit Kapila wrote: The patch for providing connection string for pg_basebackup, pg_receivexlog, pg_dump and pg_dumpall is attached with this mail. Thanks. Now that I look at this patch, I realize that we don't actually need these new functions for pg_basebackup and friends

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Peter Eisentraut
On 2/19/13 5:47 PM, Kevin Grittner wrote: > When I went to do this, I hit a shift/reduce conflict, because with > TABLE being optional it couldn't tell whether: > > TRUNCATE MATERIALIZED VIEW x, y, z; > > ... was looking for five relations or three. That goes away with > MATERIALIZED escalated t

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Josh Berkus
> And explicitly not support > > 1) Automatically rewriting queries to select from matching views > 2) Incrementally refreshing materialized views > 3) Manual DML against data in materialized views (except truncate > which is kind of DDL) > 4) Keeping track of whether the data in the materialized

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Kevin Grittner
Greg Stark wrote: > Kevin Grittner wrote: >> Tom Lane wrote: >>> The way to fix it is to not try to use the sub-production but >>> spell it all out: >>> >>>   TRUNCATE TABLE relation_expr_list ... >>> | TRUNCATE MATERIALIZED VIEW relation_expr_list ... >>> | TRUNCATE relation_expr_l

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Greg Stark
On Wed, Feb 20, 2013 at 4:26 PM, Kevin Grittner wrote: > Tom Lane wrote: > > >> The way to fix it is to not try to use the sub-production but spell it >> all out: >> >> TRUNCATE TABLE relation_expr_list ... >> | TRUNCATE MATERIALIZED VIEW relation_expr_list ... >> | TRUNCATE relatio

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Erik Rijkers
On Wed, February 20, 2013 16:28, Kevin Grittner wrote: > Peter Eisentraut wrote: > >> I suppose one should be able to expect that if one finds a view >> in the information schema, then one should be able to use DROP >> VIEW to remove it.  Which in this case wouldn't work.  So I don't >> think incl

[HACKERS] Contrib module "xml2" status

2013-02-20 Thread Ian Lawrence Barwick
Hi I'm not sure if this is a documentation or hackers issue, but the documentation page for contrib module "xml2" refers to PostgreSQL 8.4 in the future tense: "It is planned that this module will be removed in PostgreSQL 8.4 in favor of the newer standard API" http://www.postgresql.org/docs/

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Peter Eisentraut
On 2/20/13 6:13 AM, Robert Haas wrote: >> It might be useful to have an option for this, but I don't think it >> > should be the default. The default should be that the new database is >> > "ready to go". >> > >> > Then again, when would you ever actually use that option? > You'd use that option i

Re: PATCH: Split stats file per database WAS: [HACKERS] autovacuum stress-testing our system

2013-02-20 Thread Kevin Grittner
Jeff Janes wrote: > Alvaro Herrera wrote: >> >> So here's v11.  I intend to commit this shortly.  (I wanted to get it >> out before lunch, but I introduced a silly bug that took me a bit to >> fix.) > > On Windows with Mingw I get this: > > pgstat.c:4389:8: warning: variable 'found' set but not u

Re: [HACKERS] Altering Views

2013-02-20 Thread Andres Freund
On 2013-02-20 17:25:41 +0100, Misa Simic wrote: > Hi, > > If we want to "add new column" to the view - the only one way (we have > found) is: > > -drop view > -create it again ( with new column ) Since 8.4 you can add new columns to the end of a view definition using CREATE OR REPLACE VIEW. htt

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Kevin Grittner
Tom Lane wrote: > The way to fix it is to not try to use the sub-production but spell it > all out: > >       TRUNCATE TABLE relation_expr_list ... >     | TRUNCATE MATERIALIZED VIEW relation_expr_list ... >     | TRUNCATE relation_expr_list ... > > Now the parser doesn't have to make any shif

[HACKERS] Altering Views

2013-02-20 Thread Misa Simic
Hi, If we want to "add new column" to the view - the only one way (we have found) is: -drop view -create it again ( with new column ) Now, if some other view depends on the view we want change - it will not allow us to drop the view - what is fine, of course - but it does not allow us to change

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Tom Lane
Kevin Grittner writes: > Tom Lane wrote: >> Having said that, I don't think I believe your analysis of why >> this doesn't work. > Well, it wouldn't be the first time you've seen a better way to do > something in flex than I was able to see. Taking just the gram.y > part of the change which imp

Re: [HACKERS] streaming header too small

2013-02-20 Thread Heikki Linnakangas
On 20.02.2013 17:53, Selena Deckelmann wrote: On Wed, Feb 20, 2013 at 6:23 AM, Magnus Haganderwrote: Selena, was this reasonably reproducible for you? Would it be possible to get a network trace of it to show of that's the kind of package coming across, or by hacking up pg_basebackup to print t

Re: [HACKERS] streaming header too small

2013-02-20 Thread Magnus Hagander
On Wed, Feb 20, 2013 at 4:53 PM, Selena Deckelmann wrote: > > > > On Wed, Feb 20, 2013 at 6:23 AM, Magnus Hagander > wrote: >> >> >> On Feb 20, 2013 11:29 AM, "Heikki Linnakangas" >> wrote: >> > >> > On 20.02.2013 02:11, Selena Deckelmann wrote: >> >> >> >> So, I just ran into a similar issue ba

Re: [HACKERS] posix_fadvise missing in the walsender

2013-02-20 Thread Robert Haas
On Tue, Feb 19, 2013 at 5:48 PM, Simon Riggs wrote: > I agree with Merlin and Joachim - if we have the call in one place, we > should have it in both. We might want to assess whether we even want to have it one place. I've seen cases where the existing call hurts performance, because of WAL file

Re: [HACKERS] streaming header too small

2013-02-20 Thread Selena Deckelmann
On Wed, Feb 20, 2013 at 6:23 AM, Magnus Hagander wrote: > > On Feb 20, 2013 11:29 AM, "Heikki Linnakangas" > wrote: > > > > On 20.02.2013 02:11, Selena Deckelmann wrote: > >> > >> So, I just ran into a similar issue backing up a 9.2.1 server using > >> pg_basebackup version 9.2.3: > >> > >> pg_ba

Re: [HACKERS] JSON Function Bike Shedding

2013-02-20 Thread Robert Haas
On Tue, Feb 19, 2013 at 10:00 AM, Merlin Moncure wrote: > Anyways, as to overloading in general, well, SQL is heavily > overloaded. We don't have int_max, float_max, etc. and it would be > usability reduction if we did. That's true, but max(int) and max(float) are doing pretty much the same logi

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Kevin Grittner
Peter Eisentraut wrote: > I suppose one should be able to expect that if one finds a view > in the information schema, then one should be able to use DROP > VIEW to remove it.  Which in this case wouldn't work.  So I don't > think including a materialized view under views or tables is > appropria

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Kevin Grittner
Tom Lane wrote: > Kevin Grittner writes: >> When I went to do this, I hit a shift/reduce conflict, because >> with TABLE being optional it couldn't tell whether: > >> TRUNCATE MATERIALIZED VIEW x, y, z; > >> ... was looking for five relations or three.  That goes away >> with MATERIALIZED escalat

Re: [HACKERS] [RFC] indirect toast tuple support

2013-02-20 Thread Andres Freund
On 2013-02-20 10:16:45 -0500, Robert Haas wrote: > On Tue, Feb 19, 2013 at 9:26 AM, Andres Freund wrote: > > On 2013-02-19 09:12:02 -0500, Robert Haas wrote: > >> On Tue, Feb 19, 2013 at 9:00 AM, Andres Freund > >> wrote: > >> >> I'd be a little > >> >> reluctant to do it the way you propose be

Re: [HACKERS] [RFC] indirect toast tuple support

2013-02-20 Thread Robert Haas
On Tue, Feb 19, 2013 at 9:26 AM, Andres Freund wrote: > On 2013-02-19 09:12:02 -0500, Robert Haas wrote: >> On Tue, Feb 19, 2013 at 9:00 AM, Andres Freund >> wrote: >> >> So the other way that we could do this is to use something that's the >> >> same size as a TOAST pointer but has different co

Re: [HACKERS] Unarchived WALs deleted after crash

2013-02-20 Thread Jehan-Guillaume de Rorthais
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Just a quick top-post to thank you all for this fix guys ! Cheers, On 15/02/2013 18:43, Heikki Linnakangas wrote: > On 15.02.2013 19:16, Fujii Masao wrote: >> On Sat, Feb 16, 2013 at 2:07 AM, Heikki Linnakangas >> wrote: >>> On 15.02.2013 18:10, F

Re: [HACKERS] CREATE RULE "_RETURN" and toast tables

2013-02-20 Thread Andres Freund
On 2013-02-14 20:47:11 -0500, Tom Lane wrote: > Andres Freund writes: > > The current behaviour doesn't seem to be a terribly good idea. I propose > > to drop the toast table and reset the relfrozenxid in DefineQueryRewrite > > in the RelisBecomingView case. > > Yeah, probably worth doing. At th

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Peter Eisentraut
On 2/19/13 5:22 PM, David Fetter wrote: > On Tue, Feb 19, 2013 at 11:09:13PM +0100, Erik Rijkers wrote: >> On Sat, February 16, 2013 02:01, Kevin Grittner wrote: >>> matview-v4.patch.gz >> >> Hi, >> >> I was wondering if material views should not go into information_schema. I >> was thinking eith

Re: [HACKERS] streaming header too small

2013-02-20 Thread Magnus Hagander
On Feb 20, 2013 11:29 AM, "Heikki Linnakangas" wrote: > > On 20.02.2013 02:11, Selena Deckelmann wrote: >> >> So, I just ran into a similar issue backing up a 9.2.1 server using >> pg_basebackup version 9.2.3: >> >> pg_basebackup: starting background WAL receiver >> pg_basebackup: streaming header

Re: [HACKERS] FDW for PostgreSQL

2013-02-20 Thread Tom Lane
Shigeru Hanada writes: > On Fri, Feb 15, 2013 at 12:58 AM, Tom Lane wrote: >> [ rereads that... ] Hm, I did make some good points. But having seen >> the end result of this way, I'm still not very happy; it still looks >> like a maintenance problem. Maybe some additional flags in ruleutils.c >

Re: [HACKERS] FDW for PostgreSQL

2013-02-20 Thread Shigeru Hanada
On Fri, Feb 15, 2013 at 12:58 AM, Tom Lane wrote: > [ rereads that... ] Hm, I did make some good points. But having seen > the end result of this way, I'm still not very happy; it still looks > like a maintenance problem. Maybe some additional flags in ruleutils.c > is the least evil way after

Re: [HACKERS] BUG #7493: Postmaster messages unreadable in a Windows console

2013-02-20 Thread Alexander Law
Hello, 15.02.2013 02:59, Noah Misch wrote: With your proposed change, the problem will resurface in an actual SQL_ASCII database. At the problem's root is write_console()'s assumption that messages are in the database encoding. pg_bind_textdomain_codeset() tries to make that so, but it only wo

Re: [HACKERS] Review : Add hooks for pre- and post-processor executables for COPY and \copy

2013-02-20 Thread Etsuro Fujita
> + The command that input comes from or that output goes to. > + The command for COPY FROM, which input comes from, must write its > output > + to standard output. The command for COPY TO, which output goes to, > must > + read its input from standard

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Tom Lane
Robert Haas writes: > On Tue, Feb 19, 2013 at 11:01 AM, Peter Eisentraut wrote: >> This might be different if there were a command to refresh all >> materialized views, because you don't want to have to go around and type >> separate commands 47 times after a restore. > Well, it's pretty easy to

[HACKERS] Re: PostgreSql - access modified rows in prepare transaction command

2013-02-20 Thread pierpaolo.cincilla
Thank you Heikki for your reply. As you suggest, I will explain better what I'm trying to accomplish. What I'm writing a ditributed two-phase-commit termination protocol that work in this manner: 1) Each site has a replica of the database. A site A perform a transaction t1 and prepare it (PREPAR

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Robert Haas
On Tue, Feb 19, 2013 at 11:01 AM, Peter Eisentraut wrote: > On 2/19/13 8:54 AM, Robert Haas wrote: >> In the department of crazy ideas, what about having pg_dump NEVER >> refresh ANY materialized views? > > It might be useful to have an option for this, but I don't think it > should be the default

Re: [HACKERS] PostgreSql - access modified rows in prepare transaction command

2013-02-20 Thread Heikki Linnakangas
On 20.02.2013 12:48, pierpaolo.cincilla wrote: I have the problem to access modified data (updates and inserts) in a prepare transaction statement before a commit/rollback. For example consider the following block: BEGIN; do some update; do some insert; PREPARE TRANSACTION 'transaction1'; After

[HACKERS] PostgreSql - access modified rows in prepare transaction command

2013-02-20 Thread pierpaolo.cincilla
Hello, I have the problem to access modified data (updates and inserts) in a prepare transaction statement before a commit/rollback. For example consider the following block: BEGIN; do some update; do some insert; PREPARE TRANSACTION 'transaction1'; After executing the 'prepare' comma

Re: [HACKERS] Comment typo

2013-02-20 Thread Heikki Linnakangas
On 20.02.2013 05:07, Etsuro Fujita wrote: Sorry, I found one more typo. Attached is a patch. Thanks, applied. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] streaming header too small

2013-02-20 Thread Heikki Linnakangas
On 20.02.2013 02:11, Selena Deckelmann wrote: So, I just ran into a similar issue backing up a 9.2.1 server using pg_basebackup version 9.2.3: pg_basebackup: starting background WAL receiver pg_basebackup: streaming header too small: 25 I've had it happen two times in a row. I'm going to try a

Re: [HACKERS] [PATCH] Add PQconninfoParseParams and PQconninfodefaultsMerge to libpq

2013-02-20 Thread Amit Kapila
> > Tuesday, February 19, 2013 6:23 PM Amit Kapila wrote: > > On Monday, February 18, 2013 1:41 PM Heikki Linnakangas wrote: > > > On 18.02.2013 06:07, Amit Kapila wrote: > > > > On Sunday, February 17, 2013 8:44 PM Phil Sorber wrote: > > > >> On Sun, Feb 17, 2013 at 1:35 AM, Amit > > kapila > > >

Re: [HACKERS] 9.2.3 crashes during archive recovery

2013-02-20 Thread Heikki Linnakangas
On 20.02.2013 10:01, Kyotaro HORIGUCHI wrote: Sorry, Let me correct a bit. I tried to postpone smgrtruncate after the next checkpoint. This I tried to postpone smgrtruncate TO the next checktpoint. Umm, why? I don't understand this patch at all. - Heikki -- Sent via pgsql-hackers mailing

Re: [HACKERS] Materialized views WIP patch

2013-02-20 Thread Tom Lane
Kevin Grittner writes: > When I went to do this, I hit a shift/reduce conflict, because with > TABLE being optional it couldn't tell whether: > TRUNCATE MATERIALIZED VIEW x, y, z; > ... was looking for five relations or three.  That goes away with > MATERIALIZED escalated to TYPE_FUNC_NAME_KEYWO

Re: [HACKERS] 9.2.3 crashes during archive recovery

2013-02-20 Thread Kyotaro HORIGUCHI
Sorry, Let me correct a bit. > I tried to postpone smgrtruncate after the next checkpoint. This I tried to postpone smgrtruncate TO the next checktpoint. > is similar to what hotstandby feedback does to vacuum. It seems > to be working fine but I warry that it might also bloats the > table. I h