Re: [HACKERS] Temporary tables under hot standby

2012-06-10 Thread Tom Lane
Noah Misch writes: > On Fri, Jun 08, 2012 at 01:26:20PM -0400, Robert Haas wrote: >> I haven't ever heard anyone propose to redefine CREATE LOCAL TEMP >> TABLE to mean anything different than CREATE TEMP TABLE, so I'm >> disinclined to warn about that. > From a documentation perspective, it will

Re: [HACKERS] Temporary tables under hot standby

2012-06-10 Thread Noah Misch
On Fri, Jun 08, 2012 at 01:26:20PM -0400, Robert Haas wrote: > On Sun, Apr 29, 2012 at 4:02 PM, Noah Misch wrote: > > On Tue, Apr 24, 2012 at 11:55:15PM -0400, Noah Misch wrote: > >> Concerning everyone's favorite topic, how to name the new type of table, I > >> liked Tom's proposal[1] to make CRE

Re: [HACKERS] Temporary tables under hot standby

2012-06-08 Thread Kevin Grittner
Simon Riggs wrote: > On 8 June 2012 18:26, Robert Haas wrote: > >> I would be more open to warning people about CREATE GLOBAL TEMP >> TABLE - frankly, it's pretty wonky that we allow that but treat >> GLOBAL as a noise word in this first place. But I'm a little >> disinclined to have the messag

Re: [HACKERS] Temporary tables under hot standby

2012-06-08 Thread Simon Riggs
On 8 June 2012 18:26, Robert Haas wrote: > I would be more open to warning people about CREATE GLOBAL TEMP TABLE > - frankly, it's pretty wonky that we allow that but treat GLOBAL as a > noise word in this first place.  But I'm a little disinclined to have > the message speculate about what might

Re: [HACKERS] Temporary tables under hot standby

2012-06-08 Thread Robert Haas
On Sun, Apr 29, 2012 at 4:02 PM, Noah Misch wrote: > On Tue, Apr 24, 2012 at 11:55:15PM -0400, Noah Misch wrote: >> Concerning everyone's favorite topic, how to name the new type of table, I >> liked Tom's proposal[1] to make CREATE TEMP TABLE retain current behavior and >> have CREATE GLOBAL TEMP

Re: [HACKERS] Temporary tables under hot standby

2012-05-08 Thread Noah Misch
On Mon, May 07, 2012 at 09:04:28AM -0500, Merlin Moncure wrote: > On Mon, May 7, 2012 at 8:52 AM, Michael Nolan wrote: > > To cross-pollinate with another thread, if temporary tables (and > > insert/delete/update transactions to them) are to be supported on a slave, > > will the applications using

Re: [HACKERS] Temporary tables under hot standby

2012-05-07 Thread Merlin Moncure
On Tue, Apr 24, 2012 at 10:55 PM, Noah Misch wrote: > A key barrier to migrations from trigger-based replication to WAL-based > replication is the lack of temporary tables under hot standby.  I'd like to > close that gap; the changes needed will also reduce the master-side cost of > temporary tabl

Re: [HACKERS] Temporary tables under hot standby

2012-05-03 Thread Josh Berkus
> (on standby) > INSERT INTO s1 ; > SELECT ... FROM s1 WHERE ... > which seems to me to be actually harder than just rewriting as derived > table and isn't an option on Microstrategy etc, hence my observation > that GTTs don't help HS much. What I would like to see, one day, is > for temp tables t

Re: [HACKERS] Temporary tables under hot standby

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 4:11 AM, Simon Riggs wrote: > which seems to me to be actually harder than just rewriting as derived > table and isn't an option on Microstrategy etc, hence my observation > that GTTs don't help HS much. What I would like to see, one day, is > for temp tables to work without

Re: [HACKERS] Temporary tables under hot standby

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 1:57 AM, Josh Berkus wrote: > Michael, > >> What is the use case for temporary tables on a hot standby server? >> >> Perhaps this is a noobie question, but it seems to me that a hot standby >> server's use by* applications* or *users* should be limited to transactions >> tha

Re: [HACKERS] Temporary tables under hot standby

2012-05-02 Thread Josh Berkus
Michael, > What is the use case for temporary tables on a hot standby server? > > Perhaps this is a noobie question, but it seems to me that a hot standby > server's use by* applications* or *users* should be limited to transactions > that don't alter the database in any form. A very common use

Re: [HACKERS] Temporary tables under hot standby

2012-05-02 Thread Simon Riggs
On Wed, May 2, 2012 at 4:14 PM, Bruce Momjian wrote: > I think if implementing global temporary tables only for hot standby > user (#7), it might be of limited usefulness, but the ability to avoid > system table churn (#1) means global temporary tables would have a wide > usefulness, even without

Re: [HACKERS] Temporary tables under hot standby

2012-05-02 Thread Christopher Browne
On Wed, May 2, 2012 at 11:39 AM, Michael Nolan wrote: > What is the use case for temporary tables on a hot standby server? Simple... We required a "hot standby" server in order to get improved reliability. But we don't want it to sit there chewing power + money, unused. We want to *use* it to

Re: [HACKERS] Temporary tables under hot standby

2012-05-02 Thread Michael Nolan
What is the use case for temporary tables on a hot standby server? Perhaps this is a noobie question, but it seems to me that a hot standby server's use by* applications* or *users* should be limited to transactions that don't alter the database in any form. However, I can see where temporary tab

Re: [HACKERS] Temporary tables under hot standby

2012-05-02 Thread Bruce Momjian
On Thu, Apr 26, 2012 at 08:56:40AM +0100, Simon Riggs wrote: > A full GTT implementation is not required and the design differed from > that. I don't think "hideously complicated" is accurate, that's just > you're way of saying "and I disagree". Either route is pretty complex > and not much to choo

Re: [HACKERS] Temporary tables under hot standby

2012-05-01 Thread Jim Nasby
On 4/25/12 6:15 PM, Jaime Casanova wrote: On Wed, Apr 25, 2012 at 5:46 PM, Merlin Moncure wrote: > > I don't know how GTT play inside the Oracle stack such that they > aren't super popular, but if they work in the standby they will > quickly become a killer feature. IMNSHO it's annoying but

Re: [HACKERS] Temporary tables under hot standby

2012-05-01 Thread Jim Nasby
On 4/25/12 6:16 AM, Simon Riggs wrote: The way standard-like temporary tables work is exactly why I assume > Noah proposes to implement them: because they work nicely with HS. Well, following a standard that no other major DBMS has followed is not great, especially if it leads to a non-useful

Re: [HACKERS] Temporary tables under hot standby

2012-04-29 Thread Noah Misch
On Tue, Apr 24, 2012 at 11:55:15PM -0400, Noah Misch wrote: > Concerning everyone's favorite topic, how to name the new type of table, I > liked Tom's proposal[1] to make CREATE TEMP TABLE retain current behavior and > have CREATE GLOBAL TEMP TABLE and/or CREATE LOCAL TEMP TABLE request the new > S

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Robert Haas
On Thu, Apr 26, 2012 at 10:12 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Apr 26, 2012 at 3:56 AM, Simon Riggs wrote: >>> Previous discussions had Tom proposing ways of extending catalogs to >>> allow exactly this. So designs that address that point are already on >>> record. > >> Link?

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Tom Lane
Robert Haas writes: > On Thu, Apr 26, 2012 at 3:56 AM, Simon Riggs wrote: >> Previous discussions had Tom proposing ways of extending catalogs to >> allow exactly this. So designs that address that point are already on >> record. > Link? There was a thread a couple years ago ... http://archives

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Robert Haas
On Thu, Apr 26, 2012 at 9:57 AM, Greg Stark wrote: > On Thu, Apr 26, 2012 at 9:18 AM, Simon Riggs wrote: >> Implementing a feature that *requires* those things is madness and >> obscuring those crucial points is not balanced or fair. > > I think this whole discussion started the wrong way around.

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Greg Stark
On Thu, Apr 26, 2012 at 9:18 AM, Simon Riggs wrote: > Implementing a feature that *requires* those things is madness and > obscuring those crucial points is not balanced or fair. I think this whole discussion started the wrong way around. If the goal of implementing GTTs is to solve a need with r

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Simon Riggs
On Thu, Apr 26, 2012 at 12:49 PM, Kevin Grittner wrote: >> I find it a rather elegant and well-thought-out kludge. > > Global temporary tables as a feature are far more than a kludge; I > assume you're talking about that feature as a solution for the "no > temporary tables on a hot standby" probl

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Simon Riggs
On Thu, Apr 26, 2012 at 12:49 PM, Kevin Grittner wrote: >> So, I can't help wonder if what we're really missing here is a >> high-performance, log-based logical replication solution with good >> core support. On that, we do agree completely. That is exactly my goal for 9.3. > The GSoC xReader p

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Robert Haas
On Thu, Apr 26, 2012 at 3:56 AM, Simon Riggs wrote: > Previous discussions had Tom proposing ways of extending catalogs to > allow exactly this. So designs that address that point are already on > record. Link? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL C

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Kevin Grittner
> Robert Haas wrote: > Noah Misch wrote: >> Based on the range of assessments spanning your "almost useless" >> to Merlin's "killer feature", I gather that its utility is >> exceptionally site-specific. > > Well said, sir. +1 > I find it a rather elegant and well-thought-out kludge. Global

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Simon Riggs
On Thu, Apr 26, 2012 at 5:57 AM, Robert Haas wrote: > On Wed, Apr 25, 2012 at 10:04 PM, Noah Misch wrote: >> Based on the range of assessments spanning your "almost useless" to Merlin's >> "killer feature", I gather that its utility is exceptionally site-specific. > > Well said, sir. Not well sa

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Simon Riggs
On Wed, Apr 25, 2012 at 7:34 PM, Robert Haas wrote: > On Wed, Apr 25, 2012 at 1:31 PM, Simon Riggs wrote: >> However, it is a fallacy that this is a good solution for using temp >> tables on HS. I think the wish to enhance Oracle compatibility is >> making some wishful thinking happen with regard

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Heikki Linnakangas
On 25.04.2012 18:49, Robert Haas wrote: Maybe this is a silly idea, but if you're thinking about creating a local XID space and a global XID space, it might be a good idea to also make allowance for an "unlogged" XID space - that is, an XID space that is global to all backends but need not surviv

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 10:16 PM, Noah Misch wrote: >> > Past discussions have raised the issue of interaction between commands like >> > ALTER TABLE and sessions using the new-variety temporary table. ?As a first >> > cut, let's keep this simple and have ongoing use of the table block >> > opera

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 10:04 PM, Noah Misch wrote: > Based on the range of assessments spanning your "almost useless" to Merlin's > "killer feature", I gather that its utility is exceptionally site-specific. Well said, sir. It might be worth taking a couple of steps backward, here. Regardless

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Noah Misch
On Wed, Apr 25, 2012 at 11:49:23AM -0400, Robert Haas wrote: > On Tue, Apr 24, 2012 at 11:55 PM, Noah Misch wrote: > > Our temporary tables are cataloged and filled like permanent tables. ?This > > has > > the major advantage of making most code operate on tables with minimal > > regard > > for

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Noah Misch
On Wed, Apr 25, 2012 at 10:10:31AM +0100, Simon Riggs wrote: > So you are saying it is OK to not be able to *create* them on HS, just > *use* pre-defined tables? I estimated that much to cover a worthy portion of the need, yes. > That's almost useless IMHO. Based on the range of assessments span

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Jaime Casanova
On Wed, Apr 25, 2012 at 5:46 PM, Merlin Moncure wrote: > > I don't know how GTT play inside the Oracle stack such that they > aren't super popular, but if they work in the standby they will > quickly become a killer feature.  IMNSHO it's annoying but acceptable > to be forced to define them into t

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Merlin Moncure
On Wed, Apr 25, 2012 at 11:53 AM, Robert Haas wrote: > On Wed, Apr 25, 2012 at 12:30 PM, Simon Riggs wrote: >> On Wed, Apr 25, 2012 at 5:19 PM, Robert Haas wrote: >> >>> Oh, we're talking about different things, and I'm slightly confused. >>> >>> Yes, we need to support ANALYZE; what we might no

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 1:31 PM, Simon Riggs wrote: > However, it is a fallacy that this is a good solution for using temp > tables on HS. I think the wish to enhance Oracle compatibility is > making some wishful thinking happen with regard to how useful this is > going to be. We need to spend jus

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Simon Riggs
On Wed, Apr 25, 2012 at 5:53 PM, Robert Haas wrote: >> Yes, we are. Global Temp Tables won't solve the "Works on HS" problem, >> so we'd better decide fairly quickly which use case we are addressing, >> and why. ISTM Global Temp Tables is more an Oracle compatibility issue >> than a problem Postg

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 12:30 PM, Simon Riggs wrote: > On Wed, Apr 25, 2012 at 5:19 PM, Robert Haas wrote: > >> Oh, we're talking about different things, and I'm slightly confused. >> >> Yes, we need to support ANALYZE; what we might not need to support, at >> least initially, is every user of a

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Simon Riggs
On Wed, Apr 25, 2012 at 5:19 PM, Robert Haas wrote: > Oh, we're talking about different things, and I'm slightly confused. > > Yes, we need to support ANALYZE; what we might not need to support, at > least initially, is every user of a global temp table having their own > SEPARATE copy of the tab

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 12:18 PM, Robert Haas wrote: > On Wed, Apr 25, 2012 at 12:08 PM, Simon Riggs wrote: >> On Wed, Apr 25, 2012 at 4:49 PM, Robert Haas wrote: How important is support for VACUUM on these tables under hot standby?  The alternative is to fail when a session retains a

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 12:08 PM, Simon Riggs wrote: > On Wed, Apr 25, 2012 at 4:49 PM, Robert Haas wrote: >>> How important is support for VACUUM on these tables under hot standby?  The >>> alternative is to fail when a session retains a temporary table across 2B >>> local transactions.  I do no

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Simon Riggs
On Wed, Apr 25, 2012 at 4:49 PM, Robert Haas wrote: >> How important is support for VACUUM on these tables under hot standby?  The >> alternative is to fail when a session retains a temporary table across 2B >> local transactions.  I do not currently see any challenges sufficient to >> motivate n

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Robert Haas
On Tue, Apr 24, 2012 at 11:55 PM, Noah Misch wrote: > A key barrier to migrations from trigger-based replication to WAL-based > replication is the lack of temporary tables under hot standby.  I'd like to > close that gap; the changes needed will also reduce the master-side cost of > temporary tabl

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Simon Riggs
On Wed, Apr 25, 2012 at 11:08 AM, Nicolas Barbier wrote: > 2012/4/25 Simon Riggs : > >> So you are saying it is OK to not be able to *create* them on HS, just >> *use* pre-defined tables? >> >> That's almost useless IMHO. >> >> Applications expect to be able to do this all in the same transaction

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Nicolas Barbier
2012/4/25 Nicolas Barbier : > is the reason of existence for the PG-like temporary transactions. s/transactions/tables/ Nicolas -- A. Because it breaks the logical sequence of discussion. Q. Why is top posting bad? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Nicolas Barbier
2012/4/25 Simon Riggs : > So you are saying it is OK to not be able to *create* them on HS, just > *use* pre-defined tables? > > That's almost useless IMHO. > > Applications expect to be able to do this all in the same transaction > on one session > CREATE TEMP TABLE x; >  ...DML commands... > SEL

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Simon Riggs
On Wed, Apr 25, 2012 at 9:37 AM, Nicolas Barbier wrote: > The declarative creation of an “standard-like” temporary table only > happens once (it is part of the schema). Using (e.g. putting stuff in > and executing queries on) such tables can happen on the standby > without the master having to kn

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Nicolas Barbier
2012/4/25 Simon Riggs : > On Wed, Apr 25, 2012 at 4:55 AM, Noah Misch wrote: > >> I do not see a clean behind-the-scenes fix for points 1, 4 and 5.  We can >> resolve those by adding a new variety of temporary table, one coincidentally >> matching the SQL standard's notion of a temporary table.  

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Simon Riggs
On Wed, Apr 25, 2012 at 4:55 AM, Noah Misch wrote: > A third patch will permit the following commands in read-only transactions, > where they will throw an error if the subject is not a temporary table: ... > VACUUM (including VACUUM FULL) > CLUSTER (without USING clause) > REINDEX Those command

Re: [HACKERS] Temporary tables under hot standby

2012-04-25 Thread Simon Riggs
On Wed, Apr 25, 2012 at 4:55 AM, Noah Misch wrote: > 1. High catalog turnover in rapid create/drop workloads.  Heavy temporary >   table users often need to REINDEX relation-oriented catalogs.  Hot standby >   cannot assign OIDs or modify system catalogs at all. > 4. sinval traffic from every CRE

[HACKERS] Temporary tables under hot standby

2012-04-24 Thread Noah Misch
A key barrier to migrations from trigger-based replication to WAL-based replication is the lack of temporary tables under hot standby. I'd like to close that gap; the changes needed will also reduce the master-side cost of temporary table usage. Here is a high-level design for your advice and com