Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-05-05 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 > 1 seconds seems a very low default for me. I can imagine that most > distributed transactions are taking longer than this to complete. One second means it is set by default to catch *all* prepared transactions. It's simply checking how long t

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-05-05 Thread Bernd Helmle
--On Dienstag, Mai 05, 2009 15:29:15 + Greg Sabino Mullane wrote: It simply checks the age of entries in pg_prepared_xacts and gives a warning if the number is at or over the given threshhold (defaults to 1 second). I'm still a heavy +1 on making the default Postgres configuration value 0,

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-05-05 Thread Tom Lane
"Greg Sabino Mullane" writes: > Just as a followup (and I already posted this on the pgsql-admin thread), > the check_postgres script now has a specific check for this very case. > It simply checks the age of entries in pg_prepared_xacts and gives > a warning if the number is at or over the given

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-05-05 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 > The pgsql-admin list has just seen another instance where careless use > of prepared transactions brought down a database, and the DBA (who had > no idea what a prepared transaction even was) had no idea how to fix it. Just as a followup (and

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-28 Thread Robert Treat
On Wednesday 22 April 2009 15:49:32 Tom Lane wrote: > I wrote: > > Heikki Linnakangas writes: > >> Configuration affects what can be tested in installcheck, that's quite > >> natural. I would be happy with simply adding an alternative expected > >> output file for min_prepared_xacts=0 case. Like w

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-23 Thread Kevin Grittner
"Joshua D. Drake" wrote: > I am not opposed to making the default zero. +1 making zero the default for 8.4 > I am also +1 on adding the warnings. +1, but less urgent, lower priority -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subs

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Robert Haas writes: > Maybe I'm just dumb, but I don't get it. If I start a transaction and > do "SELECT * FROM foo" and then wait around for an hour or two while > someone else makes changes to foo and then do "SELECT * FROM foo" > again, I expect to see the same rows I saw the first time, which

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Robert Haas
On Wed, Apr 22, 2009 at 9:21 PM, Robert Haas wrote: > Maybe I'm just dumb, but I don't get it.  If I start a transaction and > do "SELECT * FROM foo" and then wait around for an hour or two while > someone else makes changes to foo and then do "SELECT * FROM foo" > again, I expect to see the same

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Robert Haas
On Wed, Apr 22, 2009 at 8:58 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Apr 22, 2009 at 5:44 PM, Tom Lane wrote: >>> I think we've already milked what we can from that, since a prepared >>> xact is treated exactly like an open one with no snapshot.  The point >>> is that whatever rows

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 22, 2009 at 5:44 PM, Tom Lane wrote: >> I think we've already milked what we can from that, since a prepared >> xact is treated exactly like an open one with no snapshot.  The point >> is that whatever rows it's written are still in-doubt and cannot be >> frozen,

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Robert Haas
On Wed, Apr 22, 2009 at 5:44 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Apr 22, 2009 at 2:58 PM, Heikki Linnakangas >> wrote: >>> It still does. A prepared xact is just like a idle-in-transaction backend as >>> far as vacuum is concerned. > >> Is that really necessary? It's true that y

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 22, 2009 at 2:58 PM, Heikki Linnakangas > wrote: >> It still does. A prepared xact is just like a idle-in-transaction backend as >> far as vacuum is concerned. > Is that really necessary? It's true that you can't vacuum away any > rows whose xmin is that of the

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Robert Haas
On Wed, Apr 22, 2009 at 2:58 PM, Heikki Linnakangas wrote: > Tom Lane wrote: >> >> Does a prepared xact still block vacuum cleanup in HEAD, or has that >> been fixed since 8.2? > > It still does. A prepared xact is just like a idle-in-transaction backend as > far as vacuum is concerned. Is that r

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
"Joshua D. Drake" writes: > On Wed, 2009-04-22 at 15:49 -0400, Tom Lane wrote: >> Comments? Anyone seriously opposed to making the default be zero? > I am not opposed to making the default zero. I am also +1 on adding the > warnings. What I think we could/should do about that for 8.4 is to impr

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Joshua D. Drake
On Wed, 2009-04-22 at 15:49 -0400, Tom Lane wrote: > I wrote: > Warning about very old prepared transactions is something that we > could think about doing as well; it doesn't have to be either-or. > I think the need for it would decrease quite a bit if they weren't > enabled by default, though. >

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
I wrote: > Heikki Linnakangas writes: >> Configuration affects what can be tested in installcheck, that's quite >> natural. I would be happy with simply adding an alternative expected >> output file for min_prepared_xacts=0 case. Like we've done for xml test >> cases, for example, though that's

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Heikki Linnakangas
Jeff Davis wrote: On Wed, 2009-04-22 at 21:58 +0300, Heikki Linnakangas wrote: Tom Lane wrote: Does a prepared xact still block vacuum cleanup in HEAD, or has that been fixed since 8.2? It still does. A prepared xact is just like a idle-in-transaction backend as far as vacuum is concerned. I

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Jeff Davis
On Wed, 2009-04-22 at 21:58 +0300, Heikki Linnakangas wrote: > Tom Lane wrote: > > Does a prepared xact still block vacuum cleanup in HEAD, or has that > > been fixed since 8.2? > > It still does. A prepared xact is just like a idle-in-transaction > backend as far as vacuum is concerned. I thoug

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Heikki Linnakangas
Tom Lane wrote: Does a prepared xact still block vacuum cleanup in HEAD, or has that been fixed since 8.2? It still does. A prepared xact is just like a idle-in-transaction backend as far as vacuum is concerned. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent vi

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Jeff Davis
On Wed, 2009-04-22 at 13:53 -0400, Alvaro Herrera wrote: > Another thing we could do is make autovacuum log something about those, > similar to what it does to temp tables. And if one of them gets too > near Xid wraparound, kill it. As I said in my reply to Joshua, I don't think killing a prepare

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Jeff Davis writes: > On Wed, 2009-04-22 at 11:00 -0700, Joshua D. Drake wrote: >> Then perhaps a setting like max_stale_prepared_transaction_age and once >> that threshold is met it will autorollback? > I think that defeats the safety of prepared transactions in many cases. > Let's say you PREPAR

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Heikki Linnakangas writes: > Tom Lane wrote: >> That only works for make check, not make installcheck. > Configuration affects what can be tested in installcheck, that's quite > natural. I would be happy with simply adding an alternative expected > output file for min_prepared_xacts=0 case. Lik

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Jeff Davis
On Wed, 2009-04-22 at 11:00 -0700, Joshua D. Drake wrote: > Then perhaps a setting like max_stale_prepared_transaction_age and once > that threshold is met it will autorollback? I think that defeats the safety of prepared transactions in many cases. Let's say you PREPARE TRANSACTION on two systems

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas writes: I think we should change the way we test it. Could we simply make max_prepared_transactions = 0 the default, but put "max_prepared_transactions = 5" into the config file in "make check"? That only works for make check, not make installcheck. Conf

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Andrew Dunstan
Heikki Linnakangas wrote: I think we should change the way we test it. Could we simply make max_prepared_transactions = 0 the default, but put "max_prepared_transactions = 5" into the config file in "make check"? That seems like the best alternative, it doesn't seem right to build extra co

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
Heikki Linnakangas writes: > I think we should change the way we test it. Could we simply make > max_prepared_transactions = 0 the default, but put > "max_prepared_transactions = 5" into the config file in "make check"? That only works for make check, not make installcheck. We'd really have t

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 > Anyway, maybe question zero is whether anyone else thinks this is > important enough to justify extra work in the area. Yes. For every user that complains on the list, there are a dozen other quiet ones who have been bit by the same. > The m

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
"Joshua D. Drake" writes: > On Wed, 2009-04-22 at 13:48 -0400, Tom Lane wrote: >> Another line of thought is that prepared xacts are inherently a bad >> thing to be using if you have not done careful setup of a lot of >> external infrastructure (in particular, have a transaction monitor >> running

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Heikki Linnakangas
Joshua D. Drake wrote: On Wed, 2009-04-22 at 13:48 -0400, Tom Lane wrote: The main objection to just setting max_prepared_transactions to zero by default is that it would kill our ability to test the feature in the standard regression tests. That kills it for me. Unless we want to change th

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Joshua D. Drake
On Wed, 2009-04-22 at 13:48 -0400, Tom Lane wrote: > One line of thought is just to raise the visibility of old prepared > transactions somehow. I don't think I want to go as far as, say, making > every session-start issue WARNINGs about every prepared xact that's more > than a few minutes old.

Re: [HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Alvaro Herrera
Tom Lane wrote: > Anyway, maybe question zero is whether anyone else thinks this is > important enough to justify extra work in the area. One thing that has already changed is that DROP DATABASE reports "N users and M prepared transactions", so there is more of a hint. Another thing we could do

[HACKERS] Prepared transactions vs novice DBAs, again

2009-04-22 Thread Tom Lane
The pgsql-admin list has just seen another instance where careless use of prepared transactions brought down a database, and the DBA (who had no idea what a prepared transaction even was) had no idea how to fix it. It seems to me we need to do something about making that stuff less DBA-unfriendly.