Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-13 Thread Greg Sabino Mullane
On Wed, Oct 12, 2011 at 10:50:13AM +0300, Peter Eisentraut wrote: > Actually, I'm currently personally more concerned about the breakage we > introduce in minor releases. We'd need to solve that problem before we > can even begin to think about dealing with the major release issue. +1 This b

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-12 Thread Simon Riggs
On Wed, Oct 12, 2011 at 1:44 PM, Florian Pflug wrote: > On Oct11, 2011, at 23:35 , Simon Riggs wrote: >> On Tue, Oct 11, 2011 at 10:30 PM, Florian Pflug wrote: >> >>> That experience has taught me that backwards compatibility, while very >>> important in a lot of cases, has the potential to do ju

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-12 Thread Robert Haas
On Wed, Oct 12, 2011 at 8:44 AM, Florian Pflug wrote: > With such a switch, every application that relies on true serializability for > correctness would be prone to silent data corruption should the switch ever > get set to "off" accidentally. Agreed. > Without such a switch, OTOH, all that wil

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-12 Thread Florian Pflug
On Oct11, 2011, at 23:35 , Simon Riggs wrote: > On Tue, Oct 11, 2011 at 10:30 PM, Florian Pflug wrote: > >> That experience has taught me that backwards compatibility, while very >> important in a lot of cases, has the potential to do just as much harm >> if overdone. > > Agreed. Does my suggest

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-12 Thread Simon Riggs
On Wed, Oct 12, 2011 at 8:50 AM, Peter Eisentraut wrote: > On tis, 2011-10-11 at 21:50 +0100, Simon Riggs wrote: >> I'm keen to ensure people enjoy the possibility of upgrading to the >> latest release. The continual need to retest applications mean that >> very few users upgrade quickly or with a

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-12 Thread Simon Riggs
On Wed, Oct 12, 2011 at 6:34 AM, Heikki Linnakangas wrote: > On 11.10.2011 23:21, Simon Riggs wrote: >> >> If the normal default_transaction_isolation = read committed and all >> transactions that require serializable are explicitly marked in the >> application then there is no way to turn off SSI

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-12 Thread Peter Eisentraut
On tis, 2011-10-11 at 21:50 +0100, Simon Riggs wrote: > I'm keen to ensure people enjoy the possibility of upgrading to the > latest release. The continual need to retest applications mean that > very few users upgrade quickly or with anywhere near the frequency > with which we put out new releases

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Heikki Linnakangas
On 11.10.2011 23:21, Simon Riggs wrote: If the normal default_transaction_isolation = read committed and all transactions that require serializable are explicitly marked in the application then there is no way to turn off SSI without altering the application. That is not acceptable, since it caus

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Bruce Momjian
Simon Riggs wrote: > On Tue, Oct 11, 2011 at 10:30 PM, Florian Pflug wrote: > > > That experience has taught me that backwards compatibility, while very > > important in a lot of cases, has the potential to do just as much harm > > if overdone. > > Agreed. Does my suggestion represent overdoing

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Bruce Momjian
Simon Riggs wrote: > On Tue, Oct 11, 2011 at 10:22 PM, Tom Lane wrote: > > Simon Riggs writes: > >> How could I change the viewpoint of the group without making rational > >> arguments when it matters? > > > > Well, you make your arguments, and you see if you convince anybody. > > On these specif

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Simon Riggs
On Tue, Oct 11, 2011 at 10:30 PM, Florian Pflug wrote: > That experience has taught me that backwards compatibility, while very > important in a lot of cases, has the potential to do just as much harm > if overdone. Agreed. Does my suggestion represent overdoing it? I ask for balance, not an ext

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Kevin Grittner
Simon Riggs wrote: > They ask comparative questions like "What is the risk of > upgrade?", "How much testing is required?" > I never met a customer yet that has an automated test suite > designed to stress the accuracy of results under concurrent > workloads I'll try to provide some informat

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Simon Riggs
On Tue, Oct 11, 2011 at 10:22 PM, Tom Lane wrote: > Simon Riggs writes: >> How could I change the viewpoint of the group without making rational >> arguments when it matters? > > Well, you make your arguments, and you see if you convince anybody. > On these specific points, you've failed to sway

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Florian Pflug
On Oct11, 2011, at 22:55 , Simon Riggs wrote: > Probably as a matter of policy all new features that effect semantics > should have some kind of compatibility or off switch, if easily > possible. There's a huge downside to that, though. After a while, you end up with a gazillion settings, each inf

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Simon Riggs
On Tue, Oct 11, 2011 at 10:00 PM, Kevin Grittner wrote: > Simon Riggs wrote: > >> "You'll have to retest your apps" just isn't a good answer > > For which major PostgreSQL releases have you recommended that people > deploy their apps without retesting? None. People don't always follow my advice,

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Tom Lane
Simon Riggs writes: > How could I change the viewpoint of the group without making rational > arguments when it matters? Well, you make your arguments, and you see if you convince anybody. On these specific points, you've failed to sway the consensus AFAICT, and at some point you have to accept t

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Simon Riggs
On Tue, Oct 11, 2011 at 9:53 PM, Bruce Momjian wrote: > Simon Riggs wrote: >> > Simon seems to value backward-compatibility more than the average >> > hackers poster. ?The lack of complaints about 9.1 I think means that the >> > hackers decision of _not_ providing a swich was the right one. >> >>

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Simon Riggs
On Tue, Oct 11, 2011 at 9:37 PM, Kevin Grittner wrote: > It would certainly be a trivial change to > implement; the problem is convincing others that it's a good idea. I don't want it, I just think we need it now. "You'll have to retest your apps" just isn't a good answer and we should respect t

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Kevin Grittner
Simon Riggs wrote: > "You'll have to retest your apps" just isn't a good answer For which major PostgreSQL releases have you recommended that people deploy their apps without retesting? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your sub

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Bruce Momjian
Simon Riggs wrote: > > Simon seems to value backward-compatibility more than the average > > hackers poster. ?The lack of complaints about 9.1 I think means that the > > hackers decision of _not_ providing a swich was the right one. > > So its been out 1 month and you think that is sufficient time

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Kevin Grittner
Greg Sabino Mullane wrote: > Kevin Grittner: > >> Did these transactions write anything? If not, were they >> declared to be READ ONLY? If they were, in fact, only reading, >> it would be interesting to see what the performance looks like if >> the recommendation to use the READ ONLY attrib

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Tom Lane
Simon Riggs writes: > There is no off switch and there should be. As Greg said, that ship has sailed. I believe that we specifically discussed the notion of an "off switch" via a GUC or similar during 9.1 development, and rejected it on the grounds that GUCs changing fundamental transactional be

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Simon Riggs
On Tue, Oct 11, 2011 at 9:32 PM, Bruce Momjian wrote: > Greg Sabino Mullane wrote: > -- Start of PGP signed section. >> > If the normal default_transaction_isolation = read committed and all >> > transactions that require serializable are explicitly marked in the >> > application then there is no

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Greg Sabino Mullane
On Tue, Oct 11, 2011 at 04:32:45PM -0400, Bruce Momjian wrote: ... > Simon seems to value backward-compatibility more than the average > hackers poster. The lack of complaints about 9.1 I think means that the > hackers decision of _not_ providing a swich was the right one. I wouldn't go that far:

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Kevin Grittner
Simon Riggs wrote: > Greg Sabino Mullane wrote: >> Eh? It has an off switch: repeatable read. > > You mean: if we recode the application and retest it, we can get > it to work same way as it used to. > > To most people that is the same thing as "it doesn't work with > this release", ask any a

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Bruce Momjian
Simon Riggs wrote: > On Tue, Oct 11, 2011 at 6:44 PM, Kevin Grittner > wrote: > > >> If you alter the default_transaction_isolation then you will break > >> applications like this, so it is not a valid way to turn off SSI. > > > > I don't follow you here. ?What would break? ?In what fashion? ?Sin

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Bruce Momjian
Greg Sabino Mullane wrote: -- Start of PGP signed section. > > If the normal default_transaction_isolation = read committed and all > > transactions that require serializable are explicitly marked in the > > application then there is no way to turn off SSI without altering the > > application. That

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Simon Riggs
On Tue, Oct 11, 2011 at 9:21 PM, Greg Sabino Mullane wrote: > Simon Riggs: >> Most apps use mixed mode serializable/repeatable read and therefore >> can't be changed by simple parameter. Rewriting the application isn't >> a sensible solution. >> >> I think it's clear that SSI should have had and

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Greg Sabino Mullane
> If the normal default_transaction_isolation = read committed and all > transactions that require serializable are explicitly marked in the > application then there is no way to turn off SSI without altering the > application. That is not acceptable, since it causes changes in > application behavi

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Greg Sabino Mullane
Robert Haas: > Serializable mode is much slower on this test, though. On > REL9_1_STABLE, it's about 8% slower with a single client. At 8 > clients, the difference rises to 43%, and at 32 clients, it's 51% > slower. Bummer. Thanks for putting some numbers out there; glad I was able to jump star

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Simon Riggs
On Tue, Oct 11, 2011 at 6:44 PM, Kevin Grittner wrote: >> If you alter the default_transaction_isolation then you will break >> applications like this, so it is not a valid way to turn off SSI. > > I don't follow you here.  What would break?  In what fashion?  Since > the standard allows any isol

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Kevin Grittner
Simon Riggs wrote: > It's common to find applications that have some transactions > explicitly coded to use SERIALIZABLE mode, while the rest are in > the default mode READ COMMITTED. So common that TPC-E benchmark > has been written as a representation of such workloads. I would be willing to

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Kevin Grittner
Robert Haas wrote: > Kevin Grittner wrote: >> Did these transactions write anything? If not, were they >> declared to be READ ONLY? If they were, in fact, only reading, >> it would be interesting to see what the performance looks like if >> the recommendation to use the READ ONLY attribute is

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Simon Riggs
On Tue, Oct 11, 2011 at 6:14 PM, Robert Haas wrote: > On Tue, Oct 11, 2011 at 1:11 PM, Simon Riggs wrote: >> On Mon, Oct 10, 2011 at 11:31 PM, Kevin Grittner >> wrote: >>> Simon Riggs wrote: >>> How do we turn it on/off to allow the overhead to be measured? >>> >>> User REPEATABLE READ tra

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Robert Haas
On Tue, Oct 11, 2011 at 1:11 PM, Simon Riggs wrote: > On Mon, Oct 10, 2011 at 11:31 PM, Kevin Grittner > wrote: >> Simon Riggs wrote: >> >>> How do we turn it on/off to allow the overhead to be measured? >> >> User REPEATABLE READ transactions or SERIALIZABLE transactions.  The >> easiest way, i

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Robert Haas
On Tue, Oct 11, 2011 at 12:46 PM, Kevin Grittner wrote: > Robert Haas wrote: >> I ran my good old pgbench -S, scale factor 100, shared_buffers = >> 8GB test on Nate Boley's box.  I ran it on both 9.1 and 9.2dev, >> and at all three isolation levels.  As usual, I took the median of >> three 5-minu

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Simon Riggs
On Mon, Oct 10, 2011 at 11:31 PM, Kevin Grittner wrote: > Simon Riggs wrote: > >> How do we turn it on/off to allow the overhead to be measured? > > User REPEATABLE READ transactions or SERIALIZABLE transactions.  The > easiest way, if you're doing it for all transactions (which I > recommend) is

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Kevin Grittner
Robert Haas wrote: > I ran my good old pgbench -S, scale factor 100, shared_buffers = > 8GB test on Nate Boley's box. I ran it on both 9.1 and 9.2dev, > and at all three isolation levels. As usual, I took the median of > three 5-minute runs, which I've generally found adequate to > eliminate

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-11 Thread Robert Haas
On Mon, Oct 10, 2011 at 3:59 PM, Kevin Grittner wrote: > I do have some concern about whether the performance improvements > from reduced LW locking contention elsewhere in the code may (in > whack-a-mole fashion) cause the percentages to go higher in SSI. > The biggest performance issues in some

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-10 Thread Kevin Grittner
Simon Riggs wrote: > How do we turn it on/off to allow the overhead to be measured? User REPEATABLE READ transactions or SERIALIZABLE transactions. The easiest way, if you're doing it for all transactions (which I recommend) is to set default_transaction_isolation. -Kevin -- Sent via pgsq

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-10 Thread Simon Riggs
On Mon, Oct 10, 2011 at 8:30 PM, Heikki Linnakangas wrote: > On 10.10.2011 21:25, Greg Sabino Mullane wrote: >> >> I agree it is better versus SELECT FOR, but what about repeatable read >> versus >> the new serializable? How much overhead is there in the 'monitoring of >> read/write dependencies'?

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-10 Thread Dan Ports
On Mon, Oct 10, 2011 at 04:10:18PM -0500, Kevin Grittner wrote: > Did you ever see much contention on > SerializablePredicateLockListLock, or was it just > SerializableXactHashLock? I think the former might be able to use > the non-blocking techniques, but I fear the main issue is with the > latte

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-10 Thread Kevin Grittner
Dan Ports wrote: > I spent some time thinking about this a while back, but didn't > have time to get very far. The problem isn't contention in the > predicate lock manager (which is partitioned) but the single lock > protecting the active SerializableXact state. > > It would probably help thing

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-10 Thread Dan Ports
On Mon, Oct 10, 2011 at 02:59:04PM -0500, Kevin Grittner wrote: > I do have some concern about whether the performance improvements > from reduced LW locking contention elsewhere in the code may (in > whack-a-mole fashion) cause the percentages to go higher in SSI. > The biggest performance issues

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-10 Thread Kevin Grittner
Heikki Linnakangas wrote: > On 10.10.2011 21:25, Greg Sabino Mullane wrote: >> I agree it is better versus SELECT FOR, but what about repeatable >> read versus the new serializable? How much overhead is there in >> the 'monitoring of read/write dependencies'? This is my only >> concern at the mome

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-10 Thread Dan Ports
On Mon, Oct 10, 2011 at 02:25:59PM -0400, Greg Sabino Mullane wrote: > I agree it is better versus SELECT FOR, but what about repeatable read versus > the new serializable? How much overhead is there in the 'monitoring of > read/write dependencies'? This is my only concern at the moment. Are we >

Re: [HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-10 Thread Heikki Linnakangas
On 10.10.2011 21:25, Greg Sabino Mullane wrote: I agree it is better versus SELECT FOR, but what about repeatable read versus the new serializable? How much overhead is there in the 'monitoring of read/write dependencies'? This is my only concern at the moment. Are we talking insignificant overhe

[HACKERS] Overhead cost of Serializable Snapshot Isolation

2011-10-10 Thread Greg Sabino Mullane
I'm looking into upgrading a fairly busy system to 9.1. They use serializable mode for a few certain things, and suffer through some serialization errors as a result. While looking over the new serializable/SSI documentation, one thing that stood out is: http://www.postgresql.org/docs/current/inte