Re: [PERFORM] MVCC performance issue

2010-11-29 Thread Robert Haas
On Thu, Nov 11, 2010 at 1:25 PM, Kyriacos Kyriacou wrote: > This is my first post in this mailing list and I would like to raise an > issue that in my opinion is causing performance issues of PostgreSQL > especially in a transaction processing environment. In my company we are > using PostgreSQL f

Re: [PERFORM] MVCC performance issue

2010-11-14 Thread Mladen Gogala
Marti Raudsepp wrote: Another advantage of Oracle's approach seems that they need much less tuple-level overhead. IMO the 23-byte tuple overhead is a much bigger drawback in Postgres than table fragmentation. Regards, Marti Oracle, however, does have a problem with "ORA-1555 Snapshot too o

Re: [PERFORM] MVCC performance issue

2010-11-14 Thread Marti Raudsepp
On Thu, Nov 11, 2010 at 20:25, Kyriacos Kyriacou wrote: > By definition of MVCC, when an UPDATE is performed, PostgreSQL creates a > new copy of the row in a new location. > result is to have huge fragmentation on table space, unnecessary updates > in all affected indexes, unnecessary costly I/O

Re: [PERFORM] MVCC performance issue

2010-11-14 Thread Marti Raudsepp
On Sat, Nov 13, 2010 at 07:53, Craig Ringer wrote: > Oracle's MVCC approach has its own costs. Like Pg's, those costs increase > with update/delete frequency. Instead of table bloat, Oracle suffers from > redo log growth (or redo log size management issues). Instead of increased > table scan costs

Re: [PERFORM] MVCC performance issue

2010-11-13 Thread Craig Ringer
On 11/14/2010 02:38 AM, Mladen Gogala wrote: Craig Ringer wrote: It sounds like you're describing Oracle-style MVCC, using redo logs. http://blogs.sybase.com/database/2009/04/mvcc-dispelling-some-oracle-fudunderstanding-the-cost/ Craig, this is an interesting blog page, making some valid po

Re: [PERFORM] MVCC performance issue

2010-11-13 Thread Mladen Gogala
Craig Ringer wrote: It sounds like you're describing Oracle-style MVCC, using redo logs. http://blogs.sybase.com/database/2009/04/mvcc-dispelling-some-oracle-fudunderstanding-the-cost/ Craig, this is an interesting blog page, making some valid points about the multiversioning vs. locking. T

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Rich
In reading what you are describing, don't you think PG 9 goes a long way to helping you out? On Sat, Nov 13, 2010 at 12:53 AM, Craig Ringer wrote: > On 11/12/2010 02:25 AM, Kyriacos Kyriacou wrote: > > The >> result is to have huge fragmentation on table space, unnecessary updates >> in all aff

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Craig Ringer
On 11/12/2010 02:25 AM, Kyriacos Kyriacou wrote: The result is to have huge fragmentation on table space, unnecessary updates in all affected indexes, unnecessary costly I/O operations, poor performance on SELECT that retrieves big record sets (i.e. reports etc) and slower updates. Yep. It's

[PERFORM] MVCC performance issue

2010-11-12 Thread Kyriacos Kyriacou
This is my first post in this mailing list and I would like to raise an issue that in my opinion is causing performance issues of PostgreSQL especially in a transaction processing environment. In my company we are using PostgreSQL for the last 8 year for our in-house developed billing system (telec

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Scott Carey
On Nov 12, 2010, at 9:13 AM, Kyriacos Kyriacou wrote: > My suggestion had just a single difference from what currently MVCC is > doing (btw I never said that MVCC is bad). > > NOW ===> on COMMIT previous version record is expired and the > new version record (created in new dynamically

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Scott Carey
HOT also usually requires setting FILLFACTOR to something other than the default for your table, so that there is guaranteed room in the page to modify data without allocating a new page. If you have fillfactor=75, then basically this proposal is already done -- each page has 25% temp space for

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Scott Marlowe
On Fri, Nov 12, 2010 at 9:19 AM, Ben Chobot wrote: > On Nov 12, 2010, at 8:14 AM, Kyriacos Kyriacou wrote: > > We are still using PostgreSQL 8.2.4. We are running a 24x7 system and > database size is over 200Gb so upgrade is not an easy decision! > > This is why we have slony, so you can slowly up

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Scott Marlowe
OK, in general you have to pay for MVCC one way or another. Many databases make you pay as you go, so to speak, by storing all the MVCC info in a log to be applied at some future date. Other databases you can pay later, by storing all the MVCC in the table itself. Both have similar costs, but on

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Scott Marlowe
On Fri, Nov 12, 2010 at 9:22 AM, Thom Brown wrote: > On 12 November 2010 16:14, Kyriacos Kyriacou > wrote: >> >> >> >> We are still using PostgreSQL 8.2.4. We are running a 24x7 system and >> database size is over 200Gb so upgrade is not an easy decision! >> >> I have it in my plans so in next fe

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Kyriacos Kyriacou
o:a...@squeakycode.net] > Sent: Friday, November 12, 2010 6:22 PM > To: Kyriacos Kyriacou > Cc: pgsql-performance@postgresql.org > Subject: Re: [PERFORM] MVCC performance issue > > On 11/12/2010 7:47 AM, Kyriacos Kyriacou wrote: > > > > SUGGESTION > > --

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Kyriacos Kyriacou
-performance@postgresql.org Subject: Re: [PERFORM] MVCC performance issue Ah, this is a very old version. If you can take advantage of a version with HOT support, you should be much, much happier. Cheers, Ken On Fri, Nov 12, 2010 at 06:14:00PM +0200, Kyriacos Kyriacou wrote: > > > We are st

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Tom Lane
"Kyriacos Kyriacou" writes: > We are still using PostgreSQL 8.2.4. In that case you don't have HOT updates, so it seems to me to be a little premature to be proposing a 100% rewrite of the system to fix your problems. regards, tom lane -- Sent via pgsql-performance mail

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Ben Chobot
On Nov 12, 2010, at 8:14 AM, Kyriacos Kyriacou wrote: > We are still using PostgreSQL 8.2.4. We are running a 24x7 system and > database size is over 200Gb so upgrade is not an easy decision! This is why we have slony, so you can slowly upgrade your 200Gb while you're live and then only suffer

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Thom Brown
On 12 November 2010 16:14, Kyriacos Kyriacou wrote: > > > We are still using PostgreSQL 8.2.4. We are running a 24x7 system and > database size is over 200Gb so upgrade is not an easy decision! > > I have it in my plans so in next few months I will setup new servers and > upgrade to version 9. > >

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Kenneth Marshall
Ah, this is a very old version. If you can take advantage of a version with HOT support, you should be much, much happier. Cheers, Ken On Fri, Nov 12, 2010 at 06:14:00PM +0200, Kyriacos Kyriacou wrote: > > > We are still using PostgreSQL 8.2.4. We are running a 24x7 system and > database size

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Andy Colson
On 11/12/2010 7:47 AM, Kyriacos Kyriacou wrote: SUGGESTION -- 1) When a raw UPDATE is performed, store all "new raw versions" either in separate temporary table space or in a reserved space at the end of each table (can be allocated dynamically) etc Your use of "raw" is confusi

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Kyriacos Kyriacou
: [PERFORM] MVCC performance issue 12.11.10 15:47, Kyriacos Kyriacou написав(ла): > PROBLEM DECRIPTION > -- > As an example, consider updating the "live" balance > of a customer for each phone call where the entire customer record has > to be duplicated again and

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Kyriacos Kyriacou
We are still using PostgreSQL 8.2.4. We are running a 24x7 system and database size is over 200Gb so upgrade is not an easy decision! I have it in my plans so in next few months I will setup new servers and upgrade to version 9. >> Which version of PostgreSQL are you basing this on? >> >>-

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Vitalii Tymchyshyn
12.11.10 15:47, Kyriacos Kyriacou написав(ла): PROBLEM DECRIPTION -- As an example, consider updating the "live" balance of a customer for each phone call where the entire customer record has to be duplicated again and again upon each call just for modifying a numeric value! H

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Kenneth Marshall
On Fri, Nov 12, 2010 at 07:34:36AM -0800, bricklen wrote: > On Fri, Nov 12, 2010 at 5:52 AM, Kenneth Marshall wrote: > > > > I cannot speak to your suggestion, but it sounds like you are not > > vacuuming enough and a lot of the bloat/randomization would be helped > > by making use of HOT updates

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread bricklen
On Fri, Nov 12, 2010 at 5:52 AM, Kenneth Marshall wrote: > > I cannot speak to your suggestion, but it sounds like you are not > vacuuming enough and a lot of the bloat/randomization would be helped > by making use of HOT updates in which the updates are all in the same > page and are reclaimed al

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Thom Brown
On 12 November 2010 13:47, Kyriacos Kyriacou wrote: > This is my first post in this mailing list and I would like to raise an > issue that in my opinion is causing performance issues of PostgreSQL > especially in a transaction processing environment. In my company we are > using PostgreSQL for the

Re: [PERFORM] MVCC performance issue

2010-11-12 Thread Kenneth Marshall
On Fri, Nov 12, 2010 at 03:47:30PM +0200, Kyriacos Kyriacou wrote: > This is my first post in this mailing list and I would like to raise an > issue that in my opinion is causing performance issues of PostgreSQL > especially in a transaction processing environment. In my company we are > using Post

[PERFORM] MVCC performance issue

2010-11-12 Thread Kyriacos Kyriacou
This is my first post in this mailing list and I would like to raise an issue that in my opinion is causing performance issues of PostgreSQL especially in a transaction processing environment. In my company we are using PostgreSQL for the last 8 year for our in-house developed billing system (telec