Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Greg Smith
On Thu, 26 Jun 2008, Craig Ringer wrote: I'd be interested to have this confirmed, as I don't think I've seen it documented anywhere. Is it a side-effect/benefit of HOT somehow? The documentation is in README.HOT, for example: http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Greg Smith
On Thu, 26 Jun 2008, Holger Hoffstaette wrote: How do large databases treat mass updates? AFAIK both DB2 and Oracle use MVCC (maybe a different kind?) as well An intro to the other approaches used by Oracle and DB2 (not MVCC) is at http://wiki.postgresql.org/wiki/Why_PostgreSQL_Instead_of_MyS

[PERFORM] Re: [PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread Scott Marlowe
2008/6/26 Tom Lane <[EMAIL PROTECTED]>: > "jay" <[EMAIL PROTECTED]> writes: >> I know the problem, because there are about 35 million rows , which >> cost about 12G disk space and checkpoint segments use 64, but update >> operation is in one transaction which lead fast fill up the checkpoint

[PERFORM] Re: [PERFORM] Re: [PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread Scott Marlowe
2008/6/26 Pavan Deolasee <[EMAIL PROTECTED]>: > 2008/6/26 jay <[EMAIL PROTECTED]>: > >>If we can do commit very 1000 row per round, it may resolve the >> problem. >> But PG not support transaction within function yet? >> > > Yeah, transaction control is not supported inside functions. Ther

Re: [PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread Tom Lane
"jay" <[EMAIL PROTECTED]> writes: > I know the problem, because there are about 35 million rows , which > cost about 12G disk space and checkpoint segments use 64, but update > operation is in one transaction which lead fast fill up the checkpoint > segments and lead do checkpoints frequently

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Mark Mielke
Holger Hoffstaette wrote: Hi - I have been following this thread and find some of the recommendations really surprising. I understand that MVCC necessarily creates overhead, in-place updates would not be safe against crashes etc. but have a hard time believing that this is such a huge problem fo

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Andrew Sullivan
On Thu, Jun 26, 2008 at 09:16:25PM +0800, Craig Ringer wrote: > I think Pg already does in place updates, or close, if the tuples being > replaced aren't referenced by any in-flight transaction. I noticed a while > ago that if I'm doing bulk load/update work, if there aren't any other > transac

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Andrew Sullivan
On Thu, Jun 26, 2008 at 02:40:59PM +0200, Holger Hoffstaette wrote: > large databases treat mass updates? AFAIK both DB2 and Oracle use MVCC > (maybe a different kind?) as well, but I cannot believe that large updates > still pose such big problems. DB2 does not use MVCC. This is why lock escala

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Craig Ringer
Holger Hoffstaette wrote: Hi - I have been following this thread and find some of the recommendations really surprising. I understand that MVCC necessarily creates overhead, in-place updates would not be safe against crashes etc. but have a hard time believing that this is such a huge problem fo

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Holger Hoffstaette
Hi - I have been following this thread and find some of the recommendations really surprising. I understand that MVCC necessarily creates overhead, in-place updates would not be safe against crashes etc. but have a hard time believing that this is such a huge problem for RDBMS in 2008. How do lar

Re: ??: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread Heikki Linnakangas
jay wrote: I know the problem, because there are about 35 million rows , which cost about 12G disk space and checkpoint segments use 64, but update operation is in one transaction which lead fast fill up the checkpoint segments and lead do checkpoints frequently, but checkpoints will cost

[PERFORM] Re: [PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread Pavan Deolasee
2008/6/26 jay <[EMAIL PROTECTED]>: >If we can do commit very 1000 row per round, it may resolve the > problem. > But PG not support transaction within function yet? > Yeah, transaction control is not supported inside functions. There are some hacks using dblink to do transactions inside

[PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-26 Thread jay
ort transaction within function yet? -邮件原件- 发件人: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] 代表 Heikki Linnakangas 发送时间: 2008年6月25日 18:11 收件人: jay 抄送: pgsql-performance@postgresql.org 主题: Re: [PERFORM] Postgresql update op is very very slow jay wrote: > I've a table with abou

Re: [PERFORM] Postgresql update op is very very slow

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, jay wrote: Why postgresql is so slowly? Is the PG MVCC problem? Update is extremely intensive not just because of MVCC, but because a new version of all the rows are being written out. This forces both lots of database commits and lots of complicated disk I/O to accompl

Re: [PERFORM] Postgresql update op is very very slow

2008-06-25 Thread Heikki Linnakangas
jay wrote: I've a table with about 34601755 rows ,when I execute 'update msg_table set type=0;' is very very slow, cost several hours, but still not complete? Why postgresql is so slowly? Is the PG MVCC problem? Possibly. Because of MVCC, a full-table update will actually create a new versio

[PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-25 Thread jay
Conover 发送时间: 2008年6月25日 13:02 收件人: jay 抄送: pgsql-performance@postgresql.org 主题: Re: [PERFORM] Postgresql update op is very very slow On Jun 24, 2008, at 9:12 PM, jay wrote: > > I've a table with about 34601755 rows ,when I execute 'update > msg_table set >

Re: [PERFORM] Postgresql update op is very very slow

2008-06-24 Thread Rusty Conover
On Jun 24, 2008, at 9:12 PM, jay wrote: I've a table with about 34601755 rows ,when I execute 'update msg_table set type=0;' is very very slow, cost several hours, but still not complete? Why postgresql is so slowly? Is the PG MVCC problem? But I try it on Mysql, the same table and rows

[PERFORM] Postgresql update op is very very slow

2008-06-24 Thread jay
I've a table with about 34601755 rows ,when I execute 'update msg_table set type=0;' is very very slow, cost several hours, but still not complete? Why postgresql is so slowly? Is the PG MVCC problem? But I try it on Mysql, the same table and rows, it only cost about 340 seconds. Any idea for