Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-24 Thread Tom Lane
Andrew Gierth writes: > "Greg" == Greg Stark writes: > Greg> Either of two things are true. > Greg> Either transaction 6179 committed, > [snip] > This is all missing the point. The row should have been killed by > transaction 4971, NOT 6179. By the time transaction 6179 tried to > do anything

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-23 Thread Andrew Gierth
> "Greg" == Greg Stark writes: Greg> Either of two things are true. Greg> Either transaction 6179 committed, [snip] This is all missing the point. The row should have been killed by transaction 4971, NOT 6179. By the time transaction 6179 tried to do anything with it, it was almost certain

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-23 Thread Radoslaw Zielinski
Greg Stark [2009-08-23 03:44]: > 2009/8/22 Radoslaw Zielinski : >>  $ seq 11 | xargs -ti psql bug -c "select {}, (x).* from (select >> bt_page_items('promocje_pkey',{}) as x ) as y" >>  [...] >>  psql bug -c select 11, (x).* from (select bt_page_items('promocje_pkey',11) >> as x ) as y >>  ERROR

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-23 Thread Radoslaw Zielinski
Greg Stark [2009-08-23 03:40]: > 2009/8/22 Radoslaw Zielinski : >>  bug=# select length(opis_szczeg) from promocje where id = 4300; >>  ERROR:  missing chunk number 0 for toast value 120741 in pg_toast_29644 > Sorry, what datatype is this again? And what encoding? text, UTF-8 (locale C). > Perha

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-22 Thread Greg Stark
2009/8/22 Radoslaw Zielinski : >  $ seq 11 | xargs -ti psql bug -c "select {}, (x).* from (select > bt_page_items('promocje_pkey',{}) as x ) as y" >  [...] >  psql bug -c select 11, (x).* from (select bt_page_items('promocje_pkey',11) > as x ) as y >  ERROR:  block number out of range Sorry, I f

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-22 Thread Greg Stark
2009/8/22 Radoslaw Zielinski : >  bug=# select length(opis_szczeg) from promocje where id = 4300; >  ERROR:  missing chunk number 0 for toast value 120741 in pg_toast_29644 Sorry, what datatype is this again? And what encoding? Perhaps I should have said octet_length() instead of length(). --

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-22 Thread Radoslaw Zielinski
Tom Lane [2009-08-21 20:35]: > Radoslaw Zielinski writes: >> Greg Stark [2009-08-21 18:55]: >>> Is this, perchance, new hardware? Did you test the memory in it? >> It's a "virtual private server"; the hosting provider is swearing >> everything's fine. I can't vouch for it myself, obviously. >

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-22 Thread Radoslaw Zielinski
Greg Stark [2009-08-21 20:30]: [...] > Actually I mean the key for the toast table. > Let me ask firstly do you get anything if you just do select * from > pg_toast.pg_toast_29644 where chunk_id = 120741 ? 0 rows. > And secondly, what do you get if you do "select length(htmlblob) from > tab whe

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Tom Lane
Radoslaw Zielinski writes: > Greg Stark [2009-08-21 18:55]: >> Is this, perchance, new hardware? Did you test the memory in it? > It's a "virtual private server"; the hosting provider is swearing > everything's fine. I can't vouch for it myself, obviously. Hmmm ... have you had any crashes? I

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Greg Stark
2009/8/21 Radoslaw Zielinski : > Greg Stark [2009-08-21 18:38]: > [...] >> the toast table. Not unless you can dump the whole index and find >> pointers in there or can find the details in the wal log. > > You mean the primary key index?  How do I dump it? Actually I mean the key for the toast ta

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Radoslaw Zielinski
Greg Stark [2009-08-21 18:38]: [...] > the toast table. Not unless you can dump the whole index and find > pointers in there or can find the details in the wal log. You mean the primary key index? How do I dump it? -- Radosław Zieliński pgpdIiosOzpYQ.pgp Description: PGP signature

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Radoslaw Zielinski
Greg Stark [2009-08-21 18:55]: [...] > Is this, perchance, new hardware? Did you test the memory in it? It's a "virtual private server"; the hosting provider is swearing everything's fine. I can't vouch for it myself, obviously. > Do you have other records that have "disappeared"? I haven't fo

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Greg Stark
2009/8/21 Radoslaw Zielinski : > Greg Stark [2009-08-21 18:03]: >> On Fri, Aug 21, 2009 at 4:04 PM, Radoslaw Zielinski >> wrote: >>> This particular row currently has last_modified="2009-08-11", but table_log >>> says it has been updated on "2009-08-12" -- so, data loss. > >> These are all very l

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Radoslaw Zielinski
Greg Stark [2009-08-21 18:12]: [...] > Or alternatively do you have all the wal logs and could you run > xlogdump on them looking for transaction id 6179? Nope, I don't have the wal logs. -- Radosław Zieliński pgpO5wPiNs71J.pgp Description: PGP signature

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Radoslaw Zielinski
Greg Stark [2009-08-21 18:03]: > On Fri, Aug 21, 2009 at 4:04 PM, Radoslaw Zielinski > wrote: >> This particular row currently has last_modified="2009-08-11", but table_log >> says it has been updated on "2009-08-12" -- so, data loss. > These are all very low transaction ids. you say it was a we

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Greg Stark
On Fri, Aug 21, 2009 at 5:15 PM, Alvaro Herrera wrote: > I wonder if this could be explained by xid=6179 not marked as committed > in clog.  I'd try flipping that bit and see what happens ... Well nothing's going to help much now. Firstly, once the hint bit gets set nothing second-guesses that and

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Tom Lane
Alvaro Herrera writes: > I wonder if this could be explained by xid=6179 not marked as committed > in clog. I'd try flipping that bit and see what happens ... Once the row's been marked XMAX_INVALID, changing clog won't change that marking ... regards, tom lane -- Sent

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Alvaro Herrera
Greg Stark wrote: > It looks like the row *was* updated by transaction 6179 and the new > version was stored in line pointer 12. However it's marked > XMAX_INVALID which means at least somebody at some point thought 6179 > had aborted and marked that hint bit. Hmm, but LP 12 shouldn't have len=0

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Alvaro Herrera
Radoslaw Zielinski wrote: > Hello, > > 8.4.0 installation on i686 modern Linux, a simple production app with ~4 > users entering data; normalised, no weird stuff happening there. Recently: > > ERROR: missing chunk number 0 for toast value 120741 in pg_toast_29644 > bug=# select id, last_modi

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Greg Stark
On Fri, Aug 21, 2009 at 5:03 PM, Greg Stark wrote: > On Fri, Aug 21, 2009 at 4:04 PM, Radoslaw Zielinski > wrote: > > Here's a dump of that page btw. ... [crap word-wrapped by gmail] Sorry, clearly gmail sucks for posting things that you actually care about how they're formatted. I've attached th

Re: [BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Greg Stark
On Fri, Aug 21, 2009 at 4:04 PM, Radoslaw Zielinski wrote: > This particular row currently has last_modified="2009-08-11", but table_log > says it has been updated on "2009-08-12" -- so, data loss. These are all very low transaction ids. you say it was a week of production before this happened?

[BUGS] 8.4.0 data loss / HOT-related bug

2009-08-21 Thread Radoslaw Zielinski
Hello, 8.4.0 installation on i686 modern Linux, a simple production app with ~4 users entering data; normalised, no weird stuff happening there. Recently: ERROR: missing chunk number 0 for toast value 120741 in pg_toast_29644 This table has table_log and a "before update" trigger setting "NE