Re: [HACKERS] Vacuum/visibility is busted

2013-06-20 Thread Andres Freund
On 2013-06-19 15:01:44 -0700, Jeff Janes wrote: > On Thu, Feb 7, 2013 at 12:01 PM, Andres Freund wrote: > > > On 2013-02-07 11:15:46 -0800, Jeff Janes wrote: > > > > > > Does anyone have suggestions on how to hack the system to make it > > > fast-forward the current transaction id? It would certai

Re: [HACKERS] Vacuum/visibility is busted

2013-06-19 Thread Jeff Janes
On Thu, Feb 7, 2013 at 12:01 PM, Andres Freund wrote: > On 2013-02-07 11:15:46 -0800, Jeff Janes wrote: > > > > Does anyone have suggestions on how to hack the system to make it > > fast-forward the current transaction id? It would certainly make > > testing this kind of thing faster if I could ma

Re: [HACKERS] Vacuum/visibility is busted

2013-02-08 Thread Jeff Janes
On Thu, Feb 7, 2013 at 8:38 PM, Alvaro Herrera wrote: > Alvaro Herrera escribió: >> Alvaro Herrera escribió: >> >> > Hm, if the foreign key patch is to blame, this sounds like these tuples >> > had a different set of XMAX hint bits and a different Xmax, and they >> > were clobbered by something li

Re: [HACKERS] Vacuum/visibility is busted

2013-02-08 Thread Alvaro Herrera
Tom Lane escribió: > Alvaro Herrera writes: > > xid = HeapTupleHeaderGetRawXmax(tuple); > > ! if (((tuple->t_infomask & HEAP_XMAX_IS_MULTI) && > > !MultiXactIdIsValid(xid) && > > !MultiXactIdPrecedes(xid, cutoff_multi)) || > > ! ((!(tuple->t_infomask & HEAP_

Re: [HACKERS] Vacuum/visibility is busted

2013-02-08 Thread Alvaro Herrera
Pavan Deolasee escribió: > I'm trying to reason how this bug explains what we saw. In the test, > we'd left with duplicate tuples. If I just take index 219 in the table > as an example, that tuple had three duplicates. The tuple with CTID > (150, 126) had the index pointer and the rest two were da

Re: [HACKERS] Vacuum/visibility is busted

2013-02-08 Thread Pavan Deolasee
On Fri, Feb 8, 2013 at 10:08 AM, Alvaro Herrera wrote: > Alvaro Herrera escribió: >> Alvaro Herrera escribió: >> >> > Hm, if the foreign key patch is to blame, this sounds like these tuples >> > had a different set of XMAX hint bits and a different Xmax, and they >> > were clobbered by something l

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Tom Lane
Alvaro Herrera writes: > xid = HeapTupleHeaderGetRawXmax(tuple); > ! if (((tuple->t_infomask & HEAP_XMAX_IS_MULTI) && > ! MultiXactIdIsValid(xid) && > ! MultiXactIdPrecedes(xid, cutoff_multi)) || > ! ((!(tuple->t_infomask & HEAP_XMAX_IS_MULTI)) && >

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Alvaro Herrera escribió: > Alvaro Herrera escribió: > > > Hm, if the foreign key patch is to blame, this sounds like these tuples > > had a different set of XMAX hint bits and a different Xmax, and they > > were clobbered by something like vacuum or page pruning. > > Hm, I think heap_freeze_tuple

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Jeff Janes escribió: > On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee > wrote: > > > > Right. I don't have the database handy at this moment, but earlier in > > the day I ran some queries against it and found that most of the > > duplicates which are not accessible via indexes have xmin very clos

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Simon Riggs
On 7 February 2013 19:15, Jeff Janes wrote: > On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee > wrote: >> >> Right. I don't have the database handy at this moment, but earlier in >> the day I ran some queries against it and found that most of the >> duplicates which are not accessible via indexes

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Tom Lane
Jeff Janes writes: > Does anyone have suggestions on how to hack the system to make it > fast-forward the current transaction id? What I've generally done is to stop the server then use pg_resetxlog to put the XID counter where I want it. I believe you'll need to manually create a pg_clog file c

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Andres Freund
On 2013-02-07 11:15:46 -0800, Jeff Janes wrote: > On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee > wrote: > > > > Right. I don't have the database handy at this moment, but earlier in > > the day I ran some queries against it and found that most of the > > duplicates which are not accessible via

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 10:09 AM, Pavan Deolasee wrote: > > Right. I don't have the database handy at this moment, but earlier in > the day I ran some queries against it and found that most of the > duplicates which are not accessible via indexes have xmin very close > to 2100345903. In fact, many

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Pavan Deolasee
On Thu, Feb 7, 2013 at 10:48 PM, Jeff Janes wrote: > On Thu, Feb 7, 2013 at 1:44 AM, Pavan Deolasee > wrote: >> >> jjanes=# select *, xmin, xmax, ctid from foo where index IN (select >> index from foo group by index having count(*) > 1 ORDER by index) >> ORDER by index LIMIT 3; >> index | coun

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Jeff Janes escribió: > On Thu, Feb 7, 2013 at 12:55 AM, Pavan Deolasee > wrote: > I don't see the assertion failure myself. If I do REINDEX INDEX it > gives a duplicate key violation, and if I do REINDEX TABLE or REINDEX > DATABASE I get claimed success. > > This is using either current head (a

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 9:32 AM, Jeff Janes wrote: > On Thu, Feb 7, 2013 at 12:55 AM, Pavan Deolasee > wrote: >> >> Index scans do not return any duplicates and you need to force a seq >> scan to see them. Assuming that the page level VM bit might be >> corrupted, I tried to REINDEX the table to s

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 12:55 AM, Pavan Deolasee wrote: > On Thu, Feb 7, 2013 at 11:09 AM, Jeff Janes wrote: >> While stress testing Pavan's 2nd pass vacuum visibility patch, I realized >> that vacuum/visibility was busted. But it wasn't his patch that busted it. >> As far as I can tell, the bad

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Jeff Janes
On Thu, Feb 7, 2013 at 1:44 AM, Pavan Deolasee wrote: > On Thu, Feb 7, 2013 at 2:25 PM, Pavan Deolasee > wrote: > >> >> Will look more into it, but thought this might be useful for others to >> spot the problem. >> > > And here is some more forensic info about one of the pages having > duplicate

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Alvaro Herrera escribió: > Hm, if the foreign key patch is to blame, this sounds like these tuples > had a different set of XMAX hint bits and a different Xmax, and they > were clobbered by something like vacuum or page pruning. Hm, I think heap_freeze_tuple is busted, yes. -- Álvaro Herrera

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Alvaro Herrera
Pavan Deolasee escribió: > On Thu, Feb 7, 2013 at 2:25 PM, Pavan Deolasee > wrote: > > > > > Will look more into it, but thought this might be useful for others to > > spot the problem. > > > > And here is some more forensic info about one of the pages having > duplicate tuples. > > jjanes=# s

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Pavan Deolasee
On Thu, Feb 7, 2013 at 2:25 PM, Pavan Deolasee wrote: > > Will look more into it, but thought this might be useful for others to > spot the problem. > And here is some more forensic info about one of the pages having duplicate tuples. jjanes=# select *, xmin, xmax, ctid from foo where index IN

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Pavan Deolasee
On Thu, Feb 7, 2013 at 11:09 AM, Jeff Janes wrote: > While stress testing Pavan's 2nd pass vacuum visibility patch, I realized > that vacuum/visibility was busted. But it wasn't his patch that busted it. > As far as I can tell, the bad commit was in the range > 692079e5dcb331..168d3157032879 > >

Re: [HACKERS] Vacuum/visibility is busted

2013-02-07 Thread Simon Riggs
On 7 February 2013 05:39, Jeff Janes wrote: > While stress testing Pavan's 2nd pass vacuum visibility patch, I realized > that vacuum/visibility was busted. But it wasn't his patch that busted it. > As far as I can tell, the bad commit was in the range > 692079e5dcb331..168d3157032879 Please de

Re: [HACKERS] Vacuum/visibility is busted

2013-02-06 Thread Tom Lane
Jeff Janes writes: > While stress testing Pavan's 2nd pass vacuum visibility patch, I realized > that vacuum/visibility was busted. But it wasn't his patch that busted it. > As far as I can tell, the bad commit was in the > range 692079e5dcb331..168d3157032879 > Since a run takes 12 to 24 hours