On Tue, Jun 14, 2016 at 4:02 AM, Robert Haas <robertmh...@gmail.com> wrote:
> On Sat, Jun 11, 2016 at 5:00 PM, Robert Haas <robertmh...@gmail.com> wrote:
>>> How about changing the return tuple of heap_prepare_freeze_tuple to
>>> a bitmap? Two flags: "Freeze [not] done" and "[No] more freezing
>>> needed"
>>
>> Yes, I think something like that sounds about right.
>
> Here's a patch. I took the approach of adding a separate bool out
> parameter instead. I am also attaching an update of the
> check-visibility patch which responds to assorted review comments and
> adjusting it for the problems found on Friday which could otherwise
> lead to false positives. I'm still getting occasional TIDs from the
> pg_check_visible() function during pgbench runs, though, so evidently
> not all is well with the world.
I'm still working out how half this stuff works, but I managed to get
pg_check_visible() to spit out a row every few seconds with the
following brute force approach:
CREATE TABLE foo (n int);
INSERT INTO foo SELECT generate_series(1, 100000);
Three client threads (see attached script):
1. Run VACUUM in a tight loop.
2. Run UPDATE foo SET n = n + 1 in a tight loop.
3. Run SELECT pg_check_visible('foo'::regclass) in a tight loop, and
print out any rows it produces.
I noticed that the tuples that it reported were always offset 1 in a
page, and that the page always had a maxoff over a couple of hundred,
and that we called record_corrupt_item because VM_ALL_VISIBLE returned
true but HeapTupleSatisfiesVacuum on the first tuple returned
HEAPTUPLE_DELETE_IN_PROGRESS instead of the expected HEAPTUPLE_LIVE.
It did that because HEAP_XMAX_COMMITTED was not set and
TransactionIdIsInProgress returned true for xmax.
Not sure how much of this was already obvious! I will poke at it some
more tomorrow.
--
Thomas Munro
http://www.enterprisedb.com
# create table foo (n int);
# insert into foo select generate_series(1, 100000);
import psycopg2
import threading
import time
keep_running = True
def vacuum_thread(dsn):
conn = psycopg2.connect(dsn)
conn.set_isolation_level(0)
cursor = conn.cursor()
while keep_running:
cursor.execute("VACUUM")
def update_thread(dsn):
conn = psycopg2.connect(dsn)
cursor = conn.cursor()
while keep_running:
cursor.execute("UPDATE foo SET n = n + 1")
conn.commit()
def check_thread(dsn):
conn = psycopg2.connect(dsn)
conn.set_isolation_level(0)
cursor = conn.cursor()
while keep_running:
cursor.execute("SELECT pg_check_visible('foo'::regclass)")
for row in cursor:
print row
conn.rollback()
if __name__ == "__main__":
dsn = "dbname=postgres"
t1 = threading.Thread(target = vacuum_thread, args = [dsn])
t1.start()
t2 = threading.Thread(target = update_thread, args = [dsn])
t2.start()
t3 = threading.Thread(target = check_thread, args = [dsn])
t3.start()
try:
time.sleep(86400)
except KeyboardInterrupt:
keep_running = False
t1.join()
t2.join()
t3.join()
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
