hm... could it be that the archiver for some reason is looking at rolled back blocks?
to ron: are you using temporal snapshots? i'v seen the same symptom but hadnt the nerves to look deeper into it... just reformated the fossil from last score and disabled snapshots and snaptime. :( it happend while i dumped the blocks from another fossils venti to the same venti that the failing fossil was archiving to... with snapshots enabled. have no idea if this is relevant... keep on debugging! -- cinap
--- Begin Message ---I've been seeing corruption of fossil /archive data, at a rate of about once a week. The symptom is that venti dir entries for /archive copies of some frequently referenced directories in my main fossil fs (generally it's /usr, /usr/miller or /mail/box/miller) contain incorrect venti scores. Sometimes the score is for a nonexistent block, and sometimes it's a valid score but for a different block (e.g. a data or dir block when a meta block is expected). Almost always it's the dir entry for a metadata block which gets a bad score, but occasionally it's the dir entry for a sub-dir block. This morning both the sub-dir and meta entries for one directory were written with the scores of the corresponding entries for a different directory: term% ls -lqd /n/dump/2009/0408/mail /n/dump/2009/0408/usr/miller/disk (0000000000001646 9 80) d-rwxrwxr-x M 36 upas upas 0 Jan 1 2007 /n/dump/2009/0408/mail (000000000000694e 23 80) d-rwxr-xr-x M 36 miller miller 0 Dec 31 2003 /n/dump/2009/0408/usr/miller/disk term% ls /n/dump/2009/0408/mail /n/dump/2009/0408/usr/miller/disk /n/dump/2009/0408/mail/fdisk.c /n/dump/2009/0408/mail/fdisk.c_ok /n/dump/2009/0408/mail/fdisk.c_try /n/dump/2009/0408/mail/mkfs.c /n/dump/2009/0408/usr/miller/disk/fdisk.c /n/dump/2009/0408/usr/miller/disk/fdisk.c_ok /n/dump/2009/0408/usr/miller/disk/fdisk.c_try /n/dump/2009/0408/usr/miller/disk/mkfs.c term% diff -r /n/dump/2009/0408/mail /n/dump/2009/0408/usr/miller/disk term% Here are the pairs of VtEntry structures for the two files as retrieved from venti: /n/dump/2009/0408/mail: 0000000 00000000 1ff41fe0 03000000 00000000 0000010 00000280 0830e9ae 3023dce8 3ddf6138 0000020 4fe87d59 7257e3b7 00000000 1ff42000 0000030 01000000 00000000 0000039f 21ee2227 0000040 695a8ea2 fbbd136e dcb435e0 64fbcdfb 0000050 /n/dump/2009/0408/usr/miller/disk: 0000000 00000000 1ff41fe0 03000000 00000000 0000010 000000a0 0830e9ae 3023dce8 3ddf6138 0000020 4fe87d59 7257e3b7 00000000 1ff42000 0000030 01000000 00000000 00000296 21ee2227 0000040 695a8ea2 fbbd136e dcb435e0 64fbcdfb 0000050 Note that it's just the score fields for /mail which are wrong; the size fields (280 and 39f) are correct, matching yesterday's dump: /n/dump/2009/0407/mail: 0000000 00000000 1ff41fe0 03000000 00000000 0000010 00000280 ... So, what's going on? My intuition says a fossil bug - I can't think of any disk hardware error which would lead to this kind of corruption. I have of course studied /sys/src/cmd/fossil/archive.c looking for races (my fossil+venti machine has two processors), but everything appears to be protected by locks. I would be interested to know if anyone else's archive is being quietly messed up in this way - you wouldn't necessarily know until you tried something like a history(1) command and found pieces missing. This is a quick test which may show if you have a similar problem: cd /n/dump/2009 for (i in *) { test -d $i$home/tmp || ls -d $i$home/tmp } for (i in *) { test -f $i/mail/box/$user/mbox || ls $i/mail/box/$user/mbox } In a post a while ago, Russ said > The amazing thing to me about fossil is how indestructable > it is when used with venti. > ... Once you see the data in the archive > tree, you can be very sure it's not going away. I agree with this, but I'd like a way to be reassured that my daily data has actually gone into the archive correctly. -- Richard
--- End Message ---
