I should have sent this to the mailing list as well as David. Google has fixed something that wasn't broke -- gmail. They've introduced a new UI that I haven't gotten used to yet ...
---------- Forwarded message ---------- From: Donald Allen <donaldcal...@gmail.com> Date: Sun, Dec 11, 2011 at 10:23 AM Subject: Re: Lost file-system story To: David Holland <dholland-t...@netbsd.org> On Sun, Dec 11, 2011 at 8:57 AM, David Holland <dholland-t...@netbsd.org> wrote: > On Tue, Dec 06, 2011 at 11:58:25AM -0500, Thor Lancelot Simon wrote: > > With the filesystem mounted async *nothing* pushes out most > > metadata updates, > > If this is really true, it's a bug and should be fixed. It may very well be true. I just did the following: I brought up my test laptop, running 5.1 GENERIC, with /home mounted async,noatime. I created a new file in my home directory. I should note that when I ZZ'ed out of vi, the disk light flashed momentarily, and I could hear the disk doing something. I did an ls -lt | head and the new file was there. I waited just under a minute (to let syncs happen; this is longer than any of the sysctl vfs.sync.delays, which I assume are in seconds; the man page doesn't say) and then I pulled the plug (no battery in the machine). On restart, I got no fsck errors, but the new file was not in my home directory. I then repeated this test, waiting a little over a minute this time. Same result, the new file was gone (this time I got fsck errors). Then I did the test a third time, but this time I did a sync before pulling the plug. On restart, I still got some fsck errors that were fixed, but the new file was present. This does suggest that the meta-data is not being written, at least within a minute or so of creating a new file. One thing I think we have not discussed much or at all in this thread is that the filesystem surviving a crash and how much data you lose when it does survive are separate issues. The experiments I did yesterday demonstrate that a NetBSD ffs async-mounted filesystem, together with its fsck, *can* survive a crash in bad circumstances -- lots of write activity at the time of the crash. We don't know what the probability of survival is, just that it's > 0. What I did yesterday also does not address loss of data. If Simon is correct and NetBSD is just not writing metadata until sync is explicitly called, then you could have a system up for days or weeks and lose as many as all of the files created in an async filesystem since the last re-boot. We don't know definitively what it's doing yet, but I think I've demonstrated that it's not writing meta-data within one minute windows. I will do some more playing with this, waiting longer and will report what I find. We also know from this morning's tests that a user-called sync does get the meta-data written, reducing the amount of data lost in a crash that the filesystem survives. So those who advocated periodically calling sync in a loop (Christos first suggested this to me in a private email) are correct -- it's necessary if you are going to use async mounting. More later ... /Don > > -- > David A. Holland > dholl...@netbsd.org
