On Wed, Jun 17, 2015 at 3:46 PM, Bert Huijben <b...@qqmail.nl> wrote:
> > > > -----Original Message----- > > From: stef...@apache.org [mailto:stef...@apache.org] > > Sent: woensdag 17 juni 2015 12:09 > > To: comm...@subversion.apache.org > > Subject: svn commit: r1685985 - in > /subversion/branches/fsx-1.10/subversion: > > include/private/svn_mutex.h libsvn_fs_x/batch_fsync.c > > libsvn_fs_x/batch_fsync.h libsvn_fs_x/fs.c libsvn_subr/mutex.c > > tests/libsvn_fs_x/fs-x-pack-test.c > > > > Author: stefan2 > > Date: Wed Jun 17 10:09:12 2015 > > New Revision: 1685985 > > > > URL: http://svn.apache.org/r1685985 > > Log: > > On the fsx-1.10 branch: > > Introduce a new infrastructure to FSX that allows us to do efficient > fsyncs. > > > > It basically uses a thread pool to execute multiple fsyncs concurrently. > > Interestingly, this generic implementation is faster on Linux than even > the > > POSIX-provided aio_fsync functionality on the same system. As a > centralized > > mechanism for scheduling fsyncs it also takes care of preventing > redundant > > flushes. > > > > With this commit, FSX does not actually use the new capabilities. That > will > > be in the following commits. > > Would it be possible to implement this on filehandles on Windows, instead > of just on filenames. > That is actually what I'm doing. It may not be obvious from the interface but the usage is such that files will be opened only once and fsync'ed through that handle. The only exception so far is the 'current' file which needs to be fsync'ed twice (before and after the rename). But I plan to eliminate the extra fsync even there. > Reopening a file that has just been closed is typically not fast on > Windows, as virusscanners and file indexers are often fighting to use the > same files. The next open operation then sometimes has to wait (when using > OPLOCKS) or has to be retried (when the other process opens the file in a > way that concurrent writes are denied). > I may not have answered your post when you wrote that a couple of weeks ago but I heard you and used it as input for the batch fsync design ;) > All of this can be avoided by just flushing the filehandle we already had > when the file was open for writing. By closing and re-opening we are doing > a lot of unneeded work and add a lot of potential race conditions. > One key design element is that the batch fsync container owns the open files and will open them only once. So, it can not only guarantee that we don't need to reopen files for fsync but it can even prevent reopening files in cases where different functions were to open & close the same as part of some bigger functionality. That said, this is all very much work-in-progress and the final logic may be quite different again. -- Stefan^2.
