> From: j...@freebsd.org > To: freebsd-hackers@freebsd.org > Subject: Re: syncing large mmaped files > Date: Thu, 18 Oct 2012 09:39:34 -0400 > CC: kostik...@gmail.com; tris_v...@hotmail.com > > On Thursday, October 18, 2012 4:35:37 am Konstantin Belousov wrote: > > On Thu, Oct 18, 2012 at 10:08:22AM +1000, Tristan Verniquet wrote: > > > > > > I want to work with large (1-10G) files in memory but eventually sync > > > them back out to disk. The problem is that the sync process appears to > > > lock the file in kernel for the duration of the sync, which can run > > > into minutes. This prevents other processes from reading from the file > > > (unless they already have it mapped) for this whole time. Is there > > > any way to prevent this? I think I read in a post somewhere about > > > openbsd implementing partial-writes when it hits a file with lots of > > > dirty pages in order to prevent this. Is there anything available for > > > FreeBSD or is there another way around it? > > > > > No, currently the vnode lock is held exclusive for the whole duration > > of the msync(2) syscall or its analog from the syncer. > > > > Making a change to periodically drop the vnode lock in > > vm_object_page_clean() might be possible, but requires the benchmarking > > to make sure that we do not pessimize the common case. Also, this opens > > a possibility for the vnode reclamation meantime. > > You can simulate this in userland by breaking up your msync() into multiple > msync() calls where each call just syncs a portion of the file.
Thanks, I was doing this and I thought I was getting much worse performance from the msync over the fsync, however I am trying it again now and the difference doesn't seem as large as I first imagined. It is still taking about 4x as long for the case where all the pages are dirty but catches up when the file is more sparsely written. I guess that is probably acceptable. When all pages are dirty, iostat shows that the fsync will write 128KB/Transaction, whereas msync always does 16 KB/Transaction and a lower MB/s. It will continue to do this if I only dirty every 2nd, 3rd or 4th page. When I only dirty every 5th page the fsync seems to kick into another mode and starts doing 16KB/Transaction and the time starts becoming comparable to msync. Is there anyway to get that fsync 128K/Transaction performance increase when all pages are dirty with msync? > > Anyway, note that you cannot 'work with large files in memory', even if > > you have enough RAM and no pressure to hold all the file pages resident. > > The syncer will do a writeback periodically regardless of the application > > calling msync(2) or not, with the interval of approximately 30 seconds. > > You can mmap with MAP_NOSYNC to prevent the syncer from writing the file out > every 30 seconds. Yes, I was mapping MAP_NOSYNC. > -- > John Baldwin > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
