Re: Efficient and effective fsync during commit

2015-05-29 Thread Branko Čibej
On 29.05.2015 18:23, Ivan Zhakov wrote: > On 29 May 2015 at 18:55, Stefan Fuhrmann wrote: >> You might be right. So, if you care about repository >> integrity, you should use your MSDN subscription and >> ask MS for clarification on FlushFileBuffers() behaviour. > You also may request MSDN subscri

Re: Efficient and effective fsync during commit

2015-05-29 Thread Branko Čibej
On 29.05.2015 17:55, Stefan Fuhrmann wrote: > If you assume / suspect that FlushFileBuffers() only operates on the > open handle, i.e. only flushes those changes made through that thandle, >From my dabbling with the Windows I/O stack and filesystems way back, I'd say that flushing (and all other o

Re: [RFC] new svn_client_conflict API

2015-05-29 Thread Julian Foad
Hi, Stefan. Glad to see this! Changing the API syntactic form to an opaque struct + accessor functions sounds fine, as I mentioned on IRC. I see you've done that already. As for the semantic changes, I want to help clarify our understanding of the high level 'shape' of merge scenarios. We need cl

Re: Efficient and effective fsync during commit

2015-05-29 Thread Ivan Zhakov
On 29 May 2015 at 18:55, Stefan Fuhrmann wrote: > On Fri, May 29, 2015 at 4:14 PM, Ivan Zhakov wrote: >> On 28 May 2015 at 20:47, Stefan Fuhrmann >> wrote: >>> Hi all, >>> >>> Most of us would agree that way we fsync FS changes >>> in FSFS and FSX is slow (~10 commits / sec on a SSD, >>> YMMV)

Re: Efficient and effective fsync during commit

2015-05-29 Thread Stefan Fuhrmann
On Fri, May 29, 2015 at 4:14 PM, Ivan Zhakov wrote: > On 28 May 2015 at 20:47, Stefan Fuhrmann wrote: >> Hi all, >> >> Most of us would agree that way we fsync FS changes >> in FSFS and FSX is slow (~10 commits / sec on a SSD, >> YMMV) and not even all changes are fully fsync'ed >> (repo creation

Re: svn commit: r1682265 - /subversion/trunk/subversion/libsvn_fs_fs/util.c

2015-05-29 Thread Evgeny Kotkov
Stefan Fuhrmann writes: > However, it does not tell you anything about consistency with outside > parties, say some svnsync'ed repository. The problem is that Windows may > end up not persisting the rename (of e.g. the 'current' file) after telling > the client that the commit got through and a n

Re: Efficient and effective fsync during commit

2015-05-29 Thread Ivan Zhakov
On 28 May 2015 at 20:47, Stefan Fuhrmann wrote: > Hi all, > > Most of us would agree that way we fsync FS changes > in FSFS and FSX is slow (~10 commits / sec on a SSD, > YMMV) and not even all changes are fully fsync'ed > (repo creation, upgrade). > The first question is it really a problem? I me

Re: svn commit: r1682265 - /subversion/trunk/subversion/libsvn_fs_fs/util.c

2015-05-29 Thread Ivan Zhakov
On 28 May 2015 at 20:50, Bert Huijben wrote: >> -Original Message- >> From: Philip Martin [mailto:philip.mar...@wandisco.com] >> Sent: donderdag 28 mei 2015 19:34 >> To: Ivan Zhakov >> Cc: Stefan Fuhrmann; Subversion Development; Stefan Fuhrman >> Subject: Re: svn commit: r1682265 - >> /su

Re: svn commit: r1682265 - /subversion/trunk/subversion/libsvn_fs_fs/util.c

2015-05-29 Thread Ivan Zhakov
On 28 May 2015 at 20:33, Philip Martin wrote: > Ivan Zhakov writes: > >> I meant to add platform specific code to svn_io_file_rename() to also >> fail with EXDEV on Windows for cross-volume copies. > > Will that go as far as supporting all the MoveFile, MoveFileW, > MoveFileEx, MoveFileExW varian

Re: svn commit: r1682265 - /subversion/trunk/subversion/libsvn_fs_fs/util.c

2015-05-29 Thread Ivan Zhakov
On 28 May 2015 at 23:55, Bert Huijben wrote: > In this case the log message documented that it fixed ‘a problem on > Windows’, which would suggest at least some Windows testing > [[ > Correctly fsync() after renames in FSFS on Win32. We must flush the disk > buffers after the rename, otherwise th

RE: [RFC] new svn_client_conflict API - Properties

2015-05-29 Thread Bert Huijben
> -Original Message- > From: Stefan Sperling [mailto:s...@elego.de] > Sent: vrijdag 29 mei 2015 13:16 > To: dev@subversion.apache.org > Subject: [RFC] new svn_client_conflict API > > Another sore point is property conflicts. There's still no way in our API > to resolve individual prope

[RFC] new svn_client_conflict API

2015-05-29 Thread Stefan Sperling
I would like to start transitioning away from svn_wc_conflict_description2_t in public APIs and introduce a higher-level API to eventually replace it. My end goal is to use an opaque type with methods that make it possible to implement conflict resolution in a way that is easier to use. For exampl

Re: Efficient and effective fsync during commit

2015-05-29 Thread Philip Martin
Stefan Fuhrmann writes: > On Thu, May 28, 2015 at 9:54 PM, Philip Martin > wrote: >> >> fsync() works on file descriptors rather than files, do we need to keep >> the original file descriptors open in order to fsync()? > > We could b/c there are at most 7 (4 files, 3 folders) of them for a > FSF

Re: Populating the rep-cache

2015-05-29 Thread Johan Corveleyn
On Thu, May 28, 2015 at 6:00 PM, Stefan Fuhrmann wrote: > On Wed, May 27, 2015 at 8:14 PM, Philip Martin > wrote: >> >> Julian Foad writes: >> >> > Stefan Fuhrmann wrote: >> >> * clear the rep-cache.db >> > >> > Clearing the cache and continuing operation may make subsequent >> > commits much la

Re: Queries about rep cache: get_shared_rep()

2015-05-29 Thread Stefan Fuhrmann
On Wed, May 27, 2015 at 6:35 PM, Julian Foad wrote: > Stefan Fuhrmann wrote: >> Alright. I gave it a bit more thought now. >> >> Whenever we encounter this mismatch, something pretty >> bad likely happened to the repo - such as a failed restore >> attempt. In turn, we can expect those situations t

Re: Efficient and effective fsync during commit

2015-05-29 Thread Stefan Fuhrmann
On Thu, May 28, 2015 at 9:54 PM, Philip Martin wrote: > Stefan Fuhrmann writes: > >> In the future, we should implement step 1 as simple >> non-fsync'ing file operations. Then explicitly sync every >> file, and on POSIX the folders, once. Step 2 does not >> have any atomicity requirements. Finall