On Fri, Jul 14, 2017 at 1:18 PM, Evgeny Kotkov <evgeny.kot...@visualsvn.com> wrote: > Paul Hammant <p...@hammant.org> writes: > >> 1. compression-exempt-suffixes = mp3,mp4,jpeg >> >> 2. deltification-exempt-suffixes = mp3,mp4,jpeg >> >> Regardless of the setting of 'compression-level', #1 above two mean certain >> things can skip the compression attempt. It must give up at a certain point >> right? > > Hi everyone, > > To improve the situation with slow commits of large binary and, possibly, > incompressible files I committed a patch (http://svn.apache.org/r1801940) > that adds initial support for LZ4 compression in the backend. While still > providing a decent compression ratio, LZ4 offers much faster compression > even than zlib with level=1, and can skip incompressible data chunks. > (Presumably, LZ4 is used for on-the-fly compression in different file > systems for these reasons.) > > I have seen significant (up to 3 times) speed improvement for svn import > and commit with large binary files. Sometimes, using LZ4 compression can > even outperform a configuration with disabled compression, if the file is > at least somehow compressible, as it reduces the time required to write > the data to disk. > > Currently, LZ4 compression is enabled if the fsfs.conf file specifies > compression-level=1, and all other levels still use zlib for compression. > Right now, the support for LZ4 is only implemented for the file:// protocol, > but support/negotiation for other protocols can be added later.
Nice! But how about deltification? Has anyone tried / benchmarked the effect of turning off deltification (with or without compression), to see what the effect would be on the commit time? Like I suggested in this thread yesterday (i.e. set max-deltification-walk to 0 in fsfs.conf -- or perhaps play with both max-deltification-walk and max-linear-deltification) ... -- Johan
