On 13 April 2014 08:40, <stef...@apache.org> wrote: > Author: stefan2 > Date: Sun Apr 13 04:40:40 2014 > New Revision: 1586922 > > URL: http://svn.apache.org/r1586922 > Log: > Speed up file / stream comparison, i.e. minimize the processing overhead > for finding the first mismatch. > > The approach is two-sided. Instead of fetching SVN__STREAM_CHUNK_SIZE > from all sources before comparing data, we start with a much lower 4kB > and increase the chunk size until we reach SVN__STREAM_CHUNK_SIZE while > making sure that all reads are naturally aligned. So, we quickly find > mismatches near the beginning of the file. > > On the other end side, we bump the SVN__STREAM_CHUNK_SIZE to 64kB which > gives better throughput for longer distances to the first mismatch - > without causing ill effects in APR's memory management. > > * subversion/include/svn_types.h > (SVN__STREAM_CHUNK_SIZE): Bump to 64k and add some documentation on > the general restrictions for future changes. > Hi Stefan,
You effectively reverted my recent fix (r1581296) for high memory usage with many repositories open at the same time (about 500k per repository. Also please consider r857671 and discussion before that commit: http://svn.haxx.se/dev/archive-2004-11/0123.shtml So please revert. > * subversion/include/private/svn_io_private.h > (svn_io__next_chunk_size): New utility function generating the new > read block size sequence. > > * subversion/libsvn_subr/io.c > (svn_io__next_chunk_size): Implement. > (contents_identical_p, > contents_three_identical_p): Let the new utility determine the read > block size. > > * subversion/libsvn_subr/stream.c > (svn_stream_contents_same2): Ditto. > This optimization makes sense only for FSFS case where fetching text/property content is expensive, but it doesn't makes sense for comparing on disk files. So could you please make this optimization FSFS specific, without affecting generic API and behavior. Probably it better to make just smaller chunk size for FSFS case btw. Thanks in advance. -- Ivan Zhakov CTO | VisualSVN | http://www.visualsvn.com
