Philip Martin wrote:
Julian Foad writes:
Can someone file an issue in the tracker please. I need to refer to it
in discussion with potential rc1 trials.
https://issues.apache.org/jira/browse/SVN-4723
Thanks for that and SVN-4722.
- Julian
Julian Foad writes:
> Can someone file an issue in the tracker please. I need to refer to it
> in discussion with potential rc1 trials.
https://issues.apache.org/jira/browse/SVN-4723
--
Philip
Can someone file an issue in the tracker please. I need to refer to it
in discussion with potential rc1 trials.
- Julian
On Sat, Mar 03, 2018 at 06:28:48PM +, Philip Martin wrote:
> Branko Čibej writes:
>
> > In other words ... if we wanted to make authz changes have immediate
> > effect, we'd need a better (faster, or at least non-blocking) way to
> > determine that the rules changed than reading the authz fil
Branko Čibej writes:
> In other words ... if we wanted to make authz changes have immediate
> effect, we'd need a better (faster, or at least non-blocking) way to
> determine that the rules changed than reading the authz file, even if
Relatively easy to do for in-repository authz. We could make
On 03.03.2018 19:15, James McCoy wrote:
> On Sat, Mar 03, 2018 at 06:54:06PM +0100, Branko Čibej wrote:
>> P.S.: Running tests now with the patched 1.10.x, will vote on the
>> backport as soon as that's done. If it's approved, I believe we have to
>> move our expected release date from 28th March t
On Sat, Mar 03, 2018 at 06:54:06PM +0100, Branko Čibej wrote:
> P.S.: Running tests now with the patched 1.10.x, will vote on the
> backport as soon as that's done. If it's approved, I believe we have to
> move our expected release date from 28th March to 4th April?
According to the flowchart post
On 03.03.2018 18:46, Philip Martin wrote:
> Branko Čibej writes:
>
>> So if I understand this debate correctly: The authz code is so much
>> faster now that parsing the authz file and performing the authz lookups
>> beats calculating its MD5 checksum?
> More that reading/checksumming is still too
Branko Čibej writes:
> So if I understand this debate correctly: The authz code is so much
> faster now that parsing the authz file and performing the authz lookups
> beats calculating its MD5 checksum?
More that reading/checksumming is still too slow to be done repeatedly.
1.9 reads the file o
On Sat, Mar 03, 2018 at 06:01:23PM +0100, Branko Čibej wrote:
> On 03.03.2018 17:44, Stefan Sperling wrote:
> > On Sat, Mar 03, 2018 at 04:32:35PM +, Philip Martin wrote:
> >> Stefan Sperling writes:
> >>
> >>> Which leads me to believe that r1778923 may have been based on wrong
> >>> assumpti
Philip Martin writes:
> r1825778 contains a revert of a change not in 1.10, so doesn't merge
> cleanly to 1.10. The reverse merge of -c-r1779188,-1778923 from trunk
> does merge cleanly to 1.10.
However, r1825736 is already nominated, so I just need to add r1825778
to that :-)
--
Philip
Philip Martin writes:
> Stefan Sperling writes:
>
>> I think our best course of action is to revert the change on trunk
>> and in 1.10.x. Could you do that? (I could do it, too. I'm just asking
>> you since you've probably already prepared it in a local copy.)
>
> OK, r1825778.
>
> Hmm, it doesn
On 03.03.2018 17:44, Stefan Sperling wrote:
> On Sat, Mar 03, 2018 at 04:32:35PM +, Philip Martin wrote:
>> Stefan Sperling writes:
>>
>>> Which leads me to believe that r1778923 may have been based on wrong
>>> assumptions about performance. The new authz is not fast enough to
>>> significant
Stefan Sperling writes:
> I think our best course of action is to revert the change on trunk
> and in 1.10.x. Could you do that? (I could do it, too. I'm just asking
> you since you've probably already prepared it in a local copy.)
OK, r1825778.
Hmm, it doesn't merge cleanly into 1.10...
--
P
On Sat, Mar 03, 2018 at 04:32:35PM +, Philip Martin wrote:
> Stefan Sperling writes:
>
> > Which leads me to believe that r1778923 may have been based on wrong
> > assumptions about performance. The new authz is not fast enough to
> > significantly reduce per-request overhead.
>
> My testing
On Fri, Mar 02, 2018 at 05:15:46PM +0300, Evgeny Kotkov wrote:
> Unless I am missing something, this might be worth considering before the
> 1.10 GA release.
Evgeny, you were entirely right about calling this out as a release blocker.
I am sorry for having suggested otherwise.
Stefan Sperling writes:
> Which leads me to believe that r1778923 may have been based on wrong
> assumptions about performance. The new authz is not fast enough to
> significantly reduce per-request overhead.
My testing so far was with a very small authz file -- only a handful of
rules and alias
On Sat, Mar 03, 2018 at 04:27:36PM +0100, Stefan Sperling wrote:
> Right now I have no idea how to make the current code in trunk any
> faster without reverting r1778923.
Thinking about this some more, I think reverting r1778923 is the
only way to make it faster.
The authz cache key is based on t
Philip Martin writes:
> Here are the cache stats before/after a run once the timing has
> stabilised:
>
> gets : 11770766, 4388248 hits (37.28%)
> sets : 7309448 (99.01% of misses)
> failures: 0
> used : 777 MB (94.56%) of 822 MB data cache / 958 MB total cache memory
> 2225228 entries (99.45%) o
On Sat, Mar 03, 2018 at 03:10:46PM +0100, Stefan Sperling wrote:
> Regardless, could you try one more test run with this diff against
> trunk to see if it has any impact already?
My new diff doesn't help, it brings the memory problem back.
I have done some more digging with APR POOL DEBUG.
It is
Stefan Sperling writes:
> Regardless, could you try one more test run with this diff against
> trunk to see if it has any impact already?
That's slower than current trunk:
6.2s
4.1s
4.5s
4.7s
4.7s
4.6s
4.7s
and memory use is back to several GB.
--
Philip
On 02.03.2018 19:21, Stefan Sperling wrote:
> I am just questioning the usefulness of halting the presses and restarting
> the soak for another month for something that isn't a security / data
> corruption issue.
It's a potential DOS that only needs read access. Falls under security
by my definiti
Philip Martin writes:
> Note an odd effect in the above numbers. The second run for 1.11 is
> always the fastest. The first run in each case is the slowest, the
> Subversion cache is cold. The second run is faster, the Subversion
> cache is hot. In 1.9 subsequent runs are comparable to the sec
On Sat, Mar 03, 2018 at 01:06:49PM +, Philip Martin wrote:
> Stefan Sperling writes:
>
> > I may have found a way to store an svn_repos_t object on the connection.
> > Does it help? (authz_tests pass)
>
> No significant improvement.
>
> I am timing
>
> time svn log http://localhost:/
> Yes, the problem seems to be that mod_authz_svn has no way of storing
per-connection state at present.
Please excuse the interjection here: My eyes spied "per connection" and I
wondered if something I'm doing elsewhere that is closer (but not exactly)
"per user" might help in terms of thinking.
Stefan Sperling writes:
> I may have found a way to store an svn_repos_t object on the connection.
> Does it help? (authz_tests pass)
No significant improvement.
I am timing
time svn log http://localhost:/repos/asf/subversion \
--username pm --password mp > /dev/null
and I realised
On Sat, Mar 03, 2018 at 11:43:31AM +0100, Stefan Sperling wrote:
> On Fri, Mar 02, 2018 at 09:40:28PM +, Philip Martin wrote:
> > Philip Martin writes:
> >
> > > Again, 1.10 would be nearly twice as fast, but now rereading the authz
> > > removes most of that gain.
> >
> > I think I see the
On Fri, Mar 02, 2018 at 09:40:28PM +, Philip Martin wrote:
> Philip Martin writes:
>
> > Again, 1.10 would be nearly twice as fast, but now rereading the authz
> > removes most of that gain.
>
> I think I see the underlying problem: the authz code now incorporates a
> cache based on the md5
Philip Martin writes:
> Again, 1.10 would be nearly twice as fast, but now rereading the authz
> removes most of that gain.
I think I see the underlying problem: the authz code now incorporates a
cache based on the md5 checksum of the rules, so when the rules are
unchanged the cached value can b
Philip Martin writes:
> All of those figures are the first run after starting Apache, i.e. when
> the OS cache and the Subversion cache is cold. With a hot Subversion
I meant to write
"the OS cache is hot and the Subversion cache is cold"
> cache the values are:
>
> 1.9: 6.0s
>
Stefan Sperling writes:
> On Fri, Mar 02, 2018 at 06:20:48PM +, Philip Martin wrote:
>>
>> Yes, that solves the memory use problem.
>
> Nice, I'll commit it then. This might not be a final fix but at least
> it's a step forward.
>
>> There is still a time penalty:
>>
>> Your patch:
>>
>>
On Fri, Mar 02, 2018 at 07:21:15PM +0100, Stefan Sperling wrote:
> I am just questioning the usefulness of halting the presses and restarting
> the soak for another month for something that isn't a security / data
> corruption issue. I anticipate that problems of similar severity to this
> one will
On Fri, Mar 02, 2018 at 06:20:48PM +, Philip Martin wrote:
> Stefan Sperling writes:
>
> > Hmmm. Does this help? The authz_tests pass with it.
> >
> > Index: subversion/mod_authz_svn/mod_authz_svn.c
> > ===
> > --- subversion/mod
On Fri, Mar 02, 2018 at 09:02:02PM +0300, Evgeny Kotkov wrote:
> Stefan Sperling writes:
>
> > I'd rather ship 1.10.0 at the prospected release date followed closely
> > by 1.10.1 to fix bugs such as these, than delay general access to 1.10.0
> > even further.
>
> While I do not have significant
Stefan Sperling writes:
> Hmmm. Does this help? The authz_tests pass with it.
>
> Index: subversion/mod_authz_svn/mod_authz_svn.c
> ===
> --- subversion/mod_authz_svn/mod_authz_svn.c (revision 1825730)
> +++ subversion/mod_authz_svn
Stefan Sperling writes:
> I'd rather ship 1.10.0 at the prospected release date followed closely
> by 1.10.1 to fix bugs such as these, than delay general access to 1.10.0
> even further.
While I do not have significant objections against such plan, I find the
idea of shipping a performance feat
On Fri, Mar 02, 2018 at 03:56:40PM +, Philip Martin wrote:
> Evgeny Kotkov writes:
>
> > Perhaps, a simpler reproduction script would be to issue an 'svn log' for
> > a medium-sized repository. In my environment, doing so for the trunk of
> > TortoiseSVN's repository with 25,000 revisions ca
Evgeny Kotkov writes:
> Perhaps, a simpler reproduction script would be to issue an 'svn log' for
> a medium-sized repository. In my environment, doing so for the trunk of
> TortoiseSVN's repository with 25,000 revisions causes the httpd process
> to consume up to a 1 GB of RAM while processing
On Fri, Mar 02, 2018 at 05:15:46PM +0300, Evgeny Kotkov wrote:
> Unless I am missing something, this might be worth considering before the
> 1.10 GA release.
Not about the actual bug, just a meta comment on the process:
I'd rather ship 1.10.0 at the prospected release date followed closely
by 1.1
Stefan Fuhrmann writes:
> Would it be possible for you to bisect this to find the offending revision?
> My random guess would be that in the context of mod_dav_svn, we might use
> an unsuitable pool for authz caching.
While looking through the various 1.10-related topics, I remembered about
this
On 05.12.2017 22:05, Evgeny Kotkov wrote:
Julian Foad writes:
After any issues raised in this discussion are resolved, we feel we should
go ahead and produce RC1 as soon as possible.
I think that I am seeing a 1.10 regression in terms of httpd's memory
usage during large imports.
In my envi
Julian Foad writes:
> After any issues raised in this discussion are resolved, we feel we should
> go ahead and produce RC1 as soon as possible.
I think that I am seeing a 1.10 regression in terms of httpd's memory
usage during large imports.
In my environment, when I `svn import` an unpacked v
42 matches
Mail list logo
