Actually copy Greg this time. ---------- Forwarded message --------- From: Sam Ruby <ru...@intertwingly.net> Date: Mon, Nov 25, 2019, 4:15 PM Subject: Re: Does Whimsy need to have a copy of Bills? To: Whimsy dev <dev@whimsical.apache.org>
adding Greg to email. Recap: a change is being proposed whereas whimsy will do the equivalent of the following command after every icla is processed: svn ls https://svn.apache.org/repos/private/documents/iclas --depth infinity Currently this appears to take around forty to sixty elapsed seconds to process. Questions for Greg: 1) Does this proposed workload present an unreasonable load on the svn server? 2) Are there any faster alternatives which get us a list of names but no data - Sam Ruby On Mon, Nov 25, 2019 at 3:44 PM sebb <seb...@gmail.com> wrote: > > On Mon, 25 Nov 2019 at 19:48, Sam Ruby <ru...@intertwingly.net> wrote: > > > On Mon, Nov 25, 2019 at 2:29 PM sebb <seb...@gmail.com> wrote: > > > > > > On Mon, 25 Nov 2019 at 17:59, Sam Ruby <ru...@intertwingly.net> wrote: > > > > > > > On Sun, Nov 24, 2019 at 1:17 PM sebb <seb...@gmail.com> wrote: > > > > > > > > > > I was thinking of using svn ls to create a listing file which would > > be > > > > > cached locally. > > > > > > > > Unfortunately, some observations (numbers below are approximate): > > > > > > > > svn up on a populated iclas directory: one second > > > > > > > > svn ls on iclas: two seconds, but only returns depth one > > > > > > > > svn ls on iclas --depth infinity: one minute > > > > > > > > What's most concerning is not just the elapsed time, but that this > > > > likely means that the call is expensive on the server, which may > > > > impact other users. > > > > > > > > > > > I see this as premature optimisation; we don't know whether svn list is > > > more expensive than svn update overall. > > > There may be other reasons why list is slower. Nor do we know if the > > > request will impact other users. > > > > > > Besides, if the code checks SVN info first, it will only need to fetch > > the > > > updated listing when there has been a change. > > > Those directories are not busy. > > > > > > Furthermore, every time a new test installation is set up, there is > > > definitely a large load on the server and network. > > > This network load in particular must be orders of magnitude greater than > > > for a listing. > > > > Sorry for not being clear. I would be very concerned if whimsy-vm4 > > were invoking svn ls --depth infinity every 10 minutes as the current > > cron job does. > > > That would not be the case. > > The job would use 'svn info' on the remote repo and only fetch the listing > if necessary. > > For the repos in question, changes are rare. > > > > Before any such change is deployed, it would be wise > > for us to check both with the infrastructure team and the subversion > > team (Greg likely can help with both). > > > > I'm less concerned about the overhead on development machines, and > > there I suspect that most users would be happy with a svn checkout > > --depth empty. > > > > > This would not allow testing of the functions that need to know the list of > file names. > > > > - Sam Ruby > >
