On 2 March 2016 at 13:52, Sam Ruby <ru...@intertwingly.net> wrote: > On Wed, Mar 2, 2016 at 8:09 AM, sebb <seb...@gmail.com> wrote: >> Some of the code needs access to a single file in SVN, e.g. >> >> https://svn.apache.org/repos/asf/incubator/public/trunk/content/podlings.xml >> >> AFAIK there are no other items needed in the same part of SVN. >> >> Yet one has to checkout the parent directory and keep it updated. >> >> This is a nuisance for testing purposes. >> >> For SVN files that don't need authentication, it is easy enough to >> fetch the file using HTTP(S) and cache it locally. For subsequent >> requests, the cache date can be used in an If-Modified request. >> The last checked date can also be saved to avoid checking the file too often. >> >> This technique has been used in reporter.a.o for checking the chair >> index in foundation/index.mdtext: >> >> https://svn.apache.org/repos/asf/comdev/reporter.apache.org/trunk/scripts/parsepmcs.py >> >> Might be worth cosidering re-implementing the python caching code used there. > > As very few of the resources accessed by whimsy are public, that would > only solve a small part of the problem.
Which in itself is an issue - should we be encouraging downloading all these resources for testing purposes? In particular, it seems wrong to have to download all the ICLAs. > Given that the resources needed are declared in > https://github.com/apache/whimsy/blob/master/repository.yml, perhaps a > more complete solution would be to use the 'svn:update' task defined > in https://github.com/apache/whimsy/blob/master/Rakefile#L66 ? > > At the moment, that task is assuming that you do *NOT* override :svn: > in your .whimsy config file. Perhaps it should handle that case? Better would be to have a function to report which entries are missing, so the user can decide which they want to download based on what they are testing. Maybe also with a function to report which are out of date. It might be useful if ASF::SVN raised an error instead of returning nil when it cannot find the location. It's not always easy to debug the errors that occur when the SVN dir is not found. > - Sam Ruby
