I decided to sleep on this before running into a response. On Sat, Jun 10, 2017 at 11:22 AM Sam Ruby <ru...@intertwingly.net> wrote:
> On Sat, Jun 10, 2017 at 11:10 AM, John D. Ament <johndam...@apache.org> > wrote: > > Unless someone objects, I'm going to introduce a shell script that > executes > > the contents of the cron jobs. I'm then going to raise a PR to infra to > > update this section > > > https://github.com/apache/infrastructure-puppet/blob/deployment/modules/whimsy_server/manifests/init.pp#L70 > > to > > include a call to said script. > > Not an objection, but thinking out loud. > > Some of those cronjobs are lightning fast. Others may take a while > and consume network or CPU resources. > > For the fast ones, I find it mildly amusing that they use whimsy/asf > library functions and capture the result in JSON, and you are parsing > the JSON instead of using the same library functions. What this means > is that you get stale data when you have ready access to fresher data. > Do you have a concrete list of these? I often find arbitrary statements like this hard to follow, mostly because of how unfamiliar I am with something, in this case Whimsy. > > For the slow ones, I don't think it is appropriate to run then after every > push. > > Again, having a list of these would be useful. i'm not sure I can judge without having all of the information. > What I would suggest instead is a CGI script that lists the cron jobs > that are runnable under the apache web server user id, and gives you a > button that you can push that will run that specific job on request. > What this would enable is for anybody (or perhaps just ASF members?) > to rerun the script at will. > > No, specifically I disagree with this kind of approach. If we're going to follow an automatic deploy pattern, the application needs to be self healing. If there's manual intervention something's wrong. Yesterday we got stuck because: - SVN repo list is cached in the app (but not in cron) - The cron jobs happened to run in such an order that the svn update happened after public_podlings.json was generated What I'm actually now thinking is that we should implement two bug fixes: - in ASF::SVN if the repo doesn't exist, reload the repository.yml file and try again. - in some scripts, make sure that any repos you need are up to date (rather than rely on the svn update cron) Thoughts? > I'd start by parsing the whimsy_server/manifests/cronjobs.pp. > > > John > > > > On Sat, Jun 10, 2017 at 8:58 AM John D. Ament <johndam...@apache.org> > wrote: > > > >> I just pushed up a change that requires a change to the podlings.json > >> file. However, that file is only generated in cron. I was wondering > if it > >> made sense that on deploy, to run a script that would regenerate the > static > >> files? > >> > >> John > > - Sam Ruby >
