On 09/15/2017 03:33 PM, Daniel J. Luke wrote:
On Sep 15, 2017, at 12:24 PM, David Jones <djo...@ena.com> wrote:
You kinda have to work backwards through the scripts to find what is generating
the scores-set0 file and turning it into 72_scores.cf. I am grep'ing through
the work dir on the SA server now but it contains a lot of files. I need to
find the large dirs and exclude them.
you may have already done this, but if you modify the scripts to not overwrite
(or save a copy) of the intermediate files (which may clue into exactly where
the problem is being introduced). ie. runGA lines 57-59, 124-132 (for
50_scores.cf)
I have already restructured the script to use a common tmp dir on the
server a few months ago. Before that the scripts were writing all over
the place in various home dirs on the old VM that crash/went down back
in mid March.
The issue is there is so much junk getting checked out of SVN in various
subdirs that it's hard to follow through the logic in all of the
scripts. You can look at the generate-new-scores.sh and quickly find
your eyes crossing and get lost. I made some local notes in a text file
trying to map out the scripts and the files they create but it's kinda a
mess. I guess need to dive into this again and go deeper. I spent many
hours/days a couple of months ago trying to figure this out and it wore
me out.
another 'easy' test I would try would be to set numcpus in runGA to 1 just in
case the problem is that somewhere there are multiple writers overwriting parts
of the same file
Good idea. I tried manually overriding the script to force numcpus=1
but it made no difference over the weekend.
--
David Jones
