On 05/08/2014 12:21 PM, Tomas Vondra wrote:
On 6.5.2014 23:01, Tomas Vondra wrote:
On 6.5.2014 22:24, Tom Lane wrote:
Tomas Vondra <t...@fuzzy.cz> writes:
I recall there was a call for more animals with CLOBBER_CACHE_ALWAYS
some time ago, so I went and enabled that on all three animals. Let's
see how long that will take.
I see there are more 'clobber' options in the code: CLOBBER_FREED_MEMORY
and CLOBBER_CACHE_RECURSIVELY. Would that be a good idea to enable these
as well?
The time requirements will be much higher (especially for the
RECURSIVELY option), but running that once a week shouldn't be a big
deal - the machine is pretty much dedicated to the buildfarm.
I've never had the patience to run the regression tests to completion
with CLOBBER_CACHE_RECURSIVELY at all, let alone do it on a regular
basis. (I wonder if there's some easy way to run it for just a few
regression tests...)
Now, that's a challenge ;-)
I think testing CLOBBER_FREED_MEMORY would be sensible though.
OK, I've enabled this for now.
Hmmmm, with CLOBBER_CACHE_ALWAYS + CLOBBER_FREED_MEMORY the tests take
~20h on a single branch/animal. With a single locale (e.g. "C") it would
take ~4h, but we're testing a bunch of additional czech/slovak locales.
The tests are running in sequence (magpie->treepie->fulmar) so with all
6 branches, this would take ~14 days to complete. I don't mind the
machine is running tests 100% of the time, that's why it's in buildfarm,
but I'd rather see the failures soon after the commit (and two weeks is
well over the "soon" edge, IMHO).
So I'm thinking about how to improve this. I'd like to keep the options
for all the branches (e.g. not just HEAD, as a few other animals do).
But I'm thinking about running the tests in parallel, somehow - the
machine has 4 cores, and most of the time only one of them is used. I
don't expect a perfect ~3x speedup, but getting ~2x would be nice.
Any recommendations how to do that? I see there's 'base_port' in the
config - is it enough to tweak this, or do I need to run separate the
animals using e.g. lxc?
Here is what I do on my FreeBSD VM. I have 2 animals, nightjar and
friarbird. They have the same buildroot. friarbird is set up to build
with CLOBBER_CACHE_ALWAYS, building just HEAD and just testing C locale;
nightjar builds all branches we are interested in and tests locale
cs_CZ.utf8 in addition to C.
Other than those differences they are pretty similar.
Here is the crontab that drives them:
27 5-22 * * * cd bf && ./run_branches.pl --run-all --verbose
--config=nightjarx.conf >> bf.out 2>&1
20 0 * * * cd bf && ./run_branches.pl --run-all --verbose
--config=friarbird.conf --skip-steps=install-check >> bf.out 2>&1
The buildfarm code has enough locking smarts to make sure we don't get
any build collisions doing this.
If you have an animal to do a special type of build (e.g. CLOBBER_foo)
then it's probably a good idea to set a note for that animal - see the
buildfarm program setnotes.pl. friarbird has the note set "Uses
-DCLOBBER_CACHE_ALWAYS".
If you want to do this in parallel, then you will need different
buildroots and different base ports for each animal. I would not run the
same animal on different branches concurrently, that is quite likely to
end up in port collisions.
HTH.
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
