date:20140129

[JENKINS] Jenkins the spammer

2014-01-29 Thread Hugo Trippaers

Hey guys,

Ofter the last few weeks we had a lot of problems with Jenkins actually 
reporting build problems that were actually infrastructure related. I try and 
explain the issues and how we try to prevent them from happening in the future.

The GIT plugin used by jenkins has a build in timeout of 10 minutes, if a git 
operation takes longer than this it will report an error. This timeout is not 
configurable from within Jenkins and needs to be set with a system property. 
I’ve made a patch on the cloud-jenkins plugin that will allow us to set custom 
system properties for the jclouds slaves. This resolved most of the 
intermittent build problems.

There was also the recent change to java 7 for master. This caused most of the 
master builds to fail on Jenkins as all slaves were setup to explicitly use 
java 6 (at that was our target platform). Now we made some changes so each 
build can specify which JDK version it will use at build time. This works on 
the cloudstack-buildslaves but not yet on the rpm builders. Because of this the 
debian and redhat packages are not built yet.

There were also some problems with the artifacts required for the awsapi. Due 
to a configuration in the poms for the transient dependencies (outside our 
tree) several artifacts would have be downloaded from a maven repository that 
no longer existed. However the site didn’t return a 404 that maven would 
pickup, but a 200 with an index page so maven would think it properly 
downloaded the pom/jar file. Causing grief later in the proces. We now use a 
custom settings.xml for the master build that redirects this particular 
repository to the archiva instance that david setup.

The positive news is that we now have upgraded the service offering for the 
cloudstack-buildslaves to 4 x 2.6Ghz cores and 4Gb ram, further reducing the 
main master build to just under 8 minutes (including awsapi).

It’s not completely done yet, but hopefully this will help getting Jenkins back 
into a stable state where we can rely on failures being real failures in the 
build and not in the infrastructure.

Cheers,

Hugo

[Question] Refactoring ApiDispatcher and processing parameters

2014-01-29 Thread Antonio Fornié Casarrubios

Hi,

I'm working on cloud-server#com.cloud.api.ApiDispatcher and related classes
and trying to do some refactoring here. But I just had a couple of
questions to make sure I'm not missing anything important.

1* There are several instance methods, but then there is this
#processParameters that is static. It doesn't seem a method that I would
make static. It seems more like someone wanted to call it from another
class (perhaps AutoScaleManagerImpl) and instead of getting the
ApiDispatcher object, they just made the method static, which I think would
be wrong. Are there good reasons for this method to be static? Further,
from this methods they needed to call #doAccessChecks but in this case,
instead of making #doAccessChecks also static, they added a getInstance...
which we can avoid if no method is static in the first place. The
getInstance is also strange, but instead of fixing that it's better to just
remove it since we will not need it: we should use Spring for injecting
ApiDispatcher instead.

Is there are reason not to do the change I propose? Am I missing something?

2* About the parameters received in the HTTP request, we have two methods
for "fixing" them for our needs. First, in ApiServer#verifyRequest, we have
a Map but we never need to have an array of Strings, so
the code assumes the array will always have 1 item, and always uses
params[0] without even checking (the array could have 0 or 10 Strings). All
of this also happens along ApiServlet.

Later in ApiServer#handleRequest we do a first fix in the Map (which
doesn't seem to fit well in this method). Actually we move the values to a
Map to stop having arrays. And from that point we use only
the new Map.

And finally, later in Apidispatcher we do a final fix in the Map:

unpackedParams = cmd.unpackParams(params);

So that params have values already lowercase and format validated.

So unless someone knows reason to keep it this way, I want to fix the Map
of params from the very beginning once and for all in a separate methods.
>From that point all params will be checked, lower case and Map without 1-item arrays... so future methods coming later don't need
to worry about these details. Is there a reason doing this could be wrong?
Is there any point in the middle where it will brake if my values are
Strings instead of Arrays of Strings?

89 matches

Mail list logo