Well, the idea was to tie only those jobs that need exclusive access to the database to the slave with 1 executor. The rest of the jobs can be left to run where ever. You can add more slaves to parallelize the build as much as you like and use the slave with 1 executor as the "choke point" for database access.
-- Sami Jan Seidel kirjoitti 3.7.2012 kello 9.33: > Hi folks, > > thanks for your replies :) > > @Sami: that's actually the way the jobs are handled at the moment but that's > not supporting the idea to parallelize jobs ;) > > • I will take a closer looks at locks and latches. I did actually use > it some time ago but I'm not sure if it will work the way I need it to work > as the "weight" does just dictate which job is superior but not exclusive. > • The exclusive execution plugin is not of help as it blocks all follow > up jobs until it has been excuted....on all nodes. So even non-related jobs > won't run for that time. That would create a tremendous and not copable > backlog. > • Throttle concurrent builds plugin is a nice one but also not matching. > My problem is that I need a possibility to define myself which job is > concurrent. > Maybe I should elaborate my issue a bit more. > > Our software is produced in several stages. > Two of these stages are the major ones. > • First stage is generating java code via GUIDE. That to is a visual > editor to create a GUI and its logic, and then generate the appropriate java > code from the project which is incorporated into the manually written code. > • Second stage is to build the software bundle from the entire code. > We have currently a developer and a delivery branch for each software bundle. > This way we keep our code freeze for building software down to round about > 4-6 minutes while a snapshot from the developer branch is copied to the > delivery branch. > > Everything works fine so far BUT if we have to provide a bugfix are troubles > ahead. > • The auto-generated code from GUIDE should then be stored in the > delivery branch and not the developer branch. > • The SCM module in Jenkins needs hardcoded repository paths. Dynamic > assembly of the path with variables will fail. So the job is duplicated only > with 2 different repository paths which works well so far. > • The problem I am fighting with is that the developers work on the > developer branch and every now and then also start GUIDE to > integrate/fix/validate new implementations. This must not happen in parallel > with the build job that generates into the delivery branch but will do it in > case of a bugfixing as Jenkins assumes these jobs as non-related. How should > it know that a build step accessing the database has to be exclusive. > • Non-exclusive access may break the database which is no good at all > as the database then has to be fixed manually. And that's no joy with several > thousand entries per run. > The most simple way would be if a plugin exists where you just enter other > projects and then could select block this or the other jobs if one of the > defined projects is running but I haven't seen anything like that yet. > > > > Take care > > Jan > > > Am Donnerstag, 28. Juni 2012 21:08:06 UTC+2 schrieb sti: > I'll add another way to exclude other jobs: make a slave with only one > executor and tie the jobs to restrict to the slave. Only one at a time will > run. > > -- Sami > > Jan Seidel <wakkal...@gmail.com> kirjoitti 28.6.2012 kello 13.41: > >> Hi folks, >> >> I am trying to parallelize some of our builds to speed things up. >> This particular build is quite special as it also interacts with databases. >> Multiple write access on a database will wreck the content, so this must be >> avoided by all means. It takes us in worst cast out of business for 2 weeks >> and creates loads of work and stress. >> >> Don't ask me why the DB design is as it is, that's pretty complicated, >> insane, sooo wrong and not worth discussing in order to fix that issue and >> let the build jobs just do what they are meant to do. >> Have been there, didn't like it, went away! >> >> The initial build job is a dispatcher that decides which job to run. So far >> quite easy but it also checks for 2 conditions which requires to almost >> identical jobs asides of one of the repositories location. >> So the jobs access mostly the same ressources including the database which >> must not be simultaneously! So they are a kind of "siblings" >> The default is set that inhibits to spawn duplicate jobs but with this >> second condition is it changing a bit. The second condition requires to >> block a job if a duplicate or the sibling is running >> >> This brings me to a dumb situation. Either I find a way to: >> • refer to related AND non-related jobs which may block a build >> • block on top-level (the dispatcher). This works but scraps all my >> efforts to get jobs not sharing same ressources to run in parallel >> • do not block jobs and hope that the developers don't botch it and >> wreck a database *yuk* >> Do you know a solution to block a job if a clone or a sibling already is >> running? >> >> >> >> >> >> Cheers >> >> Jan >>
