The way I've done this in past and current positions has evolved over time. Generally speaking I break all of by build systems up into a couple of parts depending on what capabilities I need these generally break down into the following 5 categories: increment - deals with code checkout (are there updates else stop), versioning (update version numbers in the code), and change log generation build - actually build the software and do any necessary packaging steps (this produces somethat that a manual QA team could use) This is also frequently a "matrix job" depending on what we're talking about building deploy - put the build on to the test system (this varies depending on whether we're talking about servers based system or clients (iOS or Android apps, or desktop applications) test - this actually does any testing which might include unit tests, functional tests, integration tests, security scanning using tools like w3af, etc All the testing happens together. Though I guess theoretically I could break this up into smaller parts but I like having it all together in a single job report - this gathers up all of the information from all the previous build steps and compiles it into a report that I can send out to my team via e-mail and/or stash into my wiki.
Each of these jobs in jenkins are build using parameters which allows me to easily pass information between the jobs. I've taken to passing the upstream build url (content in the $BUILD_URL) between jobs. The advantage of this approach is that it makes it very easy to re-run one particular job when trying to debug error using the same input data[1]. When I first started using this technique I was using the Parameterized Trigger Plugin[2] for most of my triggering needs and managing whether downstream jobs would execute or not using its property file[3] to determine whether to trigger or not (I use A LOT of shell code in my build pipeline since I find it easiest for me... but then I have over 20 years of unix experience... you might find a different technique works better for you). More recently (last year) I've started to move to using the Jenkins Build Flow Plugin[4] for job coordination to give me a bit more flexibility in job execution ordering and what I do when particular builds fail. I've combined that with an ant based templating technique[5] to make it easy to set up multiple build pipelines for use with different test environments just by adding a new test environment name to a list and running a job to generate the latest version of the templates. It also makes keeping all the jobs up to date with the latest changes since you just mod the template and then regenerate all the jobs. So that's the general "overview" of the process. Some more details on the testing part of the pipeline (since that's what you were asking about). Generally speaking I try to get all of my testing tools to generate JUnit XML output since Jenkins knows how to deal with it and it's easy to parse (even in shell scripts by making use of xmlstarlet[6]. If all your test output is in the Junit formation it's trivially easy to point to it in your jenkins test job and have it then track test data for you over time. And since it's all in a nice XML format you can then slice and dice the data as needed to generate nice reports in the reporting job. Hell, Jenkins even gathers up some of the summary data for you to you can make simple handy charts which you can then embed into you reports. I've been able to do this using a variety of tools (Junit itself, Phpunit, Behat, Sikuli[7], Python nose, shell code) to use any and all of these as part of an integrated testing framework. I've also found that wrapping your testing harness in an Ant script can be a handy technique, but again, that's probably only helpful because I've been using Ant for almost a decade as well. So again... that's just a tool choice issue more than something specific to Jenkins. I started using Ant for this long before I ever used Jenkins or a build queue of any kind for that matter (ah cron... ;-) Sorry if that was a bit long winded. Should probably turn this into a blog post for my website... Cheers, Ben [1] The Rebuild Plugin is your friend: https://wiki.jenkins-ci.org/display/JENKINS/Rebuild+Plugin [2] https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Trigger+Plugin [3] These are just a java properties style property file [4] https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin [5] http://www.blackpepper.co.uk/generating-new-jenkins-jobs-from-templates-and-parameterised-builds/ [6] http://xmlstar.sourceforge.net/ [7] Even wrote a book about this one: http://www.amazon.com/Instant-Sikuli-Test-Automation-Ben/dp/1782167870 On Sat, Feb 22, 2014 at 6:42 PM, andres reynaldy <andres.reyna...@gmail.com> wrote: > My question is about design strategy, so I want to hear the pros and cons > from those more familiar with Jenkins. Thanks. > > > On Sunday, February 23, 2014 10:11:39 AM UTC+10, Red wrote: >> >> Hi. I am also new to jenknis. I searched >> http://serverfault.com/search?q=jenkins and found 414 jenkins related post. >> >> >> On Sat, Feb 22, 2014 at 3:45 PM, andres reynaldy <andres....@gmail.com> >> wrote: >>> >>> Hi, >>> >>> I'm new to Jenkins and I'm wondering is this the user forum for jenkins? >>> I was expecting a forum, not a gmail mailing group. But anyway, I'm here to >>> ask about the best practice about how to set up integration testing in >>> Jenkins. Do we deploy that as separate build job thats run after the core >>> build job? do we put the integration testing source code in the same project >>> as the core project (like unit testing) or make it a different project >>> altogether? those are my questions for now, and if there's actually a forum >>> for jenkins community like spring forum, please let me know. Thank you very >>> much. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Jenkins Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to jenkinsci-use...@googlegroups.com. >>> >>> For more options, visit https://groups.google.com/groups/opt_out. >> >> > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jenkinsci-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
