Ashish Kumar Singh created KAFKA-1856:
-----------------------------------------

             Summary: Add PreCommit Patch Testing
                 Key: KAFKA-1856
                 URL: https://issues.apache.org/jira/browse/KAFKA-1856
             Project: Kafka
          Issue Type: Task
            Reporter: Ashish Kumar Singh
            Assignee: Ashish Kumar Singh


h1. Kafka PreCommit Patch Testing - *Don't wait for it to break*

h2. Motivation
*With great power comes great responsibility* - Uncle Ben. As Kafka user list 
is growing, mechanism to ensure quality of the product is required. Quality 
becomes hard to measure and maintain in an open source project, because of a 
wide community of contributors. Luckily, Kafka is not the first open source 
project and can benefit from learnings of prior projects.

PreCommit tests are the tests that are run for each patch that gets attached to 
an open JIRA. Based on tests results, test execution framework, test bot, +1 or 
-1 the patch. Having PreCommit tests take the load off committers to look at or 
test each patch.

h2. Tests in Kafka
h3. Unit and Integraiton Tests
[Unit and Integration 
tests|https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.9+Unit+and+Integration+Tests]
 are cardinal to help contributors to avoid breaking existing functionalities 
while adding new functionalities or fixing older ones. These tests, atleast the 
ones relevant to the changes, must be run by contributors before attaching a 
patch to a JIRA.

h3. System Tests
[System 
tests|https://cwiki.apache.org/confluence/display/KAFKA/Kafka+System+Tests] are 
much wider tests that, unlike unit tests, focus on end-to-end scenarios and not 
some specific method or class.

h2. Apache PreCommit tests
Apache provides a mechanism to automatically build a project and run a series 
of tests whenever a patch is uploaded to a JIRA. Based on test execution, the 
test framework will comment with a +1 or -1 on the JIRA.

You can read more about the framework here:
http://wiki.apache.org/general/PreCommitBuilds

h2. Plan
- Create a test-patch.py script (similar to the one used in Flume, Sqoop and 
other projects) that will take a jira as a parameter, apply on the appropriate 
branch, build the project, run tests and report results. This script should be 
committed into the Kafka code-base. To begin with, this will only run unit 
tests. We can add code sanity checks, system_tests, etc in the future.
- Create a jenkins job for running the test (as described in 
http://wiki.apache.org/general/PreCommitBuilds) and validate that it works 
manually. This must be done by a committer with Jenkins access.
- Ask an Hadoop committer (or someone else with access to 
https://builds.apache.org/job/PreCommit-Admin/) to add Kafka to the list of 
projects PreCommit-Admin triggers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to