[
https://issues.apache.org/jira/browse/KAFKA-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14094312#comment-14094312
]
Guozhang Wang commented on KAFKA-1589:
--------------------------------------
As I tagged it with "0.9", I think we can do that post 0.8.2 release. Another
issue is that currently system test is not really easy to use and run out of
box, and hence few people are using / contributing to it except LinkedIn folks.
We can also discuss how that can be improved.
> Strengthen System Tests
> -----------------------
>
> Key: KAFKA-1589
> URL: https://issues.apache.org/jira/browse/KAFKA-1589
> Project: Kafka
> Issue Type: Bug
> Reporter: Guozhang Wang
> Assignee: Gwen Shapira
> Labels: newbie
> Fix For: 0.9.0
>
>
> Although the system test code is also part of the open source repository, not
> too much attention is paid to this module today. The incurred results is that
> we keep breaking the system tests with either changes on the admin tools, or
> library upgrades that change the APIs like Zookeeper. And when the system
> tests breaks / hangs / etc, it is also hard to debug the issue. We need to
> treat the system test suite just as part of the open source code.
> Based on my personal experience trouble shooting system tests, I would
> propose doing at least the follow enhancement around system tests.
> 1. Add unit tests for all system util test tools, for example:
> kafka_system_test_utils.get_controller_attributes
> kafka_system_test_utils.get_leader_for
> 2. Add exception handling logic in the python test framework to clean-up the
> testbed upon failures, so that the subsequent test cases will not be affected.
> 3. Remove timing based mechanism such as "sleep(5000) to wait for metadata to
> be propagated" as much as possible to avoid transient failures.
> After those enhancements, we should probably also pick a very small subset
> (say one from each suite) of the system test cases into the patch reviewing
> process along with the unit tests.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
