[
https://issues.apache.org/jira/browse/KAFKA-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13754005#comment-13754005
]
Tejas Patil commented on KAFKA-1014:
------------------------------------
Here is a plan for system tests. For this feature, there would a new system
test suite : 'offset_mgmt_testsuite'
Test case #1:
----------------
Entities:
3 brokers; 1 producer; 4 consumers
2 topics: "offsets" topic (replication : 3, partitions: 1) and "test" topic
(replication : 2, partitions: 4)
Test sequence:
Have 4 consumers consuming from the "test" topic. Fail two consumers one by one
(graceful kill). This would trigger rebalance and consumers would commit and
fetch offsets thus verifying these basic methods of offset manager.
After some time, shutdown the leader broker for the partition 0 of offsets
topic so that the leadership moves to some follower and it will load the
offsets. Kill a consumer to trigger rebalance thus verifying if the loading
went correctly and offsets are served from this new leader.
Bring up 2 new consumers for "test" topic to test commit and fetch offset
functionality.
Validation:
Compare the messages produced and messages consumed. There should not be any
duplication (or lost messages) indicating that consumers could store offsets
and read from where they left off.
Test case #2:
----------------
Same as #1 but now we would do a force kill of consumers. Validation would
involve verifying messages produced and *unique* messages consumed. Those
should be same.
Test case #3:
----------------
Entities:
1 broker, 1 consumers
2 topics: "offsets" topic (replication : 1, partitions: 1) and "test" topic
(replication : 1, partitions: 4)
There would be pre-populated logs for the broker having some offsets
information. Few entries will have null payload.
Test sequence:
Broker comes up and starts loading logs of offsets topic. Consumer comes up and
sends offset fetch request. This verifies if the consumer could fetch its
offset correctly.
Let the broker run log cleaner on the logs of offsets topic. This would verify
if the log cleaner is able to dedupe the offsets logs correctly and also if the
deletion of offsets from the offset logs works correctly.
Validation:
Dump the log of offsets topic. It must have lesser entries than before. Also,
there must no entries corresponding to key with null payload.
Dump the messages consumed by consumer. This should not contain messages before
the offset saved on the offsets log.
> Junits and system tests for inbuilt offset management
> -----------------------------------------------------
>
> Key: KAFKA-1014
> URL: https://issues.apache.org/jira/browse/KAFKA-1014
> Project: Kafka
> Issue Type: Sub-task
> Components: consumer
> Reporter: Tejas Patil
> Assignee: Tejas Patil
> Priority: Minor
>
> This Jira is for 2 things:
> 1. Fix tests (junits & system tests) which fail due to inbuilt offset
> management feature.
> 2. Additional tests must be added which includes broker loading logs on
> leadership change, perform cleanup when a leader becomes follower for offsets
> topic, and other such possible scenarios.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
