[
https://issues.apache.org/jira/browse/KAFKA-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dong Lin updated KAFKA-2171:
----------------------------
Status: Patch Available (was: Open)
> System Test for Quotas
> ----------------------
>
> Key: KAFKA-2171
> URL: https://issues.apache.org/jira/browse/KAFKA-2171
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Dong Lin
> Assignee: Dong Lin
> Attachments: KAFKA-2171.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Motivation and goal:
> We want to make sure that following features are working properly for both
> consumer and producer: default quota, client-specific quota, and quota
> sharing per clientId.
> The tests and configuration described aims to cover most of the scenarios.
> More test cases with varying configurations (e.g. ackNum) can be added if
> there is good reason to do so.
> Initial setup and configuration:
> In all scenarios, we first create kafka brokers and topic as follows:
> - create two kafka broker processes (by default local)
> - create a topic with replication factor = 2 and ackNum = -1
> - let max_read = max_write = 5MB. The test machine is expected to provide
> read (write) throughput at least max_read (max_write).
> - we consider two rates are approximately the same if they differ by at most
> 5%.
> Scenario 1: Validate that max_read and max_write are provided by the test
> machine(s) using 1 producer and 1 consumer
> 1) produce data to the topic without rate limit for 30 seconds
> 2) record the rate of producer
> 3) then, consume data from the topic without rate limit until he finishes
> 4) record the rate of consumer
> 5) verify that the data consumed is identical to the data produced
> 6) verify that producer rate >= max_write and consumer rate >= max_read
> Scenario 2: validate the effectiveness of default write and read quota using
> 1 producer and 1 consumer
> 1) configure brokers to use max_write/2 as default write quota and max_read/2
> as default read quota
> 2) produce data to the topic for 30 seconds
> 3) record the rate of producer
> 4) then, consume data from the topic until he finishes
> 5) record the rate of consumer
> 6) verify that the data consumed is identical to the data produced
> 7) verify that recorded write (read) rate is within 5% of max_write/2
> (max_read/2).
> Scenario 3: validate the effectiveness of client-specific write and read
> quota using 2 producers and 2 consumers
> 1) configure brokers to use max_write/2 as default write quota and max_read/2
> as default read quota; configure brokers to use max_write/4 for producer_2
> and max_read/4 for consumer_2
> 2) both producers produce data to the topic for 30 seconds. They use
> different clientId.
> 3) record the rate of producer
> 4) both consumers consume data from the topic until they finish. They use
> different clientId and groupId.
> 5) record the rate of consumer
> 6) verify that the data consumed is identical to the data produced
> 7) verify that producer_1 and producer_2 rates are approximately max_write/2
> and max_write/4; verify that consumer_1 and consumer_2 rates are
> approximately max_read/2 and max_read/4.
> Scenario 4: validate the effectiveness of write and read quota sharing among
> clients of same clientId using 2 producers and 2 consumers.
> 1) configure brokers to use max_write/2 as default write quota and max_read/2
> as default read quota
> 2) both producers produce data to the topic for 30 seconds. They use same
> clientId.
> 3) record the rate of producer
> 4) both consumers consume data from the topic until they finish. They use
> same clientId but different groupId.
> 5) record the rate of consumer
> 6) verify that the data consumed is identical to the data produced
> 7) verify that total rate of producer_1 and producer_2 is approximately
> max_write/2; verify that total rate of consumer_1 and consumer_2 is
> approximately max_read/2.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
