[
https://issues.apache.org/jira/browse/KAFKA-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921726#comment-16921726
]
ASF GitHub Bot commented on KAFKA-8853:
---------------------------------------
scott-hendricks commented on pull request #7289: KAFKA-8853: Create sustained
connections test for Trogdor
URL: https://github.com/apache/kafka/pull/7289
This creates a test that generates sustained connections against Kafka.
There
are three different components we can stress with this, KafkaConsumer,
KafkaProducer, and AdminClient. This test tries use minimal bandwidth per
connection to reduce overhead impacts.
This test works by creating a threadpool that creates connections and then
maintains a central pool of connections at a specified keepalive rate. The
keepalive action varies by which component is being stressed:
* KafkaProducer: Sends one single produce record. The configuration for
the produce request uses the same key/value generator as the ProduceBench
test.
* KafkaConsumer: Subscribes to a single partition, seeks to the end, and
then polls a minimal number of records. Each consumer connection is its
own consumer group, and defaults to 1024 bytes as FETCH_MAX_BYTES to keep
traffic to a minimum.
* AdminClient: Makes an API call to get the nodes in the cluster.
NOTE: This test is designed to be run alongside a ProduceBench test for a
specific topic, due to the way the Consumer test polls a single partition.
There may be no data returned by the consumer test if this is run on its own.
The connection should still be kept alive, but with no data returned.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Create sustained connections test for Trogdor
> ---------------------------------------------
>
> Key: KAFKA-8853
> URL: https://issues.apache.org/jira/browse/KAFKA-8853
> Project: Kafka
> Issue Type: Improvement
> Components: system tests
> Reporter: Scott Hendricks
> Priority: Major
>
> There are currently tests to run a high amount of connects and disconnects,
> but there are no tests that create and maintain connections to bring Kafka to
> its limit.
> My plan is to write a test that will take a desired number of clients
> (KafkaConsumer, KafkaProducer, and AdminClient), the keep-alive rate for
> these connections, and the number of threads desired to maintain these
> connections.
> Each worker will spawn the desired number of threads that will find
> connections that need to be maintained and act on them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
