GitHub user edenhill opened a pull request:
https://github.com/apache/kafka/pull/2048
Pluggable verifiable clients
This adds support for pluggable VerifiableConsumer and VerifiableProducers
test client implementations
allowing third-party clients to be used in-place of the Java client in
kafkatests.
A new VerifiableClientMixin class is added and the standard Java
Verifiable{Producer,Consumer}
classes have been changed to use it.
While third-party client drivers may be implemented with a complete class
based on the Mixin, a simpler
alternative which requries no kafkatest class implementation is available
through the VerifiableClientApp class that uses ducktape's global param to
specify the client app to use (passed to ducktape through the `--globals
<json>` command line argument).
Some existing kafkatest clients for reference:
Go: https://github.com/confluentinc/confluent-kafka-go/tree/master/kafkatest
Python:
https://github.com/confluentinc/confluent-kafka-python/tree/master/confluent_kafka/kafkatest
C++:
https://github.com/edenhill/librdkafka/blob/0.9.2.x/examples/kafkatest_verifiable_client.cpp
This PR also contains documentation for the simplex JSON-based verifiable*
client protocol.
There are also some minor improvements that makes troubleshooting failing
client tests easier.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/edenhill/kafka pluggable_verifiable_clients
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/2048.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2048
----
commit 353c4e91f18e5af83daf781fc25a5df001766046
Author: Magnus Edenhill <[email protected]>
Date: 2016-09-13T20:30:09Z
Added pluggable Verifiable client support
commit bbb31d6fae7b01d40e71620eb403c58fb6152f46
Author: Magnus Edenhill <[email protected]>
Date: 2016-09-13T20:32:48Z
Added ConsumerState.Started and alive_nodes()
This extra state is needed to know which nodes succesfully started but have
not yet rebalanced.
commit 107891d261293ae424235389772939e52336f66d
Author: Magnus Edenhill <[email protected]>
Date: 2016-09-13T20:33:11Z
Add test for pluggable verifiable clients
commit 955635b61161a81dc5b24818fa3b8ce3a7ea1144
Author: Magnus Edenhill <[email protected]>
Date: 2016-09-13T20:39:57Z
verifiable_consumer: improved debug logging
commit aeb7e6522cabadea60287f0aa9900080741ce555
Author: Magnus Edenhill <[email protected]>
Date: 2016-09-19T05:53:56Z
Improved kafkatest error messages
commit 9d466c053d077f7f7e70c046ce0fc3dfdc4459ec
Author: Magnus Edenhill <[email protected]>
Date: 2016-10-03T06:52:40Z
OffsetValidationTest.test_consumer_failure: increased robustness
Allow two session timeout cycles: one for timing out the killed client
and one for the rebalance operation.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
