Apurva Mehta created KAFKA-5375: ----------------------------------- Summary: Transactions: Concurrent transactional consumer loses messages when there are broker bounces Key: KAFKA-5375 URL: https://issues.apache.org/jira/browse/KAFKA-5375 Project: Kafka Issue Type: Bug Reporter: Apurva Mehta Assignee: Apurva Mehta Priority: Blocker Fix For: 0.11.0.0
With the system test modifications in KAFKA-5366, the concurrent reader almost always consumes a fraction of the expected messages when there are broker bounces. A consumer running without concurrent writes consumes all the messages from the topic in question in the same test. {noformat} ================================================================================ test_id: kafkatest.tests.core.transactions_test.TransactionsTest.test_transactions.failure_mode=clean_bounce.bounce_target=brokers status: FAIL run time: 1 minute 59.169 seconds Input and concurrently consumed output message sets are not equal. Num input messages: 20000. Num concurrently_consumed_messages: 0 Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py", line 123, in run data = self.run_test() File "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py", line 176, in run_test return self.test_context.function(self.test) File "/usr/local/lib/python2.7/dist-packages/ducktape/mark/_mark.py", line 321, in wrapper return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs) File "/opt/kafka-dev/tests/kafkatest/tests/core/transactions_test.py", line 238, in test_transactions (len(input_message_set), len(concurrently_consumed_message_set)) AssertionError: Input and concurrently consumed output message sets are not equal. Num input messages: 20000. Num concurrently_consumed_messages: 0 {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346)