[
https://issues.apache.org/jira/browse/KAFKA-10285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293739#comment-17293739
]
Monolith commented on KAFKA-10285:
----------------------------------
Hey [~akshaysh], have you found any workaround for this ?
> consumer is not rejoining after restart of broker.
> --------------------------------------------------
>
> Key: KAFKA-10285
> URL: https://issues.apache.org/jira/browse/KAFKA-10285
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 2.3.1
> Reporter: Akshay Sharma
> Priority: Critical
>
> Running single consumer(which is internally using confluent-kafka-go and
> librdkafka 1.4.0) with static membership (group.instance.id 1 and
> session.timeout.ms is 30minutes)
> .
> Test case
> Producer is keep writing to the Kafka and consumer is reading at the same
> time. when killed Kafka-server(broker), could see error below in broker logs
> and consumer is not able to rejoin.
> *****Broker Logs*********
> ```
> ERROR given member.id 1-b4b3f832-567a-4a17-ba5b-e14fb8cb536d is identified as
> a known static member 1,but not matching the expected member.id
> 1-9248bd87-dace-40de-841e-df19c3fe2cd0
> (kafka.coordinator.group.GroupMetadata)
> ```
> Analysis.
> 1) when consumer without restarting of broker Logs:
> [2020-07-17 14:15:50,998] DEBUG Scheduling task auto-leader-rebalance-task
> with initial delay 300000 ms and period -1000 ms. (kafka.utils.KafkaScheduler)
> [2020-07-17 14:15:51,290] DEBUG Accepted connection from /127.0.0.1:46350 on
> /127.0.0.1:9092 and assigned it to processor 1, sendBufferSize
> [actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
> [102400|102400] (kafka.network.Acceptor)
> [2020-07-17 14:15:51,290] DEBUG Processor 1 listening to new connection from
> /127.0.0.1:46350 (kafka.network.Processor)
> [2020-07-17 14:15:51,308] DEBUG Accepted connection from /127.0.0.1:46352 on
> /127.0.0.1:9092 and assigned it to processor 2, sendBufferSize
> [actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
> [102400|102400] (kafka.network.Acceptor)
> [2020-07-17 14:15:51,309] DEBUG Processor 2 listening to new connection from
> /127.0.0.1:46352 (kafka.network.Processor)
> [2020-07-17 14:15:51,423] DEBUG Accepted connection from /127.0.0.1:42156 on
> /127.0.1.1:9092 and assigned it to processor 3, sendBufferSize
> [actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
> [102400|102400] (kafka.network.Acceptor)
> [2020-07-17 14:15:51,425] DEBUG Processor 3 listening to new connection from
> /127.0.0.1:42156 (kafka.network.Processor)
> [2020-07-17 14:15:51,495] DEBUG [Partition test_log-0 broker=0] High
> watermark updated to (offset=1 segment=[0:416]) (kafka.cluster.Partition)
> [2020-07-17 14:15:51,497] DEBUG [ReplicaManager broker=0] Request key
> test_log-0 unblocked 0 fetch requests. (kafka.server.ReplicaManager)
> [2020-07-17 14:15:51,498] DEBUG [ReplicaManager broker=0] Request key
> test_log-0 unblocked 0 producer requests. (kafka.server.ReplicaManager)
> [2020-07-17 14:15:51,498] DEBUG [ReplicaManager broker=0] Request key
> test_log-0 unblocked 0 DeleteRecordsRequest. (kafka.server.ReplicaManager)
> [2020-07-17 14:15:51,501] DEBUG [ReplicaManager broker=0] Produce to local
> log in 53 ms (kafka.server.ReplicaManager)
> [2020-07-17 14:15:52,312] DEBUG Accepted connection from /127.0.0.1:42158 on
> /127.0.1.1:9092 and assigned it to processor 4, sendBufferSize
> [actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
> [102400|102400] (kafka.network.Acceptor)
> [2020-07-17 14:15:52,313] DEBUG Processor 4 listening to new connection from
> /127.0.0.1:42158 (kafka.network.Processor)
> [2020-07-17 14:15:54,342] INFO [GroupCoordinator 0]: Preparing to rebalance
> group 0 in state PreparingRebalance with old generation 0
> (__consumer_offsets-48) (reason: Adding new member
> 1-9248bd87-dace-40de-841e-df19c3fe2cd0 with group instanceid Some(1))
> (kafka.coordinator.group.GroupCoordinator)
> [2020-07-17 14:15:54,357] INFO [GroupCoordinator 0]: Stabilized group 0
> generation 1 (__consumer_offsets-48)
> (kafka.coordinator.group.GroupCoordinator)
> [2020-07-17 14:15:54,383] INFO [GroupCoordinator 0]: Assignment received
> from leader for group 0 for generation 1
> (kafka.coordinator.group.GroupCoordinator)
> [2020-07-17 14:15:54,405] DEBUG [Partition __consumer_offsets-48 broker=0]
> High watermark updated to (offset=1 segment=[0:270]) (kafka.cluster.Partition)
> [2020-07-17 14:15:54,405] DEBUG [ReplicaManager broker=0] Request key
> __consumer_offsets-48 unblocked 0 fetch requests.
> (kafka.server.ReplicaManager)
> [2020-07-17 14:15:54,405] DEBUG [ReplicaManager broker=0] Request key
> __consumer_offsets-48 unblocked 0 producer requests.
> (kafka.server.ReplicaManager)
> [2020-07-17 14:15:54,405] DEBUG [ReplicaManager broker=0] Request key
> __consumer_offsets-48 unblocked 0 DeleteRecordsRequest.
> (kafka.server.ReplicaManager)
> [2020-07-17 14:15:54,405] DEBUG [ReplicaManager broker=0] Produce to local
> log in 3 ms (kafka.server.ReplicaManager)
> [2020-07-17 14:15:54,414] DEBUG Accepted connection from /127.0.0.1:42160 on
> /127.0.1.1:9092 and assigned it to processor 5, sendBufferSize
> [actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
> [102400|102400] (kafka.network.Acceptor)
> [2020-07-17 14:15:54,415] DEBUG Processor 5 listening to new connection from
> /127.0.0.1:42160 (kafka.network.Processor)
>
> 2) when killed consumer but broker is still running LOGS:
> [2020-07-17 14:18:28,266] INFO [GroupCoordinator 0]: Static member Some(1)
> with unknown member id rejoins, assigning new member id
> 1-ce854a05-5c82-42ea-ae45-6f17b2bca9b4, while old member
> 1-9248bd87-dace-40de-841e-df19c3fe2cd0 will be removed.
> (kafka.coordinator.group.GroupCoordinator)
> [2020-07-17 14:18:28,267] DEBUG [Partition test_log-0 broker=0] High
> watermark updated to (offset=22414 segment=[0:8635659])
> (kafka.cluster.Partition)
> [2020-07-17 14:18:28,267] DEBUG [ReplicaManager broker=0] Request key
> test_log-0 unblocked 0 fetch requests. (kafka.server.ReplicaManager)
> [2020-07-17 14:18:28,267] DEBUG [ReplicaManager broker=0] Request key
> test_log-0 unblocked 0 producer requests. (kafka.server.ReplicaManager)
> [2020-07-17 14:18:28,267] DEBUG [ReplicaManager broker=0] Request key
> test_log-0 unblocked 0 DeleteRecordsRequest. (kafka.server.ReplicaManager)
> [2020-07-17 14:18:28,267] DEBUG [ReplicaManager broker=0] Produce to local
> log in 0 ms (kafka.server.ReplicaManager)
> [2020-07-17 14:18:28,269] INFO [GroupCoordinator 0]: Static member joins
> during Stable stage will not trigger rebalance.
> (kafka.coordinator.group.GroupCoordinator)
> [2020-07-17 14:18:28,273] DEBUG [Partition test-0 broker=0] High watermark
> updated to (offset=11776 segment=[0:1764875]) (kafka.cluster.Partition)
> [2020-07-17 14:18:28,273] DEBUG [ReplicaManager broker=0] Request key test-0
> unblocked 0 fetch requests. (kafka.server.ReplicaManager)
> [2020-07-17 14:18:28,273] DEBUG [ReplicaManager broker=0] Request key test-0
> unblocked 0 producer requests. (kafka.server.ReplicaManager)
> [2020-07-17 14:18:28,273] DEBUG [ReplicaManager broker=0] Request key test-0
> unblocked 0 DeleteRecordsRequest. (kafka.server.ReplicaManager)
> [2020-07-17 14:18:28,273] DEBUG [ReplicaManager broker=0] Produce to local
> log in 0 ms (kafka.server.ReplicaManager)
>
> 3) Again tried killing consumer but broker is still running.
> [2020-07-17 14:34:32,645] INFO [GroupCoordinator 0]: Static member Some(1)
> with unknown member id rejoins, assigning new member id
> 1-b4b3f832-567a-4a17-ba5b-e14fb8cb536d, while old member
> 1-ce854a05-5c82-42ea-ae45-6f17b2bca9b4 will be removed.
> (kafka.coordinator.group.GroupCoordinator)
> [2020-07-17 14:34:32,645] INFO [GroupCoordinator 0]: Static member joins
> during Stable stage will not trigger rebalance.
> (kafka.coordinator.group.GroupCoordinator)
>
> 4) Consumer is running now tried killer broker logs
> [2020-07-17 14:37:36,534] INFO [GroupMetadataManager brokerId=0] Finished
> loading offsets and group metadata from __consumer_offsets-45 in 0
> milliseconds. (kafka.coordinator.group.GroupMetadataManager)
> [2020-07-17 14:37:36,597] DEBUG [GroupMetadataManager brokerId=0] Loaded
> group metadata GroupMetadata(groupId=0, generation=1,
> protocolType=Some(consumer), currentState=Stable,
> members=Map(1-9248bd87-dace-40de-841e-df19c3fe2cd0 ->
> MemberMetadata(memberId=1-9248bd87-dace-40de-841e-df19c3fe2cd0,
> groupInstanceId=Some(1), clientId=rdkafka, clientHost=/127.0.0.1,
> sessionTimeoutMs=1800000, rebalanceTimeoutMs=1800000,
> supportedProtocols=List(range), ))) with offsets Map() and pending offsets
> Map() (kafka.coordinator.group.GroupMetadataManager)
> [2020-07-17 14:37:36,599] INFO [GroupCoordinator 0]: Loading group metadata
> for 0 with generation 1 (kafka.coordinator.group.GroupCoordinator)
> [2020-07-17 14:37:36,611] INFO [GroupMetadataManager brokerId=0] Finished
> loading offsets and group metadata from __consumer_offsets-48 in 77
> milliseconds. (kafka.coordinator.group.GroupMetadataManager)
> [2020-07-17 14:37:39,485] DEBUG Accepted connection from /127.0.0.1:46498 on
> /127.0.0.1:9092 and assigned it to processor 1, sendBufferSize
> [actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
> [102400|102400] (kafka.network.Acceptor)
> [2020-07-17 14:37:39,487] DEBUG Processor 1 listening to new connection from
> /127.0.0.1:46498 (kafka.network.Processor)
> [2020-07-17 14:37:40,234] DEBUG Accepted connection from /127.0.0.1:46500 on
> /127.0.0.1:9092 and assigned it to processor 2, sendBufferSize
> [actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
> [102400|102400] (kafka.network.Acceptor)
> [2020-07-17 14:37:40,238] DEBUG Processor 2 listening to new connection from
> /127.0.0.1:46500 (kafka.network.Processor)
> [2020-07-17 14:37:41,045] DEBUG Scheduling task auto-leader-rebalance-task
> with initial delay 300000 ms and period -1000 ms. (kafka.utils.KafkaScheduler)
> [2020-07-17 14:37:44,046] DEBUG Accepted connection from /127.0.0.1:42304 on
> /127.0.1.1:9092 and assigned it to processor 3, sendBufferSize
> [actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
> [102400|102400] (kafka.network.Acceptor)
> [2020-07-17 14:37:44,047] DEBUG Processor 3 listening to new connection from
> /127.0.0.1:42304 (kafka.network.Processor)
> [2020-07-17 14:37:44,060] DEBUG Created a new full FetchContext with 1
> partition(s). Will not try to create a new session.
> (kafka.server.FetchManager)
> [2020-07-17 14:37:44,071] DEBUG Accepted connection from /127.0.0.1:42306 on
> /127.0.1.1:9092 and assigned it to processor 4, sendBufferSize
> [actual|requested]: [102400|102400] recvBufferSize [actual|requested]:
> [102400|102400] (kafka.network.Acceptor)
> [2020-07-17 14:37:44,072] DEBUG Processor 4 listening to new connection from
> /127.0.0.1:42306 (kafka.network.Processor)
> [2020-07-17 14:37:45,375] ERROR given member.id
> 1-b4b3f832-567a-4a17-ba5b-e14fb8cb536d is identified as a known static member
> 1,but not matching the expected member.id
> 1-9248bd87-dace-40de-841e-df19c3fe2cd0 (kafka.coordinator.group.GroupMetadata)
>
> 1) When consumer is killed without broker restarting , consumer is able to
> join but everytime group coordinator is assigning him a new member id.
> Is this expecting?
> 2) When consumer is running but broker is killed/restarted,
> groupmeta-datamanager is loading the metadata where i could see the expecting
> member id is the id which was assigned during first time(inital stage where
> consumer joined freshly without any restart).Is this expecting? and consumer
> is not able to join.
>
> *consumer logs*
> %7|1594908244.958|RECV|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> GroupCoordinator/1001: Received HeartbeatResponse (v3, 6 bytes, CorrId 59,
> rtt 30.92ms)
> %7|1594908244.958|HEARTBEAT|rdkafka#consumer-2| [thrd:main]: Group "0"
> heartbeat error response in state up (join state started, 1 partition(s)
> assigned): Broker: Static consumer fenced by other consumer with same
> group.instance.id
> %0|1594908244.958|FATAL|rdkafka#consumer-2| [thrd:main]: Fatal error:
> Broker: Static consumer fenced by other consumer with same group.instance.id:
> Fatal consumer error: Broker: Static consumer fenced by other consumer with
> same group.instance.id
> %7|1594908244.959|REBALANCE|rdkafka#consumer-2| [thrd:main]: Group "0" is
> rebalancing in state up (join-state started) with assignment: consumer fenced
> by newer instance
> %7|1594908244.959|PAUSE|rdkafka#consumer-2| [thrd:main]: Library pausing 1
> partition(s)
> %7|1594908244.959|BARRIER|rdkafka#consumer-2| [thrd:main]: test [0]:
> rd_kafka_toppar_op_pause_resume:2393: new version barrier v3
> %7|1594908244.959|PAUSE|rdkafka#consumer-2| [thrd:main]: Pause test [0] (v3)
> %7|1594908244.960|CGRPJOINSTATE|rdkafka#consumer-2| [thrd:main]: Group "0"
> changed join state started -> wait-unassign (v4, state up)
> %7|1594908244.960|BARRIER|rdkafka#consumer-2| [thrd:main]: Group "0":
> rd_kafka_cgrp_unassign:2476: new version barrier v5
> %7|1594908244.960|UNASSIGN|rdkafka#consumer-2| [thrd:main]: Group "0":
> unassigning 1 partition(s) (v5)
> %7|1594908244.960|BARRIER|rdkafka#consumer-2| [thrd:main]: test [0]:
> rd_kafka_toppar_op_fetch_stop:2334: new version barrier v4
> %7|1594908244.961|CONSUMER|rdkafka#consumer-2| [thrd:main]: Stop consuming
> test [0] (v4)
> %7|1594908244.962|DESP|rdkafka#consumer-2| [thrd:main]: Removing (un)desired
> topic test [0]
> %7|1594908244.962|RESUME|rdkafka#consumer-2| [thrd:main]: Library resuming 1
> partition(s)
> %7|1594908244.962|BARRIER|rdkafka#consumer-2| [thrd:main]: test [0]:
> rd_kafka_toppar_op_pause_resume:2393: new version barrier v5
> %7|1594908244.962|RESUME|rdkafka#consumer-2| [thrd:main]: Resume test [0]
> (v5)
> %7|1594908244.962|UNASSIGN|rdkafka#consumer-2| [thrd:main]: Unassign not
> done yet (1 wait_unassign, 1 assigned, 0 wait commit, join state
> wait-unassign): unassign
> %7|1594908244.963|OP|rdkafka#consumer-2| [thrd:main]: test [0] received op
> PAUSE (v3) in fetch-state active (opv2)
> %7|1594908244.963|PAUSE|rdkafka#consumer-2| [thrd:main]: Pause test [0]: at
> offset 43472 (state active, v3)
> %7|1594908244.965|RECV|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Received FetchResponse (v11, 366 bytes, CorrId 13188, rtt 11.97ms)
> %7|1594908244.965|FETCH|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001: Topic
> test [0] MessageSet size 300, error "Success", MaxOffset 46378, LSO 46378,
> Ver 2/2
> %7|1594908244.965|CONSUME|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Enqueue 2 message(s) (86 bytes, 2 ops) on test [0] fetch queue (qlen 2, v2,
> last_offset 46377, 0 ctrl msgs, uncompressed)
> %7|1594908244.965|FETCHDEC|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: Topic test [0]: fetch
> decide: updating to version 3 (was 2) at offset 43472 (was 46378)
> %7|1594908244.965|FETCH|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001: Topic
> test [0] in state active at offset 43472 (0/100000 msgs, 0/1048576 kb queued,
> opv 3) is not fetchable: paused
> %7|1594908244.965|FETCHADD|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Removed test [0] from fetch list (0 entries, opv 3): paused
> {"Component_name":"Message-Reader","Instance_id":"1","err":
> {"Code":82,"Str":"Fatal consumer error: Broker: Static consumer fenced by
> other consumer with same group.instance.id"}
> %7|1594908244.967|OP|rdkafka#consumer-2| [thrd:main]: test [0] received op
> FETCH_STOP (v4) in fetch-state active (opv3)
> %7|1594908244.967|FETCH|rdkafka#consumer-2| [thrd:main]: Stopping fetch for
> test [0] in state active (v4)
> %7|1594908244.967|PARTSTATE|rdkafka#consumer-2| [thrd:main]: Partition test
> [0] changed fetch state active -> stopping
> %7|1594908244.967|STORETERM|rdkafka#consumer-2| [thrd:main]: test [0]:
> offset store terminating
> %7|1594908244.967|PARTSTATE|rdkafka#consumer-2| [thrd:main]: Partition test
> [0] changed fetch state stopping -> stopped
> %7|1594908244.968|OP|rdkafka#consumer-2| [thrd:main]: test [0] received op
> PAUSE (v5) in fetch-state stopped (opv4)
> %7|1594908244.968|RESUME|rdkafka#consumer-2| [thrd:main]: Not resuming
> stopped test [0]: at offset 43472 (state stopped, v5)
> %7|1594908244.968|CGRPOP|rdkafka#consumer-2| [thrd:main]: Group "0" received
> op PARTITION_LEAVE in state up (join state wait-unassign, v5) for test [0]
> %7|1594908244.968|PARTDEL|rdkafka#consumer-2| [thrd:main]: Group "0": delete
> test [0]
> %7|1594908244.968|CGRPOP|rdkafka#consumer-2| [thrd:main]: Group "0" received
> op REPLY:FETCH_STOP in state up (join state wait-unassign, v5) for test [0]
> %7|1594908244.968|UNASSIGN|rdkafka#consumer-2| [thrd:main]: Group "0":
> unassign done in state up (join state wait-unassign): without new assignment:
> FETCH_STOP done
> %7|1594908244.968|CGRPJOINSTATE|rdkafka#consumer-2| [thrd:main]: Group "0"
> changed join state wait-unassign -> init (v5, state up)
> %7|1594908245.858|CONNECT|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: broker in state
> TRY_CONNECT connecting
> %7|1594908245.858|STATE|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
> %7|1594908245.858|BROADCAST|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: Broadcasting state change
> %7|1594908245.862|CONNECT|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connecting to
> ipv4#10.233.121.130:9092 (plaintext) with socket 21
> %7|1594908245.863|CONNECT|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connected to
> ipv4#10.233.121.130:9092
> %7|1594908245.863|CONNECTED|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connected (#2)
> %7|1594908245.863|STATE|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY
> %7|1594908245.863|BROADCAST|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: Broadcasting state change
> %7|1594908245.863|SEND|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Sent ApiVersionRequest (v3, 40 bytes @ 0, CorrId 8)
> %7|1594908245.866|RECV|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Received ApiVersionResponse (v3, 6 bytes, CorrId 8,
> rtt 2.29ms)
> %7|1594908245.866|PROTOERR|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Protocol parse
> failure for ApiVersion v3 at 3/6 (rd_kafka_handle_ApiVersion:1911) (incorrect
> broker.version.fallback?)
> %7|1594908245.868|PROTOERR|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiArrayCnt -1 out
> of range
> %7|1594908245.868|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiVersionRequest
> v3 failed due to UNSUPPORTED_VERSION: retrying with v0
> %7|1594908245.869|SEND|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Sent ApiVersionRequest (v0, 21 bytes @ 0, CorrId 9)
> %7|1594908245.870|RECV|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Received ApiVersionResponse (v0, 276 bytes, CorrId
> 9, rtt 1.18ms)
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Broker API support:
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey Produce (0)
> Versions 0..7
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey Fetch (1)
> Versions 0..11
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey Offset (2)
> Versions 0..5
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey Metadata
> (3) Versions 0..8
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> LeaderAndIsr (4) Versions 0..2
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey StopReplica
> (5) Versions 0..1
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> UpdateMetadata (6) Versions 0..5
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> ControlledShutdown (7) Versions 0..2
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> OffsetCommit (8) Versions 0..7
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey OffsetFetch
> (9) Versions 0..5
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> FindCoordinator (10) Versions 0..2
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey JoinGroup
> (11) Versions 0..5
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey Heartbeat
> (12) Versions 0..3
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey LeaveGroup
> (13) Versions 0..2
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey SyncGroup
> (14) Versions 0..3
> %7|1594908245.870|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> DescribeGroups (15) Versions 0..3
> %7|1594908245.871|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey ListGroups
> (16) Versions 0..2
> %7|1594908245.871|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> SaslHandshake (17) Versions 0..1
> %7|1594908245.871|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey ApiVersion
> (18) Versions 0..2
> %7|1594908245.871|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> CreateTopics (19) Versions 0..3
> %7|1594908245.871|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> DeleteTopics (20) Versions 0..3
> %7|1594908245.871|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> DeleteRecords (21) Versions 0..1
> %7|1594908245.871|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> InitProducerId (22) Versions 0..1
> %7|1594908245.871|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> OffsetForLeaderEpoch (23) Versions 0..3
> %7|1594908245.871|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> AddPartitionsToTxn (24) Versions 0..1
> %7|1594908245.874|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> AddOffsetsToTxn (25) Versions 0..1
> %7|1594908245.874|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey EndTxn (26)
> Versions 0..1
> %7|1594908245.874|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> WriteTxnMarkers (27) Versions 0..0
> %7|1594908245.874|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> TxnOffsetCommit (28) Versions 0..2
> %7|1594908245.874|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> DescribeAcls (29) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey CreateAcls
> (30) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey DeleteAcls
> (31) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> DescribeConfigs (32) Versions 0..2
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> AlterConfigs (33) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> AlterReplicaLogDirs (34) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> DescribeLogDirs (35) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> SaslAuthenticate (36) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> CreatePartitions (37) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> CreateDelegationToken (38) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> RenewDelegationToken (39) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> ExpireDelegationToken (40) Versions 0..1
> %7|1594908245.877|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> DescribeDelegationToken (41) Versions 0..1
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey
> DeleteGroups (42) Versions 0..1
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey Unknown-43?
> (43) Versions 0..0
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: ApiKey Unknown-44?
> (44) Versions 0..0
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature MsgVer1:
> Produce (2..2) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature MsgVer1:
> Fetch (2..2) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> MsgVer1
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature MsgVer2:
> Produce (3..3) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature MsgVer2:
> Fetch (4..4) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> MsgVer2
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> ApiVersion: ApiVersion (0..0) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> ApiVersion
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> BrokerGroupCoordinator: FindCoordinator (0..0) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> BrokerGroupCoordinator
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> BrokerBalancedConsumer: FindCoordinator (0..0) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> BrokerBalancedConsumer
> %7|1594908245.878|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> ThrottleTime: Produce (1..2) supported by broker
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> ThrottleTime: Fetch (1..2) supported by broker
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> ThrottleTime
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature Sasl:
> JoinGroup (0..0) supported by broker
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> Sasl
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> SaslHandshake: SaslHandshake (0..0) supported by broker
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> SaslHandshake
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature LZ4:
> FindCoordinator (0..0) supported by broker
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> LZ4
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> OffsetTime: Offset (1..1) supported by broker
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> OffsetTime
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> IdempotentProducer: InitProducerId (0..0) supported by broker
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> IdempotentProducer
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature ZSTD:
> Produce (7..7) supported by broker
> %7|1594908245.879|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature ZSTD:
> Fetch (10..10) supported by broker
> %7|1594908245.880|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> ZSTD
> %7|1594908245.880|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> SaslAuthReq: SaslHandshake (1..1) supported by broker
> %7|1594908245.880|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Feature
> SaslAuthReq: SaslAuthenticate (0..0) supported by broker
> %7|1594908245.880|APIVERSION|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Enabling feature
> SaslAuthReq
> %7|1594908245.880|STATE|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Broker changed state APIVERSION_QUERY -> UP
> %7|1594908245.880|BROADCAST|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: Broadcasting state change
> %7|1594908245.880|METADATA|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: Skipping metadata refresh of 1 topic(s):
> connected: already being requested
> %7|1594908360.596|METADATA|rdkafka#consumer-2| [thrd:main]: Requesting
> metadata for 1/1 topics: periodic topic and broker list refresh
> %7|1594908360.597|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Request metadata for 1 topic(s): periodic topic and
> broker list refresh
> %7|1594908360.597|SEND|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001: Sent
> MetadataRequest (v2, 31 bytes @ 0, CorrId 13189)
> %7|1594908360.601|RECV|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Received MetadataResponse (v2, 153 bytes, CorrId 13189, rtt 3.74ms)
> %7|1594908360.602|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: ===== Received metadata (for 1 requested topics):
> periodic topic and broker list refresh =====
> %7|1594908360.603|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: ClusterId: n05vTLn3R0qdkQZQ235wTQ, ControllerId: 1001
> %7|1594908360.603|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: 1 brokers, 1 topics
> %7|1594908360.603|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Broker #0/1: localhost:9092 NodeId 1001
> %7|1594908360.603|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Topic #0/1: test with 1 partitions
> %7|1594908360.603|METADATA|rdkafka#consumer-2| [thrd:main]: Topic test
> partition 0 Leader 1001
> %7|1594908360.603|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: 1/1 requested topic(s) seen in metadata
> %7|1594908660.597|METADATA|rdkafka#consumer-2| [thrd:main]: Requesting
> metadata for 1/1 topics: periodic topic and broker list refresh
> %7|1594908660.597|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Request metadata for 1 topic(s): periodic topic and
> broker list refresh
> %7|1594908660.598|SEND|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001: Sent
> MetadataRequest (v2, 31 bytes @ 0, CorrId 13190)
> %7|1594908660.602|RECV|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Received MetadataResponse (v2, 153 bytes, CorrId 13190, rtt 4.12ms)
> %7|1594908660.602|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: ===== Received metadata (for 1 requested topics):
> periodic topic and broker list refresh =====
> %7|1594908660.602|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: ClusterId: n05vTLn3R0qdkQZQ235wTQ, ControllerId: 1001
> %7|1594908660.603|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: 1 brokers, 1 topics
> %7|1594908660.603|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Broker #0/1: localhost:9092 NodeId 1001
> %7|1594908660.603|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Topic #0/1: test with 1 partitions
> %7|1594908660.603|METADATA|rdkafka#consumer-2| [thrd:main]: Topic test
> partition 0 Leader 1001
> %7|1594908660.604|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: 1/1 requested topic(s) seen in metadata
> %7|1594908844.966|BROKERFAIL|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> GroupCoordinator: failed: err: Local: Broker transport failure: (errno:
> Connection reset by peer)
> %7|1594908844.967|FAIL|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> GroupCoordinator: Disconnected (after 602775ms in state UP)
> %7|1594908844.967|STATE|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> GroupCoordinator: Broker changed state UP -> DOWN
> %7|1594908844.967|BROADCAST|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> Broadcasting state change
> %7|1594908844.967|BUFQ|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> GroupCoordinator/1001: Purging bufq with 0 buffers
> %7|1594908844.967|BUFQ|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> GroupCoordinator/1001: Purging bufq with 0 buffers
> %7|1594908844.967|BUFQ|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> GroupCoordinator/1001: Updating 0 buffers on connection reset
> %7|1594908844.967|METADATA|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> Requesting metadata for 1/1 topics: broker down
> %7|1594908844.967|METADATA|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> localhost:9092/1001: Request metadata for 1 topic(s): broker down
> %7|1594908844.967|STATE|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> GroupCoordinator: Broker changed state DOWN -> INIT
> %7|1594908844.968|BROADCAST|rdkafka#consumer-2| [thrd:GroupCoordinator]:
> Broadcasting state change
> %7|1594908844.968|SEND|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001: Sent
> MetadataRequest (v2, 31 bytes @ 0, CorrId 13191)
> %7|1594908844.974|CGRPSTATE|rdkafka#consumer-2| [thrd:main]: Group "0"
> changed state up -> query-coord (v5, join-state init)
> %7|1594908844.974|BROADCAST|rdkafka#consumer-2| [thrd:main]: Broadcasting
> state change
> %7|1594908844.974|CGRPQUERY|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Group "0": querying for coordinator: intervaled in state
> query-coord
> %7|1594908844.974|CGRPSTATE|rdkafka#consumer-2| [thrd:main]: Group "0"
> changed state query-coord -> wait-coord (v5, join-state init)
> %7|1594908844.974|BROADCAST|rdkafka#consumer-2| [thrd:main]: Broadcasting
> state change
> %7|1594908844.976|SEND|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001: Sent
> FindCoordinatorRequest (v2, 25 bytes @ 0, CorrId 13192)
> %7|1594908844.982|RECV|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Received MetadataResponse (v2, 153 bytes, CorrId 13191, rtt 12.73ms)
> %7|1594908844.983|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: ===== Received metadata (for 1 requested topics): broker
> down =====
> %7|1594908844.983|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: ClusterId: n05vTLn3R0qdkQZQ235wTQ, ControllerId: 1001
> %7|1594908844.983|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: 1 brokers, 1 topics
> %7|1594908844.983|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Broker #0/1: localhost:9092 NodeId 1001
> %7|1594908844.983|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Topic #0/1: test with 1 partitions
> %7|1594908844.983|METADATA|rdkafka#consumer-2| [thrd:main]: Topic test
> partition 0 Leader 1001
> %7|1594908844.983|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: 1/1 requested topic(s) seen in metadata
> %7|1594908844.988|RECV|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Received FindCoordinatorResponse (v2, 88 bytes, CorrId 13192, rtt 12.27ms)
> %7|1594908844.988|CGRPCOORD|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Group "0" coordinator is localhost:9092 id 1001
> %7|1594908844.988|CGRPSTATE|rdkafka#consumer-2| [thrd:main]: Group "0"
> changed state wait-coord -> wait-broker-transport (v5, join-state init)
> %7|1594908844.988|BROADCAST|rdkafka#consumer-2| [thrd:main]: Broadcasting
> state change
> %7|1594908845.871|BROKERFAIL|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: failed: err:
> Local: Broker transport failure: (errno: Connection reset by peer)
> %7|1594908845.871|FAIL|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Disconnected (after 599990ms in state UP)
> %7|1594908845.871|STATE|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Broker changed state UP -> DOWN
> %7|1594908845.871|BROADCAST|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: Broadcasting state change
> %7|1594908845.871|BUFQ|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Purging bufq with 0 buffers
> %7|1594908845.871|BUFQ|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Purging bufq with 0 buffers
> %7|1594908845.871|BUFQ|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Updating 0 buffers on connection reset
> %7|1594908845.871|METADATA|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: Requesting metadata for 1/1 topics: broker
> down
> %7|1594908845.871|METADATA|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: localhost:9092/1001: Request metadata for 1
> topic(s): broker down
> %7|1594908845.871|STATE|rdkafka#consumer-2| [thrd:localhost:9092/bootstrap]:
> localhost:9092/bootstrap: Broker changed state DOWN -> INIT
> %7|1594908845.871|BROADCAST|rdkafka#consumer-2|
> [thrd:localhost:9092/bootstrap]: Broadcasting state change
> %7|1594908845.872|SEND|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001: Sent
> MetadataRequest (v2, 31 bytes @ 0, CorrId 13193)
> %7|1594908845.876|RECV|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Received MetadataResponse (v2, 153 bytes, CorrId 13193, rtt 4.91ms)
> %7|1594908845.877|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: ===== Received metadata (for 1 requested topics): broker
> down =====
> %7|1594908845.877|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: ClusterId: n05vTLn3R0qdkQZQ235wTQ, ControllerId: 1001
> %7|1594908845.877|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: 1 brokers, 1 topics
> %7|1594908845.877|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Broker #0/1: localhost:9092 NodeId 1001
> %7|1594908845.877|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Topic #0/1: test with 1 partitions
> %7|1594908845.877|METADATA|rdkafka#consumer-2| [thrd:main]: Topic test
> partition 0 Leader 1001
> %7|1594908845.877|METADATA|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: 1/1 requested topic(s) seen in metadata
> %7|1594908845.976|CGRPQUERY|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Group "0": querying for coordinator: intervaled in state
> wait-broker-transport
> %7|1594908845.977|SEND|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001: Sent
> FindCoordinatorRequest (v2, 25 bytes @ 0, CorrId 13194)
> %7|1594908846.018|RECV|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Received FindCoordinatorResponse (v2, 88 bytes, CorrId 13194, rtt 40.71ms)
> %7|1594908846.018|CGRPCOORD|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Group "0" coordinator is localhost:9092 id 1001
> %7|1594908846.977|CGRPQUERY|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Group "0": querying for coordinator: intervaled in state
> wait-broker-transport
> %7|1594908846.977|SEND|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001: Sent
> FindCoordinatorRequest (v2, 25 bytes @ 0, CorrId 13195)
> %7|1594908846.984|RECV|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Received FindCoordinatorResponse (v2, 88 bytes, CorrId 13195, rtt 7.08ms)
> %7|1594908846.985|CGRPCOORD|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Group "0" coordinator is localhost:9092 id 1001
> %7|1594908847.977|CGRPQUERY|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Group "0": querying for coordinator: intervaled in state
> wait-broker-transport
> %7|1594908847.977|SEND|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001: Sent
> FindCoordinatorRequest (v2, 25 bytes @ 0, CorrId 13196)
> %7|1594908847.983|RECV|rdkafka#consumer-2|
> [thrd:-kafka-0.localhost.default.svc.cluster.lo]: localhost:9092/1001:
> Received FindCoordinatorResponse (v2, 88 bytes, CorrId 13196, rtt 5.66ms)
> %7|1594908847.983|CGRPCOORD|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Group "0" coordinator is localhost:9092 id 1001
> %7|1594908848.977|CGRPQUERY|rdkafka#consumer-2| [thrd:main]:
> localhost:9092/1001: Group "0": querying for coordinator: intervaled in state
> wait-broker-transport
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
