Donny Nadolny created KAFKA-13263:
-------------------------------------
Summary: TimeoutException when calling initializeTransactions() in
a healthy cluster, persisting until a broker was restarted
Key: KAFKA-13263
URL: https://issues.apache.org/jira/browse/KAFKA-13263
Project: Kafka
Issue Type: Bug
Affects Versions: 2.6.2
Reporter: Donny Nadolny
Summary:
- we had multiple transactional kafka clients (different Apache Flink
applications) stop working at the same time, all of them logging
{{org.apache.kafka.common.errors.TimeoutException: Timeout expired while
initializing transactional state in 60000ms.}}
- this problem persisted for several hours
- restarting the kafka process on a single broker (the leader for a
{{__transaction_state}} partition that seemed to be the culprit) immediately
the problem
Some details:
- the kafka cluster seemed healthy the whole time: no brokers were down, no
under-replicated partitions, etc. No non-transactional clients seemed to be
affected
- we first tried changing the {{transactional.id}} that the clients use, to
see if some existing transaction state was the problem - it didn't work, same
exception
- our transaction & cleanup timeouts are longer than the default: 1 hour for
{{transaction.max.timeout.ms}} and 1 day for
{{transaction.remove.expired.transaction.cleanup.interval.ms}}. The problem
lasted for several hours but less than 1 day
- we diagnosed this by noticing that the "Initialized transactionalId" log
lines in kafka from the TransactionCoordinator were present for all partitions
except for one partition: {{__transaction_state-29}}. The leader of that
partition (broker 1266) was the broker we restarted which fixed this.
- immediately after restarting the process on broker 1266, a follower took
over, and shortly after that 1266 (the preferred leader) again became leader,
and we didn't get any more timeouts
Logs:
All logs cluster-wide with {{__transaction_state-29}} from the time when this
started happening, lightly redacted with {{--removed--}}:
{code:java}
[2021-08-31 18:22:09.596915] INFO TransactionCoordinator:66 -
[TransactionCoordinator id=1232] Initialized transactionalId Source: flinkapp
-> Try.get -> flat map to entry -> Sink: kafka-sink:
flinkapp-b7463a48c0aedd250dc469cc1ffc08dc-50 with producerId 2116001 and
producer epoch 11068 on partition __transaction_state-29
[2021-08-31 18:22:30.069676] DEBUG KafkaController:62 - [Controller id=1214]
Topics not in preferred replica for broker 1266 Map(--removed--,
__transaction_state-29 -> ArrayBuffer(1266, 1232, 1187), --removed--.
[2021-08-31 18:22:30.070326] INFO KafkaController:66 - [Controller id=1214]
Starting replica leader election (PREFERRED) for partitions
--removed--,__transaction_state-29,--removed triggered by AutoTriggered
[2021-08-31 18:22:30.096160] INFO logger:66 - [Controller id=1214 epoch=157]
Changed partition __transaction_state-29 from OnlinePartition to
OnlinePartition with state LeaderAndIsr(leader=1266, leaderEpoch=222,
isr=List(1187, 1232, 1266), zkVersion=403)
[2021-08-31 18:22:30.101069] TRACE logger:54 - [Controller id=1214 epoch=157]
Sending become-follower LeaderAndIsr request
LeaderAndIsrPartitionState(topicName='__transaction_state', partitionIndex=29,
controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266],
zkVersion=403, replicas=[1266, 1232, 1187], addingReplicas=[],
removingReplicas=[], isNew=false) to broker 1187 for partition
__transaction_state-29
[2021-08-31 18:22:30.102259] TRACE logger:54 - [Broker id=1187] Handling
LeaderAndIsr request correlationId 9187 from controller 1214 epoch 157 starting
the become-follower transition for partition __transaction_state-29 with leader
1266
[2021-08-31 18:22:30.102374] INFO logger:66 - [Broker id=1187] Follower
__transaction_state-29 starts at leader epoch 222 from offset 2199146 with high
watermark 2199146. Previous leader epoch was 221.
[2021-08-31 18:22:30.103121] INFO ReplicaFetcherManager:66 -
[ReplicaFetcherManager on broker 1187] Removed fetcher for partitions
Set(pmplatform_recon.reconciliation_state_updated.bson-3,
__transaction_state-29)
[2021-08-31 18:22:30.106119] INFO ReplicaFetcherManager:66 -
[ReplicaFetcherManager on broker 1187] Added fetcher to broker 1266 for
partitions Map(__transaction_state-29 -> (offset=2199146, leaderEpoch=222))
[2021-08-31 18:22:30.106277] TRACE logger:54 - [Broker id=1187] Completed
LeaderAndIsr request correlationId 9187 from controller 1214 epoch 157 for the
become-follower transition for partition __transaction_state-29 with leader 1266
[2021-08-31 18:22:30.109601] INFO ReplicaFetcherThread:66 - [ReplicaFetcher
replicaId=1187, leaderId=1266, fetcherId=1] Retrying leaderEpoch request for
partition __transaction_state-29 as the leader reported an error:
UNKNOWN_LEADER_EPOCH
[2021-08-31 18:22:30.109999] INFO TransactionStateManager:66 - [Transaction
State Manager 1187]: No cached transaction metadata found for
__transaction_state-29 during become-follower transition
[2021-08-31 18:22:30.123016] TRACE logger:54 - [Controller id=1214 epoch=157]
Sending become-leader LeaderAndIsr request
LeaderAndIsrPartitionState(topicName='__transaction_state', partitionIndex=29,
controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266],
zkVersion=403, replicas=[1266, 1232, 1187], addingReplicas=[],
removingReplicas=[], isNew=false) to broker 1266 for partition
__transaction_state-29
[2021-08-31 18:22:30.133232] TRACE logger:54 - [Controller id=1214 epoch=157]
Sending become-follower LeaderAndIsr request
LeaderAndIsrPartitionState(topicName='__transaction_state', partitionIndex=29,
controllerEpoch=157, leader=1266, leaderEpoch=222, isr=[1187, 1232, 1266],
zkVersion=403, replicas=[1266, 1232, 1187], addingReplicas=[],
removingReplicas=[], isNew=false) to broker 1232 for partition
__transaction_state-29
[2021-08-31 18:22:30.134325] TRACE logger:54 - [Broker id=1232] Handling
LeaderAndIsr request correlationId 8854 from controller 1214 epoch 157 starting
the become-follower transition for partition __transaction_state-29 with leader
1266
[2021-08-31 18:22:30.134665] INFO logger:66 - [Broker id=1232] Follower
__transaction_state-29 starts at leader epoch 222 from offset 2199146 with high
watermark 2199146. Previous leader epoch was 221.
[2021-08-31 18:22:30.135064] INFO ReplicaFetcherManager:66 -
[ReplicaFetcherManager on broker 1232] Removed fetcher for partitions
Set(--removed--, __transaction_state-29)
[2021-08-31 18:22:30.137097] INFO ReplicaFetcherThread:66 - [ReplicaFetcher
replicaId=1266, leaderId=1232, fetcherId=1] Partition __transaction_state-29
has an older epoch (221) than the current leader. Will await the new
LeaderAndIsr state before resuming fetching.
[2021-08-31 18:22:30.137356] WARN ReplicaFetcherThread:70 - [ReplicaFetcher
replicaId=1266, leaderId=1232, fetcherId=1] Partition __transaction_state-29
marked as failed
[2021-08-31 18:22:30.140900] INFO ReplicaFetcherManager:66 -
[ReplicaFetcherManager on broker 1232] Added fetcher to broker 1266 for
partitions Map(__transaction_state-29 -> (offset=2199146, leaderEpoch=222))
[2021-08-31 18:22:30.142692] TRACE logger:54 - [Broker id=1232] Completed
LeaderAndIsr request correlationId 8854 from controller 1214 epoch 157 for the
become-follower transition for partition __transaction_state-29 with leader 1266
[2021-08-31 18:22:30.142935] INFO ReplicaFetcherThread:66 - [ReplicaFetcher
replicaId=1232, leaderId=1266, fetcherId=1] Retrying leaderEpoch request for
partition __transaction_state-29 as the leader reported an error:
UNKNOWN_LEADER_EPOCH
[2021-08-31 18:22:30.143842] INFO TransactionStateManager:66 - [Transaction
State Manager 1232]: Unloaded transaction metadata
TxnMetadataCacheEntry(coordinatorEpoch=220, numTransactionalEntries=21) for
__transaction_state-29 on become-follower transition
[2021-08-31 18:22:30.146822] TRACE logger:54 - [Broker id=1266] Handling
LeaderAndIsr request correlationId 8863 from controller 1214 epoch 157 starting
the become-leader transition for partition __transaction_state-29
[2021-08-31 18:22:30.147722] TRACE logger:54 - [Broker id=1237] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8873
[2021-08-31 18:22:30.148472] TRACE logger:54 - [Broker id=1196] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8920
[2021-08-31 18:22:30.148704] TRACE logger:54 - [Broker id=1243] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8833
[2021-08-31 18:22:30.148825] TRACE logger:54 - [Broker id=1231] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8834
[2021-08-31 18:22:30.148852] TRACE logger:54 - [Broker id=1206] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9115
[2021-08-31 18:22:30.148997] TRACE logger:54 - [Broker id=1260] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9109
[2021-08-31 18:22:30.148999] TRACE logger:54 - [Broker id=1275] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8895
[2021-08-31 18:22:30.149063] TRACE logger:54 - [Broker id=1254] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8818
[2021-08-31 18:22:30.149100] TRACE logger:54 - [Broker id=1195] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9049
[2021-08-31 18:22:30.149123] TRACE logger:54 - [Broker id=1187] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9188
[2021-08-31 18:22:30.149166] TRACE logger:54 - [Broker id=1262] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8804
[2021-08-31 18:22:30.149449] TRACE logger:54 - [Broker id=1218] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9308
[2021-08-31 18:22:30.149573] TRACE logger:54 - [Broker id=1258] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8818
[2021-08-31 18:22:30.149793] TRACE logger:54 - [Broker id=1250] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9040
[2021-08-31 18:22:30.149805] TRACE logger:54 - [Broker id=1256] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9029
[2021-08-31 18:22:30.150028] TRACE logger:54 - [Broker id=1210] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8934
[2021-08-31 18:22:30.150155] TRACE logger:54 - [Broker id=1264] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8769
[2021-08-31 18:22:30.150325] TRACE logger:54 - [Broker id=1268] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9103
[2021-08-31 18:22:30.150325] TRACE logger:54 - [Broker id=1216] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9153
[2021-08-31 18:22:30.150353] TRACE logger:54 - [Broker id=1271] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8747
[2021-08-31 18:22:30.150385] TRACE logger:54 - [Broker id=1252] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8778
[2021-08-31 18:22:30.150530] TRACE logger:54 - [Broker id=1198] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9072
[2021-08-31 18:22:30.150534] TRACE logger:54 - [Broker id=1242] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8837
[2021-08-31 18:22:30.150718] TRACE logger:54 - [Broker id=1200] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9093
[2021-08-31 18:22:30.150802] TRACE logger:54 - [Broker id=1235] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8769
[2021-08-31 18:22:30.150817] TRACE logger:54 - [Broker id=1219] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9263
[2021-08-31 18:22:30.150926] TRACE logger:54 - [Broker id=1204] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8897
[2021-08-31 18:22:30.151076] TRACE logger:54 - [Broker id=1265] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8758
[2021-08-31 18:22:30.151112] TRACE logger:54 - [Broker id=1227] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9086
[2021-08-31 18:22:30.151216] TRACE logger:54 - [Broker id=1191] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8993
[2021-08-31 18:22:30.151270] TRACE logger:54 - [Broker id=1221] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8826
[2021-08-31 18:22:30.151368] TRACE logger:54 - [Broker id=1192] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9107
[2021-08-31 18:22:30.151380] TRACE logger:54 - [Broker id=1213] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8857
[2021-08-31 18:22:30.151407] TRACE logger:54 - [Broker id=1269] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9035
[2021-08-31 18:22:30.151461] TRACE logger:54 - [Broker id=1277] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8889
[2021-08-31 18:22:30.151505] TRACE logger:54 - [Broker id=1228] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8829
[2021-08-31 18:22:30.151651] TRACE logger:54 - [Broker id=1217] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9351
[2021-08-31 18:22:30.151866] TRACE logger:54 - [Broker id=1236] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9099
[2021-08-31 18:22:30.151918] TRACE logger:54 - [Broker id=1280] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
10851
[2021-08-31 18:22:30.151979] TRACE logger:54 - [Broker id=1267] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9049
[2021-08-31 18:22:30.152051] TRACE logger:54 - [Broker id=1222] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8855
[2021-08-31 18:22:30.152085] TRACE logger:54 - [Broker id=1244] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9438
[2021-08-31 18:22:30.152126] TRACE logger:54 - [Broker id=1240] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9065
[2021-08-31 18:22:30.152202] TRACE logger:54 - [Broker id=1261] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9060
[2021-08-31 18:22:30.152309] TRACE logger:54 - [Broker id=1272] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9140
[2021-08-31 18:22:30.152364] INFO ReplicaFetcherManager:66 -
[ReplicaFetcherManager on broker 1266] Removed fetcher for partitions
Set(--removed--, __transaction_state-29, --removed--)
[2021-08-31 18:22:30.152415] TRACE logger:54 - [Broker id=1257] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8777
[2021-08-31 18:22:30.152462] TRACE logger:54 - [Broker id=1276] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8826
[2021-08-31 18:22:30.152588] TRACE logger:54 - [Broker id=1259] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9011
[2021-08-31 18:22:30.152766] TRACE logger:54 - [Broker id=1224] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8819
[2021-08-31 18:22:30.152830] TRACE logger:54 - [Broker id=1193] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8864
[2021-08-31 18:22:30.153018] TRACE logger:54 - [Broker id=1247] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8998
[2021-08-31 18:22:30.153084] TRACE logger:54 - [Broker id=1215] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9120
[2021-08-31 18:22:30.153107] TRACE logger:54 - [Broker id=1197] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9134
[2021-08-31 18:22:30.153163] TRACE logger:54 - [Broker id=1188] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9179
[2021-08-31 18:22:30.153211] TRACE logger:54 - [Broker id=1274] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9084
[2021-08-31 18:22:30.153270] TRACE logger:54 - [Broker id=1263] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9095
[2021-08-31 18:22:30.153296] TRACE logger:54 - [Broker id=1251] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9005
[2021-08-31 18:22:30.153428] TRACE logger:54 - [Broker id=1201] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8841
[2021-08-31 18:22:30.153569] TRACE logger:54 - [Broker id=1248] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9105
[2021-08-31 18:22:30.153641] TRACE logger:54 - [Broker id=1246] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9142
[2021-08-31 18:22:30.153701] TRACE logger:54 - [Broker id=1209] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8831
[2021-08-31 18:22:30.153847] TRACE logger:54 - [Broker id=1202] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8792
[2021-08-31 18:22:30.153995] TRACE logger:54 - [Broker id=1203] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8857
[2021-08-31 18:22:30.154230] TRACE logger:54 - [Broker id=1207] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9060
[2021-08-31 18:22:30.154265] TRACE logger:54 - [Broker id=1186] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9167
[2021-08-31 18:22:30.154301] TRACE logger:54 - [Broker id=1255] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8765
[2021-08-31 18:22:30.154673] TRACE logger:54 - [Broker id=1253] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8792
[2021-08-31 18:22:30.154771] TRACE logger:54 - [Broker id=1220] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9128
[2021-08-31 18:22:30.154827] TRACE logger:54 - [Broker id=1239] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9145
[2021-08-31 18:22:30.155245] TRACE logger:54 - [Broker id=1233] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9138
[2021-08-31 18:22:30.156050] TRACE logger:54 - [Broker id=1223] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8850
[2021-08-31 18:22:30.156134] TRACE logger:54 - [Broker id=1278] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8826
[2021-08-31 18:22:30.156245] TRACE logger:54 - [Broker id=1232] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8855
[2021-08-31 18:22:30.156833] TRACE logger:54 - [Broker id=1270] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9109
[2021-08-31 18:22:30.156855] TRACE logger:54 - [Broker id=1226] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9124
[2021-08-31 18:22:30.157341] TRACE logger:54 - [Broker id=1205] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9189
[2021-08-31 18:22:30.157533] TRACE logger:54 - [Broker id=1211] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9074
[2021-08-31 18:22:30.159415] INFO KafkaController:66 - [Controller id=1214]
Partition __transaction_state-29 completed preferred replica leader election.
New leader is 1266
[2021-08-31 18:22:30.160907] TRACE logger:54 - [Broker id=1241] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8866
[2021-08-31 18:22:30.163844] TRACE logger:54 - [Broker id=1184] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8952
[2021-08-31 18:22:30.165204] TRACE logger:54 - [Broker id=1214] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
9187
[2021-08-31 18:22:30.179043] TRACE logger:54 - [Broker id=1238] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8839
[2021-08-31 18:22:30.179435] TRACE logger:54 - [Broker id=1185] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8954
[2021-08-31 18:22:30.182935] TRACE logger:54 - [Broker id=1199] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8785
[2021-08-31 18:22:30.186117] INFO logger:66 - [Broker id=1266] Leader
__transaction_state-29 starts at leader epoch 222 from offset 2199146 with high
watermark 2199146 ISR [1187,1232,1266] addingReplicas [] removingReplicas [].
Previous leader epoch was 221.
[2021-08-31 18:22:30.230864] TRACE logger:54 - [Broker id=1266] Completed
LeaderAndIsr request correlationId 8863 from controller 1214 epoch 157 for the
become-leader transition for partition __transaction_state-29
[2021-08-31 18:22:30.355035] INFO TransactionStateManager:66 - [Transaction
State Manager 1266]: Loading transaction metadata from __transaction_state-29
at epoch 222
[2021-08-31 18:22:30.370655] TRACE logger:54 - [Broker id=1266] Cached leader
info UpdateMetadataPartitionState(topicName='__transaction_state',
partitionIndex=29, controllerEpoch=157, leader=1266, leaderEpoch=222,
isr=[1187, 1232, 1266], zkVersion=403, replicas=[1266, 1232, 1187],
offlineReplicas=[]) for partition __transaction_state-29 in response to
UpdateMetadata request sent by controller 1214 epoch 157 with correlation id
8864
[2021-08-31 18:22:31.111207] INFO Log:66 - [Log
partition=__transaction_state-29, dir=/pay/kafka/data] Truncating to 2199146
has no effect as the largest offset in the log is 2199145
[2021-08-31 18:22:31.145411] INFO Log:66 - [Log
partition=__transaction_state-29, dir=/pay/kafka/data] Truncating to 2199146
has no effect as the largest offset in the log is 2199145
[2021-08-31 18:22:33.362421] INFO TransactionStateManager:66 - [Transaction
State Manager 1266]: Finished loading 21 transaction metadata from
__transaction_state-29 in 3008 milliseconds, of which 0 milliseconds was spent
in the scheduler.
[2021-08-31 18:22:33.362600] INFO TransactionStateManager:66 - [Transaction
State Manager 1266]: Completed loading transaction metadata from
__transaction_state-29 for coordinator epoch 222
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
