Xavier Lange created KAFKA-4412:
-----------------------------------
Summary: Replication fetch stuck in loop on offset null
Key: KAFKA-4412
URL: https://issues.apache.org/jira/browse/KAFKA-4412
Project: Kafka
Issue Type: Bug
Components: replication
Reporter: Xavier Lange
I kicked off a cluster rebalance and it never completed. I had to look at node
eth0 traffic to see there was a constant 60MB/s (I'm usually at about 5MB/s
ingest). The /kafka/logs/server.log was looping like this, then I deleted the
topic in question to make it stop:
{quote}
[2016-11-15 18:21:27,745] ERROR Found invalid messages during fetch for
partition [cisco-2016.11.13,19] offset 861323 error null
(kafka.server.ReplicaFetcherThread)
[2016-11-15 18:21:27,755] ERROR Found invalid messages during fetch for
partition [cisco-2016.11.13,19] offset 861323 error null
(kafka.server.ReplicaFetcherThread)
[2016-11-15 18:21:27,773] ERROR Found invalid messages during fetch for
partition [cisco-2016.11.13,19] offset 861323 error null
(kafka.server.ReplicaFetcherThread)
[2016-11-15 18:21:27,788] ERROR Found invalid messages during fetch for
partition [cisco-2016.11.13,19] offset 861323 error null
(kafka.server.ReplicaFetcherThread)
[2016-11-15 18:21:27,847] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,19]
(kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:27,852] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,11]
(kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:27,853] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,9] (kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:27,855] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,3] (kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:27,856] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,16]
(kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:27,857] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,2] (kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:27,858] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,19]
(kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:28,012] INFO Deleting index
/data/cisco-2016.11.13-19/00000000000000000000.index (kafka.log.OffsetIndex)
[2016-11-15 18:21:28,016] INFO Deleted log for partition [cisco-2016.11.13,19]
in /data/cisco-2016.11.13-19. (kafka.log.LogManager)
[2016-11-15 18:21:28,024] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,11]
(kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:28,165] INFO Deleting index
/data/cisco-2016.11.13-11/00000000000000000000.index (kafka.log.OffsetIndex)
[2016-11-15 18:21:28,165] INFO Deleted log for partition [cisco-2016.11.13,11]
in /data/cisco-2016.11.13-11. (kafka.log.LogManager)
[2016-11-15 18:21:28,167] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,9] (kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:28,232] INFO Deleting index
/data/cisco-2016.11.13-9/00000000000000000000.index (kafka.log.OffsetIndex)
[2016-11-15 18:21:28,232] INFO Deleted log for partition [cisco-2016.11.13,9]
in /data/cisco-2016.11.13-9. (kafka.log.LogManager)
[2016-11-15 18:21:28,242] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,3] (kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:28,341] INFO Deleting index
/data/cisco-2016.11.13-3/00000000000000000000.index (kafka.log.OffsetIndex)
[2016-11-15 18:21:28,342] INFO Deleted log for partition [cisco-2016.11.13,3]
in /data/cisco-2016.11.13-3. (kafka.log.LogManager)
[2016-11-15 18:21:28,375] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,16]
(kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:28,465] INFO Deleting index
/data/cisco-2016.11.13-16/00000000000000000000.index (kafka.log.OffsetIndex)
[2016-11-15 18:21:28,466] INFO Deleted log for partition [cisco-2016.11.13,16]
in /data/cisco-2016.11.13-16. (kafka.log.LogManager)
[2016-11-15 18:21:28,469] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [cisco-2016.11.13,2] (kafka.server.ReplicaFetcherManager)
[2016-11-15 18:21:28,486] INFO Deleting index
/data/cisco-2016.11.13-2/00000000000000000000.index (kafka.log.OffsetIndex)
[2016-11-15 18:21:28,486] INFO Deleted log for partition [cisco-2016.11.13,2]
in /data/cisco-2016.11.13-2. (kafka.log.LogManager)
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
