Hi all,
I have a kafka 0.9.0.0 cluster with 11 nodes.
First,I found server logs as below，
server.log.2016-10-17-22:[2016-10-17 22:22:13,885] WARN
[ReplicaFetcherThread-0-4], Error in fetch
kafka.server.ReplicaFetcherThread$FetchRequest@367c9f98. Possible cause:
org.apache.kafka.common.protocol.types.SchemaException: Error reading field
'responses': Error reading array of size 1786735, only 2389 bytes available
(kafka.server.ReplicaFetcherThread)
server.log.2016-10-17-22:[2016-10-17 22:22:15,456] WARN
[ReplicaFetcherThread-0-5], Error in fetch
kafka.server.ReplicaFetcherThread$FetchRequest@12088f91. Possible cause:
org.apache.kafka.common.protocol.types.SchemaException: Error reading field
'responses': Error reading array of size 1338722, only 5662 bytes available
(kafka.server.ReplicaFetcherThread)
server.log.2016-10-17-22:[2016-10-17 22:22:15,888] WARN
[ReplicaFetcherThread-0-4], Error in fetch
kafka.server.ReplicaFetcherThread$FetchRequest@60069db2. Possible cause:
org.apache.kafka.common.protocol.types.SchemaException: Error reading field
'responses': Error reading array of size 1786735, only 2389 bytes available
(kafka.server.ReplicaFetcherThread)
server.log.2016-10-17-22:[2016-10-17 22:22:17,460] WARN
[ReplicaFetcherThread-0-5], Error in fetch
kafka.server.ReplicaFetcherThread$FetchRequest@4a5991cb. Possible cause:
org.apache.kafka.common.protocol.types.SchemaException: Error reading field
'responses': Error reading array of size 1338722, only 5662 bytes available
(kafka.server.ReplicaFetcherThread)
Then I jstack pid，and I see
"ReplicaFetcherThread-0-3" prio=10 tid=0x00007f1254319800 nid=0xfdb runnable
[0x00007f0ee36d7000]
"ReplicaFetcherThread-0-8" prio=10 tid=0x00007f1278141800 nid=0x66f runnable
[0x00007f0ee2ecf000]
"ReplicaFetcherThread-0-9" prio=10 tid=0x00007f1278127000 nid=0x66e runnable
[0x00007f0ee2fd0000]
"ReplicaFetcherThread-0-4" prio=10 tid=0x00007f127810c800 nid=0x66d waiting on
condition [0x00007f0ee30d1000]
"ReplicaFetcherThread-0-1" prio=10 tid=0x00007f12780ef800 nid=0x66c runnable
[0x00007f0ee31d2000]
"ReplicaFetcherThread-0-7" prio=10 tid=0x00007f12780d4800 nid=0x66b runnable
[0x00007f0ee32d3000]
"ReplicaFetcherThread-0-5" prio=10 tid=0x00007f12780b9800 nid=0x66a waiting on
condition [0x00007f0ee33d4000]
"ReplicaFetcherThread-0-6" prio=10 tid=0x00007f127809f000 nid=0x669 runnable
[0x00007f0ee34d5000]
"ReplicaFetcherThread-0-2" prio=10 tid=0x00007f1278084800 nid=0x668 runnable
[0x00007f0ee35d6000]
"ReplicaFetcherThread-0-10" prio=10 tid=0x00007f127804c800 nid=0x666 runnable
[0x00007f0ee37d8000]
the log shows that there are 2 replicaFetcherThreads waiting on condition.
my cluster have no broker version compatible problem. From log, I thought
there are some exceptions in broker 4 and broker 5, so I restart then, and
everything goes right.
what does this log means and how can it occurs?
Will appreciate if anyone has any insight on what's happening here.
Thanks.
