Shivangi Singh created KAFKA-9367:
-------------------------------------
Summary: CRC failure
Key: KAFKA-9367
URL: https://issues.apache.org/jira/browse/KAFKA-9367
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 2.0.0
Reporter: Shivangi Singh
We have a 14 node kafka(2.0.0) cluster
In our case
*Leader* : *Broker Id* : 1003 *Ip*: 10.84.198.238
*Replica* : *Broker Id* : 1014 *Ip*: 10.22.2.74
A request was sent from replica -> leader to which leader(10.84.198.238) had
the following exception
var/log/kafka/server.log.2019-12-26-00:[2019-12-26 00:13:04,386] ERROR Closing
socket for 10.84.198.238:6667-10.22.2.74:53118-121025 because of error
(kafka.network.Processor)
/var/log/kafka/server.log.2019-12-26-00-org.apache.kafka.common.errors.InvalidRequestException:
Error getting request for apiKey: FETCH, apiVersion: 8, connectionId:
10.84.198.238:6667-10.22.2.74:53118-121025, listenerName:
ListenerName(PLAINTEXT), principal: User:ANONYMOUS
/var/log/kafka/server.log.2019-12-26-00-Caused by:
org.apache.kafka.common.protocol.types.SchemaException: *Error reading field
'forgotten_topics_data':* Error reading array of size 23668, only 69 bytes
available
/var/log/kafka/server.log.2019-12-26-00- at
org.apache.kafka.common.protocol.types.Schema.read(Schema.java:77)
/var/log/kafka/server.log.2019-12-26-00- at
org.apache.kafka.common.protocol.ApiKeys.parseRequest(ApiKeys.java:290)
/var/log/kafka/server.log.2019-12-26-00- at
org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:63)
------------------------------------------------------------------------
In response to this, replica (10.22.2.74) had the following log in it
[2019-12-26 00:13:04,390] WARN [ReplicaFetcher replicaId=1014, leaderId=1003,
fetcherId=0] Error in response for fetch request (type=FetchRequest,
replicaId=1014, maxWait=500, minBytes=1, maxBytes=10485760,
fetchData={_topic_name_=(offset=50344687, logStartOffset=24957467,
maxBytes=1048576)}, isolationLevel=READ_UNCOMMITTED, toForget=,
metadata=(sessionId=1747349875, epoch=183382033))
(kafka.server.ReplicaFetcherThread)
java.io.IOException: Connection to 1003 was disconnected before the response
was read
at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:97)
at
kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:240)
at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:43)
at
kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:149)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:114)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
------------------------------------------------------------------------
Post this broker 1003 had the following exception
/var/log/kafka/server.log.2019-12-26-00:[2019-12-26 00:16:37,828] ERROR
[ReplicaFetcher replicaId=1003, leaderId=1014, fetcherId=0] Found invalid
messages during etch for partition _topic_name_ offset 91200983
(kafka.server.ReplicaFetcherThread)
/var/log/kafka/server.log.2019-12-26-00-*org.apache.kafka.common.record.InvalidRecordException:
Record is corrupt (stored crc = 1460037823, computed crc = 114378201)*
/var/log/kafka/server.log.2019-12-26-00:[2019-12-26 00:16:40,690] ERROR Closing
socket for 10.84.198.238:6667-10.22.2.74:49850-740543 because of error
(kafka.network.Processor)
/var/log/kafka/server.log.2019-12-26-00-org.apache.kafka.common.errors.InvalidRequestException:
Error getting request for apiKey: FETCH, apiVersion: 8, connectionId:
10.84.198.238:6667-10.22.2.74:49850-740543, listenerName:
ListenerName(PLAINTEXT), principal: User:ANONYMOUS
Could you help us with the above issue?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
