[
https://issues.apache.org/jira/browse/KAFKA-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Randall Hauch updated KAFKA-10273:
----------------------------------
Description:
When runtime exceptions are thrown from converters, the current error messages
are descriptive, but not actionable. For example:
{noformat}
Error encountered in task task-0. Executing stage 'VALUE_CONVERTER' with class
'org.apache.kafka.connect.json.JsonConverter', where consumed record is
<snipped>
Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[] to
Kafka Connect data failed due to serialization error:
at
org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:334)
<snipped>
Caused by: org.apache.kafka.common.errors.SerializationException:
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('b' (code
98)): was expecting double-quote to start field name
at [Source: (byte[])"{brokenjson-:"bar 1"}"; line: 1, column: 3]{noformat}
This describes the error accurately, explaining when it happened, and what
caused the exception to be thrown. However, after receiving this error it's
still unclear what remediation step to take next.
Instead, in addition to the existing descriptive context and instigating error
being displayed, these error messages should give users a lead to begin
investigating and remediating the error.
There are a few changes that users may use to remediate exceptions from
converters:
* Reconfigure the converter (change schemas.enable on the JsonConverter to
align with the topic contents)
* Use different data (advancing sink/source offsets manually to skip bad
records, or changing topics)
* Remediate the converter's environment failure (resolving network faults or
reconfiguring external systems)
An error message which suggests manual inspection of the converter
configuration would be user-actionable, and could be generically applied across
converters without knowledge of specific configurations. This would point users
towards the most common remediation step, reconfiguring the converter.
For sink connectors we can also indicate that bad data in a topic may be
skipped in a number of ways, such as changing offsets, ignoring errors, writing
to a DLQ, etc. It's unclear which of these is worth suggesting, and it may
change if an alternative strategy feature exists in the future.
Non-Goals/ Out-of-scope:
The third type of remediation of the converter's environment is outside the
scope of this issue, and requires converter-specific knowledge in order to make
a suggestion.
This change is not attempting to add converter-specific error messaging, and
the framework messaging here needs to be converter-agnostic, and as such can
only point out general classes of errors and remediation steps, not specific
ones. We can programmatically generate lists of relevant configurations, but
can't single any out as being the cause. We can say that the data is
incompatible with the converter, but can't say why exactly.
was:
When runtime exceptions are thrown from converters, the current error messages
are descriptive, but not actionable. For example:
{noformat}
Error encountered in task task-0. Executing stage 'VALUE_CONVERTER' with class
'org.apache.kafka.connect.json.JsonConverter', where consumed record is
<snipped>
Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[] to
Kafka Connect data failed due to serialization error:
at
org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:334)
<snipped>
Caused by: org.apache.kafka.common.errors.SerializationException:
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('b' (code
98)): was expecting double-quote to start field name
at [Source: (byte[])"{brokenjson-:"bar 1"}"; line: 1, column: 3]{noformat}
This describes the error accurately, explaining when it happened, and what
caused the exception to be thrown. However, after receiving this error it's
still unclear what remediation step to take next.
Instead, in addition to the existing descriptive context and instigating error
being displayed, these error messages should give users a lead to begin
investigating and remediating the error.
There are a few changes that users may use to remediate exceptions from
converters:
* Reconfigure the converter (change schemas.enable on the JsonConverter to
align with the topic contents)
* Use different data (advancing sink/source offsets manually to skip bad
records, or changing topics)
* Remediate the converter's environment failure (resolving network faults or
reconfiguring external systems)
An error message which suggests manual inspection of the converter
configuration would be user-actionable, and could be generically applied across
converters without knowledge of specific configurations. This would point users
towards the most common remediation step, reconfiguring the converter.
We can also indicate that bad data in a topic may be skipped in a number of
ways, such as changing offsets, ignoring errors, writing to a DLQ, etc. It's
unclear which of these is worth suggesting, and it may change if an alternative
strategy feature exists in the future.
Non-Goals/ Out-of-scope:
The third type of remediation of the converter's environment is outside the
scope of this issue, and requires converter-specific knowledge in order to make
a suggestion.
This change is not attempting to add converter-specific error messaging, and
the framework messaging here needs to be converter-agnostic, and as such can
only point out general classes of errors and remediation steps, not specific
ones. We can programmatically generate lists of relevant configurations, but
can't single any out as being the cause. We can say that the data is
incompatible with the converter, but can't say why exactly.
> Connect Converters should produce actionable error messages
> -----------------------------------------------------------
>
> Key: KAFKA-10273
> URL: https://issues.apache.org/jira/browse/KAFKA-10273
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Greg Harris
> Priority: Minor
>
> When runtime exceptions are thrown from converters, the current error
> messages are descriptive, but not actionable. For example:
> {noformat}
> Error encountered in task task-0. Executing stage 'VALUE_CONVERTER' with
> class 'org.apache.kafka.connect.json.JsonConverter', where consumed record is
> <snipped>
> Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[]
> to Kafka Connect data failed due to serialization error:
> at
> org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:334)
> <snipped>
> Caused by: org.apache.kafka.common.errors.SerializationException:
> com.fasterxml.jackson.core.JsonParseException: Unexpected character ('b'
> (code 98)): was expecting double-quote to start field name
> at [Source: (byte[])"{brokenjson-:"bar 1"}"; line: 1, column: 3]{noformat}
> This describes the error accurately, explaining when it happened, and what
> caused the exception to be thrown. However, after receiving this error it's
> still unclear what remediation step to take next.
> Instead, in addition to the existing descriptive context and instigating
> error being displayed, these error messages should give users a lead to begin
> investigating and remediating the error.
> There are a few changes that users may use to remediate exceptions from
> converters:
> * Reconfigure the converter (change schemas.enable on the JsonConverter to
> align with the topic contents)
> * Use different data (advancing sink/source offsets manually to skip bad
> records, or changing topics)
> * Remediate the converter's environment failure (resolving network faults or
> reconfiguring external systems)
> An error message which suggests manual inspection of the converter
> configuration would be user-actionable, and could be generically applied
> across converters without knowledge of specific configurations. This would
> point users towards the most common remediation step, reconfiguring the
> converter.
> For sink connectors we can also indicate that bad data in a topic may be
> skipped in a number of ways, such as changing offsets, ignoring errors,
> writing to a DLQ, etc. It's unclear which of these is worth suggesting, and
> it may change if an alternative strategy feature exists in the future.
> Non-Goals/ Out-of-scope:
> The third type of remediation of the converter's environment is outside the
> scope of this issue, and requires converter-specific knowledge in order to
> make a suggestion.
> This change is not attempting to add converter-specific error messaging, and
> the framework messaging here needs to be converter-agnostic, and as such can
> only point out general classes of errors and remediation steps, not specific
> ones. We can programmatically generate lists of relevant configurations, but
> can't single any out as being the cause. We can say that the data is
> incompatible with the converter, but can't say why exactly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
