[
https://issues.apache.org/jira/browse/FLINK-35504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated FLINK-35504:
----------------------------------
Description:
Currently all logs are in DEBUG level. Some of those messages are very helpful
to get the progress and errors, which can be changed to INFO or WARN level. We
can also include error details into DEBUG level messages so it's easier to
debug with more context.
Meanwhile, we can update the metric to track
{{numRecordsSendPartialFailureCounter}}, which is records that were returned by
Elasticsearch as failed and were retried. FWIW, the base class tracks following
metrics already so we don't need to implement them: {{CurrentSendTime}} Gauge,
{{NumBytesOut}} and {{NumRecordsOut}} Counters.
was:
Currently all logs are in DEBUG level. Some of those messages are very helpful
to get the progress and errors, which can be changed to INFO or WARN level. We
can also include error details into DEBUG level messages so it's easier to
debug with more context.
Meanwhile, we can update the metric to track {{numRecordsSend}}. FWIW, the base
class tracks following metrics already so we don't need to implement them:
{{CurrentSendTime}} Gauge, {{NumBytesOut}} and {{NumRecordsOut}} Counters.
> Improve Elasticsearch 8 connector observability
> -----------------------------------------------
>
> Key: FLINK-35504
> URL: https://issues.apache.org/jira/browse/FLINK-35504
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / ElasticSearch
> Affects Versions: elasticsearch-3.1.0
> Reporter: Mingliang Liu
> Priority: Major
> Labels: pull-request-available
>
> Currently all logs are in DEBUG level. Some of those messages are very
> helpful to get the progress and errors, which can be changed to INFO or WARN
> level. We can also include error details into DEBUG level messages so it's
> easier to debug with more context.
> Meanwhile, we can update the metric to track
> {{numRecordsSendPartialFailureCounter}}, which is records that were returned
> by Elasticsearch as failed and were retried. FWIW, the base class tracks
> following metrics already so we don't need to implement them:
> {{CurrentSendTime}} Gauge, {{NumBytesOut}} and {{NumRecordsOut}} Counters.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
