That's absolutely no problem Tzu-Li. Either of them would work. Thank you!
On Thu, Apr 19, 2018 at 4:56 PM Tzu-Li (Gordon) Tai
wrote:
> @Alexander
> Sorry about that, that would be my mistake. I’ll close FLINK-9204 as a
> duplicate and leave my thoughts on FLINK-9155. Thanks for pointing out!
>
@Alexander
Sorry about that, that would be my mistake. I’ll close FLINK-9204 as a
duplicate and leave my thoughts on FLINK-9155. Thanks for pointing out!
On 19 April 2018 at 2:00:51 AM, Elias Levy (fearsome.lucid...@gmail.com) wrote:
Either proposal would work. In the later case, at a minimum
Either proposal would work. In the later case, at a minimum we'd need a
way to identify the source within the metric. The basic error metric would
then allow us to go into the logs to determine the cause of the error, as
we already record the message causing trouble in the log.
On Mon, Apr 16,
ouch, i forgot to mention I opened
https://issues.apache.org/jira/browse/FLINK-9155 to track this. Should it
be a duplicate of 9204 then?
On Wed, Apr 18, 2018 at 3:32 PM Tzu-Li (Gordon) Tai
wrote:
> Hi,
>
> These are valid concerns. And yes, AFAIK users have been writing to logs
> within the des
Hi,
These are valid concerns. And yes, AFAIK users have been writing to logs within
the deserialization schema to track this. The connectors as of now have no
logging themselves in case of a skipped record.
I think we can implement both logging and metrics to track this, most of which
you have
Thanks for starting the discussion Elias.
I see two ways to address this issue.
1) Add an interface that a deserialization schema can implement to register
metrics. Each source would need to check for the interface and call it to
setup metrics.
2) Check for null returns in the source functions an
I have the same question. In case of kafka source, it would be good to know
topic name and offset of the corrupted message for further investigation.
Looks like the only option is to write messages into a log file
On Fri, Apr 6, 2018 at 9:12 PM Elias Levy
wrote:
> I was wondering how are folks t
