Thanks, I figured that was the preferred approach. I’ve noticed that at times
it doesn’t seem like it’s catching the exceptions (just have the wrapped sink
wrapped with a try-catch block at the moment).
Did you have to do anything special on that front? I’d assumed that catching
the IOException
Hi Rion,
We're using plain Kafka producer to send records to DLQ.
Regards,
Maciek
wt., 3 sie 2021 o 18:07 Rion Williams napisał(a):
>
> Thanks Maciek,
>
> It looks like my initial issue had another problem with a bad interface that
> was being used (or an improper one), but after changing that
Thanks Maciek,
It looks like my initial issue had another problem with a bad interface that
was being used (or an improper one), but after changing that and ensuring all
of the fields were implemented it worked as expected.
I know in my particular case, I'm planning on writing to Kafka, howeve
Hey.
As far as I see, you're not overriding functions like open,
setRuntimeContext, snapshotState, initializeState - the calls needs to
be passed to the inner sink function.
pon., 2 sie 2021 o 19:31 Rion Williams napisał(a):
>
> Hi again Maciek (and all),
>
> I just recently returned to start in
Hi again Maciek (and all),
I just recently returned to start investigating this approach, however I can't
seem to get the underlying invocation to work as I would normally expect. I'll
try to share a bit more as what I currently have and perhaps I'm just missing
something minor that someone may
This is the idea.
Of course you need to wrap more functions like: open, close,
notifyCheckpointComplete, snapshotState, initializeState and
setRuntimeContext.
The problem is that if you want to catch problematic record you need
to set batch size to 1, which gives very bad performance.
Regards,
Ma
Hi Maciej,
Thanks for the quick response. I wasn't aware of the idea of using a
SinkWrapper, but I'm not quite certain that it would suit this specific use
case (as a SinkFunction / RichSinkFunction doesn't appear to support
side-outputs). Essentially, what I'd hope to accomplish would be to pick
Hi Rion,
We have implemented such a solution with Sink Wrapper.
Regards,
Maciek
śr., 14 lip 2021 o 16:21 Rion Williams napisał(a):
>
> Hi all,
>
> Recently I've been encountering an issue where some external dependencies or
> process causes writes within my JDBCSink to fail (e.g. something is
Hi all,
Recently I've been encountering an issue where some external dependencies
or process causes writes within my JDBCSink to fail (e.g. something is
being inserted with an explicit constraint that never made it's way there).
I'm trying to see if there's a pattern or recommendation for handling
