Thanks John. Is it the same case if i want to write in a postgres table with the sql connector?
El mié, 6 dic 2023 a las 11:05, John Casey (<theotherj...@google.com>) escribió: > It is, but it's not possible to to take an existing transform, and simply > configure it to do this. > > For example (and this is what I'm doing), it's possible to write a > transform that tries to write to kafka, and upon failure, emits the failure > to an alternate pcollection. > > It's not possible (yet) to take an existing PTransform that's part of the > library, and configure it to do something other than simply retrying > failures > > On Wed, Dec 6, 2023 at 10:44 AM Juan Romero <jsrf...@gmail.com> wrote: > >> But , is it not possible to get the message that can't reach the target >> sink and put it in another target (eg: kafka error topic where we can >> verify which messages failed to be delivered to the target)? >> >> >> El mié, 6 dic 2023 a las 10:40, John Casey via user (< >> user@beam.apache.org>) escribió: >> >>> I'm currently implementing improvements on Kafka, File, Spanner, and >>> Bigtable IOs. >>> >>> I'm planning on tackling PubSub and BQ next year. >>> >>> All of this is still in progress though, so there aren't easy >>> workarounds for the moment. >>> >>> On Tue, Dec 5, 2023 at 5:56 PM Robert Bradshaw <rober...@google.com> >>> wrote: >>> >>>> Currently error handling is implemented on sinks in an ad-hoc basis >>>> (if at all) but John (cc'd) is looking at improving things here. >>>> >>>> On Mon, Dec 4, 2023 at 10:25 AM Juan Romero <jsrf...@gmail.com> wrote: >>>> > >>>> > Hi guys. I want to ask you about how to deal with the scenario when >>>> the target sink (eg: jdbc, kafka, bigquery, pubsub etc) fails for any >>>> reason and i don't want to lost the message and create a bottleneck with >>>> many errors due an hypothetical target sink problem, and i want to use >>>> with_excpetion_handling in order to get the message that failing to reach >>>> the target and send the message to an other error topic. Any idea to solve >>>> this scenario? >>>> >>>
