Re: Kafka streams - how to handle application level exception in event processing

2020-09-23 Thread Bruno Cadonna
Hi Pushkar, if you do not want to lose any event, you should cache the events somewhere (e.g. a state store) in case there is an issue with an external system you connect to (e.g. database issue). If the order of the event is important, you must ensure that the events in your cache are proces

Re: Kafka streams - how to handle application level exception in event processing

2020-09-23 Thread Pushkar Deole
Thanks Bruno.. yeah, I think I could figure it out... For dependencies such as database, for which all the events will be blocked, we are planning to put a retry mechanism, so processing will wait until the database connection is backup. If the problem is with the incoming event, like bad format et

Re: Two MirrorMakers 2 for two DCs

2020-09-23 Thread Oleg Osipov
Ryanne, should I use Connect to run MM2? Now I use connect-mirror-maker.sh driver. I noticed when I restart M2Ms, all topics that I created before start syncing. But It doesn't work for newly created topics (after restart). On 2020/09/21 22:12:13, Ryanne Dolan wrote: > Oleg, yes you can ru