I would go with Kafka Streams. Kafka Streams is just a library. So you can make a simple or complex application. You can include a rules engine or machine learning model as a dependency as well as a kafka dependency and based on your message send to being different topics. Run a number of your application equal to your partitions on different machines and let the magic happen.
On Mon, Mar 11, 2019, 10:28 AM Greenhorn Techie <greenhorntec...@gmail.com> wrote: > Hi, > > Our Kafka message has 10-15 fields. On the consumer side, we need to take > some action based on a combination of fields. For example, if Field A = X > and Field B = Y, then push the message to Topic T1. However, if Field C = P > or Field D = Q, then push the message into Topic T2. Likewise, we have many > such rules depending on which certain action has to be taken. > > Wondering whether there is standard pattern to implement such business > rules logic in any of the Kafka ecosystem components? For Kafka Streams? > Even otherwise, would be glad to understand how companies implement such > decision trees in practice on streaming data. > > Thanks >
