Re: Capturing and storing these Kafka events for query.

2018-01-11 Thread Hans Jespersen
Another approach would be to create the query first (in something like KSQL) and then send the Kafka data through the pre-existing streaming query. In this case the results would be going into various result topics. Tools like KSQL also let you query historical data but you need to be sure that

Re: Capturing and storing these Kafka events for query.

2018-01-11 Thread Manoj Khangaonkar
Hi, If I understood the question correctly , then the better approach is to consume events from topic and store in your favorite database. Then query the database as needed. Querying the topic for messages in kafka is not recommended as that will be a linear search. regards On Thu, Jan 11, 2018

Capturing and storing these Kafka events for query.

2018-01-11 Thread Maria Pilar
Hi all, I have a requirement to be able to capture and store events for query, and I'm trying to choose the best option for that: 1) Capture the events from a separate topic, store events a state, in order to convert a stream to a table, that means materializing the stream. The option for it, tha