Re: Kafka streams store migration - best practices

2017-08-01 Thread Debasish Ghosh
Thanks for the confirmation. I guess listener would make sense if I did some caching of the store and needed to refresh it for every change in the underlying store. On Tue, Aug 1, 2017 at 6:10 PM, Damian Guy wrote: > No you don't need to set a listener. Was just mentioning as it an option > if y

Re: Kafka streams store migration - best practices

2017-08-01 Thread Damian Guy
No you don't need to set a listener. Was just mentioning as it an option if you wan't to know that the metadata needs refreshing, On Tue, 1 Aug 2017 at 13:25 Debasish Ghosh wrote: > Regarding the last point, do I need to set up the listener ? > > All I want is to do a query from the store. For t

Re: Kafka streams store migration - best practices

2017-08-01 Thread Debasish Ghosh
Regarding the last point, do I need to set up the listener ? All I want is to do a query from the store. For that I need to invoke streams.store() first, which can potentially throw an InvalidStateStoreException during rebalancing / migration of stores. If I call streams.store() with retries till

Re: Kafka streams store migration - best practices

2017-08-01 Thread Damian Guy
Hi, On Tue, 1 Aug 2017 at 08:34 Debasish Ghosh wrote: > Hi - > > I have a Kafka Streams application that needs to run on multiple instances. > It fetches metadata from all local stores and has an http query layer for > interactive queries. In some cases when I have new instances deployed, > stor

Kafka streams store migration - best practices

2017-08-01 Thread Debasish Ghosh
Hi - I have a Kafka Streams application that needs to run on multiple instances. It fetches metadata from all local stores and has an http query layer for interactive queries. In some cases when I have new instances deployed, store migration takes place making the current metadata invalid. Here ar