Regarding the last point, do I need to set up the listener ? All I want is to do a query from the store. For that I need to invoke streams.store() first, which can potentially throw an InvalidStateStoreException during rebalancing / migration of stores. If I call streams.store() with retries till the rebalancing is done or I exceed some max retry count, then I think I should good.
Or am I missing something ? regards. On Tue, Aug 1, 2017 at 1:10 PM, Damian Guy <damian....@gmail.com> wrote: > Hi, > > On Tue, 1 Aug 2017 at 08:34 Debasish Ghosh <ghosh.debas...@gmail.com> > wrote: > >> Hi - >> >> I have a Kafka Streams application that needs to run on multiple >> instances. >> It fetches metadata from all local stores and has an http query layer for >> interactive queries. In some cases when I have new instances deployed, >> store migration takes place making the current metadata invalid. Here are >> my questions regarding some of the best practices to be followed to handle >> this issue of store migration - >> >> - When the migration is in process, a query for the metadata may result >> in InvalidStateStoreException - is it a good practice to always have a >> retry semantics based query for the metadata ? >> > > Yes. Whenever the application is rebalancing the stores will be > unavailable, so retrying is the right thing to do. > > >> - Should I check KafkaStreams.state() and only assume that I have got >> the correct metadata when the state() call returns Running. If it >> returns Rebalancing, then I should re-query. Is this correct approach ? >> > > Correct again! If the state is rebalancing, then the metadata (for some > stores at least) is going to change, so you should get it again. You can > set a StateListener on the KafkaStreams instance to listen to these events. > > >> >> regards. >> >> -- >> Debasish Ghosh >> http://manning.com/ghosh2 >> http://manning.com/ghosh >> >> Twttr: @debasishg >> Blog: http://debasishg.blogspot.com >> Code: http://github.com/debasishg >> > -- Debasish Ghosh http://manning.com/ghosh2 http://manning.com/ghosh Twttr: @debasishg Blog: http://debasishg.blogspot.com Code: http://github.com/debasishg
