Thanks Bill, the StateRestoreListener is exactly the tool needed for my use case.
Patrik, thanks for the heads-up on that issue. I guess until it's fixed that makes it even easier to wait until the cache is warmed :-). Chris On Tue, Nov 13, 2018 at 10:40 PM Patrik Kleindl <pklei...@gmail.com> wrote: > Hi Chris > > We are using them like you described. > Performance is very good compared to the database used before. > Beware that until https://issues.apache.org/jira/browse/KAFKA-7380 > is done the startup will be blocked until all global stores are restored > (sequentially). > This can take a little for larger topic and/or multiple global stores. > > We are blocking access until they are available although this is not ideal > in terms of timeout tuning. > > Any ideas are welcome. > > Best regards > > Patrik > > > Am 14.11.2018 um 00:17 schrieb Chris Toomey <ctoo...@gmail.com>: > > > > We're considering using GlobalKTables / KeyValueStores for locally > caching > > topic content in services. The topics would be compacted such that only > the > > latest key/value pair would exist for a given key. > > > > One question that's come up is how to determine, when bootstrapping the > > app, when the cache has been populated with the latest content from the > > topic (so we start with a "warm" cache). ReadOnlyKeyValueStore has > > an approximateNumEntries() method that we could use to see how much we've > > got, but trying to figure out how much there is in the topic looks much > > more difficult -- the only way I can see via the APIs / code is to use an > > AdminClient to get the topic partitions and then the KafkaConsumer to get > > the end offsets for those. > > > > Does anyone have experience doing this kind of caching? How did you > handle > > the bootstrapping issue? > > > > Any thoughts on easier or better ways to determine when the cache is > warm? > > > > thx, > > Chris >
