Thanks Bill, the StateRestoreListener is exactly the tool needed for my use
case.

Patrik, thanks for the heads-up on that issue. I guess until it's fixed
that makes it even easier to wait until the cache is warmed :-).

Chris

On Tue, Nov 13, 2018 at 10:40 PM Patrik Kleindl <pklei...@gmail.com> wrote:

> Hi Chris
>
> We are using them like you described.
> Performance is very good compared to the database used before.
> Beware that until https://issues.apache.org/jira/browse/KAFKA-7380
> is done the startup will be blocked until all global stores are restored
> (sequentially).
> This can take a little for larger topic and/or multiple global stores.
>
> We are blocking access until they are available although this is not ideal
> in terms of timeout tuning.
>
> Any ideas are welcome.
>
> Best regards
>
> Patrik
>
> > Am 14.11.2018 um 00:17 schrieb Chris Toomey <ctoo...@gmail.com>:
> >
> > We're considering using GlobalKTables / KeyValueStores for locally
> caching
> > topic content in services. The topics would be compacted such that only
> the
> > latest key/value pair would exist for a given key.
> >
> > One question that's come up is how to determine, when bootstrapping the
> > app, when the cache has been populated with the latest content from the
> > topic (so we start with a "warm" cache). ReadOnlyKeyValueStore has
> > an approximateNumEntries() method that we could use to see how much we've
> > got, but trying to figure out how much there is in the topic looks much
> > more difficult -- the only way I can see via the APIs / code is to use an
> > AdminClient to get the topic partitions and then the KafkaConsumer to get
> > the end offsets for those.
> >
> > Does anyone have experience doing this kind of caching? How did you
> handle
> > the bootstrapping issue?
> >
> > Any thoughts on easier or better ways to determine when the cache is
> warm?
> >
> > thx,
> > Chris
>

Reply via email to