Thanks Bill, the StateRestoreListener is exactly the tool needed for my use
case.
Patrik, thanks for the heads-up on that issue. I guess until it's fixed
that makes it even easier to wait until the cache is warmed :-).
Chris
On Tue, Nov 13, 2018 at 10:40 PM Patrik Kleindl wrote:
> Hi Chris
>
>
Hi Chris
We are using them like you described.
Performance is very good compared to the database used before.
Beware that until https://issues.apache.org/jira/browse/KAFKA-7380
is done the startup will be blocked until all global stores are restored
(sequentially).
This can take a little for larg
Hi Chris
Yes, for every state store in a kafka streams application, the state
restore listener is executed.
-Bill
On Tue, Nov 13, 2018 at 10:36 PM Chris Toomey wrote:
> Thanks Bill. So poking around the code a bit, it looks like perhaps any
> kafka streams execution that produces a state store
Thanks Bill. So poking around the code a bit, it looks like perhaps any
kafka streams execution that produces a state store would execute this
"restore state store" operation, even creating a new state store, is that
correct? If so that indeed could be just what I need.
On Tue, Nov 13, 2018 at 6:
Hi Chris,
I'm not sure I totally understand your requirements but the
StateRestoreListener (
https://kafka.apache.org/20/javadoc/org/apache/kafka/streams/processor/StateRestoreListener.html)
class provides callbacks when restoring state stores (including
GlobalStores) and may provide what you are
Definitely Ryanne -- that's what I meant by "topics would be compacted".
But that doesn't obviate checking bootstrapping progress.
On Tue, Nov 13, 2018 at 5:04 PM Ryanne Dolan wrote:
> Chris, consider using log compaction.
>
> Ryanne
>
> On Tue, Nov 13, 2018, 3:17 PM Chris Toomey wrote:
>
> >
Chris, consider using log compaction.
Ryanne
On Tue, Nov 13, 2018, 3:17 PM Chris Toomey wrote:
> We're considering using GlobalKTables / KeyValueStores for locally caching
> topic content in services. The topics would be compacted such that only the
> latest key/value pair would exist for a giv
We're considering using GlobalKTables / KeyValueStores for locally caching
topic content in services. The topics would be compacted such that only the
latest key/value pair would exist for a given key.
One question that's come up is how to determine, when bootstrapping the
app, when the cache has
