Ok... got it... is there any hook that I can attach to the global k table
or global store? What I mean here is I want to know when the global store
is updated with data from topic in that case the hook that I specified
should be invoked so i can do some activity like logging that, this will
allow me to know how long the global store took to sync up with topic after
the event has been put on the topic.

On Tue, May 26, 2020 at 10:58 PM Matthias J. Sax <mj...@apache.org> wrote:

> For example it could be some "static" information, like a mapping from
> zip code to city name.
>
> Something that does usually not change over time.
>
>
> -Matthias
>
> On 5/25/20 9:55 PM, Pushkar Deole wrote:
> > Matthias,
> >
> > I am wondering what you mean by "Global store hold "axially" data that is
> > provided from "outside" of the
> > app"
> >
> > will you be able to give some example use case here as to what you mean
> by
> > axially data provided from outside app?
> >
> > On Sat, May 2, 2020 at 1:58 AM Matthias J. Sax <mj...@apache.org> wrote:
> >
> >> Both stores sever a different purpose.
> >>
> >> Regular stores allow you to store state the application computes.
> >> Writing into the changelog is a fault-tolerance mechanism.
> >>
> >> Global store hold "axially" data that is provided from "outside" of the
> >> app. There is no changelog topic, but only the input topic (that is used
> >> to re-create the global state).
> >>
> >> Local stores are sharded and updates are "sync" as they don't need to be
> >> shared with anybody else.
> >>
> >> For global stores, as all instances need to be updated, updates are
> >> async (we don't know when which instance will update it's own global
> >> store replica).
> >>
> >>>> Say one stream thread updates the topic for global store and starts
> >>>> processing next event wherein the processor tries to read the global
> >> store
> >>>> which may not have been synced with the topic?
> >>
> >> Correct. There is no guarantee when the update to the global store will
> >> be applied. As said, global stores are not designed to hold data the
> >> application computes.
> >>
> >>
> >> -Matthias
> >>
> >>
> >> On 4/30/20 11:11 PM, Pushkar Deole wrote:
> >>> thanks... will try with GlobalKTable.
> >>> As a side question, I didn't really understand the significance of
> global
> >>> state store which kind of works in a reverse way to local state store
> >> i.e.
> >>> local state store is updated and then saved to changelog topic whereas
> in
> >>> case of global state store the topic is updated first and then synced
> to
> >>> global state store. Do these two work in sync i.e. the update to topic
> >> and
> >>> global state store ?
> >>>
> >>> Say one stream thread updates the topic for global store and starts
> >>> processing next event wherein the processor tries to read the global
> >> store
> >>> which may not have been synced with the topic?
> >>>
> >>> On Fri, May 1, 2020 at 3:35 AM Matthias J. Sax <mj...@apache.org>
> wrote:
> >>>
> >>>> Yes.
> >>>>
> >>>> A `GlobalKTable` uses a global store internally.
> >>>>
> >>>> You can also use `StreamsBuilder.addGlobalStore()` or
> >>>> `Topology.addGlobalStore()` to add a global store "manually".
> >>>>
> >>>>
> >>>> -Matthias
> >>>>
> >>>>
> >>>> On 4/30/20 7:42 AM, Pushkar Deole wrote:
> >>>>> Thanks Matthias.
> >>>>> Can you elaborate on the replicated caching layer part?
> >>>>> When you say global stores, do you mean GlobalKTable created from a
> >> topic
> >>>>> e.g. using StreamsBuilder.globalTable(String topic) method ?
> >>>>>
> >>>>> On Thu, Apr 30, 2020 at 12:44 PM Matthias J. Sax <mj...@apache.org>
> >>>> wrote:
> >>>>>
> >>>>>> It's not possible to modify state store from "outside".
> >>>>>>
> >>>>>> If you want to build a "replicated caching layer", you could use
> >> global
> >>>>>> stores and write into the corresponding topics to update all stores.
> >> Of
> >>>>>> course, those updates would be async.
> >>>>>>
> >>>>>>
> >>>>>> -Matthias
> >>>>>>
> >>>>>> On 4/29/20 10:52 PM, Pushkar Deole wrote:
> >>>>>>> Hi All,
> >>>>>>>
> >>>>>>> I am wondering if this is possible: i have been asked to use state
> >>>> stores
> >>>>>>> as a general replicated cache among multiple instances of service
> >>>>>> instances
> >>>>>>> however the state store is created through streambuilder but is not
> >>>>>>> actually modified through stream processor topology however it is
> to
> >> be
> >>>>>>> modified from outside the stream topology. So, essentially, the
> state
> >>>>>> store
> >>>>>>> is just to be created from streambuilder and then to be used as an
> >>>>>>> application level cache that will get replicated between
> application
> >>>>>>> instances. Is this possible using state stores?
> >>>>>>>
> >>>>>>> Secondly, if possible, is this a good design approach?
> >>>>>>>
> >>>>>>> Appreciate your response since I don't know the internals of state
> >>>>>> stores.
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >
>
>

Reply via email to