Just read the KIP wiki again: https://cwiki.apache.org/confluence/display/KAFKA/KIP-63%3A+Unify+store+and+downstream+caching+in+streams
Just one minor comments but otherwise I'm +1: In "proposed changes" section: "The cache has two functions. First, it continues to serve as a read buffer for data that is sent to the state store, just like today. Second, it serves as a write deduplicator for the state store (just like today) as well as for the downstream processor node(s). " I feel this is still a bit confusing. The caching layer is only turned on for state stores used in aggregate operators in the DSL, right? For example in KStream-KStream joins, we will not turn on caching since there is no updates on the state store on the same key. So could we just propose "removing the caching layer inside the persistent state store engines (i.e. RocksDB), and instead add a per-thread global cache which will only be activated for state stores used in Streams DSL's aggregate operator as a write deduplicator for both the state store and to the downstream operators"? Guozhang On Thu, Sep 8, 2016 at 10:07 AM, Eno Thereska <eno.there...@gmail.com> wrote: > There have been a couple of changes to KIP-63 since the voting started, > after more feedback, most notably the fact that this KIP applies to the DSL > only, and not to the Processor API. > > At this point I'd like to restart the voting process. > > Thanks > Eno > > > On 31 Aug 2016, at 17:16, Jim Jagielski <j...@jagunet.com> wrote: > > > > +1 > >> On Aug 25, 2016, at 6:57 AM, Eno Thereska <eno.there...@gmail.com> > wrote: > >> > >> Hi folks, > >> > >> We'd like to start the vote for KIP-63. At this point the Wiki addresses > >> all previous questions and we believe the PoC is feature-complete. > >> > >> Thanks > >> Eno > > > > -- -- Guozhang
