Hi Eno, Does this mean that Kafka Streams will disable the RocksDB write buffer?
Is it currently safe to recover a Kafka Streams application after SIGKILL on the same machine? If not, will this make it safe to do so? If RocksDB is not flushed before offsets are commited in Kafka and is killed with SIGKILL, will the data in the write buffer be lost (since Kafka Streams disables the transaction log)? That data will be present in the Kafka changelog but will it get applied to the recovered RocksDB database on restart? Thanks, Roger On Fri, Jun 3, 2016 at 2:39 PM, Eno Thereska <eno.there...@gmail.com> wrote: > Hi Gwen, > > Yes. As an example, if cache.max.bytes.buffering set to X, and if users > have A aggregation operators and T KTable.to() operators, then X*(A + T) > total bytes will be allocated for caching. > > Eno > > > On 3 Jun 2016, at 21:37, Gwen Shapira <g...@confluent.io> wrote: > > > > Just to clarify: "cache.max.bytes.buffering" is per processor? > > > > > > On Thu, Jun 2, 2016 at 11:30 AM, Eno Thereska <eno.there...@gmail.com> > wrote: > >> Hi there, > >> > >> I have created KIP-63: Unify store and downstream caching in streams > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-63%3A+Unify+store+and+downstream+caching+in+streams > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-63:+Unify+store+and+downstream+caching+in+streams > > > >> > >> > >> Feedback is appreciated. > >> > >> Thank you > >> Eno > >
