Re: kafka streams, docker/k18s and rocksdb - storage performance

2018-02-28 Thread Pegerto Fernandez Torres
Hi Nicu, I think the best you can do is start with an empty dir, that will give you access to a not layered filesystem, but you need to take in consideration that you will stream data every time that the container restart, so a rolling update can be slow. My suggestion is to check with you platfo

Re: [Streams] partition expansion of input streams

2018-02-09 Thread Pegerto Fernandez Torres
t via the reset tool and clean local state. Change number of partitions. Restart the app. All approaches are difficult... There might be other approaches too. But you cannot change the number of input topic partitions on the fly while the application is running. It would crash. -Matthias On 2/9/18 6

[Streams] partition expansion of input streams

2018-02-09 Thread Pegerto Fernandez Torres
Hello, I start to think in the problem of expanding our source partitions, it looks like a stateless stream will handle the expansion properly after Rebalancing. In the case of a state-full then the problem seems different, first problem is the changelog itself, the expected definition will miss

kafka-streams using throught

2018-02-05 Thread Pegerto Fernandez Torres
Hello all, Can someone help me to understand the throught operation? At the documentation, the operation is equivalent to use #to# and define a #stream#. And according to the code that is exactly what it does, invoke to and return a stream. Now I want to store a intermediate result of the topol