Re: [Streams] partition expansion of input streams

2018-02-09 Thread Pegerto Fernandez Torres
Hello Matthias, No is not too late, I understand the complexity of change of modify the topic size. The approach to oversize the partition is implemented from day one, but having multiple topics one each domain subject, I want to consider the case of someone miss sizing a specific domain. I thin

Re: [Streams] partition expansion of input streams

2018-02-09 Thread Matthias J. Sax
Changing source topic partitions for stateful application is very difficult. The best thing is to overpartition source topics to avoid this issue in the first place. (Too late for you I guess.) If you really need to change the partitions count, the cleanest solution is, to create a second input t

[Streams] partition expansion of input streams

2018-02-09 Thread Pegerto Fernandez Torres
Hello, I start to think in the problem of expanding our source partitions, it looks like a stateless stream will handle the expansion properly after Rebalancing. In the case of a state-full then the problem seems different, first problem is the changelog itself, the expected definition will miss