I guess you can do thus "manually" by splitting you code into multiple applications so you can scale each part independently.
I am not against an improvement in Streams itself, but I am not sure how this could be done atm. -Matthias On 4/28/17 8:42 AM, Eno Thereska wrote: > Hi Jeyhun, > > You make a good observation and I think a discussion/contribution around this > would be very much appreciated by the community. Are you thinking of a KIP > perhaps? > > Eno > >> On 28 Apr 2017, at 16:13, Jeyhun Karimov <je.kari...@gmail.com> wrote: >> >> Hi community, >> >> I have a question regarding with streams library. >> >> Currently, in kafka-streams we run the whole topology in one instance and >> there can be several topologies or tasks in a single node. However, there >> can be use-cases with very complex topologies with costly operators. So, >> when we want to scale-up, instead of copying the whole topology to run in >> parallel, we may need to scale-up specific operators (or subgraphs in >> tasks) in topology (it depends on a defined cost function). >> So my question is that, is the specified use-case is compatible with >> kafka-streams motivation and would it be appreciated by community the >> relevant contribution? >> >> >> Cheers, >> Jeyhun >> -- >> -Cheers >> >> Jeyhun >
signature.asc
Description: OpenPGP digital signature
