Re: Operator order

2016-06-20 Thread Yasuhiro Matsuda
You can do a conditional branching by using KStream.branch(Predicate...). You can then merge multiple streams using KStreamBuilder.merge(KStream...). -Yasuhiro On Mon, Jun 20, 2016 at 4:45 AM, Jeyhun Karimov wrote: > Hi Guozhang, > > Thank you for your reply. Yes, it is correct. Your solution i

Re: Operator order

2016-06-20 Thread Jeyhun Karimov
Hi Guozhang, Thank you for your reply. Yes, it is correct. Your solution is match for my use case. I will try to use the topology you mentioned in a more dynamic way. Thanks Jeyhun On Mon, Jun 20, 2016 at 1:59 AM Guozhang Wang wrote: > Hello Jeyhun, > > Another way to do this "dynamic routing

Re: Operator order

2016-06-19 Thread Guozhang Wang
Hello Jeyhun, Another way to do this "dynamic routing" is to specify your topology using the lower-level processor API: http://docs.confluent.io/3.0.0/streams/developer-guide.html#processor-api More specifically, you can for example specify both A and D as parents of E when adding processor E, a

Re: Operator order

2016-06-19 Thread Matthias J. Sax
Thanks for clarification. Still don't have an better answer as before. How much overhead my suggestion gives is hard to predict. However, the filter operators will run in the same thread (it's more or less just another chained method call), thus, it should not be too large. Furthermore, it should

Re: Operator order

2016-06-18 Thread Jeyhun Karimov
Hi Matthias, Thank you for your answer. In my use-case, depending on statistics of every operator, some tuples can be escaped for specific operators, so that we can get approximate but faster result. I think this is somehow similar to TelegraphCQ in dynamism of operators. In my case, the goal is

Re: Operator order

2016-06-18 Thread Matthias J. Sax
Hi Jeyhun, there is no support by the library itself. But you could build a custom solution by building the DAG with all required edges (ie, additional edges from A->E, and B->sink etc.). For this, each output message from A would be duplicate and send to B and E. Therefore, A should "tag" each me