Re: Splitting stream

2021-05-10 Thread Arvid Heise
Hi Nikola, side outputs definitively are at least as efficient as using two filters but they are also harder to implement and maintain. Do you actually have a use case where every bit of performance counts? If so, please also check enableObjectReuse [1] and look into serialization [2]. Also if y

Re: Splitting stream

2021-05-10 Thread Taher Koitawala
I think what your looking for is a a side output. Change the logic to a process function. What is true goes to collector false can go to a side output. Which now gives you 2 streams On Mon, May 10, 2021, 8:14 PM Nikola Hrusov wrote: > Hi Arvid, > > In my case it's the latter, thus I have also th

Re: Splitting stream

2021-05-10 Thread Nikola Hrusov
Hi Arvid, In my case it's the latter, thus I have also thought about using the filter (map is not useful in my case). What I am not sure which is better to be used? In what case would you split a stream with side output and in what case with filter? Would there be any performance gain/pain based

Re: Splitting stream

2021-05-10 Thread Arvid Heise
Hi Nikola, if you just want to apply a different user function to the records depending on the property "exist" the simplest way is to use source -> map(if exist do this else that) -> sink If it turns out that you want to apply a different subgraph, you can do source -> filter(if exist) -> do t