Re: Multiple Streams Connect Watermark

2018-04-27 Thread Chengzhi Zhao
Got it, Thanks a lot Fabian. Looking forward to seeing your book. Best, Chengzhi On Thu, Apr 26, 2018 at 4:02 PM, Fabian Hueske wrote: > You can also merge all three types into an nary-Either type and union all > three inputs together. > However, Flink only supports a binary Either, so you'd ha

Re: Multiple Streams Connect Watermark

2018-04-26 Thread Fabian Hueske
You can also merge all three types into an nary-Either type and union all three inputs together. However, Flink only supports a binary Either, so you'd have to implement a custom TypeInformation and TypeSerializer to make that work. Best, Fabian 2018-04-26 20:44 GMT+02:00 Chengzhi Zhao : > Thank

Re: Multiple Streams Connect Watermark

2018-04-26 Thread Chengzhi Zhao
Thanks Fabian for the explanation. If I have data with different schemas, it seems the only option I have is to use connect to perform joins (inner, outer), is there any operators that can put more than two streams together (all different schema)? Best, Chengzhi On Thu, Apr 26, 2018 at 6:05 AM,

Re: Multiple Streams Connect Watermark

2018-04-26 Thread Fabian Hueske
Hi Chengzhi, Functions in Flink are implemented in a way to preserve the timestamps of elements or assign timestamps which are aligned with the existing watermarks. For example, the result of a time window aggregation has the end timestamp of the window as a timestamp and records emitted by the on

Multiple Streams Connect Watermark

2018-04-25 Thread Chengzhi Zhao
Hi, everyone, I am trying to do some join-like pipeline using flink connect operator and CoProcessFunction, I have use case that I need to connect 3+ streams. So I am having something like this: A ===> C B ==> E D So two streams A and B connect at first with 3 h