But the join requirement is to match the records from two streams occurring
within one hour (besides the normal join key condition), if I use the
second join window, those records wouldn't be in the same window any more.
On Thu, Apr 28, 2016 at 11:47 PM, Ashutosh Kumar <kmr.ashutos...@gmail.com>
wrote:
> Time unit can be in seconds as well. Is there specific need to get bursts
> hourly?
>
> On Fri, Apr 29, 2016 at 11:48 AM, Henry Cai <h...@pinterest.com> wrote:
>
>> For the below standard stream/stream join, does flink store the results
>> of stream 1 and stream 2 into state store for the current hour and at the
>> end of the hour window it will fire the window by iterating through all
>> stored elements in the state store to find join matches?
>>
>> My concern is during most of the time in the hour, the output (assuming
>> the output is going to another stream) will be idle and on each hour mark
>> there will be huge outputs of joined records emitted, any way to make it
>> more gradual?
>>
>>
>> dataStream.join(otherStream)
>> .where(0).equalTo(1)
>> .window(TumblingEventTimeWindows.of(Time.hours(1)))
>> .apply (new JoinFunction () {...});
>>
>>
>
