Yes, that would be perfect. Thanks!
--
Aljoscha
> On 26 Feb 2016, at 20:53, Zach Cox wrote:
>
> Sure, want me to open a jira issue and then PR a new page into
> https://github.com/apache/flink/tree/master/docs/internals, following these
> instructions? http://flink.apache.org/contribute-docume
Sure, want me to open a jira issue and then PR a new page into
https://github.com/apache/flink/tree/master/docs/internals, following these
instructions? http://flink.apache.org/contribute-documentation.html
-Zach
On Fri, Feb 26, 2016 at 1:13 PM Aljoscha Krettek
wrote:
> Cool, that’s a nice wri
Cool, that’s a nice write up. Would you maybe be interested in integrating this
as some sort of internal documentation in Flink? So that prospective
contributors can get to know this stuff.
Cheers,
Aljoscha
> On 26 Feb 2016, at 18:32, Zach Cox wrote:
>
> Thanks for the confirmation Aljoscha! I
Thanks for the confirmation Aljoscha! I wrote up results from my little
experiment: https://github.com/zcox/flink-repartition-watermark-example
-Zach
On Fri, Feb 26, 2016 at 2:58 AM Aljoscha Krettek
wrote:
> Hi,
> yes, your description is spot on!
>
> Cheers,
> Aljoscha
> > On 26 Feb 2016, at
Hi,
yes, your description is spot on!
Cheers,
Aljoscha
> On 26 Feb 2016, at 00:19, Zach Cox wrote:
>
> I think I found the information I was looking for:
>
> RecordWriter broadcasts each emitted watermark to all outgoing channels [1].
>
> StreamInputProcessor tracks the max watermark received
I think I found the information I was looking for:
RecordWriter broadcasts each emitted watermark to all outgoing channels [1].
StreamInputProcessor tracks the max watermark received on each incoming
channel separately, and computes the task's watermark as the min of all
incoming watermarks [2].
Hi - how are watermarks passed along parallel tasks where there is a
repartition? For example, say I have a simple streaming job computing
hourly counts per key, something like this:
val environment = StreamExecutionEnvironment.getExecutionEnvironment
environment.setParallelism(2)
environment.setS
