To be more clear...
A single source in a Flink program is a logical concept. Flink jobs are
run with some level of parallelism meaning that multiple copies of your
source (and all other) functions are run distributed across a cluster. So
if you have a streaming program with two sources and you r
All streams can be parallelized in Flink even with only one source. You
can have multiple sinks as well.
On Thu, Dec 15, 2016 at 7:56 AM, Meghashyam Sandeep V <
vr1meghash...@gmail.com> wrote:
> 1. If we have multiple sources, can the streams be parallelized ?
> 2. Can we have multiple sinks as
1. If we have multiple sources, can the streams be parallelized ?
2. Can we have multiple sinks as well?
On Dec 14, 2016 10:46 PM, wrote:
> Got it. Thanks!
>
> On Dec 15, 2016, at 02:58, Jamie Grier wrote:
>
> Ahh, sorry, for #2: A single Flink job can have as many sources as you
> like. They c
Got it. Thanks!
> On Dec 15, 2016, at 02:58, Jamie Grier wrote:
>
> Ahh, sorry, for #2: A single Flink job can have as many sources as you like.
> They can be combined in multiple ways, via things like joins, or connect(),
> etc. They can also be completely independent — in other words the dat
Ahh, sorry, for #2: A single Flink job can have as many sources as you
like. They can be combined in multiple ways, via things like joins, or
connect(), etc. They can also be completely independent — in other words
the data flow graph can be completely disjoint. You never to need to call
execute()
Hey Jamie,
Ok with #1. I guess #2 is just not possible.
I got it about #3. I just checked the code for the tumbling window assigner
and I noticed it's just its default trigger that gets overwritten when
using a custom trigger, not the way it assigns windows, it makes sense now.
Regarding #4, aft
For #1 there are a couple of ways to do this. The easiest is probably
stream1.connect(stream2).map(...) where the MapFunction maps the two input
types to a common type that you can then process uniformly.
For #3 There must always be a WindowAssigner specified. There are some
convenient ways to d
Hello people,
I've written down some quick questions for which I couldn't find much or
anything in the documentation. I hope you can answer some of them!
*# Multiple consumers*
*1.* Is it possible to .union() streams of different classes? It is useful
to create a consumer that counts elements on
