Re: No error on dangling dataflow parts

2015-10-08 Thread Matthias J. Sax
I just skipped over the discussion. If I got it right, the question was about if a dedicated sink operator must be the last in a flow or not. In the case I described below, it is about distinct flows in a single program. This was not discussed in the PR (or did I miss it?). For sink/no-sink I don'

Re: No error on dangling dataflow parts

2015-10-08 Thread Aljoscha Krettek
We had some discussion on the Jira Issue when I changed the translation from operators to StreamGraph: https://issues.apache.org/jira/browse/FLINK-2398 There are arguments for both ways of doing it, I'm not partial towards any solution but if you want this changed you can start a discussion. (If w

Re: No error on dangling dataflow parts

2015-10-08 Thread Matthias J. Sax
Dropping would be less strange. However, raising an exception would be natural (at least to me) On 10/08/2015 04:30 PM, Aljoscha Krettek wrote: > What do you mean? The current behavior is strange or the other way round > would be strange? > > I think it is in line with what other Stream Proces

Re: No error on dangling dataflow parts

2015-10-08 Thread Aljoscha Krettek
What do you mean? The current behavior is strange or the other way round would be strange? I think it is in line with what other Stream Processing Systems provide. For example Storm and Google Dataflow behave similarly. > On 08 Oct 2015, at 16:25, Matthias J. Sax wrote: > > Well. This behavio

Re: No error on dangling dataflow parts

2015-10-08 Thread Matthias J. Sax
Well. This behavior would also be kind of strange... (at least to me) On 10/08/2015 04:22 PM, Aljoscha Krettek wrote: > Hi, > I think Flink does in fact not drop the dangling parts. In streaming it is > allowed to have dangling operators that are not sinks. They are executed and > the output is

Re: No error on dangling dataflow parts

2015-10-08 Thread Aljoscha Krettek
Hi, I think Flink does in fact not drop the dangling parts. In streaming it is allowed to have dangling operators that are not sinks. They are executed and the output is just discarded. Cheers, Aljoscha > On 08 Oct 2015, at 16:18, Matthias J. Sax wrote: > > Hi, > > I just hit a problem in St