Gyula Fora created FLINK-2497:
-
Summary: Rework streaming iteration flow control
Key: FLINK-2497
URL: https://issues.apache.org/jira/browse/FLINK-2497
Project: Flink
Issue Type: Improvement
Hey,
The JobGraph that is executed cannot have cycles (due to scheduling
reasons), that's why there is no edge between the head and tail operators.
What we do instead is we add an extra source to the head and a sink to the
tail (with the same parallelism), and the feedback data is passed outside
o
Hi,guysI am reading the code about streaming iteration.I find there is no
StreamEdge the StreamIterationHead and StreamIterationTail.Why? Because the
graph can't have any ring?Or any other concerns?
thanks
;>>>> creation. For example:
>>>>>
>>>>> DataStream source = ..
>>>>> DataStream mapper = source.map(noOpMapper)
>>>>> DataStream feedback = mapper.filter(...)
>>>>> source.union(feedback)
>>>>
back = mapper.filter(...)
> > > > source.union(feedback)
> > > >
> > > > This would basically mean that a DataStream is mutable and can be
> > > extended
> > > > after creation with more streams.
> > > >
> > > > On Tue, 7 Jul 2015
ataStream is mutable and can be
> > extended
> > > after creation with more streams.
> > >
> > > On Tue, 7 Jul 2015 at 16:12 Aljoscha Krettek
> > wrote:
> > >
> > > > I think this would be good yes. I was just about to open an Issue
at 16:12 Aljoscha Krettek
> wrote:
> >
> > > I think this would be good yes. I was just about to open an Issue for
> > > changing the Streaming Iteration API. :D
> > >
> > > Then we should also make the implementation very straightforward and
>
hink this would be good yes. I was just about to open an Issue for
> > changing the Streaming Iteration API. :D
> >
> > Then we should also make the implementation very straightforward and
> > simple, right now, the implementation of the iterations is all over the
> >
after creation with more streams.
On Tue, 7 Jul 2015 at 16:12 Aljoscha Krettek wrote:
> I think this would be good yes. I was just about to open an Issue for
> changing the Streaming Iteration API. :D
>
> Then we should also make the implementation very straightforward and
> si
;
>> I think this would be good yes. I was just about to open an Issue for
>> changing the Streaming Iteration API. :D
>>
>> Then we should also make the implementation very straightforward and
>> simple, right now, the implementation of the iterations is all ov
ink this would be good yes. I was just about to open an Issue for
> changing the Streaming Iteration API. :D
>
> Then we should also make the implementation very straightforward and
> simple, right now, the implementation of the iterations is all over the
> place.
>
> On Tue, 7
+1 for rethinking the iterations in DataStream
However, wouldn't this proposal allow the definition of arbitrary loops
(e.g., nested loops) that are not well behaved afaik?
On Tue, Jul 7, 2015 at 4:12 PM, Stephan Ewen wrote:
> I see that the newly proposed API makes some things easier to define
I think this would be good yes. I was just about to open an Issue for
changing the Streaming Iteration API. :D
Then we should also make the implementation very straightforward and
simple, right now, the implementation of the iterations is all over the
place.
On Tue, 7 Jul 2015 at 15:57 Gyula
I see that the newly proposed API makes some things easier to define.
There is one source of confusion, though, in my opinion:
The new API suggests that the data stream actually refers to the operator
that created it.
The "DataStream mapper = source.map(noOpMapper)" here refers to the map
operato
Hey,
Along with the suggested changes to the streaming API structure I think we
should also rework the "iteration" api. Currently the iteration api tries
to mimic the syntax of the batch API while the runtime behaviour is quite
different.
What we create instead of iterations is really just cyclic
Gyula Fora created FLINK-1854:
-
Summary: Planvisualizer cannot visualize the streaming iteration
example
Key: FLINK-1854
URL: https://issues.apache.org/jira/browse/FLINK-1854
Project: Flink
Paris Carbone created FLINK-1686:
Summary: Streaming iteration heads cannot be instantiated
Key: FLINK-1686
URL: https://issues.apache.org/jira/browse/FLINK-1686
Project: Flink
Issue Type
17 matches
Mail list logo
