As far as I remember when we were implementing the iteration for streaming we deliberately "went off" the jobgraph and implemented the backward edge as an in-memory hook (practically the same in batch but not blocking in the backward channel).
When we asked the reasoning here on the dev list we got some reasoning for keeping the check for the DAG property of the Jobgraph, but I could not find the thread now. On Wed, Jan 21, 2015 at 11:57 AM, Paris Carbone <par...@kth.se> wrote: > Hello, > > While implementing the SAMOA adapter for Flink-Streaming we stumbled upon > the need to allow loops (or circular dependencies) in the job graph. Many > incremental machine learning tasks define loops already and there is no > trivial way of getting around it. In the streaming job graph builder there > is only a check that does not allow the user to submit graphs with loops, > however, from what Gyula told me, if the check is removed the streaming job > runs as expected. Is there (still) a major reason for having this check, at > least in the streaming component? > > Paris > > >
