Disabling operator chaining is also the only way to see metrics from each step right now.
On Monday, February 8, 2016, Stephan Ewen <se...@apache.org> wrote: > There is no fundamental reason to not implement this for batch as well. > > In Streaming, users seem to want more control about threads and resources > (given that these are often continuous pipelines), while in batch that was > not requested so far. > > But I see that a non-chained function is more safe with respect to > accidental object reuse by a user in their functions... > > On Mon, Feb 8, 2016 at 8:19 PM, Greg Hogan <c...@greghogan.com > <javascript:;>> wrote: > > > Is there a reason to not also implement this for batch processing? This > > would allow object reuse to be truly disabled. > > > > On Mon, Feb 8, 2016 at 2:02 PM, Stephan Ewen <se...@apache.org > <javascript:;>> wrote: > > > > > It may be useful for example when you have two MapFunctions and each > does > > > something CPU intensive, or communicates with an external service. > > > > > > Without chaining, you will have two threads and an elastic channel > > between > > > the functions to buffer some records, which may help in such a case. > > > > > > > > > On Mon, Feb 8, 2016 at 7:57 PM, Greg Hogan <c...@greghogan.com > <javascript:;>> wrote: > > > > > > > When is this useful in streaming? > > > > > > > > On Mon, Feb 8, 2016 at 1:46 PM, Nick Dimiduk <ndimi...@apache.org > <javascript:;>> > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > https://ci.apache.org/projects/flink/flink-docs-release-0.10/api/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.html#disableOperatorChaining() > > > > > > > > > > On Mon, Feb 8, 2016 at 10:34 AM, Greg Hogan <c...@greghogan.com > <javascript:;>> > > > wrote: > > > > > > > > > > > Is it possible to force operator chaining to be disabled? Similar > > to > > > > how > > > > > > object reuse can be enabled or disabled? > > > > > > > > > > > > Greg > > > > > > > > > > > > > > > > > > > > >
