Re: Thoughts About Object Reuse and Collection Execution

2015-03-02 Thread Ted Dunning
On Mon, Mar 2, 2015 at 5:17 PM, Stephan Ewen wrote: > There are two execution modes for the runtime: "reuse" and "non-reuse". That makes a fair bit of sense.

Re: Thoughts About Object Reuse and Collection Execution

2015-03-02 Thread Stephan Ewen
@Ted Here is a bit of background about how things are currently done in the Flink runtime: There are two execution modes for the runtime: "reuse" and "non-reuse". - The "non-reuse" mode will create new objects for every record received from the network, or taken out of a sort-buffer or hash-tab

Re: Thoughts About Object Reuse and Collection Execution

2015-02-28 Thread Aljoscha Krettek
@Stephan, they are not copied when object reuse is enabled. This might be a problem, though, so maybe we should just change it in that place. On Sat, Feb 28, 2015 at 7:57 AM, Ted Dunning wrote: > This is going to have profound performance implications if this is the only > path for iteration. > >

Re: Thoughts About Object Reuse and Collection Execution

2015-02-27 Thread Ted Dunning
This is going to have profound performance implications if this is the only path for iteration. On Fri, Feb 27, 2015 at 10:58 PM, Stephan Ewen wrote: > I vote to have the key extractor return a new value each time. That means > that objects are not reused everywhere where it is possible, but s

Re: Thoughts About Object Reuse and Collection Execution

2015-02-27 Thread Stephan Ewen
I vote to have the key extractor return a new value each time. That means that objects are not reused everywhere where it is possible, but still in most places, which still helps. What still puzzles me: I thought that the collection execution stores copies of the returned records by default (reuse

Thoughts About Object Reuse and Collection Execution

2015-02-27 Thread Aljoscha Krettek
Hello Nation of Flink, while figuring out this bug: https://issues.apache.org/jira/browse/FLINK-1569 I came upon some difficulties. The problem is that the KeyExtractorMappers always return the same tuple. This is problematic, since Collection Execution does simply store the returned values in a li