Re: Tuple performance and the curious JIT compiler

2016-03-10 Thread Márton Balassi
If the community can agree that the proposal that Gábor Horváth has suggested is a nice approach and can accept that the results will be coming around mid summer, then I would strongly suggest "reserving" him this topic. His previous experience makes him a strong candidate for the task. To add to

Re: Tuple performance and the curious JIT compiler

2016-03-10 Thread Ufuk Celebi
Very nice proposal! On Wed, Mar 9, 2016 at 6:35 PM, Stephan Ewen wrote: > Thanks for posting this. > > I think it is not super urgent (in the sense of weeks or few months), so > results around mid summer is probably good. > The background in LLVM is a very good base for this! > > On Wed, Mar 9, 2

Re: Tuple performance and the curious JIT compiler

2016-03-09 Thread Stephan Ewen
Thanks for posting this. I think it is not super urgent (in the sense of weeks or few months), so results around mid summer is probably good. The background in LLVM is a very good base for this! On Wed, Mar 9, 2016 at 3:56 PM, Gábor Horváth wrote: > Hi, > > In the meantime I sent out the curren

Re: Tuple performance and the curious JIT compiler

2016-03-09 Thread Gábor Horváth
Hi, In the meantime I sent out the current version of the proposal draft [1]. Hopefully it will help you triage this task and contribute to the discussion of the problem. How urgent is this issue? In what time frame should there be results? Best Regards, Gábor [1] http://apache-flink-mailing-lis

Re: Tuple performance and the curious JIT compiler

2016-03-09 Thread Stephan Ewen
Do we have consensus that we want to "reserve" this topic for a GSoC student? It is becoming a feature that gains more importance. To see we can "hold off" on working on that, would be good to know a bit more, like - when is it decided whether this project takes place? - when would results be

Re: Tuple performance and the curious JIT compiler

2016-03-08 Thread Márton Balassi
@Fabian: That is my bad, but I think we should be still on time. Pinged Uli just to make sure. Proposal from Gabor and Jira from me are coming soon. On Tue, Mar 8, 2016 at 11:43 AM, Fabian Hueske wrote: > Hi Gabor, > > I did not find any Flink proposals for this year's GSoC in JIRA (should be >

Re: Tuple performance and the curious JIT compiler

2016-03-08 Thread Fabian Hueske
Hi Gabor, I did not find any Flink proposals for this year's GSoC in JIRA (should be labeled with gsoc2016). I am also not sure if any of the Flink committers signed up as a GSoC mentor. Maybe it is still time to do that but as it looks right now there are no GSoC projects offered by Flink. Best,

Re: Tuple performance and the curious JIT compiler

2016-03-08 Thread Gábor Horváth
Hi! I am planning to do GSoC and I would like to work on the serializers. More specifically I would like to implement code generation. I am planning to send the first draft of the proposal to the mailing list early next week. If everything is going well, that will include some preliminary benchmar

Re: Tuple performance and the curious JIT compiler

2016-03-08 Thread Stephan Ewen
Ah, very good, that makes sense! I would guess that this performance difference could probably be seen at various points where generic serializers and comparators are used (also for Comparable, Writable) or where the TupleSerializer delegates to a sequence of other TypeSerializers. I guess creati

Re: Tuple performance and the curious JIT compiler

2016-03-07 Thread Greg Hogan
The issue is not with the Tuple hierarchy (running Gelly examples had no effect on runtime, and as you note there aren't any subclass overrides) but with CopyableValue. I had been using IntValue exclusively but had switched to using LongValue for graph generation. CopyableValueComparator and Copyab

Re: Tuple performance and the curious JIT compiler

2016-03-07 Thread Stephan Ewen
Hi Greg! Sounds very interesting. Do you have a hunch what "virtual" Tuple methods are being used that become less jit-able? In many cases, tuples use only field accesses (like "vakle.f1") in the user functions. I have to dig into the serializers, to see if they could suffer from that. The "getF

Tuple performance and the curious JIT compiler

2016-03-04 Thread Greg Hogan
I am noticing what looks like the same drop-off in performance when introducing TupleN subclasses as expressed in "Understanding the JIT and tuning the implementation" [1]. I start my single-node cluster, run an algorithm which relies purely on Tuples, and measure the runtime. I execute a separate