Query Optimisation layer for flink streaming

2015-03-27 Thread Wepngong Benaiah
hello I have updated the proposal on melange and will like that you appraise the proposal so that i will do any necessary refinements before its late for any corrections Thanks @gyfora , @rmetzger, @senorcarbone @mbalassi -- Wepngong Ngeh Benaiah

Re: GSoC project proposal: Query optimisation layer for Flink Streaming

2015-03-24 Thread Wepngong Benaiah
t; Flink streaming currently only supports a limited set of optimisations >> applied on the streaming programs such as *operator chaining*, and >> several optimisations for *windowing* *computations*. >> >> Also, there is currently no optimizer as a separate module on

Re: GSoC project proposal: Query optimisation layer for Flink Streaming

2015-03-24 Thread Márton Balassi
e: > >> Hello, >> I cam out with the following proposal which I believe needs alot of >> review. I will appreciate if you can help me make appropriate corrections >> before the deadline for submission. >> Thanks @gyfora, @pariscarbone >> >> >>

Re: GSoC project proposal: Query optimisation layer for Flink Streaming

2015-03-24 Thread Robert Metzger
ns > before the deadline for submission. > Thanks @gyfora, @pariscarbone > > > *GSoC project: Query optimisation layer for Flink Streaming > <https://issues.apache.org/jira/browse/FLINK-1617>* > > NAME: Wepngong Ngeh Benaiah > > EMAIL: bwepng...@gmail.com > &

GSoC project proposal: Query optimisation layer for Flink Streaming

2015-03-21 Thread Wepngong Benaiah
Hello, I cam out with the following proposal which I believe needs alot of review. I will appreciate if you can help me make appropriate corrections before the deadline for submission. Thanks @gyfora, @pariscarbone *GSoC project: Query optimisation layer for Flink Streaming <ht

Re: Questions on GSoC project: Query optimisation layer for Flink Streaming

2015-03-20 Thread Wepngong Benaiah
Hello, I am thinking of implementing the following optimization algorithms for GSOC2015 These are supposed to do statistical graph analysis and streaming graph optimization 1.  OPERATOR REORDERING Means changing the order in which the operators appear in the stream graph to eliminate overheads.

Re: Questions on GSoC project: Query optimisation layer for Flink Streaming

2015-03-20 Thread Paris Carbone
Hi Wepngong, This is an interesting proposal. There are indeed many streaming optimisations out there but as Gyula said we should focus on a few and engineer them in a nice way. Perhaps for the time being it makes sense to focus on a streaming job graph optimiser that basically applies optimisa

Re: Questions on GSoC project: Query optimisation layer for Flink Streaming

2015-03-20 Thread Gyula Fóra
Hey, Of course the aim of the project would not be to implement all possible optimizations because that would be impossible to do so in such short time :) It would be nice if one could carefully select some optimizations that would make the most impact on the performance and implement those. Reg

Questions on GSoC project: Query optimisation layer for Flink Streaming

2015-03-19 Thread Wepngong Benaiah
hello , I have been making some research on https://issues.apache.org/jira/browse/FLINK-1617 using http://hirzels.com/martin/papers/csur14-streamopt.pdf and others. I find out that there are many optimization techniques available like 1. OPERATOR REORDERING 2. REDUNDANCY ELIMINATION 3. OPERATOR SEP