Re: How to make a generic key for groupBy

2015-05-11 Thread Stephan Ewen
t I need (such as getHeadKey(), >>> getLastl(), or makeKey(Key,Object)) to “navigate” in the key space, and a >>> cached hash code to make hash maps faster. My generic algorithms do not >>> need to know how many fields are exposed in the Key, but they need to be >>

RE: How to make a generic key for groupBy

2015-05-01 Thread LINZ, Arnaud
ilto:user@flink.apache.org> Objet : Re: How to make a generic key for groupBy Hey Arnaud! I have made a quick sample implementation of how you can very efficiently support generic keys, like yours. I put the code in this repository: https://github.com/StephanEwen/flink-generic-keys It implemen

Re: How to make a generic key for groupBy

2015-04-29 Thread Stephan Ewen
. My generic algorithms do not >> need to know how many fields are exposed in the Key, but they need to be >> able to construct another key from two keys. >> >> >> >> Arnaud >> >> >> >> *De :* ewenstep...@gmail.com [mailto:ewenstep...@gmail.c

Re: How to make a generic key for groupBy

2015-04-27 Thread Stephan Ewen
han Ewen > *Envoyé :* vendredi 24 avril 2015 11:14 > *À :* user@flink.apache.org > *Objet :* Re: How to make a generic key for groupBy > > > > Hi Arnaud! > > > > Thank you for the warm words! Let's find a good way to get this to work... > > > > As a bit

RE: How to make a generic key for groupBy

2015-04-27 Thread LINZ, Arnaud
[mailto:ewenstep...@gmail.com] De la part de Stephan Ewen Envoyé : vendredi 24 avril 2015 11:14 À : user@flink.apache.org Objet : Re: How to make a generic key for groupBy Hi Arnaud! Thank you for the warm words! Let's find a good way to get this to work... As a bit of background: In Flink, the API nee

Re: How to make a generic key for groupBy

2015-04-24 Thread Stephan Ewen
Hi Arnaud! Thank you for the warm words! Let's find a good way to get this to work... As a bit of background: In Flink, the API needs to now a bit about the types that go through the functions, because Flink pre-generates and configures serializers, and validates that things fit together. It is

Re: How to make a generic key for groupBy

2015-04-23 Thread Soumitra Kumar
Will you elaborate on your use case? It would help to find out where Flink shines. IMO, its a great project, but needs more differentiation from Spark. On Thu, Apr 23, 2015 at 7:25 AM, LINZ, Arnaud wrote: > Hello, > > > > After a quite successful benchmark yesterday (Flink being about twice > f

How to make a generic key for groupBy

2015-04-23 Thread LINZ, Arnaud
Hello, After a quite successful benchmark yesterday (Flink being about twice faster than Spark on my use cases), I’ve turned instantly from spark-fan to flink-fan – great job, committers! So I’ve decided to port my existing Spark tools to Flink. Happily, most of the difficulty was renaming clas