Re: Flink the right tool for the job ? Huge Data window lateness

2017-02-24 Thread Patrick Brunmayr
f results are >> sent out >> of Flink and stored in an external db, if you can design the db writes to >> be idempotent, >> then it’ll effectively be a “last write wins”. It depends mostly on your >> pipeline and >> use case. >> >> *- Computations per m

Flink the right tool for the job ? Huge Data window lateness

2017-02-24 Thread Patrick Brunmayr
Hello I've done my first steps with Flink and i am very impressed of its capabilities. Thank you for that :) I want to use it for a project we are currently working on. After reading some documentation i am not sure if it's the right tool for the job. We have an IoT application in which we are mon

Re: Difference between partition and groupBy

2017-02-24 Thread Patrick Brunmayr
he partitioning or grouping. Usually >> there are more partitions than parallel instances, so each instance will >> take care of multiple partitions. >> >> >> >> On Thu, Feb 23, 2017 at 6:16 PM, Patrick Brunmayr >> wrote: >> >>> What is the basic difference between partitioning datasets by key or >>> grouping them by key ? >>> >>> Does it make a difference in terms of paralellism ? >>> >>> Thx >>> >> >> >

Difference between partition and groupBy

2017-02-23 Thread Patrick Brunmayr
What is the basic difference between partitioning datasets by key or grouping them by key ? Does it make a difference in terms of paralellism ? Thx