Hi Gyula, Thanks for your response.
So if i use partitionBy then data point with the same will receive exactly by the same instance of operator ? Another question is if i execute reduce() operator on after partitionBy, will that reduce operator guarantee ordering within the same key ? Cheers On Fri, Jul 3, 2015 at 4:14 PM, Gyula Fóra <gyula.f...@gmail.com> wrote: > Hey! > > Both groupBy and partitionBy will trigger a shuffle over the network based > on some key, assuring that elements with the same keys end up on the same > downstream processing operator. > > The difference between the two is that groupBy in addition to this returns > a GroupedDataStream which lets you execute some special operations, such as > key based rolling aggregates. > > PartitionBy is useful when you are using simple operators but still want > to control the messages received by parallel instances (in a mapper for > example). > > Cheers, > Gyula > > tambunanw <if05...@gmail.com> ezt írta (időpont: 2015. júl. 3., P, 10:32): > >> Hi All, >> >> I'm trying to digest what's the difference between this two. From my >> experience in Spark GroupBy will cause shuffling on the network. Is that >> the >> same case in Flink ? >> >> I've watch videos and read a couple docs about Flink that's actually Flink >> will compile the user code into it's own optimized graph structure so i >> think Flink engine will take care of this one ? >> >> From the docs for Partitioning >> >> >> http://ci.apache.org/projects/flink/flink-docs-master/apis/streaming_guide.html#partitioning >> >> Is that true that GroupBy is more advanced than PartitionBy ? Can someone >> elaborate ? >> >> I think this one is really confusing for me that come from Spark world. >> Any >> help would be really appreciated. >> >> Cheers >> >> >> >> >> >> -- >> View this message in context: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Streaming-PartitionBy-vs-GroupBy-differences-tp1927.html >> Sent from the Apache Flink User Mailing List archive. mailing list >> archive at Nabble.com. >> > -- Welly Tambunan Triplelands http://weltam.wordpress.com http://www.triplelands.com <http://www.triplelands.com/blog/>
