Re: Data manipulation in Hive over Tez

2016-12-02 Thread Robert Grandl
Thanks Rajesh for your answer. That was really helpful. I would like to ask you few more questions. I am trying to better understand how the pairs are propagated and processed at various vertices. Edge:- encodes the data movement logic Processing logic:- process and partition the output key

Re: Data manipulation in Hive over Tez

2016-11-29 Thread Rajesh Balamohan
Hi Robert, 1. At high level, you can refer to https://github.com/apache/ hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/ exec/tez/DagUtils.java where different vertices, edges etc gets created as per the execution plan. Consider a vertex as a combination of input, processing logic and outp