Re: Serializing dags

2015-08-31 Thread canan chen
Hi Raajay, Regarding how hive build DAG, you can check https://github.com/apache/hive/blob/72f97fc7760134465333983fc40766e9e864e643/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java Check method TezTask#build Do you mind share what kind of changes you want to make on DAGs ? On Tue, Se

How can I see the operator tree of hive ?

2015-06-29 Thread canan chen
Is there any way to see the operator tree of hive before and after optimization ? Thanks

Is hive binary only for hadoop 1.x ?

2015-06-27 Thread canan chen
I mean if I want to run on hadoop 2.x, do I must to build it from source with hadoop 2.x ?

Any reason for "no two aggregations can have different DISTINCT columns"

2015-03-17 Thread canan chen
Hive manual says that hive don't support "no two aggregations can have different DISTINCT columns". I think theoretically it is possible to implement this, is there any specific reason for that ? Performance reason or something else ?

Question on hive query correlation optimization

2015-03-12 Thread canan chen
I use the following sql with mr engine and find that it would invoke 3 mr jobs. But as my understanding the join and group by operator could be done in the same mr job since they are using the same key. So not sure why here still 3 mr jobs, anyone know that ? Thanks select s2.name,count(1) as cn