STAGE DEPENDENCIES: Stage-12 is a root stage Stage-3 depends on stages: Stage-12 Stage-0 depends on stages: Stage-3 Stage-4 depends on stages: Stage-0 STAGE PLANS: Stage: Stage-12 Map Reduce Local Work Alias -> Map Local Tables: ad Fetch Operator limit: -1 c Fetch Operator limit: -1 c1 Fetch Operator limit: -1 da Fetch Operator limit: -1 mb Fetch Operator limit: -1 Alias -> Map Local Operator Tree: ad TableScan alias: ad Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Filter Operator predicate: (cur_flag = 1) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE HashTable Sink Operator filter predicates: 0 {(nav_tcd = 9)} {(nav_tcd = 11)} {(nav_refer_page_type_id) IN (51, 52)} {(nav_tcd = 10)} {(nav_tcd) IN (13, '-')} {(nav_refer_page_type_id) IN (51, 52)} 1 2 3 4 keys: 0 UDFToDouble(nav_tcdt) (type: double) 1 UDFToDouble(site_categ_id) (type: double) 2 UDFToDouble(site_categ_id) (type: double) 3 UDFToDouble(mg_brand_id) (type: double) 4 UDFToDouble(attr_detl_id) (type: double) c TableScan alias: c Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Filter Operator predicate: (cur_flag = 1) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE HashTable Sink Operator filter predicates: 0 {(nav_tcd = 9)} {(nav_tcd = 11)} {(nav_refer_page_type_id) IN (51, 52)} {(nav_tcd = 10)} {(nav_tcd) IN (13, '-')} {(nav_refer_page_type_id) IN (51, 52)} 1 2 3 4 keys: 0 UDFToDouble(nav_tcdt) (type: double) 1 UDFToDouble(site_categ_id) (type: double) 2 UDFToDouble(site_categ_id) (type: double) 3 UDFToDouble(mg_brand_id) (type: double) 4 UDFToDouble(attr_detl_id) (type: double) c1 TableScan alias: c1 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Filter Operator predicate: (cur_flag = 1) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE HashTable Sink Operator filter predicates: 0 {(nav_tcd = 9)} {(nav_tcd = 11)} {(nav_refer_page_type_id) IN (51, 52)} {(nav_tcd = 10)} {(nav_tcd) IN (13, '-')} {(nav_refer_page_type_id) IN (51, 52)} 1 2 3 4 keys: 0 UDFToDouble(nav_tcdt) (type: double) 1 UDFToDouble(site_categ_id) (type: double) 2 UDFToDouble(site_categ_id) (type: double) 3 UDFToDouble(mg_brand_id) (type: double) 4 UDFToDouble(attr_detl_id) (type: double) da TableScan alias: da Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Filter Operator predicate: (cur_flag = 1) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE HashTable Sink Operator keys: 0 _col87 (type: bigint) 1 attr_id (type: bigint) mb TableScan alias: mb Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Filter Operator predicate: (cur_flag = 1) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE HashTable Sink Operator filter predicates: 0 {(nav_tcd = 9)} {(nav_tcd = 11)} {(nav_refer_page_type_id) IN (51, 52)} {(nav_tcd = 10)} {(nav_tcd) IN (13, '-')} {(nav_refer_page_type_id) IN (51, 52)} 1 2 3 4 keys: 0 UDFToDouble(nav_tcdt) (type: double) 1 UDFToDouble(site_categ_id) (type: double) 2 UDFToDouble(site_categ_id) (type: double) 3 UDFToDouble(mg_brand_id) (type: double) 4 UDFToDouble(attr_detl_id) (type: double) Stage: Stage-3 Map Reduce Map Operator Tree: TableScan alias: a Statistics: Num rows: 43775244 Data size: 7060894848 Basic stats: COMPLETE Column stats: NONE Map Join Operator condition map: Left Outer Join0 to 1 Left Outer Join0 to 2 Left Outer Join0 to 3 Left Outer Join0 to 4 filter predicates: 0 {(nav_tcd = 9)} {(nav_tcd = 11)} {(nav_refer_page_type_id) IN (51, 52)} {(nav_tcd = 10)} {(nav_tcd) IN (13, '-')} {(nav_refer_page_type_id) IN (51, 52)} 1 2 3 4 keys: 0 UDFToDouble(nav_tcdt) (type: double) 1 UDFToDouble(site_categ_id) (type: double) 2 UDFToDouble(site_categ_id) (type: double) 3 UDFToDouble(mg_brand_id) (type: double) 4 UDFToDouble(attr_detl_id) (type: double) outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col18, _col21, _col22, _col23, _col24, _col25, _col32, _col53, _col73, _col87, _col88 Statistics: Num rows: 192611077 Data size: 31067938004 Basic stats: COMPLETE Column stats: NONE Map Join Operator condition map: Left Outer Join0 to 1 keys: 0 _col87 (type: bigint) 1 attr_id (type: bigint) outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col18, _col21, _col22, _col23, _col24, _col25, _col32, _col53, _col73, _col88, _col103 Statistics: Num rows: 211872189 Data size: 34174732545 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: _col0 (type: string), _col1 (type: bigint), _col2 (type: int), _col3 (type: string), _col10 (type: string), _col11 (type: string), _col13 (type: string), _col12 (type: string), _col6 (type: string), _col7 (type: string), _col8 (type: string), _col9 (type: string), _col15 (type: string), _col32 (type: string), _col53 (type: string), _col73 (type: string), _col14 (type: string), _col88 (type: string), _col103 (type: string), _col4 (type: string), _col5 (type: string), _col16 (type: string), _col18 (type: string), _col24 (type: bigint), _col25 (type: int), _col21 (type: string), _col22 (type: double), _col23 (type: bigint) outputColumnNames: _col0, _col1, _col2, _col3, _col10, _col11, _col13, _col12, _col6, _col7, _col8, _col9, _col15, _col32, _col53, _col73, _col14, _col88, _col103, _col4, _col5, _col16, _col18, _col24, _col25, _col21, _col22, _col23 Statistics: Num rows: 211872189 Data size: 34174732545 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count(_col4), count(DISTINCT _col4), count(DISTINCT _col5), count(CASE WHEN ((length(_col16) > 0)) THEN (_col4) ELSE (null) END), count(DISTINCT CASE WHEN ((length(_col16) > 0)) THEN (_col4) ELSE (null) END), count(DISTINCT CASE WHEN ((length(_col16) > 0)) THEN (_col5) ELSE (null) END), count(CASE WHEN (((length(_col16) > 0) or (length(_col18) > 0))) THEN (_col4) ELSE (null) END), count(DISTINCT CASE WHEN (((length(_col16) > 0) or (length(_col18) > 0))) THEN (_col4) ELSE (null) END), count(DISTINCT CASE WHEN (((length(_col16) > 0) or (length(_col18) > 0))) THEN (_col5) ELSE (null) END), count(CASE WHEN ((length(_col18) > 0)) THEN (_col4) ELSE (null) END), count(DISTINCT CASE WHEN ((length(_col18) > 0)) THEN (_col4) ELSE (null) END), count(DISTINCT CASE WHEN ((length(_col18) > 0)) THEN (_col5) ELSE (null) END), count(CASE WHEN (((length(_col18) > 0) and (length(_col16) > 0))) THEN (_col4) ELSE (null) END), count(DISTINCT CASE WHEN (((length(_col18) > 0) and (length(_col16) > 0))) THEN (_col4) ELSE (null) END), count(DISTINCT CASE WHEN (((length(_col18) > 0) and (length(_col16) > 0))) THEN (_col5) ELSE (null) END), count(CASE WHEN (((length(_col18) > 0) and (length(_col16) = 0))) THEN (_col4) ELSE (null) END), count(DISTINCT CASE WHEN (((length(_col18) > 0) and (length(_col16) = 0))) THEN (_col4) ELSE (null) END), count(DISTINCT CASE WHEN (((length(_col18) > 0) and (length(_col16) = 0))) THEN (_col5) ELSE (null) END), count(DISTINCT _col24), count(DISTINCT CASE WHEN ((_col25 = 1)) THEN (_col24) ELSE (null) END), sum(_col22), count(DISTINCT CASE WHEN ((((length(_col21) = 0) or _col21 is null) or (_col21 = '-'))) THEN (null) ELSE (_col21) END), sum(_col23) keys: _col0 (type: string), _col1 (type: bigint), _col2 (type: int), _col3 (type: string), _col10 (type: string), _col11 (type: string), _col13 (type: string), _col12 (type: string), _col6 (type: string), _col7 (type: string), _col8 (type: string), _col9 (type: string), CASE WHEN ((_col15 = 9)) THEN (_col32) WHEN (((_col15 = 11) and (_col6) IN (51, 52))) THEN (_col53) WHEN ((_col15 = 10)) THEN (_col73) WHEN ((((_col15) IN (13, '-') and (_col6) IN (51, 52)) and (_col14 rlike '-'))) THEN (_col88) ELSE (_col14) END (type: string), CASE WHEN ((_col15 = 9)) THEN ('-') WHEN ((_col15 = 10)) THEN ('-') WHEN (((_col15 = 11) and (_col6) IN (51, 52))) THEN ('-') WHEN ((((_col15) IN (13, '-') and (_col6) IN (51, 52)) and (_col14 rlike '-'))) THEN (_col103) WHEN (((_col15) IN (13, '-') and (_col6) IN (51, 52))) THEN ('-') WHEN ((_col15 = 11)) THEN ('-') WHEN ((_col15 = 5)) THEN ('-') WHEN ((_col15 = 15)) THEN ('-') WHEN ((_col15 = 16)) THEN ('-') WHEN ((_col15 = 17)) THEN ('-') WHEN ((_col15 = 14)) THEN ('-') WHEN ((_col15 = 12)) THEN ('-') ELSE (_col15) END (type: string), _col4 (type: string), _col5 (type: string), CASE WHEN ((length(_col16) > 0)) THEN (_col4) ELSE (null) END (type: string), CASE WHEN ((length(_col16) > 0)) THEN (_col5) ELSE (null) END (type: string), CASE WHEN (((length(_col16) > 0) or (length(_col18) > 0))) THEN (_col4) ELSE (null) END (type: string), CASE WHEN (((length(_col16) > 0) or (length(_col18) > 0))) THEN (_col5) ELSE (null) END (type: string), CASE WHEN ((length(_col18) > 0)) THEN (_col4) ELSE (null) END (type: string), CASE WHEN ((length(_col18) > 0)) THEN (_col5) ELSE (null) END (type: string), CASE WHEN (((length(_col18) > 0) and (length(_col16) > 0))) THEN (_col4) ELSE (null) END (type: string), CASE WHEN (((length(_col18) > 0) and (length(_col16) > 0))) THEN (_col5) ELSE (null) END (type: string), CASE WHEN (((length(_col18) > 0) and (length(_col16) = 0))) THEN (_col4) ELSE (null) END (type: string), CASE WHEN (((length(_col18) > 0) and (length(_col16) = 0))) THEN (_col5) ELSE (null) END (type: string), _col24 (type: bigint), CASE WHEN ((_col25 = 1)) THEN (_col24) ELSE (null) END (type: bigint), CASE WHEN ((((length(_col21) = 0) or _col21 is null) or (_col21 = '-'))) THEN (null) ELSE (_col21) END (type: string) mode: hash outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32, _col33, _col34, _col35, _col36, _col37, _col38, _col39, _col40, _col41, _col42, _col43, _col44, _col45, _col46, _col47, _col48, _col49, _col50, _col51 Statistics: Num rows: 211872189 Data size: 34174732545 Basic stats: COMPLETE Column stats: NONE Reduce Output Operator key expressions: _col0 (type: string), _col1 (type: bigint), _col2 (type: int), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string), _col7 (type: string), _col8 (type: string), _col9 (type: string), _col10 (type: string), _col11 (type: string), _col12 (type: string), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: string), _col18 (type: string), _col19 (type: string), _col20 (type: string), _col21 (type: string), _col22 (type: string), _col23 (type: string), _col24 (type: string), _col25 (type: string), _col26 (type: bigint), _col27 (type: bigint), _col28 (type: string) sort order: +++++++++++++++++++++++++++++ Map-reduce partition columns: _col0 (type: string), _col1 (type: bigint), _col2 (type: int), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string), _col7 (type: string), _col8 (type: string), _col9 (type: string), _col10 (type: string), _col11 (type: string), _col12 (type: string), _col13 (type: string) Statistics: Num rows: 211872189 Data size: 34174732545 Basic stats: COMPLETE Column stats: NONE value expressions: _col29 (type: bigint), _col32 (type: bigint), _col35 (type: bigint), _col38 (type: bigint), _col41 (type: bigint), _col44 (type: bigint), _col49 (type: double), _col51 (type: bigint) Local Work: Map Reduce Local Work Reduce Operator Tree: Group By Operator aggregations: count(VALUE._col0), count(DISTINCT KEY._col14:0._col0), count(DISTINCT KEY._col14:1._col0), count(VALUE._col3), count(DISTINCT KEY._col14:2._col0), count(DISTINCT KEY._col14:3._col0), count(VALUE._col6), count(DISTINCT KEY._col14:4._col0), count(DISTINCT KEY._col14:5._col0), count(VALUE._col9), count(DISTINCT KEY._col14:6._col0), count(DISTINCT KEY._col14:7._col0), count(VALUE._col12), count(DISTINCT KEY._col14:8._col0), count(DISTINCT KEY._col14:9._col0), count(VALUE._col15), count(DISTINCT KEY._col14:10._col0), count(DISTINCT KEY._col14:11._col0), count(DISTINCT KEY._col14:12._col0), count(DISTINCT KEY._col14:13._col0), sum(VALUE._col20), count(DISTINCT KEY._col14:14._col0), sum(VALUE._col22) keys: KEY._col0 (type: string), KEY._col1 (type: bigint), KEY._col2 (type: int), KEY._col3 (type: string), KEY._col4 (type: string), KEY._col5 (type: string), KEY._col6 (type: string), KEY._col7 (type: string), KEY._col8 (type: string), KEY._col9 (type: string), KEY._col10 (type: string), KEY._col11 (type: string), KEY._col12 (type: string), KEY._col13 (type: string) mode: mergepartial outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32, _col33, _col34, _col35, _col36 Statistics: Num rows: 105936094 Data size: 17087366191 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: _col0 (type: string), _col1 (type: bigint), UDFToLong(_col2) (type: bigint), UDFToLong(_col3) (type: bigint), 1 (type: int), UDFToLong(_col4) (type: bigint), _col5 (type: string), _col6 (type: string), _col7 (type: string), UDFToLong(_col8) (type: bigint), _col9 (type: string), UDFToLong(_col10) (type: bigint), _col11 (type: string), _col14 (type: bigint), _col15 (type: bigint), _col16 (type: bigint), _col17 (type: bigint), _col18 (type: bigint), _col19 (type: bigint), _col20 (type: bigint), _col21 (type: bigint), _col22 (type: bigint), _col23 (type: bigint), _col24 (type: bigint), _col25 (type: bigint), _col26 (type: bigint), _col27 (type: bigint), _col28 (type: bigint), _col29 (type: bigint), _col30 (type: bigint), _col31 (type: bigint), _col32 (type: bigint), _col33 (type: bigint), _col34 (type: double), _col35 (type: bigint), _col36 (type: bigint), UDFToLong(0) (type: bigint), '2015-08-27 15:50:39' (type: string), _col12 (type: string), _col13 (type: string) outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32, _col33, _col34, _col35, _col36, _col37, _col38, _col39 Statistics: Num rows: 105936094 Data size: 17087366191 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false Statistics: Num rows: 105936094 Data size: 17087366191 Basic stats: COMPLETE Column stats: NONE table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: testbic.trfc_tpa_d Stage: Stage-0 Move Operator tables: partition: ds - step - replace: true table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: testbic.trfc_tpa_d Stage: Stage-4 Stats-Aggr Operator