Pengcheng Xiong created HIVE-13988:
--------------------------------------
Summary: zero length file is being created for empty bucket in tez
mode
Key: HIVE-13988
URL: https://issues.apache.org/jira/browse/HIVE-13988
Project: Hive
Issue Type: Bug
Reporter: Pengcheng Xiong
Assignee: Pengcheng Xiong
Even though bucket is empty, zero length file is being created in tez mode.
steps to reproduce the issue:
{noformat}
hive> set hive.execution.engine;
hive.execution.engine=tez
hive> drop table if exists emptybucket_orc;
OK
Time taken: 5.416 seconds
hive> create table emptybucket_orc(age int) clustered by (age) sorted by (age)
into 99 buckets stored as orc;
OK
Time taken: 0.493 seconds
hive> insert into table emptybucket_orc select distinct(age) from studenttab10k
limit 0;
Query ID = hrt_qa_20160523231955_8b981be7-68c4-4416-8a48-5f8c7ff551c3
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1464045121842_0002)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... llap SUCCEEDED 1 1 0 0
0 0
Reducer 2 ...... llap SUCCEEDED 1 1 0 0
0 0
Reducer 3 ...... llap SUCCEEDED 1 1 0 0
0 0
Reducer 4 ...... llap SUCCEEDED 99 99 0 0
0 0
----------------------------------------------------------------------------------------------
VERTICES: 04/04 [==========================>>] 100% ELAPSED TIME: 11.00 s
----------------------------------------------------------------------------------------------
Loading data to table default.emptybucket_orc
OK
Time taken: 16.907 seconds
hive> dfs -ls /apps/hive/warehouse/emptybucket_orc;
Found 99 items
-rwxrwxrwx 3 hrt_qa hdfs 0 2016-05-23 23:20
/apps/hive/warehouse/emptybucket_orc/000000_0
-rwxrwxrwx 3 hrt_qa hdfs 0 2016-05-23 23:20
/apps/hive/warehouse/emptybucket_orc/000001_0
......
{noformat}
Expected behavior:
In tez mode, zero length file shouldn't get created on hdfs if bucket is empty
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
