HI Gopal,
Another question which i have is whenever we run Union All statement,
apart from Folders we also see Zero Bytes Files in HDFS. Are there locks
file (LCK) ?
Mahender
On 2/24/2016 4:26 PM, Gopal Vijayaraghavan wrote:
>> SET mapred.input.dir.recursive=TRUE;
> ...
>> Can we set above set
Thanks Gopal will look into it
On 2/24/2016 4:26 PM, Gopal Vijayaraghavan wrote:
SET mapred.input.dir.recursive=TRUE;
...
Can we set above setting as tblProperties or Hive Table properties.
Not directly, those are MapReduce properties - they are not settable via
Hive tables.
That said, you c
> SET mapred.input.dir.recursive=TRUE;
...
> Can we set above setting as tblProperties or Hive Table properties.
Not directly, those are MapReduce properties - they are not settable via
Hive tables.
That said, you can write your own SemanticAnalyzerHooks to do pretty much
anything you want like t
Thanks Gopal. This is a architectural change from Hive 0.13 to hive
1.2. We are migrating our hive query from 0.13 to 1.2. Previously it is
running perfectly against 0.13 but same query in 1.2 is failing due to
union/union-all performance improvement. because of creation of sub
directories. W
>Is there anyway to avoid creating sub-directories while running in tez?
>Or this is by design and can not be changed?
Yes, this is by design. The Tez execution of UNION is entirely parallel &
the task-ids overlaps - so the files created have to have unique names.
But the total counts for "Map 1
Hi
Below insert with union will create sub-directories while executing in Tez.
set hive.execution.engine=tez;
insert overwrite table t3
select * from t1 limit 1
union
select * from t2 limit 2 ;
Is there anyway to avoid creating sub-directories while running in tez? Or this
is by d
> Is there anyway to avoid creating sub-directories? Or this is by design
>and can not be changed?
This is because of the way file-formats generate hadoop name files without
collisions.
For instance, any change to that would break Parquet-MR for Tez. That's
why we generate a compatible, but colli
Hi Team,
Below insert with union-all will create sub-directories:
set hive.execution.engine=tez;
create table h1_passwords_target like h1_passwords;
insert overwrite table h1_passwords_target
select * from
(select * from h1_passwords limit 1
union all
select * from h1_passwords limit 2 ) sub
