[
https://issues.apache.org/jira/browse/HIVE-12632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-12632:
------------------------------------
Attachment: HIVE-12632.patch
The patch. [~prasanth_j] looks like LLAP IO was also broken since MiniLlap
cluster integration, it was not initialized. I am fixing QTestUtils to
remove all the defaults from ctors to avoid accidental settings, and
adding back LLAP IO init. That works now; if some other tests break
because it was not running with them I will move that into separate bug
(and assign to you :P)
The main patch changes the explain output, and also checks splits for
acidity to determine which input format to return, falling back to the
wrapped one for ACID splits.
However, this part is not really tested because ACID code doesn't run
properly for the attached test. I modeled this on other ACID tests, except
for using the alter statements to get some partitions with original files
and some with base files.
I see no signs of ACID in the logs and LLAP just logs several original
files for repeated inserts.
[~alangates] [~ekoifman] is this a correct usage of ACID? (explains, drop
etc. omitted for brevity):
{noformat}
set hive.mapred.mode=nonstrict; SET
hive.vectorized.execution.enabled=true; SET hive.llap.io.enabled=false;
SET hive.exec.orc.default.buffer.size=32768; SET
hive.exec.orc.default.row.index.stride=1000; SET
hive.optimize.index.filter=true; set hive.fetch.task.conversion=none;
set hive.support.concurrency=true;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
CREATE TABLE orc_llap (...) partitioned by (csmallint smallint) clustered
by (cint) into 2 buckets stored as orc;
insert into table orc_llap partition (csmallint = 1) select ... from
alltypesorc ...;
insert into table orc_llap partition (csmallint = 2) select ... from
alltypesorc ...;
alter table orc_llap SET TBLPROPERTIES ('transactional'='true');
insert into table orc_llap partition (csmallint = 3) select ... from
alltypesorc ...;
SET hive.llap.io.enabled=true;
...
select cint, csmallint, cbigint from orc_llap where cint is not null;
...
insert into table orc_llap partition (csmallint = 1) values (1, 1, 1, 1);
...
select cint, csmallint, cbigint from orc_llap where cint is not null;
{noformat}
> LLAP: don't use IO elevator for ACID tables
> --------------------------------------------
>
> Key: HIVE-12632
> URL: https://issues.apache.org/jira/browse/HIVE-12632
> Project: Hive
> Issue Type: Bug
> Reporter: Takahiko Saito
> Assignee: Sergey Shelukhin
> Attachments: HIVE-12632.patch
>
>
> Until HIVE-12631 is fixed, we need to avoid ACID tables in IO elevator. Right
> now, a FileNotFound error is thrown.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
