Mark Grover created HIVE-3104:
---------------------------------
Summary: Predicate pushdown doesn't work with multi-insert
statements using LATERAL VIEW
Key: HIVE-3104
URL: https://issues.apache.org/jira/browse/HIVE-3104
Project: Hive
Issue Type: Bug
Components: CLI
Affects Versions: 0.9.0
Environment: Apache Hive 0.9.0, Apache Hadoop 0.20.205.0
Reporter: Mark Grover
Fix For: 0.10.0
Predicate pushdown seems to work for single-insert queries using LATERAL VIEW.
It also seems to work for multi-insert queries *not* using LATERAL VIEW.
However, it doesn't work for multi-insert queries using LATERAL VIEW.
Here are some examples. In the below examples, I make use of the fact that a
query with no partition filtering when run under "hive.mapred.mode=strict"
fails.
--Table creation and population
DROP TABLE IF EXISTS test;
CREATE TABLE test (col1 array<int>, col2 int) PARTITIONED BY (part_col int);
INSERT OVERWRITE TABLE test PARTITION (part_col=1) SELECT array(1,2), count(*)
FROM test;
INSERT OVERWRITE TABLE test PARTITION (part_col=2) SELECT array(2,4,6),
count(*) FROM test;
-- Query 1
-- This succeeds (using LATERAL VIEW with single insert)
set hive.mapred.mode=strict;
FROM partition_test
LATERAL VIEW explode(col1) tmp AS exp_col1
INSERT OVERWRITE DIRECTORY '/test/1'
SELECT exp_col1
WHERE (part_col=2);
-- Query 2
-- This succeeds (NOT using LATERAL VIEW with multi-insert)
set hive.mapred.mode=strict;
FROM partition_test
INSERT OVERWRITE DIRECTORY '/test/1'
SELECT col1
WHERE (part_col=2)
INSERT OVERWRITE DIRECTORY '/test/2'
SELECT col1
WHERE (part_col=2);
-- Query 3
-- This fails (using LATERAL VIEW with multi-insert)
set hive.mapred.mode=strict;
FROM partition_test
LATERAL VIEW explode(col1) tmp AS exp_col1
INSERT OVERWRITE DIRECTORY '/test/1'
SELECT exp_col1
WHERE (part_col=2)
INSERT OVERWRITE DIRECTORY '/test/2'
SELECT exp_col1
WHERE (part_col=2);
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
