Hi there,
We are in the process of migrating to HDP 3.1.4, from an older HDP stack.
We have noticed strange behaviour regarding the CBP and how it treats null
checks against complex types.
consider the following:
CREATE TABLE test_null_array (null_array ARRAY<STRING>);
INSERT OVERWRITE TABLE test_null_array SELECT EXPLODE(ARRAY(NULL,
ARRAY('foobar')));
SELECT * FROM test_null_array; -- returns two rows (NULL, ["foobar"])
SET hive.cbo.enable=false;
SELECT * FROM test_null_array WHERE null_array IS NULL; -- returns one row
(NULL) as expected
EXPLAIN SELECT * FROM test_null_array WHERE null_array IS NULL; -- line 9:
predicate:null_array is null
SET hive.cbo.enable=true;
SELECT * FROM test_null_array WHERE null_array IS NULL; -- returns zero
rows *unexpected*
EXPLAIN SELECT * FROM test_null_array WHERE null_array IS NULL; -- line 9:
predicate:false
We have a suspicion that this is new behaviour for HIVE 3. Has anyone else
noticed this behaviour, can direct me to documentation about this?
Thank you, Eyal Ben Ivri
