[
https://issues.apache.org/jira/browse/HIVE-27943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794301#comment-17794301
]
Stamatis Zampetakis commented on HIVE-27943:
--------------------------------------------
I raised a PR with a fix for this particular problem but I am afraid that its
not the only one. For sure, adding more tests with map joins and dates will
uncover more runtime failures but let's treat each under its own dedicated
ticket.
> NPE in VectorMapJoinCommonOperator.setUpHashTable when running query with
> join on date
> --------------------------------------------------------------------------------------
>
> Key: HIVE-27943
> URL: https://issues.apache.org/jira/browse/HIVE-27943
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 4.0.0-beta-1
> Reporter: Stamatis Zampetakis
> Assignee: Stamatis Zampetakis
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-27943_plan.txt
>
>
> The error can be reproduced on [current
> master|https://github.com/apache/hive/commit/fd6ced288dbf9ce7f3c3a2ca948d78f3b88f170f]
> using the following steps.
> {code:sql}
> set hive.auto.convert.join=true;
> CREATE TABLE person (fname string, birthDate date);
> INSERT INTO person VALUES ('Victor', '2023-11-27'), ('Alexandre',
> '2023-11-28');
> SELECT * FROM person p1 INNER JOIN person p2 ON p1.birthDate=p2.birthDate;
> {code}
> The stacktrace is shown below.
> {noformat}
> 2023-12-07T07:06:39,744 ERROR [TezTR-592758_1_2_1_0_0] tez.TezProcessor:
> Failed initializeAndRunProcessor
> java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:351)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:292)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:276)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:381)
> ~[tez-runtime-internals-0.10.2.jar:0.10.2]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:82)
> ~[tez-runtime-internals-0.10.2.jar:0.10.2]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:69)
> ~[tez-runtime-internals-0.10.2.jar:0.10.2]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[?:1.8.0_261]
> at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_261]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
> ~[hadoop-common-3.3.6.jar:?]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:69)
> ~[tez-runtime-internals-0.10.2.jar:0.10.2]
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:39)
> ~[tez-runtime-internals-0.10.2.jar:0.10.2]
> at
> org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> ~[tez-common-0.10.2.jar:0.10.2]
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:118)
> ~[hive-llap-server-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[?:1.8.0_261]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ~[?:1.8.0_261]
> at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_261]
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinCommonOperator.setUpHashTable(VectorMapJoinCommonOperator.java:673)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinCommonOperator.completeInitializationOp(VectorMapJoinCommonOperator.java:634)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:450)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:382)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:549)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:503)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:369)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:332)
> ~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
> ... 16 more
> {noformat}
> This is probably caused by HIVE-23852 which added the DATE type as a possible
> key for the HashTable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
