[
https://issues.apache.org/jira/browse/HIVE-22410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964567#comment-16964567
]
fanshilun commented on HIVE-22410:
----------------------------------
[~samuelan]
The problem does not appear in the Sql query, but appears in the HMS startup
warm-up，I will give some detailed instructions.
1.The problem occurs in the hot load function prewarm (RawStore rawStore) of
the CachedStore. After the DataBase is loaded, the table information loading
starts.
2.sharedCache.populateTableInCache(table, tableColStats...) CachedStore 559
Lines Throw NPE.
3.Method populateTableInCache calculates the memory of the object, and throws
NPE when performing OBJECT_ARRAY calculation.
> CachedStore Prewarm Failure NPE
> -------------------------------
>
> Key: HIVE-22410
> URL: https://issues.apache.org/jira/browse/HIVE-22410
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
> Affects Versions: 3.1.2, 2.3.7
> Environment: JDK 1.7.0_40
> Reporter: fanshilun
> Assignee: fanshilun
> Priority: Major
> Labels: cachedstore
> Fix For: 2.3.7, 3.1.3
>
> Attachments: HIVE-22410.1.patch
>
>
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimateArrayElements(IncrementalObjectSizeEstimator.java:501)
> ~[hive-metastore-2.3.5.jar:2.3.5]
> at
> org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:443)
> ~[hive-metastore-2.3.5.jar:2.3.5]
> at
> org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:433)
> ~[hive-metastore-2.3.5.jar:2.3.5]
> at
> org.apache.hadoop.hive.metastore.utils.IncrementalObjectSizeEstimator$ObjectEstimator.estimate(IncrementalObjectSizeEstimator.java:369)
> ~[hive-metastore-2.3.5.jar:2.3.5]
> at
> org.apache.hadoop.hive.metastore.cached.SharedCache.populateTableInCache(SharedCache.java:1028)
> ~[hive-metastore-2.3.5.jar:2.3.5]
> at
> org.apache.hadoop.hive.metastore.cached.CachedStore.prewarm(CachedStore.java:314)
> ~[hive-metastore-2.3.5.jar:2.3.5]
> at
> org.apache.hadoop.hive.metastore.cached.CachedStore$CacheUpdateMasterWork.run(CachedStore.java:492)
> [hive-metastore-2.3.5.jar:2.3.5]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [?:1.7.0_40]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [?:1.7.0_40]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> [?:1.7.0_40]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [?:1.7.0_40]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [?:1.7.0_40]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [?:1.7.0_40]
> at java.lang.Thread.run(Thread.java:724) [?:1.7.0_40]
> *reason:*
> Methods estimateArrayElements throw NPE
> I found that the last class does not appear in the parent (HashMap<Class<?>,
> Object Estimator> ), and the parent does not contain the information of the
> ConcurrentHashMap Class.
> The solution is to initialize the class information before using it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
