[
https://issues.apache.org/jira/browse/HIVE-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15686282#comment-15686282
]
Rui Li commented on HIVE-15259:
-------------------------------
With Spark 2.0, you don't have to copy all the jars to Hive lib. Please refer
to our wiki
https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started.
The log you posted is from Spark. I guess it uploads the required jars to
prepare the classpath for containers. Not sure if it's related to the
deserialization time. And on Hive side we don't have much control on it - we
basically just specify that hive-exec is needed. Others are up to Spark.
> The deserialization time of HOS20 is longer than what in HOS16
> ---------------------------------------------------------------
>
> Key: HIVE-15259
> URL: https://issues.apache.org/jira/browse/HIVE-15259
> Project: Hive
> Issue Type: Improvement
> Reporter: liyunzhang_intel
> Attachments: Deserialization_HOS16.PNG, Deserialization_HOS20.PNG
>
>
> deploy Hive on Spark on spark 1.6 version and spark 2.0 version.
> run query and in latest code(with spark2.0) the deserialization time of a
> task is 4 sec while the deserialization time of spark1.6 is 1 sec. The detail
> is in attached picture.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
