[
https://issues.apache.org/jira/browse/HIVE-12880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-12880:
------------------------------------
Description:
It looks like spark-assembly contains versions of Hive classes (e.g. HiveConf),
and these sometimes (always?) come from older versions of Hive.
We've seen problems where depending on classpath perturbations, NoSuchField
errors may be thrown for recently added ConfVars because the HiveConf class
comes from spark-assembly.
Would making sure spark-assembly comes last in the classpath solve the problem?
Otherwise, can we depend on something that does not package older Hive classes?
Currently, HIVE-12179 provides a workaround (in non-Spark use case, at least; I
am assuming this issue can also affect Hive-on-Spark).
was:
It looks like spark-assembly contains versions of Hive classes (e.g. HiveConf),
and these sometimes (always?) come from older versions of Hive.
We've seen problems where depending on classpath changes, NoSuchField errors
are thrown for recently added configs because the HiveConf class comes from
spark-assembly.
Would making sure spark-assembly comes last in the classpath solve the problem?
Otherwise, can we depend on something that does not package Hive classes?
Currently, HIVE-12179 provides a workaround (in non-Spark use case, at least; I
am assuming this issue can also affect Hive-on-Spark).
> spark-assembly causes Hive class version problems
> -------------------------------------------------
>
> Key: HIVE-12880
> URL: https://issues.apache.org/jira/browse/HIVE-12880
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
>
> It looks like spark-assembly contains versions of Hive classes (e.g.
> HiveConf), and these sometimes (always?) come from older versions of Hive.
> We've seen problems where depending on classpath perturbations, NoSuchField
> errors may be thrown for recently added ConfVars because the HiveConf class
> comes from spark-assembly.
> Would making sure spark-assembly comes last in the classpath solve the
> problem?
> Otherwise, can we depend on something that does not package older Hive
> classes?
> Currently, HIVE-12179 provides a workaround (in non-Spark use case, at least;
> I am assuming this issue can also affect Hive-on-Spark).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
