[
https://issues.apache.org/jira/browse/HIVE-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576629#comment-16576629
]
Aihua Xu commented on HIVE-19767:
---------------------------------
[~szehon] Sure. Seems the second patch has the following issue: the
configurations are applied to HS2 service, but the other HiveConf won't get the
same overrides. In HiveConf:initialize() function, we call
applySystemProperties() to read properties from system properties. Now since
the patch doesn't set the values to system properties, the following freshly
created HiveConf won't get such values.
Of course, ideally I feel HiveConf will get initialized on the HS2 service
level once and then the new one gets cloned from that. But I think we have
some/many places create HiveConf object by calling new HiveConf(). The patch
#1 also won't be able to propagate the configuration to the following freshly
created HiveConf. It may or may not cause inconsistency.
A simple fix is probably through system property. We can register those
overrides as expected like OverrideVars. The ideal fix probably requires to get
rid of new HiveConf(), but that's a lot of change.
{noformat}
public static Map<String, String> getConfSystemProperties() {
Map<String, String> systemProperties = new HashMap<String, String>();
for (ConfVars oneVar : ConfVars.values()) {
if (System.getProperty(oneVar.varname) != null) {
if (System.getProperty(oneVar.varname).length() > 0) {
systemProperties.put(oneVar.varname,
System.getProperty(oneVar.varname));
}
}
for (ConfVars oneVar : OverrideVars.values()) {
if (System.getProperty(oneVar.varname) != null) {
if (System.getProperty(oneVar.varname).length() > 0) {
systemProperties.put(oneVar.varname,
System.getProperty(oneVar.varname));
}
}
}
return systemProperties;
}
{noformat}
> HiveServer2 should take hiveconf for non Hive properties
> --------------------------------------------------------
>
> Key: HIVE-19767
> URL: https://issues.apache.org/jira/browse/HIVE-19767
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 1.2.2, 3.0.0, 2.3.2
> Reporter: Szehon Ho
> Assignee: Szehon Ho
> Priority: Major
> Attachments: HIVE-19767.2.patch, HIVE-19767.patch
>
>
> The -hiveconf command line option works in HiveServer2 with properties in
> HiveConf.java, but not so well with other properties (like mapred properties
> or spark properties to control underlying execution engine, or custom
> properties understood by custom listeners)
> It is inconsistent with HiveCLI.
> HiveCLI behavior:
> {noformat}
> ./bin/hive --hiveconf a=b
> hive> set a;
> a=b {noformat}
> HiveServer2 behavior:
> {noformat}
> ./bin/hiveserver2 --hiveconf a=b
> beeline> set a;
> +-----------------+
> | set |
> +-----------------+
> | a is undefined |
> +-----------------+{noformat}
> Although it is possible to set up hive-site.xml or even mapred-site.xml to
> fill in the relevant properties, it is more convenient when testing HS2 with
> different configuration to be able to use --hiveconf to change on the fly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
