[
https://issues.apache.org/jira/browse/HIVE-21377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787567#comment-16787567
]
Bo commented on HIVE-21377:
----------------------------
[~pvary] Even we pass the logic for this issue, we also see the other
issue(blob-cast-error.jpg).
So, could we open the jira for the implementing the DirectSQL feature with
Oracle?
The blog-case.error-jpg, this issues is related to MetastoreDirectSqlUtils.java
{code:java}
static void setPartitionParameters(String PARTITION_PARAMS, boolean
convertMapNullsToEmptyStrings,
PersistenceManager pm, String partIds, TreeMap<Long, Partition>
partitions)
throws MetaException {
String queryText;
queryText = "select \"PART_ID\", \"PARAM_KEY\", \"PARAM_VALUE\" from " +
PARTITION_PARAMS + ""
+ " where \"PART_ID\" in (" + partIds + ") and \"PARAM_KEY\" is not
null"
+ " order by \"PART_ID\" asc";
loopJoinOrderedResult(pm, partitions, queryText, 0, new
ApplyFunc<Partition>() {
@Override
public void apply(Partition t, Object[] fields) {
t.putToParameters((String)fields[1], fields[2])); <<< fields[2] is
related to the WARN log
}});
// Perform conversion of null map values
for (Partition t : partitions.values())
{code}
By the way, this code is already changed in last version of Hive accordingly:
{code:java}
public void apply(Partition t, Object[] fields) {
t.putToParameters((String)fields[1], extractSqlClob(fields[2]));
}});
// Perform conversion of null map values
for (Partition t : partitions.values()) {
t.setParameters(MetaStoreServerUtils.trimMapNulls(t.getParameters(),
convertMapNullsToEmptyStrings));
{code}
By the way, could we consider to open the JIRA about the implementing DirectSQL
features for Oracle DB?
> Using Oracle as HMS DB with DirectSQL
> -------------------------------------
>
> Key: HIVE-21377
> URL: https://issues.apache.org/jira/browse/HIVE-21377
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 3.0.0, 3.1.0
> Reporter: Bo
> Assignee: Rajkumar Singh
> Priority: Major
> Attachments: HIVE-21377.01.patch, HIVE-21377.patch
>
>
> When we use the Oracle as HMS DB, we saw this kind of contents in the HMS log
> accordingly:
> {code:java}
> 2019-02-02 T08:23:57,102 WARN [Thread-12]: metastore.ObjectStore
> (ObjectStore.java:handleDirectSqlError(3741)) - Falling back to ORM path due
> to direct SQL failure (this is not an error): Cannot extract boolean from
> column value 0 at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.extractSqlBoolean(MetaStoreDirectSql.java:1031)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsFromPartitionIds(MetaStoreDirectSql.java:728)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.access$300(MetaStoreDirectSql.java:109)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql$1.run(MetaStoreDirectSql.java:471)
> at org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:73)
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsViaSqlFilter(MetaStoreDirectSql.java:462)
> at
> org.apache.hadoop.hive.metastore.ObjectStore$8.getSqlResult(ObjectStore.java:3392)
> {code}
> In Hive, we handle the Postgres, MySQL and Derby for the extractSqlBoolean.
> But Oracle return the 0 or 1 for Boolean. So we need to modify the
> MetastoreDirectSqlUtils.java - [1]
> So, could add this snip in this code?
> {code:java}
> static Boolean extractSqlBoolean(Object value) throws MetaException {
> if (value == null) {
> return null;
> }
> if (value instanceof Boolean) {
> return (Boolean)value;
> }
> if (value instanceof Number) { // add
> try {
> return BooleanUtils.toBooleanObject((BigDecimal) value, 1, 0, null);
> } catch(IllegalArugmentExeception iae){
> // NOOP
> }
> if (value instanceof String) {
> try {
> return BooleanUtils.toBooleanObject((String) value, "Y", "N", null);
> } catch (IllegalArgumentException iae) {
> // NOOP
> }
> }
> throw new MetaException("Cannot extract boolean from column value " +
> value);
> }
> {code}
> [1] -
> https://github.com/apache/hive/blob/f51f108b761f0c88647f48f30447dae12b308f31/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDirectSqlUtils.java#L501-L527
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
