[
https://issues.apache.org/jira/browse/HIVE-23406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-23406:
----------------------------------
Description:
SharedWorkOptimizer does not checks null sort order in ReduceSinkDesc when
compares ReduceSink operators:
[https://github.com/apache/hive/blob/ca9aba606c4d09b91ee28bf9ee1ae918db8cdfb9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java#L1444]
{code:java}
ReduceSinkDesc op1Conf = ((ReduceSinkOperator) op1).getConf();
ReduceSinkDesc op2Conf = ((ReduceSinkOperator) op2).getConf();
if (StringUtils.equals(op1Conf.getKeyColString(),
op2Conf.getKeyColString()) &&
StringUtils.equals(op1Conf.getValueColsString(),
op2Conf.getValueColsString()) &&
StringUtils.equals(op1Conf.getParitionColsString(),
op2Conf.getParitionColsString()) &&
op1Conf.getTag() == op2Conf.getTag() &&
StringUtils.equals(op1Conf.getOrder(), op2Conf.getOrder()) &&
op1Conf.getTopN() == op2Conf.getTopN() &&
canDeduplicateReduceTraits(op1Conf, op2Conf)) {
return true;
} else {
return false;
}
{code}
An expression like
{code:java}
StringUtils.equals(op1Conf.getNullOrder(), op2Conf.getNullOrder()) &&
{code}
should be added.
was:
SharedWorkOptimizer does not checks null sort order in ReduceSinkDesc when
compares to ReduceSink operators:
[https://github.com/apache/hive/blob/ca9aba606c4d09b91ee28bf9ee1ae918db8cdfb9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java#L1444]
{code:java}
ReduceSinkDesc op1Conf = ((ReduceSinkOperator) op1).getConf();
ReduceSinkDesc op2Conf = ((ReduceSinkOperator) op2).getConf();
if (StringUtils.equals(op1Conf.getKeyColString(),
op2Conf.getKeyColString()) &&
StringUtils.equals(op1Conf.getValueColsString(),
op2Conf.getValueColsString()) &&
StringUtils.equals(op1Conf.getParitionColsString(),
op2Conf.getParitionColsString()) &&
op1Conf.getTag() == op2Conf.getTag() &&
StringUtils.equals(op1Conf.getOrder(), op2Conf.getOrder()) &&
op1Conf.getTopN() == op2Conf.getTopN() &&
canDeduplicateReduceTraits(op1Conf, op2Conf)) {
return true;
} else {
return false;
}
{code}
An expression like
{code:java}
StringUtils.equals(op1Conf.getNullOrder(), op2Conf.getNullOrder()) &&
{code}
should be added.
> SharedWorkOptimizer should check nullSortOrders when comparing ReduceSink
> operators
> -----------------------------------------------------------------------------------
>
> Key: HIVE-23406
> URL: https://issues.apache.org/jira/browse/HIVE-23406
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
> Reporter: Krisztian Kasa
> Priority: Major
>
> SharedWorkOptimizer does not checks null sort order in ReduceSinkDesc when
> compares ReduceSink operators:
>
> [https://github.com/apache/hive/blob/ca9aba606c4d09b91ee28bf9ee1ae918db8cdfb9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java#L1444]
> {code:java}
> ReduceSinkDesc op1Conf = ((ReduceSinkOperator) op1).getConf();
> ReduceSinkDesc op2Conf = ((ReduceSinkOperator) op2).getConf();
> if (StringUtils.equals(op1Conf.getKeyColString(),
> op2Conf.getKeyColString()) &&
> StringUtils.equals(op1Conf.getValueColsString(),
> op2Conf.getValueColsString()) &&
> StringUtils.equals(op1Conf.getParitionColsString(),
> op2Conf.getParitionColsString()) &&
> op1Conf.getTag() == op2Conf.getTag() &&
> StringUtils.equals(op1Conf.getOrder(), op2Conf.getOrder()) &&
> op1Conf.getTopN() == op2Conf.getTopN() &&
> canDeduplicateReduceTraits(op1Conf, op2Conf)) {
> return true;
> } else {
> return false;
> }
> {code}
> An expression like
> {code:java}
> StringUtils.equals(op1Conf.getNullOrder(), op2Conf.getNullOrder()) &&
> {code}
> should be added.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
