[
https://issues.apache.org/jira/browse/IMPALA-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy updated IMPALA-14070:
---------------------------------------
Description:
The problem is with this code fragment in SortNode.computeStats:
{noformat}
long partNdv = numPartitionExprs_ == 0 ? 1 :
Expr.getNumDistinctValues(partExprs);
if (partNdv >= 0) {
long maxRowsInHeaps = partNdv * getPerPartitionLimit(); <====
maxRowsInHeaps can overflow and can become less than -1
if (cardinality_ < 0 || cardinality_ > maxRowsInHeaps) {
cardinality_ = maxRowsInHeaps;
}
}
{noformat}
If maxRowsInHeaps overflows and becomes less than -1 then the cardinality will
be invalid and we will hit this precondition check later in
PlanNode.smallestValidCardinality:
{noformat}
protected static long smallestValidCardinality(long cardinality1, long
cardinality2) {
Preconditions.checkArgument(
cardinality1 >= -1, "cardinality1 is invalid: %s", cardinality1);
Preconditions.checkArgument(
cardinality2 >= -1, "cardinality2 is invalid: %s", cardinality2);
...
{noformat}
was:
The problem is with this code fragment:
{noformat}
long partNdv = numPartitionExprs_ == 0 ? 1 :
Expr.getNumDistinctValues(partExprs);
if (partNdv >= 0) {
long maxRowsInHeaps = partNdv * getPerPartitionLimit(); <====
maxRowsInHeaps can overflow and can become less than -1
if (cardinality_ < 0 || cardinality_ > maxRowsInHeaps) {
cardinality_ = maxRowsInHeaps;
}
}
{noformat}
If maxRowsInHeaps overflows and becomes less than -1 then the cardinality will
be invalid and we will hit this precondition check later in
PlanNode.smallestValidCardinality:
{noformat}
protected static long smallestValidCardinality(long cardinality1, long
cardinality2) {
Preconditions.checkArgument(
cardinality1 >= -1, "cardinality1 is invalid: %s", cardinality1);
Preconditions.checkArgument(
cardinality2 >= -1, "cardinality2 is invalid: %s", cardinality2);
...
{noformat}
> SortNode's computeStats can calculate invalid cardinality
> ---------------------------------------------------------
>
> Key: IMPALA-14070
> URL: https://issues.apache.org/jira/browse/IMPALA-14070
> Project: IMPALA
> Issue Type: Bug
> Reporter: Zoltán Borók-Nagy
> Priority: Major
>
> The problem is with this code fragment in SortNode.computeStats:
> {noformat}
> long partNdv = numPartitionExprs_ == 0 ? 1 :
> Expr.getNumDistinctValues(partExprs);
> if (partNdv >= 0) {
> long maxRowsInHeaps = partNdv * getPerPartitionLimit(); <====
> maxRowsInHeaps can overflow and can become less than -1
> if (cardinality_ < 0 || cardinality_ > maxRowsInHeaps) {
> cardinality_ = maxRowsInHeaps;
> }
> }
> {noformat}
> If maxRowsInHeaps overflows and becomes less than -1 then the cardinality
> will be invalid and we will hit this precondition check later in
> PlanNode.smallestValidCardinality:
> {noformat}
> protected static long smallestValidCardinality(long cardinality1, long
> cardinality2) {
> Preconditions.checkArgument(
> cardinality1 >= -1, "cardinality1 is invalid: %s", cardinality1);
> Preconditions.checkArgument(
> cardinality2 >= -1, "cardinality2 is invalid: %s", cardinality2);
> ...
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
