yashmayya opened a new issue, #16419: URL: https://github.com/apache/pinot/issues/16419
- Function calls with literal-only operands get evaluated in the broker during query planning / optimization itself instead of during query execution in the servers as an optimization. - This is done through the [PinotEvaluateLiteralRule](https://github.com/apache/pinot/blob/9a18becd1591340f653b1e6caf53ba069b1645e4/pinot-query-planner/src/main/java/org/apache/pinot/calcite/rel/rules/PinotEvaluateLiteralRule.java#L59). - However, there's a bug that prevents `CAST` functions from being evaluated. The `CAST` operator in Calcite only has one operand (the value to be cast) and the type to be cast to is determined by the operator's return type. Pinot's `CAST` function implementation requires two arguments: the value to be cast and the target type. - This discrepancy causes the `CAST` function to not be found in the rule implementation, and the function call is retained as is. - This can cause a myriad of downstream issues. For instance, take a query like: ``` select AirlineID from airlineStats where ts > '2000-05-16T00:00:00' and ts < '2025-05-16T00:00:00'; ``` - The filter becomes `AND(>($82, CAST(_UTF-8'2000-05-16T00:00:00'):TIMESTAMP(0) NOT NULL), <($82, CAST(_UTF-8'2025-05-16T00:00:00'):TIMESTAMP(0) NOT NULL))` where the casts are added by Calcites to convert the string literals to the appropriate type. - Since the `CAST` doesn't get evaluated during query planning, it makes it down all the way to the leaf stage, where [PredicateComparisonRewriter](https://github.com/apache/pinot/blob/9a18becd1591340f653b1e6caf53ba069b1645e4/pinot-common/src/main/java/org/apache/pinot/sql/parsers/rewriter/PredicateComparisonRewriter.java#L33) rewrites the filter expression to: ``` FilterAnd FilterExpression(predicate=[minus(ts,cast('2000-05-16T00:00:00','TIMESTAMP')) > '0'], operator=[RANGE]) FilterExpression(predicate=[minus(ts,cast('2025-05-16T00:00:00','TIMESTAMP')) < '0'], operator=[RANGE]) ``` - This is a big issue because it'll prevent the filter from using range indexes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
