mrhhsg opened a new pull request, #68579:
URL: https://github.com/apache/doris/pull/68579
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Several functions validate some arguments in
`checkLegalityBeforeTypeCoercion`, which runs during analysis before constant
folding, and require those arguments to be literals. A constant expression
that
folds to a valid literal is therefore rejected, for example:
select sha2('abc', 200 + 56);
-- the second parameter of sha2 must be a literal
select date_trunc(dt, concat('mon', 'th')) from t;
-- the time unit parameter ... must be a string constant
select array_apply([1, 2, 3], concat('>', '='), 2);
-- Can not build function ... op support const value only
select orthogonal_bitmap_expr_calculate(to_bitmap(k), cast(k as
varchar), concat('1', '|2')) from t;
-- Can not found function (the constructor only accepted VarcharLiteral)
set debug_skip_fold_constant = true; select topn(s, 1 + 1) from t;
-- topn requires second parameter must be a constant positive integer
The same happens for split_by_regexp (limit), regexp_replace /
regexp_replace_one
(4th argument), tokenize (properties), rand / random, uniform (min / max),
width_bucket (bucket count), now / utc_timestamp / utc_time (precision), the
AI
scalar functions and ai_agg (resource name, task), sequence_match /
sequence_count (pattern) and topn / topn_array / topn_weighted (N).
Fix: add a `FoldLiteralArguments` marker interface through which a function
declares the argument positions that must be literals. Before the legality
checks run (`TypeCoercionUtils.processBoundFunction` and
`ExpressionAnalyzer.visit`), such an argument is folded on FE with
`FoldConstantRuleOnFE.evaluateWithoutContext` when it is a constant
expression,
and replaced only when the result is a literal. The existing checks then see
the literal and still validate its value, and BE keeps receiving a literal as
before (or a cast of it when the literal type is wider than the parameter
type,
which the rewrite phase folds). Because the folding happens during analysis,
it
also applies to INSERT ... VALUES, which skips the rewrite phase, and under
`debug_skip_fold_constant`. Non-constant arguments and constant expressions
FE
cannot fold (for example ones calling crc32) are rejected with the same
errors
as before. date_trunc folds a string argument only when it may be the time
unit (the other argument is not a literal, or is a string literal that is
not a
time unit), so a string date value next to a literal time unit stays unfolded
and the derived return type does not change. utc_time folds its precision
only
when it is at most INT, because its return scale is derived from an INT
literal.
array_apply now validates `op` in `checkLegalityBeforeTypeCoercion` instead
of
its constructors, and the orthogonal_bitmap_expr_calculate(_count)
constructors
take an Expression and check for a string literal there.
Not covered here: the aggregate combinators (`_state`, `_merge`, `_union`,
`_foreach`) of the functions above, which build their nested function from
the
unfolded arguments; curtime, whose signature only accepts a TINYINT
precision;
and, under `debug_skip_fold_constant`, a folded literal whose type is wider
than
the parameter type (for example the SMALLINT result of `2 + 3` as the TINYINT
bucket count of width_bucket), which keeps its narrowing cast and is still
rejected by the after-rewrite checks.
`test_split_by_regexp` asserted that `1 + 2` is rejected as the limit; it now
uses `1 - 2`, which still folds to an invalid value.
### Release note
Functions that require literal arguments (such as sha2, date_trunc,
split_by_regexp, regexp_replace, array_apply, width_bucket, now,
sequence_match
and topn) now accept constant expressions that fold to a valid literal.
### Check List (For Author)
- Test:
- Unit Test: FoldLiteralArgumentsTest, plus the existing related FE UTs
(TypeCoercionUtilsTest, FoldConstantTest, CombineCombinatorTest,
InferPredicateFromMonotonicFunctionTest, MergeDateTruncTest,
PartitionColumnTraceTest, StackTest, ...) pass
- Regression test: new nereids_function_p0/fold_literal_arguments, and 23
existing suites touching these functions pass locally
- Behavior changed: Yes (constant expressions that fold to a valid literal
are
accepted where only literals were accepted before)
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
