This is an automated email from the ASF dual-hosted git repository.

jacktengg pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
     new f468c7d397c [fix](nereids) Honor cast mode for string literal 
arithmetic (#70518)
f468c7d397c is described below

commit f468c7d397ce5c65ad02c62e9f5e0187ba8f950f
Author: TengJianPing <[email protected]>
AuthorDate: Mon Nov 16 15:02:56 2026 +0800

    [fix](nereids) Honor cast mode for string literal arithmetic (#70518)
    
    Nereids converts arithmetic operands to numeric types during analysis.
    String literals can use the other operand's type when conversion
    succeeds. Other strings need a CAST expression. CAST evaluation uses
    enable_strict_cast to handle invalid input.
    
    Issue Number: None
    Related PR: None
    PR Type: Bugfix
    
    Problem Summary:
    
    SELECT 'abc' + 1 fails during analysis even when
    enable_strict_cast=false. A separate BigDecimal check rejects the value
    before CAST evaluation. Remove this check and use the existing CAST
    rules. The query then returns NULL in non-strict mode and reports a
    conversion error in strict mode.
    
    - **Frequency:** Every invalid numeric string literal that reaches the
    separate arithmetic check fails during analysis.
    - **Root cause:** The value check uses BigDecimal and does not read the
    CAST mode. It also rejects spaces that DOUBLE conversion accepts.
    - **Why existing tests missed it:** The inspected tests cover explicit
    CAST and valid arithmetic. They do not cover invalid string literals at
    this arithmetic analysis entry.
    - **Fix:** Remove the duplicate value check. Keep numeric type selection
    and CAST construction. Add comments with valid and invalid examples.
    - **Other paths and branches:** Addition, subtraction, multiplication,
    modulo, and bitwise arithmetic share this method. Division and explicit
    CAST already use CAST evaluation. The same check exists in tag
    2.1.11-rc01. That version can fall back to the legacy planner, but the
    reported installation's planner path is unverified.
    - **Behavior change:** Invalid numeric strings now follow the existing
    CAST mode. Strict mode still reports an error. Valid DOUBLE text with
    surrounding spaces is accepted. Integer strings retain integer
    arithmetic.
    
    - Before the fix, the new SQL suite fails with the reported 'abc'
    analysis error. Both new unit tests fail on the removed check.
    - After the fix, build.sh --fe passes with Checkstyle enabled.
    - run-fe-ut.sh passes all 99 tests in TypeCoercionUtilsTest,
    ExpressionAnalyzerTest, FoldConstantTest, and StringLikeLiteralTest.
    - run-regression-test.sh generates the new suite's output. The new suite
    and three existing arithmetic and CAST suites then pass against their
    output files.
    - The new SQL suite needs no table or stored data. It checks both
    operand positions and CAST modes with FE folding and BE evaluation.
    - Direct SQL and EXPLAIN confirm NULL in non-strict mode and conversion
    errors in strict mode on the new local FE.
    
    String literal arithmetic now follows enable_strict_cast. For example,
    SELECT 'abc' + 1 returns NULL when the setting is false. It reports a
    conversion error when the setting is true. Valid DOUBLE text with
    surrounding spaces is accepted.
    
    - Behavior changed: Yes (see Details)
    - Does this need documentation: No
    - Pre-submission status: Ready
    
    - Tool: Codex (GPT-6)
    - Generated: Code, comments, tests, analysis, and this description
    - Human review: Pending
    
    ### Background
    
    <!-- How the feature or module this PR touches works today, in two to
    four plain sentences, for a reviewer who doesn't know it. Link the user
    doc if there is one. Leave the problem and the fix to Problem Summary.
    -->
    
    ### What problem does this PR solve?
    
    <!-- Issue Number: "close #N" if this PR resolves the issue, "#N" if it
    only relates to it, "None" if there is none. A Feature needs an issue.
    -->
    <!-- Related PR: "#N (relation)" for each PR a reviewer should know
    about, such as "#123 (introduced the bug)", separated by commas. Write
    "None" if no related PR is known. -->
    <!-- PR Type: Bugfix, Performance, Feature, Enhancement, Refactor, Test,
    Build/CI/Chore, Revert, Release/Dependency, or Other. -->
    
    Issue Number:
    Related PR:
    PR Type:
    
    Problem Summary:
    
    <!-- Two to five sentences: the problem or need and who is affected; the
    approach in one sentence; when the diff is larger than that suggests,
    what else it had to handle, or that most of it is tests; and what
    existing users see after an upgrade. -->
    
    ### Details
    
    <!--
    For a diff of more than about 300 changed lines, start with "- **Change
    map:**": how many lines are production code, tests, generated files, and
    docs, then the code in parts, each with the problem it handles and its
    size.
    Then answer each field of your PR type on its own line, as "- **Field:**
    answer", in this order:
    Bugfix: Frequency, Root cause, Why existing tests missed it, Fix, Other
    paths and branches
    Performance: Why it was slow, Optimization, Results, Applicability and
    cost
      Feature: Scope, Design, Usage, Default behavior, Compatibility
    Enhancement: Before and after, Configs and defaults, Impact on existing
    users
    Refactor: Moves vs. logic changes, Behavior change, Compatibility,
    Performance impact, Rollback
    Test, adding tests: Scenarios, What existing tests miss, Fails when
    broken
      Test, fixing a test: Failure, Cause, Stability evidence
    Build/CI/Chore: Affected, Developer and release impact, License and
    third-party, Rollback
      Revert: Why not fix forward, Original author notified, Follow-up
    Release/Dependency: Versions, Upstream changes, Compatibility, Rollback
      Other: Why Other
    "None" is a valid answer when it is true. Don't restate the diff.
    -->
    
    ### Testing
    
    <!-- The tests, suites, or commands that ran, what each shows, and where
    it ran (locally or in CI). For a fix, the test that fails before it and
    passes after. -->
    
    ### Notes for Reviewers
    
    <!-- Optional; delete this section if there is nothing to say. Why each
    non-obvious edit was made, what you are unsure about, and where to start
    reading if Details has no Change map. -->
    
    ### Release note
    
    <!-- One paragraph on what users will notice, or None. -->
    
    ### Check List (For Author)
    
    <!-- Behavior changed: "No", or "Yes (see Details)" with a Behavior
    change field in Details. -->
    <!-- Does this need documentation: "No", or "Yes" with the
    apache/doris-website PR or the plan. -->
    <!-- Pre-submission status: "Ready"; "Partially verified: " and what is
    not verified; or "Not ready: " and why, for a Draft PR. -->
    
    - Behavior changed:
    - Does this need documentation:
    - Pre-submission status:
    
    ### AI Disclosure
    
    <!-- Tool: the agent and model, or None. Generated: the parts it wrote
    (code, tests, this description), or None. Human review: what you
    reviewed of the generated parts, or "Not applicable" without AI tools.
    You may add "- Prompts:" to summarize how you directed the agent. -->
    
    - Tool:
    - Generated:
    - Human review:
---
 .../doris/nereids/util/TypeCoercionUtils.java      | 15 +----
 .../doris/nereids/util/TypeCoercionUtilsTest.java  | 43 ++++++++++++++
 .../nereids_p0/test_string_literal_arithmetic.out  | 37 ++++++++++++
 .../test_string_literal_arithmetic.groovy          | 68 ++++++++++++++++++++++
 .../suites/nereids_syntax_p0/cast.groovy           | 12 +++-
 5 files changed, 159 insertions(+), 16 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/util/TypeCoercionUtils.java 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/util/TypeCoercionUtils.java
index a1c4cd0b4ef..61f3dd4c2fa 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/util/TypeCoercionUtils.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/util/TypeCoercionUtils.java
@@ -65,7 +65,6 @@ import 
org.apache.doris.nereids.trees.expressions.literal.MapLiteral;
 import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
 import org.apache.doris.nereids.trees.expressions.literal.Result;
 import org.apache.doris.nereids.trees.expressions.literal.SmallIntLiteral;
-import org.apache.doris.nereids.trees.expressions.literal.StringLikeLiteral;
 import org.apache.doris.nereids.trees.expressions.literal.StringLiteral;
 import org.apache.doris.nereids.trees.expressions.literal.TimeStampNsLiteral;
 import org.apache.doris.nereids.trees.expressions.literal.TimeV2Literal;
@@ -1023,18 +1022,8 @@ public class TypeCoercionUtils {
         // but if string literal could be cast to small type, we could use 
smaller type than double.
         binaryArithmetic = 
TypeCoercionUtils.processCharacterLiteralInBinaryOperator(binaryArithmetic);
 
-        // check string literal can cast to double
-        for (Expression expr : binaryArithmetic.children()) {
-            if (expr instanceof StringLikeLiteral) {
-                try {
-                    new BigDecimal(((StringLikeLiteral) 
expr).getStringValue());
-                } catch (NumberFormatException e) {
-                    throw new IllegalStateException(String.format(
-                            "string literal %s cannot be cast to double", 
expr.toSql()));
-                }
-            }
-        }
-
+        // Keep other strings for the CAST expressions below. CAST evaluation 
must use enable_strict_cast.
+        // For example, 'abc' + 1 returns NULL in non-strict mode and reports 
an error in strict mode.
         Expression left = binaryArithmetic.left();
         Expression right = binaryArithmetic.right();
         // 1. choose default numeric type for left and right
diff --git 
a/fe/fe-core/src/test/java/org/apache/doris/nereids/util/TypeCoercionUtilsTest.java
 
b/fe/fe-core/src/test/java/org/apache/doris/nereids/util/TypeCoercionUtilsTest.java
index 890f97c4693..c3cb4ae5eac 100644
--- 
a/fe/fe-core/src/test/java/org/apache/doris/nereids/util/TypeCoercionUtilsTest.java
+++ 
b/fe/fe-core/src/test/java/org/apache/doris/nereids/util/TypeCoercionUtilsTest.java
@@ -1149,6 +1149,49 @@ public class TypeCoercionUtilsTest {
                 TypeCoercionUtils.castIfNotSameType(new 
StringLiteral("string"), VarcharType.createVarcharType(100)));
     }
 
+    @Test
+    public void testStringArithmeticDefersValueConversionToCast() {
+        ConnectContext previousContext = ConnectContext.get();
+        ConnectContext connectContext = new ConnectContext();
+        connectContext.setThreadLocalInfo();
+        try {
+            // Analysis must keep a CAST in both modes. CAST evaluation 
returns NULL or an error.
+            // The same rule applies to STRING, VARCHAR, and CHAR literals on 
either side.
+            for (boolean strict : new boolean[] {false, true}) {
+                connectContext.getSessionVariable().enableStrictCast = strict;
+                for (Expression string : ImmutableList.of(new 
StringLiteral("abc"),
+                        new VarcharLiteral("abc"), new CharLiteral("abc", 3))) 
{
+                    IntegerLiteral one = new IntegerLiteral(1);
+                    Expression left = 
TypeCoercionUtils.processBinaryArithmetic(new Add(string, one));
+                    Expression right = 
TypeCoercionUtils.processBinaryArithmetic(new Add(one, string));
+                    Assertions.assertEquals(new Cast(string, 
DoubleType.INSTANCE), left.child(0));
+                    Assertions.assertEquals(new Cast(string, 
DoubleType.INSTANCE), right.child(1));
+                    Assertions.assertEquals(DoubleType.INSTANCE, 
left.getDataType());
+                    Assertions.assertEquals(DoubleType.INSTANCE, 
right.getDataType());
+                }
+            }
+        } finally {
+            ConnectContext.remove();
+            if (previousContext != null) {
+                previousContext.setThreadLocalInfo();
+            }
+        }
+    }
+
+    @Test
+    public void testStringArithmeticKeepsNumericTypeSelection() {
+        // An integer string must still use integer arithmetic. It must not 
become DOUBLE.
+        Expression integer = TypeCoercionUtils.processBinaryArithmetic(
+                new Add(new StringLiteral("1"), new IntegerLiteral(1)));
+        Assertions.assertEquals(BigIntType.INSTANCE, integer.getDataType());
+
+        // DOUBLE accepts spaces around a number. BigDecimal rejects this 
input before CAST runs.
+        StringLiteral spaced = new StringLiteral(" 1 ");
+        Expression floating = TypeCoercionUtils.processBinaryArithmetic(new 
Add(spaced, new IntegerLiteral(1)));
+        Assertions.assertEquals(new Cast(spaced, DoubleType.INSTANCE), 
floating.child(0));
+        Assertions.assertEquals(DoubleType.INSTANCE, floating.getDataType());
+    }
+
     @Test
     public void testDecimalArithmetic() {
         Multiply multiply = new Multiply(new DecimalLiteral(new 
BigDecimal("987654.321")),
diff --git a/regression-test/data/nereids_p0/test_string_literal_arithmetic.out 
b/regression-test/data/nereids_p0/test_string_literal_arithmetic.out
new file mode 100644
index 00000000000..2c726aba922
--- /dev/null
+++ b/regression-test/data/nereids_p0/test_string_literal_arithmetic.out
@@ -0,0 +1,37 @@
+-- This file is automatically generated. You should know what you did if you 
want to edit this
+-- !invalid_add --
+\N     \N
+
+-- !invalid_shared --
+\N     \N      \N      \N      \N      \N      \N      \N      \N      \N      
\N      \N
+
+-- !existing_paths --
+\N     \N      \N
+
+-- !string_pairs --
+\N     \N
+
+-- !valid_numbers --
+2      2.5     11      2
+
+-- !valid_numbers --
+2      2.5     11      2
+
+-- !invalid_add --
+\N     \N
+
+-- !invalid_shared --
+\N     \N      \N      \N      \N      \N      \N      \N      \N      \N      
\N      \N
+
+-- !existing_paths --
+\N     \N      \N
+
+-- !string_pairs --
+\N     \N
+
+-- !valid_numbers --
+2      2.5     11      2
+
+-- !valid_numbers --
+2      2.5     11      2
+
diff --git 
a/regression-test/suites/nereids_p0/test_string_literal_arithmetic.groovy 
b/regression-test/suites/nereids_p0/test_string_literal_arithmetic.groovy
new file mode 100644
index 00000000000..5cb9f8d4e37
--- /dev/null
+++ b/regression-test/suites/nereids_p0/test_string_literal_arithmetic.groovy
@@ -0,0 +1,68 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_string_literal_arithmetic") {
+    // Use literals only. No table or stored row is required. The original 
failure is 'abc' + 1.
+    // Save the variables so this suite does not change the next suite's 
execution mode.
+    def originalSettings = ["enable_strict_cast", "enable_fold_constant_by_be",
+            "debug_skip_fold_constant", "enable_sql_cache"].collectEntries { 
name ->
+        [(name): sql("show variables like '${name}'")[0][1]]
+    }
+    try {
+        // Disable the cache so each query tests the selected CAST and 
execution modes.
+        sql "set enable_sql_cache=false"
+        sql "set enable_fold_constant_by_be=false"
+        for (skipFold in [false, true]) {
+            // false tests FE constant folding. true tests BE evaluation of 
the same literals.
+            sql "set debug_skip_fold_constant=${skipFold}"
+            sql "set enable_strict_cast=false"
+
+            // An invalid numeric string becomes NULL. Check both operand 
positions.
+            order_qt_invalid_add "select 'abc' + 1, 1 + 'abc'"
+            // These operators share the type conversion code used by addition.
+            order_qt_invalid_shared """select 'abc' - 1, 1 - 'abc', 'abc' * 1, 
1 * 'abc',
+                    'abc' % 1, 1 % 'abc', 'abc' & 1, 1 & 'abc',
+                    'abc' | 1, 1 | 'abc', 'abc' ^ 1, 1 ^ 'abc'"""
+            // Division and explicit CAST already use the required conversion 
rules.
+            order_qt_existing_paths "select 'abc' / 1, 'abc' div 1, cast('abc' 
as double) + 1"
+            // Two string operands and an empty string must also reach CAST 
evaluation.
+            order_qt_string_pairs "select 'abc' + '1', '' + 1"
+
+            for (strict in [false, true]) {
+                sql "set enable_strict_cast=${strict}"
+                // Keep integer type selection. Fractional and exponent forms 
use DOUBLE.
+                // Spaces are valid for DOUBLE, but the old BigDecimal check 
rejected them.
+                order_qt_valid_numbers "select '1' + 1, '1.5' + 1, '1e1' + 1, 
' 1 ' + 1"
+            }
+
+            // Strict mode must still fail. Match the input value because FE 
and BE use
+            // different CAST error messages. Test each operator and operand 
position alone.
+            for (op in ["+", "-", "*", "%", "&", "|", "^"]) {
+                test {
+                    sql "select 'abc' ${op} 1"
+                    exception "abc"
+                }
+                test {
+                    sql "select 1 ${op} 'abc'"
+                    exception "abc"
+                }
+            }
+        }
+    } finally {
+        originalSettings.each { name, value -> sql "set ${name}=${value}" }
+    }
+}
diff --git a/regression-test/suites/nereids_syntax_p0/cast.groovy 
b/regression-test/suites/nereids_syntax_p0/cast.groovy
index 30e57d58e60..428f58f5eaa 100644
--- a/regression-test/suites/nereids_syntax_p0/cast.groovy
+++ b/regression-test/suites/nereids_syntax_p0/cast.groovy
@@ -186,9 +186,15 @@ suite("cast") {
 
     qt_group31 "select count(*) from cast_test where (k11='2015-03-13 
12:36:38' or k11 = '2000-01-01 00:00:00') and k5 is not null group by k1%2, 
k2%2, k3%3, k4%3, k11%2 order by count(*)"
 
-    test {
-        sql "select true + 1 + 'x'"
-        exception "string literal 'x' cannot be cast to double"
+    def originalEnableStrictCast = sql("select @@enable_strict_cast")[0][0]
+    try {
+        sql "set enable_strict_cast = true"
+        test {
+            sql "select true + 1 + 'x'"
+            exception "can't cast to double in strict mode"
+        }
+    } finally {
+        sql "set enable_strict_cast = ${originalEnableStrictCast}"
     }
 
     qt_sql_test_DecimalV3_mode """select cast(1 as DECIMALV3(1, 0)) % 2.1;""";


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to