github-actions[bot] commented on code in PR #68874:
URL: https://github.com/apache/doris/pull/68874#discussion_r4256301159
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/PluginDrivenInsertCommandContext.java:
##########
@@ -84,16 +91,69 @@ public void
setStaticPartitionSpecFromExpressions(Map<String, Expression> partit
// Commit metadata must use the same semantic bytes as the
materialized partition row.
literal = (Literal)
ConnectorWriteValueConverter.convert(column, literal);
}
- // Binary partition keys must bypass character decoding,
including empty and non-UTF-8 bytes.
- spec.put(entry.getKey(), literal instanceof VarBinaryLiteral
- ? "0x" + literal.toString() :
literal.getStringValue());
+ spec.put(entry.getKey(), specValue(literal));
+ literals.put(entry.getKey(), literal);
if (entry.getValue() instanceof NullLiteral) {
nullKeys.add(entry.getKey());
}
}
}
this.staticPartitionSpec = spec;
this.staticPartitionNullKeys = nullKeys;
+ this.staticPartitionLiterals = literals;
+ }
+
+ public void setBoundTargetSchema(List<Column> boundTargetSchema) {
+ this.boundTargetSchema = boundTargetSchema;
+ }
+
+ /**
+ * Casts the static partition values to the types of their columns in the
bound target schema; see
+ * {@code ConnectorWriteHandle#getCastStaticPartitionSpec}. Each value is
the one BindSink materializes into
+ * the rows, after constant folding, but a value that does not fit its
type fails here instead of becoming
+ * NULL.
+ */
+ public Map<String, String> castStaticPartitionSpec() {
+ Preconditions.checkState(staticPartitionLiterals.isEmpty() ||
!boundTargetSchema.isEmpty(),
+ "static partition values are cast before the sink is bound to
a target schema");
+ Map<String, String> spec = new LinkedHashMap<>();
+ for (Map.Entry<String, Literal> entry :
staticPartitionLiterals.entrySet()) {
+ if (staticPartitionNullKeys.contains(entry.getKey())) {
+ continue;
+ }
+ Literal value = entry.getValue();
+ Optional<Column> column = boundTargetSchema.stream()
+ .filter(candidate ->
candidate.getName().equalsIgnoreCase(entry.getKey()))
+ .findFirst();
+ if (column.isPresent()) {
+ value = writtenValue(value,
DataType.fromCatalogType(column.get().getType()));
+ }
+ spec.put(entry.getKey(), specValue(value));
+ }
+ return spec;
+ }
+
+ private static String specValue(Literal literal) {
+ // Binary partition keys must bypass character decoding, including
empty and non-UTF-8 bytes.
+ return literal instanceof VarBinaryLiteral ? "0x" + literal.toString()
: literal.getStringValue();
+ }
+
+ private static Literal writtenValue(Literal value, DataType columnType) {
+ if (!value.getDataType().isStringLikeType() ||
!columnType.isStringLikeType()) {
+ Expression cast = value.checkedCastTo(columnType);
Review Comment:
[P2] Match static partition metadata to surviving BE casts
With `debug_skip_fold_constant=true`, row casts run in BE, but this call
still independently casts the metadata in FE. For
`PARTITION(p=0.5000000298023223876953125001)` on a Paimon `FLOAT` column,
`BigDecimal.floatValue()` produces `0.50000006`; BE's
`CastToFloat::_from_decimalv3` (`cast_to_basic_number_common.h:340-345`)
converts through double and rounds the midpoint to `0.5`.
Reduced bound plan inferred from `BindSink`:
```text
ConnectorSink(p FLOAT)
Project(id, CAST(decimal_literal AS FLOAT) AS p)
Filter(id = -1)
Scan(src)
```
A runtime-empty scan/filter therefore makes `withOverwrite` delete the
existing `0.50000006` partition while the row cast names `0.5`; a populated
overwrite fails the outside-target check. The same path gives metadata `"12"`
for numeric `PARTITION(p=123)` on `VARCHAR(2)`, while BE's string cast
preserves `"123"`. Before this change, Paimon's parser rejected the raw decimal
for precision loss and the raw `"123"` for excess length; the new strings pass.
Derive metadata from the row's actual coercion semantics, or reject unmatched
surviving casts before dispatch, and cover runtime-empty FLOAT/string
overwrites with folding disabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
