Copilot commented on code in PR #17867:
URL: https://github.com/apache/pinot/pull/17867#discussion_r2923973114
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/JsonExtractScalarTransformFunction.java:
##########
@@ -120,6 +158,37 @@ public TransformResultMetadata getResultMetadata() {
return _resultMetadata;
}
+ @Override
+ public @Nullable RoaringBitmap getNullBitmap(ValueBlock valueBlock) {
+ if (!_defaultIsNull) {
+ return super.getNullBitmap(valueBlock);
+ }
+ RoaringBitmap bitmap = new RoaringBitmap();
+ for (TransformFunction arg : _arguments.subList(1, _arguments.size() - 1))
{
+ RoaringBitmap argBitmap = arg.getNullBitmap(valueBlock);
+ if (argBitmap != null) {
+ bitmap.or(argBitmap);
+ }
+ }
+ int numDocs = valueBlock.getNumDocs();
+ RoaringBitmap nullBitmap = new RoaringBitmap();
+ IntFunction<Object> resultExtractor = getResultExtractor(valueBlock);
+ for (int i = 0; i < numDocs; i++) {
+ Object result = null;
+ try {
+ result = resultExtractor.apply(i);
+ } catch (Exception ignored) {
+ }
+ if (result == null) {
+ nullBitmap.add(i);
+ }
+ }
+ if (!nullBitmap.isEmpty()) {
+ bitmap.or(nullBitmap);
+ }
Review Comment:
getNullBitmap() always returns a RoaringBitmap instance even when it is
empty. Per the TransformFunction contract (and BaseTransformFunction behavior),
it should return null when there are no null rows to avoid extra allocations
and to preserve the semantic meaning of a null return value.
```suggestion
}
if (bitmap.isEmpty()) {
return null;
}
```
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/JsonExtractScalarTransformFunction.java:
##########
@@ -110,7 +114,41 @@ public void init(List<TransformFunction> arguments,
Map<String, ColumnContext> c
+ "/DOUBLE_ARRAY/STRING_ARRAY", resultsType));
}
if (arguments.size() == 4) {
- _defaultValue = dataType.convert(((LiteralTransformFunction)
arguments.get(3)).getStringLiteral());
+ LiteralTransformFunction literalTransformFun =
(LiteralTransformFunction) arguments.get(3);
+ _defaultIsNull = literalTransformFun.isNull() && _nullHandlingEnabled;
+ switch (dataType) {
+ case INT:
+ _defaultValue = literalTransformFun.getIntLiteral();
+ break;
+ case LONG:
+ _defaultValue = literalTransformFun.getLongLiteral();
+ break;
+ case FLOAT:
+ _defaultValue = literalTransformFun.getFloatLiteral();
+ break;
+ case DOUBLE:
+ case TIMESTAMP:
+ _defaultValue = literalTransformFun.getDoubleLiteral();
+ break;
Review Comment:
Default value parsing for TIMESTAMP uses getDoubleLiteral(), which can lose
precision for large epoch values and also regresses support for string
timestamp literals that previously worked via DataType.TIMESTAMP.convert(...).
Consider using the long/timestamp literal accessor (or the existing
DataType.TIMESTAMP.convert on the string literal) so TIMESTAMP defaults are
handled consistently with other Pinot timestamp parsing rules.
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/JsonExtractScalarTransformFunction.java:
##########
@@ -38,6 +38,8 @@
import org.apache.pinot.core.util.NumericException;
import org.apache.pinot.spi.data.FieldSpec.DataType;
import org.apache.pinot.spi.utils.JsonUtils;
+import org.jspecify.annotations.Nullable;
+import org.roaringbitmap.RoaringBitmap;
Review Comment:
This file introduces org.jspecify.annotations.Nullable, but
TransformFunction/BaseTransformFunction (and most Pinot core code) use
javax.annotation.Nullable. Using a different `@Nullable` annotation here makes
nullness annotations inconsistent and can reduce tooling effectiveness;
consider switching to javax.annotation.Nullable for the override and removing
the jspecify import.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
