gortiz opened a new pull request, #17867:
URL: https://github.com/apache/pinot/pull/17867
This pull request enhances the `JsonExtractScalarTransformFunction` to
provide better support for handling `null` default values, especially when
Pinot's null handling feature is enabled. It introduces logic to correctly
propagate nulls when a `null` default is specified, and adds comprehensive
tests to validate this behavior.
**Enhancements to null handling in JSON extraction:**
* Added a `_defaultIsNull` flag and logic to detect when the default value
for `jsonExtractScalar` is explicitly set to `null`, and to handle this case
differently depending on whether null handling is enabled
(`_nullHandlingEnabled`).
[[1]](diffhunk://#diff-d46a454e5b8d36c10e064b4a33c2cf26e5921a3ef71c8fbf75b1dc2a9dd7bf86R77)
[[2]](diffhunk://#diff-d46a454e5b8d36c10e064b4a33c2cf26e5921a3ef71c8fbf75b1dc2a9dd7bf86L113-R151)
* Overrode the `getNullBitmap` method to ensure rows with a `null` result
are correctly marked as null when a `null` default is used and null handling is
enabled.
* Updated the `init` method signature to accept the `nullHandlingEnabled`
parameter, aligning with the new null handling logic.
**Testing improvements:**
* Added new test records with `null` values in the JSON field to the test
dataset.
* Introduced two new tests to verify the behavior of `jsonExtractScalar`
when the default value is `null`, both with null handling enabled and disabled,
ensuring correct output in each scenario.
**Miscellaneous:**
* Added the `@Language("sql")` annotation to improve static analysis of SQL
queries in tests.
* Imported `@Nullable` and `RoaringBitmap` to support the new null handling
logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
