Ramin Gharib created FLINK-40616:
------------------------------------
Summary: Make BITMAP and VARIANT conversion handling consistent
and remove redundant checks
Key: FLINK-40616
URL: https://issues.apache.org/jira/browse/FLINK-40616
Project: Flink
Issue Type: Improvement
Components: API / Type Serialization System, Table SQL / Runtime
Reporter: Ramin Gharib
Assignee: Ramin Gharib
BITMAP and VARIANT are the two sealed built-in binary types. Each has a single
canonical internal representation, RoaringBitmapData and BinaryVariant. Their
handling across the type system had drifted apart, and the canonical-form
invariant was enforced in several overlapping places for BITMAP but only one
for VARIANT.
Concrete divergences:
- Conversion-class validation used a per-type static whitelist set
(INPUT_OUTPUT_CONVERSION). BitmapType listed both the interface and the
concrete RoaringBitmapData, while VariantType listed only the interface, so the
two produced asymmetric extraction and bridging results.
- DataStructureConverters registered VARIANT as a plain identity converter but
BITMAP through a bespoke BitmapBitmapConverter that additionally re-validated
the runtime value.
- In DataFormatConverters, BitmapConverter carried a per-record instanceof
RoaringBitmapData guard that VariantConverter did not have.
- BitmapSerializer.serialize did not enforce the canonical form, whereas
VariantSerializer.serialize does via toBinaryVariant.
This change makes the two types follow the same paths and enforces the
canonical form once per type, in the serializer:
- BitmapType/VariantType: supportsInputConversion/supportsOutputConversion use
Class.isAssignableFrom, dropping the static whitelist sets.
- ClassDataTypeConverter/ValueDataTypeConverter: both types resolve to their
interface conversion class uniformly, removing the duplicated per-value
branches.
- DataStructureConverters: VARIANT and BITMAP both return a shared, stateless
IdentityConverter.INSTANCE; the redundant switch guard is dropped.
IdentityConverter gains that singleton, reused for all identity conversions.
- DataFormatConverters: BitmapConverter becomes a plain identity converter,
matching VariantConverter.
- BitmapSerializer.serialize routes through
RoaringBitmapData.toRoaringBitmapData, matching VariantSerializer.
- The now-unused BitmapBitmapConverter is removed.
Note on behavior: There is no user-facing change for the supported
implementations, RoaringBitmapData and BinaryVariant. Custom implementations
remain unsupported; the Bitmap/Variant Javadoc contract is unchanged. The only
difference is that a non-canonical conversion class is now rejected at the
serialization boundary instead of by a plan-time whitelist.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
