[
https://issues.apache.org/jira/browse/SPARK-57924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18093367#comment-18093367
]
jiangxintong commented on SPARK-57924:
--------------------------------------
I'm working on this. Will submit a PR following the BitmapAndAgg pattern
(SPARK-53877).
> Add bitmap_xor_agg aggregation function
> ---------------------------------------
>
> Key: SPARK-57924
> URL: https://issues.apache.org/jira/browse/SPARK-57924
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: jiangxintong
> Priority: Major
>
> Introduce bitmap_xor_agg, a bitwise XOR (symmetric difference) aggregation
> function for flat bitmaps, completing the function set alongside
> bitmap_or_agg (3.5.0), bitmap_construct_agg (3.5.0), bitmap_count (3.5.0),
> and bitmap_and_agg (4.1.0, SPARK-53877).
> The bitmap_xor_agg function outputs a bitmap representing the bitwise XOR of
> all bitmaps in the input column. The input column must contain bitmaps
> generated from bitmap_construct_agg().
> {code:sql}
> > SELECT substring(hex(bitmap_xor_agg(col)), 0, 6) FROM VALUES (X '10'), (X
> > '30'), (X '40') AS tab(col);
> 600000
> > SELECT substring(hex(bitmap_xor_agg(col)), 0, 6) FROM VALUES (X '10'), (X
> > '10') AS tab(col);
> 000000
> {code}
> (The second example demonstrates symmetric difference: two identical values
> XOR to zero.)
> h2. Use Cases
> Symmetric difference is useful for:
> * Finding users who belong to exactly one of two groups (not both)
> * Detecting changes between two snapshots
> * Any scenario requiring XOR semantics on bitmaps
> h2. Implementation
> Follow the exact pattern of BitmapAndAgg (SPARK-53877):
> * Merge operation: buffer[i] ^= input[i]
> * Initialize buffer to all zeros (same as BitmapOrAgg)
> * Both update() and merge() paths must use XOR
> Relates to: SPARK-53877
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
