linrrzqqq opened a new pull request, #70774: URL: https://github.com/apache/doris/pull/70774
### Background Multiple consumers can access the same bitmap column. Measuring its serialized size optimized the underlying Roaring map in place, and copying a BitmapValue could also modify the source's sharing flag. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix Keep reads of a shared bitmap independent of other consumers while preserving compact output at the boundaries that need it. ### Details - **Change map:** Bitmap ownership, encoding and aggregate-state reads are in bitmap_value.h and bitmap_intersect.h. Producer, Segment and SerDe changes use these APIs. Block serialization prepares compact bitmap columns for Exchange. - **Frequency:** The race requires overlapping accesses to shared bitmap data. The previous optimization also ran during single-threaded sizing. - **Root cause:** Const size and copy operations could modify the source bitmap or its sharing state. - **Why existing tests missed it:** Sequential value checks exercise the contents of the set without checking concurrent access to its representation. - **Fix:** Detach shared Roaring maps on mutation. Size and write the current representation directly. Optimize completed aggregate results and bitmaps constructed from arrays or strings. When compact bytes are requested for an unoptimized shared input, size and write one local optimized copy. - **Other paths and branches:** Segment readers reuse compact native bitmap representations. Ordinary decoding skips optional length calculation. Intersection-state readers request the consumed byte count for both numeric and string keys. Exchange sizes and writes the same compact bitmap snapshot, including bitmaps nested in other columns. - **Behavior change:** An intermediate bitmap may retain its original layout until a compact output is requested. The encoding versions and bitmap set contents are preserved. ### Testing The Release build and 109 ASAN block and bitmap tests passed. The LZ4 block test checks both the allocation size and the input to the codec, round-trips a batch, and checks that the source stays unchanged. A targeted TSan run passed with four threads copying, sizing and writing the same source, checking both the original and compact bytes. Earlier validation included the bitmap scalar and bitmap_agg SQL suites. ### Release note Shared bitmap values can be sized and serialized concurrently without modifying their representation. ### Check List (For Author) - Behavior changed: Yes (see Details). - Does this need documentation: No. - Pre-submission status: Ready. ### AI Disclosure - Tool: Codex (GPT-6). - Generated: Code, tests and commit message. - Human review: Pending. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
