Gabriel39 opened a new pull request, #70654: URL: https://github.com/apache/doris/pull/70654
### Background The native MongoDB reader copies borrowed BSON values into Doris columns and polls the scanner's IOContext for early termination. IOContext is also copied by cache and writeback paths. ### What problem does this PR solve? Issue Number: #70251 Related PR: #70604 (M1 implementation) PR Type: Bugfix Problem Summary: A scanner consumer can write IOContext::should_stop while a MongoDB reader polls the same ordinary bool, creating a C++ data race during cancellation. Replace the flag with a copyable atomic signal that preserves independent snapshots when an IOContext is copied. Add missing cancellation, BSON ownership, datetime precision/timezone, and decimal precision/scale coverage. Existing D06 policies and limits remain unchanged. ### Details - **Change map:** 20 changed production lines; 366 changed test/fixture lines; 264 generated golden-result lines; no documentation changes. - **Frequency:** Reachable when scanner termination overlaps an active reader; the unsynchronized access is undefined behavior regardless of whether a particular query visibly fails. - **Root cause:** FileScannerV2::try_stop writes a plain bool from the consumer thread while reader polling occurs on the scanner thread. - **Why existing tests missed it:** Cancellation tests stopped before reading or exercised source failures without a simultaneous stop writer; they did not validate IOContext copy semantics under concurrent access. - **Fix:** Use a sequentially consistent atomic flag with explicit snapshot copy/assignment, retaining aggregate initialization and existing bool-style callers without an allocation. - **Other paths and branches:** The shared IOContext fix also covers other file readers and cache paths. No RPC or storage format changes. This PR targets master; backports are separate. - **Behavior change:** Cancellation polling is synchronized. BSON conversion, read-policy defaults, resource limits, and source admission behavior are unchanged. - **Coverage:** DATETIME(0..6), negative epochs, millisecond precision rejection, UTC/fixed-offset/named timezones, a DST boundary, decimal precisions 1/9/10/18/19/38 and scales 0/intermediate/precision, Decimal128's 34 significant digits, exponent extremes, overflow/rounding/NULL policy, and exact-or-local predicate constants. Unsupported DATE/TIME/TIMESTAMPTZ and decimal precision above 38 remain explicit errors. ### Testing - Complete local FE and ASAN BE builds using build.sh. - 66 MongoDB connector FE tests, including Checkstyle, plus 28 connector-session/read-policy FE tests: passed. - 61 selected ASAN BE tests: passed with leak detection, stack-use-after-return detection, and strict string checks. - All four MongoDB SQL suites: passed with zero failed/fatal/skipped suites. - Generated golden output: 68 pushdown-on/off result pairs and 42 cross-timezone comparisons match. An independent exact-arithmetic/BSON-source check verified all 187 base precision-matrix cells. - Actual source commands retain Int64, Decimal128 and BSON Date types; collection candidates use simple collation and views retain their collation. - Real-driver fault runner: EOF/LIMIT/decode failure, getMore failure, socket/query timeout, cancellation after an observed blocked getMore, concurrent readers, cursor cleanup, and recovery passed. - The same concurrent probe compiled against the original IOContext header reports a should_stop race under Helgrind (exit 99); the fixed header passes (exit 0). - clang-format 16, clang-tidy on affected files, build hygiene, and diff checks passed locally. ### Release note Synchronize external-scan cancellation to remove a data race while preserving existing MongoDB query semantics. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: OpenAI Codex (GPT-6) - Generated: Code, tests, fixtures, generated regression output, and this description. - Human review: Not yet performed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
