sam-1112 commented on PR #5415: URL: https://github.com/apache/datafusion-comet/pull/5415#issuecomment-5594938292
Thanks for the detailed reproducer. I addressed this by accounting for captures outside counted repetition rather than adding a native compile-then-fallback path. Capture-state cost is now charged when each capturing group is parsed, including groups under `*`, `+`, and `?`. The complete cost then propagates through concatenation and alternation. Counted repetition multiplies the already-accounted inner size without charging the capturing group again. I added: - boundary coverage for bare, uncounted, counted, concatenated, and alternated captures; - the exact 4,096-branch / 303,103-byte ASCII pattern, which the analyzer now classifies as `Incompatible` before native conversion; and - routing and fallback coverage: a 64-branch reduction of the same pattern shape stays on the JVM dispatcher by default, while the full pattern on Spark 4.0/JDK 21 falls back to Spark when the dispatcher is disabled, without reaching native compilation or raising `CompiledTooBig`. The focused analyzer, routing/fallback, and parity suites pass locally. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
