asolimando opened a new pull request, #23651: URL: https://github.com/apache/datafusion/pull/23651
## Which issue does this PR close? - Closes #21443. - Part of #8227. ## Rationale for this change `StatisticsRegistry` (#21483) was opt-in behind `datafusion.optimizer.use_statistics_registry` and ran as a separate tree walk from the main statistics computation, so the two could disagree and the registry only reached code that explicitly consulted it. Since #23051 collapsed statistics traversal into a single `StatisticsContext` walk, the registry can just be folded into that walk instead. This makes it the default path: providers are consulted at each node before falling back to the operator's `statistics_from_inputs`, so there's one walk and the flag is no longer needed. ## What changes are included in this PR? - The registry is consulted at each node of the single `StatisticsContext` walk: the first provider that returns `Computed` sets the node's stats, otherwise the walk falls back to the operator's `statistics_from_inputs`. `join_selection` uses this walk. - The walk carries `ExtendedStatistics` (core `Statistics` plus a type-keyed extension map) internally; the `statistics_from_inputs` signature is unchanged. Extensions are empty by default (the walk never forwards them, so a stale extension can't propagate silently); providers opt into propagation, e.g. `PassthroughStatisticsProvider` across cardinality-preserving operators. - `StatisticsProvider` gains a defaulted, partition-aware `compute_statistics_with_args` (overall-only by default, delegates per partition so a partition-blind provider can't leak an overall estimate into one partition). - `use_statistics_registry` is deprecated (kept but ignored, warns on use); `configs.md`, `information_schema.slt`, and the 55.0.0 upgrade guide are updated. - `DefaultStatisticsProvider` is removed; the walk's native fallback to `statistics_from_inputs` replaces it. - Two examples under `datafusion-examples/examples/statistics/`, each with a before/after EXPLAIN flip: a row-count override provider (core `Statistics`) and a custom NDV-sketch provider carried through the extension map. ## Are these changes tested? Unit tests in `operator_statistics`/`statistics` (provider chaining, partition-aware providers, per-partition `UnionExec` with a registry), the `statistics_registry.slt` file, and both examples. Full extended suite passes locally. ## Are there any user-facing changes? - `datafusion.optimizer.use_statistics_registry` is deprecated and ignored (accepted with a warning, removed later). Default behavior is unchanged: no providers registered means the registry is a no-op. See the 55.0.0 upgrade guide. - API change: `StatisticsProvider` gains a defaulted `compute_statistics_with_args`; `DefaultStatisticsProvider` is removed (register nothing for the default, or `StatisticsRegistry::default_with_builtin_providers()`). ---- Disclaimer: I used AI to assist in the code generation, I have manually reviewed the output and it matches my intention and understanding. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
