alexandrefimov opened a new issue, #25049: URL: https://github.com/apache/datafusion/issues/25049
### Describe the bug The Substrait producer emits aggregate calls with `output_type: None`. The logical plan already has a known result type, but the exported call does not carry it. This affects `count`, `sum`, `avg` and `min` in the reproducer below. Reproduced on main at `8a9228164fddeaa03c9ee510b641990e403f2eb5`. ### To reproduce From a DataFusion checkout, using its pinned Rust toolchain: ```sh git clone https://github.com/alexandrefimov/substrait-conformance-cases conformance-cases git -C conformance-cases checkout c4d3ffd74842aa38079b3d523913199dc615a352 mkdir -p datafusion/substrait/examples cp conformance-cases/probe/datafusion_producer_probe.rs datafusion/substrait/examples/corpus_producer.rs cargo run --locked -p datafusion-substrait --example corpus_producer -- --aggregate-output-types ``` Use an unused example filename if `corpus_producer.rs` already exists. The probe registers an empty named table with a required Int64 column and calls `to_substrait_plan` for each aggregate. It inspects the produced protobuf directly, without converting it back through a consumer. | Query | Logical output type | Exported AggregateFunction.output_type | | --- | --- | --- | | `SELECT count(i) FROM t` | Int64 | None | | `SELECT sum(i) FROM t` | Int64 | None | | `SELECT avg(i) FROM t` | Float64 | None | | `SELECT min(i) FROM t` | Int64 | None | The final diagnostic is `{"cases":4,"missing_output_types":4}`. ### Expected behavior [`AggregateFunction.output_type`](https://github.com/substrait-io/substrait/blob/v0.87.0/proto/substrait/algebra.proto#L1806) must be set to the return type derived from the referenced function declaration. The producer should write a conforming type, or report that it cannot represent that function contract. [`from_aggregate_function`](https://github.com/apache/datafusion/blob/8a9228164fddeaa03c9ee510b641990e403f2eb5/datafusion/substrait/src/logical_plan/producer/expr/aggregate_function.rs) currently sets `output_type: None` explicitly. A consumer that validates required fields can reject these plans; one that uses the declaration for schema inference has no type to use. This report is about the missing declaration. It does not prescribe an integer AVG return type or address aggregate phases. #24967 and #25045 concern phase handling in the consumer; #15831 fixed the corresponding missing-output-type issue for BinaryExpr. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
