bigluck opened a new issue, #25937: URL: https://github.com/apache/datafusion/issues/25937
### Describe the bug `EmptyRelationNode` only stores `produce_one_row`, so `LogicalPlan::EmptyRelation` is serialized without its schema and decoded with `LogicalPlanBuilder::empty`, which has no columns: - https://github.com/apache/datafusion/blob/242b96c231636b5738125678a83197896a5cc64b/datafusion/proto-models/proto/datafusion.proto#L182-L184 - https://github.com/apache/datafusion/blob/242b96c231636b5738125678a83197896a5cc64b/datafusion/proto/src/logical_plan/mod.rs#L799-L801 The optimizer reduces queries like `LIMIT 0` or `WHERE false` to an `EmptyRelation` that keeps the schema of its input. After a round trip, a plan that is only an `EmptyRelation` comes back with no columns, and a plan where another node refers to those columns fails to decode. ### To Reproduce With a `t.csv` file containing a header `id,name` and one row, on datafusion 55.1.0: ```rust use datafusion::error::Result; use datafusion::prelude::*; use datafusion_proto::bytes::{logical_plan_from_bytes, logical_plan_to_bytes}; #[tokio::main] async fn main() -> Result<()> { let ctx = SessionContext::new(); ctx.register_csv("t", "t.csv", CsvReadOptions::new()).await?; // 1. The plan is reduced to an `EmptyRelation` and comes back without columns let plan = ctx.sql("SELECT * FROM t LIMIT 0").await?.into_optimized_plan()?; let bytes = logical_plan_to_bytes(&plan)?; let restored = logical_plan_from_bytes(&bytes, &ctx.task_ctx())?; println!("{:?}", plan.schema().field_names()); println!("{:?}", restored.schema().field_names()); // 2. A node above the `EmptyRelation` refers to its columns: decoding fails let plan = ctx .sql("SELECT max(id), count(*) FROM t WHERE false") .await? .into_optimized_plan()?; let bytes = logical_plan_to_bytes(&plan)?; logical_plan_from_bytes(&bytes, &ctx.task_ctx())?; Ok(()) } ``` Output: ``` ["t.id", "t.name"] [] Error: SchemaError(FieldNotFound { field: Column { relation: Some(Bare { table: "t" }), name: "id" }, valid_fields: [] }, Some("")) ``` ### Expected behavior The decoded plan has the same schema as the original one. `SELECT * FROM t LIMIT 0` keeps the columns `id` and `name`, as it does when the plan is not serialized, and the second plan decodes. ### Additional context We found it in a service that sends logical plans to a separate executor: `SELECT * FROM t LIMIT 0` returned a table without columns. - On the physical side, `EmptyExecNode` already carries its schema. - Substrait has a similar issue: #13251. I have a fix ready and will open a PR shortly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
