jfahne commented on issue #16017: URL: https://github.com/apache/datafusion/issues/16017#issuecomment-2886965734
Just an update. I was able to reproduce the error with the following goofy test added to the [dataframe tests](https://github.com/apache/datafusion/blob/main/datafusion/core/tests/dataframe/mod.rs) ```rust #[tokio::test] async fn bad_describe_behavior() -> Result<()> { let config = SessionConfig::from_string_hash_map(&HashMap::from([( "datafusion.sql_parser.enable_ident_normalization".to_owned(), "true".to_owned(), )]))?; let ctx = SessionContext::new_with_config(config); let name = "aggregate_test_100"; register_aggregate_csv(&ctx, name).await?; let df = ctx.table(name); let df = df .await? .filter(col("c2").eq(lit(3)).and(col("c1").eq(lit("a"))))? .limit(0, Some(1))? .sort(vec![ // make the test deterministic col("c1").sort(true, true), col("c2").sort(true, true), col("c3").sort(true, true), ])? .select_columns(&["c1"])?; let df_renamed = df.clone().with_column_renamed("c1", "CoLu.Mn1")?; let res = &df_renamed.clone().collect().await?; println!("{:?}", df_renamed.describe().await.unwrap()); Ok(()) } ``` I get the a similar error to the one you shared @johnkerl : ``` thread 'dataframe::bad_describe_behavior' panicked at datafusion/core/tests/dataframe/mod.rs:1883:50: called `Result::unwrap()` on an `Err` value: Execution("Schema error: No field named colu.mn1. Valid fields are aggregate_test_100.\"CoLu.Mn1\".") note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace test dataframe::bad_describe_behavior ... FAILED failures: failures: dataframe::bad_describe_behavior test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 606 filtered out; finished in 0.10s ``` One of my initial thoughts for where this error would be coming from is an unexpected behavior of the identifier normalization which is enabled by default. This seems to be the right thread to pull at as disabling `datafusion.sql_parser.enable_ident_normalization` and running the same test leads to an identical error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
