kosiew commented on code in PR #16583:
URL: https://github.com/apache/datafusion/pull/16583#discussion_r2181741126
##########
datafusion/core/src/datasource/listing/table.rs:
##########
@@ -67,8 +69,62 @@ pub enum SchemaSource {
/// Configuration for creating a [`ListingTable`]
///
+/// # Schema Evolution Support
///
-#[derive(Debug, Clone)]
+/// This configuration supports schema evolution through the optional
+/// [`SchemaAdapterFactory`]. You might want to override the default factory
when:
+///
+/// - **Reading files with evolving schemas**: When your data files have been
written
+/// over time with different but compatible schemas (e.g., added columns,
renamed fields)
+/// - **Type coercion requirements**: When you need custom logic for
converting between
+/// different Arrow data types (e.g., Int32 ↔ Int64, Utf8 ↔ LargeUtf8)
+/// - **Column mapping**: When files have different column names or ordering
than
+/// your expected table schema
+/// - **Backwards compatibility**: When newer table schemas need to read older
file
+/// formats gracefully
+///
+/// If not specified, a [`DefaultSchemaAdapterFactory`] will be used, which
handles
+/// basic schema compatibility cases.
+///
+/// # Complete Example: Schema Evolution Setup
+/// ```rust
+/// # use std::sync::Arc;
+/// # use datafusion::datasource::listing::{ListingTableConfig,
ListingOptions, ListingTableUrl};
+/// # use datafusion::datasource::file_format::parquet::ParquetFormat;
+/// # use datafusion::datasource::schema_adapter::{SchemaAdapterFactory,
SchemaAdapter};
+/// # use arrow::datatypes::{Schema, Field, DataType, SchemaRef};
+/// #
+/// # // Custom schema adapter for handling schema evolution
+/// # #[derive(Debug)]
+/// # struct EvolutionSchemaAdapterFactory;
+/// # impl SchemaAdapterFactory for EvolutionSchemaAdapterFactory {
+/// # fn create(&self, projected_table_schema: SchemaRef, file_schema:
SchemaRef) -> Box<dyn SchemaAdapter> {
+/// # unimplemented!("Custom schema adapter implementation")
+/// # }
+/// # }
+/// #
Review Comment:
Deleted this example, because there is an almost similar example
appropriately located near `fn with_schema_adapter_factory`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
