alamb commented on code in PR #13063:
URL: https://github.com/apache/datafusion/pull/13063#discussion_r1811255682
##########
datafusion/core/src/datasource/schema_adapter.rs:
##########
@@ -99,8 +107,86 @@ pub trait SchemaMapper: Debug + Send + Sync {
) -> datafusion_common::Result<RecordBatch>;
}
-/// Implementation of [`SchemaAdapterFactory`] that maps columns by name
-/// and casts columns to the expected type.
+/// Default [`SchemaAdapterFactory`] for mapping schemas.
+///
+/// This can be used to adapt file-level record batches to a table schema and
+/// implement schema evolution.
+///
+/// Given an input file schema and a table schema, this factor can make
+/// [`SchemaMapper`]s that:
+///
+/// 1. Reorder columns
+/// 2. Cast columns to the correct type
+/// 3. Fill missing columns with nulls
+///
+/// # Illustration of Schema Mapping
+///
+/// ```text
+/// ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─
─
+/// ┌───────┐ ┌───────┐ │ ┌───────┐ ┌───────┐
┌───────┐ │
+/// ││ 1.0 │ │ "foo" │ ││ NULL │ │ "foo" │ │ "1.0" │
+/// ├───────┤ ├───────┤ │ Schema mapping ├───────┤ ├───────┤
├───────┤ │
+/// ││ 2.0 │ │ "bar" │ ││ NULL │ │ "bar" │ │ "2.0" │
+/// └───────┘ └───────┘ │────────────────▶ └───────┘ └───────┘
└───────┘ │
+/// │ │
+/// column "c" column "b"│ column "a" column "b" column
"c"│
+/// │ Float64 Utf8 │ Int32 Utf8 Utf8
+/// ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─
─ ┘
Review Comment:
I use https://monodraw.helftone.com/ (though I hear https://asciiflow.com is
pretty good too)
I feel like I should get a commission for all the times I recommend Monodraw
😆
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
