qzyu999 commented on code in PR #23656:
URL: https://github.com/apache/datafusion/pull/23656#discussion_r3679385929
##########
datafusion/datasource/src/sink.rs:
##########
@@ -40,6 +41,32 @@ use async_trait::async_trait;
use datafusion_physical_plan::execution_plan::{EvaluationType, SchedulingType};
use futures::StreamExt;
+/// Metadata about a single file produced by a [`DataSink`] write operation.
+///
+/// This struct is format-agnostic. The [`Self::format_metadata`] field carries
+/// serialized format-specific metadata (e.g., a Parquet file footer serialized
+/// via Thrift Compact Protocol).
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct FileWriteMetadata {
+ /// Object-store path where the file was written.
+ pub path: String,
+ /// Number of rows written to this specific file.
+ pub row_count: u64,
+ /// Sum of compressed row group sizes in bytes.
+ ///
+ /// Note: this may differ slightly from the actual on-disk file size as it
+ /// excludes the Parquet footer, page indexes, and other metadata overhead.
+ pub byte_size: u64,
Review Comment:
Rewrote the doc to be format-neutral:
```rust
/// Best-effort total encoded data size in bytes.
///
/// How this value is computed depends on the format. For Parquet it is the
/// sum of compressed row-group sizes (which excludes the footer and page
/// indexes); for other formats it may be the full file size or unavailable
/// (zero). Consumers should treat this as an approximation.
pub byte_size: u64,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
