qzyu999 opened a new issue, #24010:
URL: https://github.com/apache/datafusion/issues/24010

   ## Background
   
   PR #23656 added FileWriteMetadata with a ormat_metadata: Option<Bytes> 
field designed to carry format-specific metadata (e.g., Thrift-serialized 
Parquet FileMetaData). The current implementation sets this to None because the 
typed ParquetMetaData is already accessible via ParquetSink::written() for Rust 
consumers.
   
   ## Proposal
   
   When a concrete FFI use case emerges (e.g., Python via PyO3, or C via 
datafusion-ffi), serialize the Parquet FileMetaData using Thrift Compact 
Protocol and populate ormat_metadata so that non-Rust consumers can access 
per-file column statistics without re-reading file footers.
   
   ## Context
   
   - PR: #23656
   - Review comment: 
https://github.com/apache/datafusion/pull/23656#discussion_r3683548423


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to