ding-young commented on code in PR #16268:
URL: https://github.com/apache/datafusion/pull/16268#discussion_r2149653552
##########
datafusion/common/src/config.rs:
##########
@@ -330,6 +386,13 @@ config_namespace! {
/// the new schema verification step.
pub skip_physical_aggregate_schema_check: bool, default = false
+ /// Sets the compression codec used when spilling data to disk.
+ ///
+ /// Since datafusion writes spill files using the Arrow IPC Stream
format,
+ /// only codecs supported by the Arrow IPC Stream Writer are allowed.
+ /// Valid values are: uncompressed, lz4_frame, zstd
+ pub spill_compression: SpillCompression, default =
SpillCompression::Uncompressed
Review Comment:
Thank you! After doing some performance investigation, I'll set the optimal
option as default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
