Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-14 Thread via GitHub
adamreeve commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2206231243 ## datafusion/datasource-parquet/src/file_format.rs: ## @@ -959,14 +953,18 @@ pub async fn fetch_parquet_metadata( store: &dyn ObjectStore, meta: &Obje

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-14 Thread via GitHub
adamreeve commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2206050403 ## datafusion/core/Cargo.toml: ## @@ -61,13 +61,21 @@ default = [ "unicode_expressions", "compression", "parquet", +"parquet_encryption", Rev

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-14 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2206010727 ## datafusion/datasource-parquet/src/file_format.rs: ## @@ -959,14 +953,18 @@ pub async fn fetch_parquet_metadata( store: &dyn ObjectStore, meta: &Obje

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-14 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2206009134 ## datafusion/core/Cargo.toml: ## @@ -61,13 +61,21 @@ default = [ "unicode_expressions", "compression", "parquet", +"parquet_encryption", Rev

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-14 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2206008277 ## datafusion/core/Cargo.toml: ## @@ -61,13 +61,21 @@ default = [ "unicode_expressions", "compression", "parquet", +"parquet_encryption", Rev

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-14 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2206008277 ## datafusion/core/Cargo.toml: ## @@ -61,13 +61,21 @@ default = [ "unicode_expressions", "compression", "parquet", +"parquet_encryption", Rev

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-14 Thread via GitHub
alamb commented on PR #16649: URL: https://github.com/apache/datafusion/pull/16649#issuecomment-3070584588 Thanks again everyone -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-14 Thread via GitHub
alamb merged PR #16649: URL: https://github.com/apache/datafusion/pull/16649 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-12 Thread via GitHub
alamb commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2202528175 ## datafusion/datasource-parquet/src/file_format.rs: ## @@ -959,14 +953,18 @@ pub async fn fetch_parquet_metadata( store: &dyn ObjectStore, meta: &ObjectMe

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-10 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2196418502 ## datafusion/core/Cargo.toml: ## @@ -61,13 +61,21 @@ default = [ "unicode_expressions", "compression", "parquet", +"parquet_encryption", Rev

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-10 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2198712011 ## datafusion/common/src/encryption.rs: ## @@ -0,0 +1,76 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreem

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-09 Thread via GitHub
adamreeve commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r219642 ## datafusion/core/Cargo.toml: ## @@ -61,13 +61,21 @@ default = [ "unicode_expressions", "compression", "parquet", +"parquet_encryption", Rev

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-09 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2196418502 ## datafusion/core/Cargo.toml: ## @@ -61,13 +61,21 @@ default = [ "unicode_expressions", "compression", "parquet", +"parquet_encryption", Rev

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-09 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2196417839 ## datafusion/datasource-parquet/src/file_format.rs: ## @@ -959,14 +953,18 @@ pub async fn fetch_parquet_metadata( store: &dyn ObjectStore, meta: &Obje

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-09 Thread via GitHub
adamreeve commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2196392195 ## datafusion/datasource-parquet/src/file_format.rs: ## @@ -959,14 +953,18 @@ pub async fn fetch_parquet_metadata( store: &dyn ObjectStore, meta: &Obje

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-09 Thread via GitHub
adamreeve commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2196388182 ## datafusion/core/Cargo.toml: ## @@ -61,13 +61,21 @@ default = [ "unicode_expressions", "compression", "parquet", +"parquet_encryption", Rev

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-09 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2196387219 ## datafusion/datasource-parquet/src/file_format.rs: ## @@ -959,14 +953,18 @@ pub async fn fetch_parquet_metadata( store: &dyn ObjectStore, meta: &Obje

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-09 Thread via GitHub
adamreeve commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2196363819 ## datafusion/datasource-parquet/src/file_format.rs: ## @@ -959,14 +953,18 @@ pub async fn fetch_parquet_metadata( store: &dyn ObjectStore, meta: &Obje

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-03 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2184109462 ## datafusion/datasource-parquet/src/file_format.rs: ## @@ -350,15 +352,17 @@ impl FileFormat for ParquetFormat { Some(time_unit) => Some(parse_coe

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-03 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2183972933 ## datafusion/common/src/config.rs: ## @@ -29,12 +29,16 @@ use std::error::Error; use std::fmt::{self, Display}; use std::str::FromStr; -#[cfg(feature = "par

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-03 Thread via GitHub
corwinjoy commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2183847859 ## datafusion/core/src/test/mod.rs: ## @@ -38,6 +38,7 @@ use crate::test_util::{aggr_test_schema, arrow_test_data}; use arrow::array::{self, Array, ArrayRef, De

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-03 Thread via GitHub
corwinjoy commented on PR #16649: URL: https://github.com/apache/datafusion/pull/16649#issuecomment-3033798159 > Thanks @corwinjoy ! I have a few suggestions on how to maybe make this code a bit easier to deal with -- let me know what you think > > I think this would benefit from:

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-02 Thread via GitHub
alamb commented on code in PR #16649: URL: https://github.com/apache/datafusion/pull/16649#discussion_r2180912655 ## datafusion/datasource-parquet/src/file_format.rs: ## @@ -350,15 +352,17 @@ impl FileFormat for ParquetFormat { Some(time_unit) => Some(parse_coerce_

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-01 Thread via GitHub
corwinjoy commented on PR #16649: URL: https://github.com/apache/datafusion/pull/16649#issuecomment-3026163921 ## Pull Request Overview from Copilot This PR introduces an optional `parquet_encryption` feature that wires encryption/decryption properties through the Parquet reader and w

Re: [PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-01 Thread via GitHub
corwinjoy commented on PR #16649: URL: https://github.com/apache/datafusion/pull/16649#issuecomment-3026209585 @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[PR] feat: Add a configuration to make parquet encryption optional [datafusion]

2025-07-01 Thread via GitHub
corwinjoy opened a new pull request, #16649: URL: https://github.com/apache/datafusion/pull/16649 ## Which issue does this PR close? This is a followup to PR https://github.com/apache/datafusion/pull/16351 . In that review, it was suggested that we add a configuration to make encrypt