Re: [PR] docs: Add changelog for 0.7.0 release [datafusion-comet]

2025-03-17 Thread via GitHub
andygrove merged PR #1527: URL: https://github.com/apache/datafusion-comet/pull/1527 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [I] Enable parquet filter pushdown by default [datafusion]

2025-03-17 Thread via GitHub
adriangb commented on issue #3463: URL: https://github.com/apache/datafusion/issues/3463#issuecomment-2730550280 I don't think this needs to block but I'll point out that I have a PR up for a bug from the interaction between `SchemaAdapter` and parquet filter pushdown: https://github.com/ap

Re: [I] Migrate the following tests to `insta` [datafusion]

2025-03-17 Thread via GitHub
blaginin commented on issue #15282: URL: https://github.com/apache/datafusion/issues/15282#issuecomment-2730556871 Thanks, added to the list 🌻 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] chore: Enable Spark SQL tests for native_iceberg_compat [datafusion-comet]

2025-03-17 Thread via GitHub
codecov-commenter commented on PR #1541: URL: https://github.com/apache/datafusion-comet/pull/1541#issuecomment-2730586578 ## [Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/1541?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_ca

[I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]

2025-03-17 Thread via GitHub
andygrove opened a new issue, #1542: URL: https://github.com/apache/datafusion-comet/issues/1542 ### Describe the bug This issue is to track Spark SQL test failures in native_iceberg_compat mode. - Comet tries to read JSON files with Parquet reader ### Steps to reprod

Re: [PR] fix: remove code duplication in native_datafusion and native_iceberg_compat implementations [datafusion-comet]

2025-03-17 Thread via GitHub
mbutrovich commented on code in PR #1443: URL: https://github.com/apache/datafusion-comet/pull/1443#discussion_r1999472634 ## native/core/src/parquet/mod.rs: ## @@ -46,23 +47,22 @@ use self::util::jni::TypePromotionInfo; use crate::execution::operators::ExecutionError; use cra

Re: [I] Spark SQL test failures in native_iceberg_compat mode [datafusion-comet]

2025-03-17 Thread via GitHub
andygrove commented on issue #1542: URL: https://github.com/apache/datafusion-comet/issues/1542#issuecomment-2730625465 @parthchandra @mbutrovich fyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Fix predicate pushdown for custom SchemaAdapters [datafusion]

2025-03-17 Thread via GitHub
alamb commented on code in PR #15263: URL: https://github.com/apache/datafusion/pull/15263#discussion_r1999523980 ## datafusion/core/src/datasource/physical_plan/parquet.rs: ## @@ -224,6 +224,64 @@ mod tests { ) } +#[tokio::test] +async fn test_pushdown_w

Re: [I] `native_datafusion` scan is only enabled when `spark.comet.exec.enabled` is set [datafusion-comet]

2025-03-17 Thread via GitHub
andygrove closed issue #1536: `native_datafusion` scan is only enabled when `spark.comet.exec.enabled` is set URL: https://github.com/apache/datafusion-comet/issues/1536 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[PR] Minor: consistently apply `clippy::clone_on_ref_ptr` in all crates [datafusion]

2025-03-17 Thread via GitHub
alamb opened a new pull request, #15284: URL: https://github.com/apache/datafusion/pull/15284 ## Which issue does this PR close? ## Rationale for this change - Found while reviewing https://github.com/apache/datafusion/pull/15263 from @adriangb Some of the newer Dat

Re: [PR] Migrate user_defined tests to insta [datafusion]

2025-03-17 Thread via GitHub
shruti2522 commented on PR #15255: URL: https://github.com/apache/datafusion/pull/15255#issuecomment-2730590045 > > I tried allow_duplicates!(), but it gets a bit tricky with async functions > > Can you please explain more on this? I tried modifying _async_ `run_and_compare_query` an

Re: [I] [Epic] Add snapshot tests (migrate to `insta` for tests) [datafusion]

2025-03-17 Thread via GitHub
alamb commented on issue #15178: URL: https://github.com/apache/datafusion/issues/15178#issuecomment-2730830528 > [@alamb](https://github.com/alamb) can I ask you to put "good first issue" on tickets in the list if you're happy with them? I don't think I have permission to do that Do

Re: [I] Run / test Datafusion with JSON Bench from ClickHouse [datafusion]

2025-03-17 Thread via GitHub
alamb commented on issue #14874: URL: https://github.com/apache/datafusion/issues/14874#issuecomment-2730836455 > Add specialized support in arrow-rs for variant binary types (specifically for the metadata columns) I think this will be a fun project for the right type of person. I

Re: [I] Add SQL examples to window functions: `nth_value`, etc [datafusion]

2025-03-17 Thread via GitHub
alamb commented on issue #13399: URL: https://github.com/apache/datafusion/issues/13399#issuecomment-2730849740 > [@sageraven1](https://github.com/sageraven1) , Are you still working on this? I see that PR was marked as stale and got closed. If you aren't working on this, I would like to pi

Re: [PR] Minor: consistently apply `clippy::clone_on_ref_ptr` in all crates [datafusion]

2025-03-17 Thread via GitHub
alamb commented on code in PR #15284: URL: https://github.com/apache/datafusion/pull/15284#discussion_r1999606913 ## datafusion/catalog/src/session.rs: ## @@ -145,7 +145,7 @@ impl From<&dyn Session> for TaskContext { state.scalar_functions().clone(), st

<    1   2   3