Re: [PR] fix: Capture nullability in `Values` node planning [datafusion]

2025-02-05 Thread via GitHub
rkrishn7 commented on PR #14472: URL: https://github.com/apache/datafusion/pull/14472#issuecomment-2636003587 `Rust / build with wasm-pack (pull_request)` is currently failing, but looks like this is due to the recent upstream release of `uuid`. Should be fixed in #14494 -- This is an au

Re: [PR] Support WITHIN GROUP syntax to standardize certain existing aggregate functions [datafusion]

2025-02-05 Thread via GitHub
berkaysynnada commented on PR #13511: URL: https://github.com/apache/datafusion/pull/13511#issuecomment-2636004482 > I will check if I can add more new test cases. (Additionally, I had no option but to modify some existing test cases; otherwise, they would fail due to the changed syntax)

Re: [PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-05 Thread via GitHub
xudong963 commented on code in PR #14499: URL: https://github.com/apache/datafusion/pull/14499#discussion_r1942439311 ## datafusion/sqllogictest/test_files/aggregate.slt: ## @@ -5032,18 +5032,17 @@ logical_plan 03)Aggregate: groupBy=[[aggregate_test_100.c3]], aggr=[[min(ag

[PR] chore(deps): bump aws-config from 1.5.15 to 1.5.16 in /datafusion-cli [datafusion]

2025-02-05 Thread via GitHub
dependabot[bot] opened a new pull request, #14500: URL: https://github.com/apache/datafusion/pull/14500 Bumps [aws-config](https://github.com/smithy-lang/smithy-rs) from 1.5.15 to 1.5.16. Commits See full diff in https://github.com/smithy-lang/smithy-rs/commits";>compare view

Re: [PR] fix(ci): build error with wasm [datafusion]

2025-02-05 Thread via GitHub
xudong963 merged PR #14494: URL: https://github.com/apache/datafusion/pull/14494 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@data

Re: [I] Update ClickBench benchmarks with DataFusion `44.0.0` [datafusion]

2025-02-05 Thread via GitHub
pmcgleenon commented on issue #13983: URL: https://github.com/apache/datafusion/issues/13983#issuecomment-2636181803 I've re-run this again following the [discussion on the ClickBench PR](https://github.com/ClickHouse/ClickBench/pull/301#discussion_r1941921542) Q28 needed an update,

Re: [PR] Fix link to volcano parallelism paper [datafusion]

2025-02-05 Thread via GitHub
alamb merged PR #14497: URL: https://github.com/apache/datafusion/pull/14497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Fix link to volcano parallelism paper [datafusion]

2025-02-05 Thread via GitHub
alamb commented on PR #14497: URL: https://github.com/apache/datafusion/pull/14497#issuecomment-2636374977 @Lordworms fixed the wasm build in https://github.com/apache/datafusion/pull/14494 ❤️ so merging this one in -- This is an automated message from the Apache Git Service. To respond

Re: [PR] refactor: remove uses of `arrow_buffer` & `arrow_array` and use reexport in arrow instead [datafusion]

2025-02-05 Thread via GitHub
alamb commented on code in PR #14503: URL: https://github.com/apache/datafusion/pull/14503#discussion_r1942654228 ## datafusion/common/Cargo.toml: ## @@ -52,7 +52,6 @@ apache-avro = { version = "0.17", default-features = false, features = [ ], optional = true } arrow = { work

Re: [PR] refactor: remove uses of `arrow_buffer` & `arrow_array` and use reexport in arrow instead [datafusion]

2025-02-05 Thread via GitHub
Chen-Yuan-Lai commented on PR #14503: URL: https://github.com/apache/datafusion/pull/14503#issuecomment-2636401146 > Any chance you are willing to do the same thing for arrow-array? Sure : ) -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] chore(deps): bump aws-config from 1.5.15 to 1.5.16 in /datafusion-cli [datafusion]

2025-02-05 Thread via GitHub
alamb merged PR #14500: URL: https://github.com/apache/datafusion/pull/14500 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] feat: Add `array_max` function support [datafusion]

2025-02-05 Thread via GitHub
alamb commented on PR #14470: URL: https://github.com/apache/datafusion/pull/14470#issuecomment-2636407620 FYI @findepi I think you mentioned this feature recently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Run test from `adding-udfs.md` documentation [datafusion]

2025-02-05 Thread via GitHub
alamb commented on PR #14485: URL: https://github.com/apache/datafusion/pull/14485#issuecomment-2636413486 I took the liberty of running prettier and pushing the change to this branch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] refactor: remove uses of `arrow_buffer` & `arrow_array` and use reexport in arrow instead [datafusion]

2025-02-05 Thread via GitHub
Chen-Yuan-Lai commented on PR #14495: URL: https://github.com/apache/datafusion/pull/14495#issuecomment-2636284822 I wrongly deleted this branch, please review the new PR #14503 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942582463 ## benchmarks/bench.sh: ## @@ -34,7 +34,7 @@ COMMAND= BENCHMARK=all DATAFUSION_DIR=${DATAFUSION_DIR:-$SCRIPT_DIR/..} DATA_DIR=${DATA_DIR:-$SCRIPT_DIR/data} -CAR

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
findepi commented on PR #14483: URL: https://github.com/apache/datafusion/pull/14483#issuecomment-2636298157 @mbrobbel many changes in this PR are about adding `--locked` to various cargo invocations. I think it should be possible to verify dependencies in one place, and then keep eg run

Re: [I] Incorrect backslash treatment in string literals in DataFusion CLI [datafusion]

2025-02-05 Thread via GitHub
alamb commented on issue #13286: URL: https://github.com/apache/datafusion/issues/13286#issuecomment-2636286656 BTW @pmcgleenon hit this when updating ClickBench as well: https://github.com/ClickHouse/ClickBench/pull/301 -- This is an automated message from the Apache Git Service.

Re: [PR] Accept any uncorrelated plan when checking subquery correlation [datafusion]

2025-02-05 Thread via GitHub
findepi commented on PR #14463: URL: https://github.com/apache/datafusion/pull/14463#issuecomment-2636317619 > This looks quite similar to #14474 Yes, that PR includes this as the first commit. Will close in favor. -- This is an automated message from the Apache Git Service.

Re: [PR] Support uncorrelated EXISTS [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14474: URL: https://github.com/apache/datafusion/pull/14474#discussion_r1942608923 ## datafusion/expr/src/logical_plan/invariants.rs: ## @@ -272,26 +267,34 @@ fn check_inner_plan(inner_plan: &LogicalPlan, can_contain_outer_ref: bool) -> Re

Re: [PR] Support uncorrelated EXISTS [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14474: URL: https://github.com/apache/datafusion/pull/14474#discussion_r1942608923 ## datafusion/expr/src/logical_plan/invariants.rs: ## @@ -272,26 +267,34 @@ fn check_inner_plan(inner_plan: &LogicalPlan, can_contain_outer_ref: bool) -> Re

Re: [PR] Support uncorrelated EXISTS [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14474: URL: https://github.com/apache/datafusion/pull/14474#discussion_r1942609981 ## datafusion/expr/src/logical_plan/invariants.rs: ## @@ -272,26 +267,34 @@ fn check_inner_plan(inner_plan: &LogicalPlan, can_contain_outer_ref: bool) -> Re

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
jayzhan211 commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942524406 ## datafusion/functions/src/string/ascii.rs: ## @@ -93,6 +95,33 @@ impl ScalarUDFImpl for AsciiFunc { make_scalar_function(ascii, vec![])(args) }

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
mbrobbel commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942615574 ## benchmarks/bench.sh: ## @@ -34,7 +34,7 @@ COMMAND= BENCHMARK=all DATAFUSION_DIR=${DATAFUSION_DIR:-$SCRIPT_DIR/..} DATA_DIR=${DATA_DIR:-$SCRIPT_DIR/data} -CA

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942615130 ## ci/scripts/rust_clippy.sh: ## @@ -18,6 +18,4 @@ # under the License. set -ex -cargo clippy --all-targets --workspace --features avro,pyarrow,integration-te

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942615130 ## ci/scripts/rust_clippy.sh: ## @@ -18,6 +18,4 @@ # under the License. set -ex -cargo clippy --all-targets --workspace --features avro,pyarrow,integration-te

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
mbrobbel commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942614160 ## .github/workflows/extended.yml: ## @@ -87,7 +87,7 @@ jobs: - name: Run tests run: | cd datafusion - cargo test --profile c

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
mbrobbel commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942615076 ## README.md: ## @@ -152,26 +152,13 @@ deprecate methods before removing them, according to the [deprecation guidelines [deprecation guidelines]: https://dat

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942619187 ## .github/workflows/extended.yml: ## @@ -87,7 +87,7 @@ jobs: - name: Run tests run: | cd datafusion - cargo test --profile ci

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
mbrobbel commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942624629 ## .github/workflows/extended.yml: ## @@ -87,7 +87,7 @@ jobs: - name: Run tests run: | cd datafusion - cargo test --profile c

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
mbrobbel commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942627197 ## ci/scripts/rust_clippy.sh: ## @@ -18,6 +18,4 @@ # under the License. set -ex -cargo clippy --all-targets --workspace --features avro,pyarrow,integration-t

Re: [PR] Accept any uncorrelated plan when checking subquery correlation [datafusion]

2025-02-05 Thread via GitHub
findepi closed pull request #14463: Accept any uncorrelated plan when checking subquery correlation URL: https://github.com/apache/datafusion/pull/14463 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Support uncorrelated EXISTS [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14474: URL: https://github.com/apache/datafusion/pull/14474#discussion_r1942596627 ## datafusion/optimizer/src/decorrelate_predicate_subquery.rs: ## @@ -1844,6 +1864,69 @@ mod tests { assert_optimized_plan_equal(plan, expected) }

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
mbrobbel commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942624629 ## .github/workflows/extended.yml: ## @@ -87,7 +87,7 @@ jobs: - name: Run tests run: | cd datafusion - cargo test --profile c

[I] Do We Need partition_by Field in (Bounded)WindowAggExec ? [datafusion]

2025-02-05 Thread via GitHub
berkaysynnada opened a new issue, #14505: URL: https://github.com/apache/datafusion/issues/14505 ### Is your feature request related to a problem or challenge? Both WindowAggExec and BoundedWindowAggExec has a field of ``` /// Partition Keys pub partition_keys: Vec>,

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942577243 ## .github/workflows/extended.yml: ## @@ -87,7 +87,7 @@ jobs: - name: Run tests run: | cd datafusion - cargo test --profile ci

[PR] Add more LIKE with escape tests [datafusion]

2025-02-05 Thread via GitHub
findepi opened a new pull request, #14501: URL: https://github.com/apache/datafusion/pull/14501 Add test cases for the bug recently fixed in Arrow. - closes https://github.com/apache/datafusion/issues/13304 -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
jayzhan211 commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942525348 ## datafusion/functions/src/string/bit_length.rs: ## @@ -106,6 +108,33 @@ impl ScalarUDFImpl for BitLengthFunc { } } +fn coerce_types(&self,

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
jayzhan-synnada commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942523715 ## datafusion/functions/src/string/ascii.rs: ## @@ -93,6 +95,33 @@ impl ScalarUDFImpl for AsciiFunc { make_scalar_function(ascii, vec![])(args)

Re: [PR] feat: metadata columns [datafusion]

2025-02-05 Thread via GitHub
chenkovsky commented on PR #14057: URL: https://github.com/apache/datafusion/pull/14057#issuecomment-2636206099 > ( I'd like to see some tests against `SchemaAdapter`) @adriangb feel free to correct me, I know maybe I'm wrong. it seems that schema adapater has no relationship with met

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
jayzhan211 commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942530489 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -584,23 +541,36 @@ fn get_valid_types( match target_type_class {

Re: [PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-05 Thread via GitHub
zhuqi-lucas commented on code in PR #14499: URL: https://github.com/apache/datafusion/pull/14499#discussion_r1942519687 ## datafusion/sqllogictest/test_files/aggregate.slt: ## @@ -5032,18 +5032,17 @@ logical_plan 03)Aggregate: groupBy=[[aggregate_test_100.c3]], aggr=[[min(

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
jayzhan211 commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942524406 ## datafusion/functions/src/string/ascii.rs: ## @@ -93,6 +95,33 @@ impl ScalarUDFImpl for AsciiFunc { make_scalar_function(ascii, vec![])(args) }

Re: [PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-05 Thread via GitHub
zhuqi-lucas commented on code in PR #14499: URL: https://github.com/apache/datafusion/pull/14499#discussion_r1942531772 ## datafusion/physical-plan/src/stream.rs: ## @@ -472,6 +498,7 @@ impl Stream for ObservedStream { cx: &mut Context<'_>, ) -> Poll> { le

[PR] impl limit pushdown for MemoryExec [datafusion]

2025-02-05 Thread via GitHub
zjregee opened a new pull request, #14502: URL: https://github.com/apache/datafusion/pull/14502 ## Which issue does this PR close? Closes #14337. ## Rationale for this change ## What changes are included in this PR? Implement limit pushdown for MemoryExec, add unit test and modify

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
shehabgamin commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942535860 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -584,23 +541,36 @@ fn get_valid_types( match target_type_class {

Re: [I] Incorrect LIKE result when pattern contains `\\` double backslash [datafusion]

2025-02-05 Thread via GitHub
findepi commented on issue #13304: URL: https://github.com/apache/datafusion/issues/13304#issuecomment-2636194261 This is now fixed, but let's add `'\a' LIKE '\\%'` to the regression tests: https://github.com/apache/datafusion/pull/14501 -- This is an automated message from the Apache Git

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
shehabgamin commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942535860 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -584,23 +541,36 @@ fn get_valid_types( match target_type_class {

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
shehabgamin commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942541549 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -584,23 +541,36 @@ fn get_valid_types( match target_type_class {

[PR] Fix a clippy warning in `datafusion-sqllogictest` [datafusion]

2025-02-05 Thread via GitHub
mbrobbel opened a new pull request, #14506: URL: https://github.com/apache/datafusion/pull/14506 ## Which issue does this PR close? ``` ❯ cargo clippy -p datafusion-sqllogictest --all-features -- -Dwarnings Checking datafusion-sqllogictest v45.0.0 (/Users/matthijsbrobbel/cod

Re: [PR] 14044/enhancement/add xxhash algorithms in expression api [datafusion]

2025-02-05 Thread via GitHub
alamb commented on PR #14367: URL: https://github.com/apache/datafusion/pull/14367#issuecomment-2636414961 This PR isn't passing CI checks so I am marking it as draft to clear the review queue. -- This is an automated message from the Apache Git Service. To respond to the message, please

[PR] minor: improve PR template [datafusion]

2025-02-05 Thread via GitHub
alamb opened a new pull request, #14507: URL: https://github.com/apache/datafusion/pull/14507 ## Which issue does this PR close? Closes #. ## Rationale for this change When reading a PR it is often helpful to see the title of the related ticket. I would like to b

Re: [PR] feat: Add `array_max` function support [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14470: URL: https://github.com/apache/datafusion/pull/14470#discussion_r1942729891 ## datafusion/functions-nested/src/max.rs: ## @@ -0,0 +1,173 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agre

Re: [PR] Fix: Avoid recursive external error wrapping [datafusion]

2025-02-05 Thread via GitHub
alamb commented on code in PR #14371: URL: https://github.com/apache/datafusion/pull/14371#discussion_r1942700622 ## datafusion/common/src/error.rs: ## @@ -131,6 +131,10 @@ pub enum DataFusionError { /// Errors from either mapping LogicalPlans to/from Substrait plans /

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942726264 ## .github/workflows/extended.yml: ## @@ -87,7 +87,7 @@ jobs: - name: Run tests run: | cd datafusion - cargo test --profile ci

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
mbrobbel commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942744884 ## .github/workflows/extended.yml: ## @@ -87,7 +87,7 @@ jobs: - name: Run tests run: | cd datafusion - cargo test --profile c

Re: [I] Project Ideas for GSoC 2025 [datafusion]

2025-02-05 Thread via GitHub
alamb commented on issue #14478: URL: https://github.com/apache/datafusion/issues/14478#issuecomment-2637309500 Another potential project that I think would be huge and very much in the realm is Spark Functions - https://github.com/apache/datafusion/issues/5600 This set of function

Re: [PR] Improve error messages to include the function name. [datafusion]

2025-02-05 Thread via GitHub
Omega359 commented on PR #14511: URL: https://github.com/apache/datafusion/pull/14511#issuecomment-2637344587 > I would prefer the function name to be highlighted some how, it can uppercase or square brackets, otherwise the function name got lost in the message 1. `Function [test] ex

Re: [I] Add Memory Profiling Functionality [datafusion]

2025-02-05 Thread via GitHub
comphead commented on issue #14510: URL: https://github.com/apache/datafusion/issues/14510#issuecomment-2637366054 That is nice @berkaysynnada I love it. To implement it we have to use a `memory reservation` in most of intensive operations, like Sorting, Join, hash, scan, etc, now we

Re: [PR] Collection of errors [datafusion]

2025-02-05 Thread via GitHub
alamb commented on code in PR #14439: URL: https://github.com/apache/datafusion/pull/14439#discussion_r1943247569 ## datafusion/sqllogictest/src/engines/datafusion_engine/error.rs: ## @@ -30,7 +30,7 @@ pub enum DFSqlLogicTestError { #[error("SqlLogicTest error(from sqllogic

Re: [PR] Add `DataFusionError::Collection` to return multiple `DataFusionError`s [datafusion]

2025-02-05 Thread via GitHub
alamb commented on code in PR #14439: URL: https://github.com/apache/datafusion/pull/14439#discussion_r1943361006 ## datafusion/common/src/error.rs: ## @@ -334,6 +343,14 @@ impl Error for DataFusionError { DataFusionError::Context(_, e) => Some(e.as_ref()),

Re: [PR] feat: Add `datafusion-spark` crate [datafusion]

2025-02-05 Thread via GitHub
andygrove commented on PR #14392: URL: https://github.com/apache/datafusion/pull/14392#issuecomment-2637613868 I filed an issue in Comet to do the necessary work (testing and examples) to prepare to move the crate into DataFusion repo. https://github.com/apache/datafusion-comet/issues

Re: [PR] 14044/enhancement/add xxhash algorithms in expression api [datafusion]

2025-02-05 Thread via GitHub
Spaarsh commented on PR #14367: URL: https://github.com/apache/datafusion/pull/14367#issuecomment-2637627050 > Beyond the comments I've added it would be very welcome to include .slt tests for these hash functions. Search for md5 or sha256 for examples of current tests. @Omega359 I f

Re: [PR] Validate and unpack function arguments tersely [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14513: URL: https://github.com/apache/datafusion/pull/14513#discussion_r1943185688 ## datafusion/functions/src/utils.rs: ## @@ -18,10 +18,50 @@ use arrow::array::ArrayRef; use arrow::datatypes::DataType; -use datafusion_common::{Result, Scala

Re: [I] Failure while parsing GIN Index [datafusion-sqlparser-rs]

2025-02-05 Thread via GitHub
LucaCappelletti94 commented on issue #1706: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1706#issuecomment-2637372258 Right now, an Index only supports columns as defined by a `OrderByExpr`, which I do not believe is general enough to also handle the aforementioned cases.

Re: [PR] Improve error messages to include the function name. [datafusion]

2025-02-05 Thread via GitHub
comphead commented on PR #14511: URL: https://github.com/apache/datafusion/pull/14511#issuecomment-2637374150 > > I would prefer the function name to be highlighted some how, it can uppercase or square brackets, otherwise the function name got lost in the message > > 1. `Function tes

Re: [PR] Require space after -- to start single line comment in MySQL [datafusion-sqlparser-rs]

2025-02-05 Thread via GitHub
hansott commented on PR #1705: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1705#issuecomment-2637492463 @iffyio doing a check for which characters are allowed to start comment (using a script against mysql) -- This is an automated message from the Apache Git Service. To r

Re: [PR] Improve error messages to include the function name. [datafusion]

2025-02-05 Thread via GitHub
mbrobbel commented on code in PR #14511: URL: https://github.com/apache/datafusion/pull/14511#discussion_r1943055656 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -243,7 +251,7 @@ fn try_coerce_types( // none possible -> Error plan_err!( "Failed to

Re: [PR] Validate and unpack function arguments tersely [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14513: URL: https://github.com/apache/datafusion/pull/14513#discussion_r1943188641 ## datafusion/functions/src/utils.rs: ## @@ -18,10 +18,50 @@ use arrow::array::ArrayRef; use arrow::datatypes::DataType; -use datafusion_common::{Result, Scala

Re: [PR] chore(deps): Update sqlparser to `0.54.0` [datafusion]

2025-02-05 Thread via GitHub
goldmedal merged PR #14255: URL: https://github.com/apache/datafusion/pull/14255 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@data

Re: [PR] chore(deps): Update sqlparser to `0.54.0` [datafusion]

2025-02-05 Thread via GitHub
goldmedal commented on PR #14255: URL: https://github.com/apache/datafusion/pull/14255#issuecomment-2637260336 🚀 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [I] Project Ideas for GSoC 2025 [datafusion]

2025-02-05 Thread via GitHub
comphead commented on issue #14478: URL: https://github.com/apache/datafusion/issues/14478#issuecomment-2637276320 > [@comphead](https://github.com/comphead), would you be willing to mentor a student on a project to study our codebase and dependencies to reduce DF binary size? Hi @oz

Re: [PR] Validate and unpack function arguments tersely [datafusion]

2025-02-05 Thread via GitHub
comphead commented on code in PR #14513: URL: https://github.com/apache/datafusion/pull/14513#discussion_r1943305019 ## datafusion/functions/src/datetime/make_date.rs: ## @@ -127,9 +121,11 @@ impl ScalarUDFImpl for MakeDateFunc { ColumnarValue::Array(a) => Some(

Re: [PR] Validate and unpack function arguments tersely [datafusion]

2025-02-05 Thread via GitHub
comphead commented on code in PR #14513: URL: https://github.com/apache/datafusion/pull/14513#discussion_r1943306577 ## datafusion/functions/src/lib.rs: ## @@ -1,4 +1,5 @@ // Licensed to the Apache Software Foundation (ASF) under one +// Licensed to the Apache Software Foundati

Re: [PR] Feature Unifying source execution plans [datafusion]

2025-02-05 Thread via GitHub
ion-elgreco commented on PR #14224: URL: https://github.com/apache/datafusion/pull/14224#issuecomment-2637240016 Looks great! Migrating from the ParquetExec to DataSourceExec looks straightforward to me -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] Validate and unpack function arguments tersely [datafusion]

2025-02-05 Thread via GitHub
comphead commented on code in PR #14513: URL: https://github.com/apache/datafusion/pull/14513#discussion_r1943308369 ## datafusion/functions/src/core/nvl2.rs: ## @@ -104,27 +105,19 @@ impl ScalarUDFImpl for NVL2Func { } fn coerce_types(&self, arg_types: &[DataType])

Re: [I] Buildable release builds [datafusion]

2025-02-05 Thread via GitHub
findepi commented on issue #14479: URL: https://github.com/apache/datafusion/issues/14479#issuecomment-2637572096 Let's do this. we upgrade compiler version manually anyway, so we can update one more place i believe. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] fix: Capture nullability in `Values` node planning [datafusion]

2025-02-05 Thread via GitHub
rkrishn7 commented on code in PR #14472: URL: https://github.com/apache/datafusion/pull/14472#discussion_r1943660946 ## datafusion/expr/src/logical_plan/builder.rs: ## @@ -296,12 +298,14 @@ impl LogicalPlanBuilder { field_types.push(common_type.unwrap_or(DataType::N

Re: [PR] fix: Capture nullability in `Values` node planning [datafusion]

2025-02-05 Thread via GitHub
rkrishn7 commented on code in PR #14472: URL: https://github.com/apache/datafusion/pull/14472#discussion_r1943660946 ## datafusion/expr/src/logical_plan/builder.rs: ## @@ -296,12 +298,14 @@ impl LogicalPlanBuilder { field_types.push(common_type.unwrap_or(DataType::N

Re: [I] nullable Expr being constant fold to value can cause schema change and internal error [datafusion]

2025-02-05 Thread via GitHub
tv42 commented on issue #13190: URL: https://github.com/apache/datafusion/issues/13190#issuecomment-2638031512 For others stumbling on this, you can disable this check with ```rust let options = session_config.options_mut(); options.execution.skip_physical_aggrega

Re: [I] Project Ideas for GSoC 2025 (Google Summer of Code) [datafusion]

2025-02-05 Thread via GitHub
ozankabak commented on issue #14478: URL: https://github.com/apache/datafusion/issues/14478#issuecomment-2638951722 @XiangpengHao, do you think you can take a look at how we can divide up the work of larger-than-memory aggregations into smaller tasks? If this is possible, we can make it a p

Re: [I] Implement Nicer / DuckDB style explain plans [datafusion]

2025-02-05 Thread via GitHub
irenjj commented on issue #9371: URL: https://github.com/apache/datafusion/issues/9371#issuecomment-2638964832 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] fix: order by expr rewrite fix [datafusion]

2025-02-05 Thread via GitHub
berkaysynnada commented on PR #14486: URL: https://github.com/apache/datafusion/pull/14486#issuecomment-2639043430 I checked the newly added tests and they were failing before 👍🏻 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-05 Thread via GitHub
zhuqi-lucas commented on code in PR #14499: URL: https://github.com/apache/datafusion/pull/14499#discussion_r1942551654 ## datafusion/sqllogictest/test_files/aggregate.slt: ## @@ -5032,18 +5032,17 @@ logical_plan 03)Aggregate: groupBy=[[aggregate_test_100.c3]], aggr=[[min(

Re: [PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-05 Thread via GitHub
zhuqi-lucas commented on code in PR #14499: URL: https://github.com/apache/datafusion/pull/14499#discussion_r1942554373 ## datafusion/physical-plan/src/stream.rs: ## @@ -444,18 +444,44 @@ impl Stream for EmptyRecordBatchStream { pub(crate) struct ObservedStream { inner: Se

Re: [PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-05 Thread via GitHub
zhuqi-lucas commented on PR #14499: URL: https://github.com/apache/datafusion/pull/14499#issuecomment-2636239227 We may can add a follow-up to reuse LimitStream which is used by SortPreservingMergeExec ```rust impl Stream for LimitStream { type Item = Result; fn p

Re: [PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-05 Thread via GitHub
zhuqi-lucas commented on code in PR #14499: URL: https://github.com/apache/datafusion/pull/14499#discussion_r1942554373 ## datafusion/physical-plan/src/stream.rs: ## @@ -444,18 +444,44 @@ impl Stream for EmptyRecordBatchStream { pub(crate) struct ObservedStream { inner: Se

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
jayzhan211 commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942559050 ## datafusion/functions/src/string/ascii.rs: ## @@ -93,6 +95,33 @@ impl ScalarUDFImpl for AsciiFunc { make_scalar_function(ascii, vec![])(args) }

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
shehabgamin commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942537530 ## datafusion/functions/src/string/ascii.rs: ## @@ -93,6 +95,33 @@ impl ScalarUDFImpl for AsciiFunc { make_scalar_function(ascii, vec![])(args) }

Re: [PR] Fix Type Coercion for UDF Arguments [datafusion]

2025-02-05 Thread via GitHub
jayzhan211 commented on code in PR #14268: URL: https://github.com/apache/datafusion/pull/14268#discussion_r1942560239 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -584,23 +541,36 @@ fn get_valid_types( match target_type_class {

Re: [PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-05 Thread via GitHub
zhuqi-lucas commented on code in PR #14499: URL: https://github.com/apache/datafusion/pull/14499#discussion_r1942554373 ## datafusion/physical-plan/src/stream.rs: ## @@ -444,18 +444,44 @@ impl Stream for EmptyRecordBatchStream { pub(crate) struct ObservedStream { inner: Se

Re: [PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-05 Thread via GitHub
zhuqi-lucas commented on code in PR #14499: URL: https://github.com/apache/datafusion/pull/14499#discussion_r1942519687 ## datafusion/sqllogictest/test_files/aggregate.slt: ## @@ -5032,18 +5032,17 @@ logical_plan 03)Aggregate: groupBy=[[aggregate_test_100.c3]], aggr=[[min(

[PR] refactor: remove uses of `arrow_buffer` & `arrow_array` and use reexport in arrow instead [datafusion]

2025-02-05 Thread via GitHub
Chen-Yuan-Lai opened a new pull request, #14503: URL: https://github.com/apache/datafusion/pull/14503 ## Which issue does this PR close? Closes #14115. ## Rationale for this change As https://github.com/apache/datafusion/issues/14115#issuecomment-26348

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
findepi commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942581503 ## README.md: ## @@ -152,26 +152,13 @@ deprecate methods before removing them, according to the [deprecation guidelines [deprecation guidelines]: https://data

Re: [I] Browser-accessible official DataFusion playground / DataFusion fiddle [datafusion]

2025-02-05 Thread via GitHub
findepi commented on issue #13818: URL: https://github.com/apache/datafusion/issues/13818#issuecomment-2636087000 This looks very cool, @gabotechs! And even better than the CLI itself (https://github.com/apache/datafusion/issues/13286) :) I took the privilege to file the first enhanceme

Re: [PR] Support uncorrelated EXISTS [datafusion]

2025-02-05 Thread via GitHub
alamb commented on code in PR #14474: URL: https://github.com/apache/datafusion/pull/14474#discussion_r1942586153 ## datafusion/sqllogictest/test_files/subquery.slt: ## @@ -389,19 +389,25 @@ where o_orderstatus in ( 2 3 +# uncorrelated exists +query I +SELECT 1 WHERE EXISTS

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-05 Thread via GitHub
mbrobbel commented on code in PR #14483: URL: https://github.com/apache/datafusion/pull/14483#discussion_r1942595648 ## ci/scripts/rust_clippy.sh: ## @@ -18,6 +18,4 @@ # under the License. set -ex -cargo clippy --all-targets --workspace --features avro,pyarrow,integration-t

Re: [PR] Feat: Add fetch to CoalescePartitionsExec [datafusion]

2025-02-05 Thread via GitHub
mertak-synnada commented on code in PR #14499: URL: https://github.com/apache/datafusion/pull/14499#discussion_r1942597562 ## datafusion/sqllogictest/test_files/aggregate.slt: ## @@ -5032,18 +5032,17 @@ logical_plan 03)Aggregate: groupBy=[[aggregate_test_100.c3]], aggr=[[m

Re: [PR] fix(doc): remove AWS_PROFILE from supported S3 configuration [datafusion]

2025-02-05 Thread via GitHub
alamb merged PR #14492: URL: https://github.com/apache/datafusion/pull/14492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] fix(ci): build error with wasm [datafusion]

2025-02-05 Thread via GitHub
alamb commented on PR #14494: URL: https://github.com/apache/datafusion/pull/14494#issuecomment-2636314142 Thank you @Lordworms and @xudong963 ❤️ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] refactor: switch BooleanBufferBuilder to NullBufferBuilder in MaybeNu… [datafusion]

2025-02-05 Thread via GitHub
Chen-Yuan-Lai opened a new pull request, #14504: URL: https://github.com/apache/datafusion/pull/14504 …llBufferBuilder ## Which issue does this PR close? Closes #14115 . ## Rationale for this change As mentioned in #14115 , several examples in DataF

  1   2   3   >