Re: [PR] Update extending-operators.md [datafusion]

2025-07-18 Thread via GitHub
github-actions[bot] commented on PR #15832: URL: https://github.com/apache/datafusion/pull/15832#issuecomment-3091393278 Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or

Re: [PR] DataFusion `48.0.0` release post [datafusion-site]

2025-07-18 Thread via GitHub
alamb commented on PR #91: URL: https://github.com/apache/datafusion-site/pull/91#issuecomment-3090552542 Amazing -- thank you @Omega359 -- I plan to review this carefully in the next hour or so (when I am on the train) -- This is an automated message from the Apache Git Service. To resp

Re: [PR] DataFusion `48.0.0` release post [datafusion-site]

2025-07-18 Thread via GitHub
Omega359 commented on PR #91: URL: https://github.com/apache/datafusion-site/pull/91#issuecomment-3090557283 I hope to start filling out the sections, fixme's, etc this weekend. I would really like to get the clickbench comparison over time done for this release as well but that requires so

Re: [PR] feat: Add JNI-based Hadoop FileSystem support for S3 and other Hadoop-compatible stores [datafusion-comet]

2025-07-18 Thread via GitHub
mbutrovich commented on PR #1992: URL: https://github.com/apache/datafusion-comet/pull/1992#issuecomment-3090572205 Should we set this to draft as we discuss the design going forward? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] DataFusion `48.0.0` release post [datafusion-site]

2025-07-18 Thread via GitHub
alamb commented on PR #91: URL: https://github.com/apache/datafusion-site/pull/91#issuecomment-3090618288 > I hope to start filling out the sections, fixme's, etc this weekend. I would really like to get the clickbench comparison over time done for this release as well but that requires

Re: [PR] DataFusion `49.0.0` release post [datafusion-site]

2025-07-18 Thread via GitHub
alamb commented on code in PR #91: URL: https://github.com/apache/datafusion-site/pull/91#discussion_r2216920283 ## content/blog/2025-07-25-datafusion-49.0.0.md: ## @@ -0,0 +1,207 @@ + --- +layout: post +title: Apache DataFusion 49.0

[PR] Replace π-related bound constants with next_up/next_down [datafusion]

2025-07-18 Thread via GitHub
rthummaluru opened a new pull request, #16823: URL: https://github.com/apache/datafusion/pull/16823 ## Which issue does this PR close? Closes #16712. ## Rationale for this change Rust 1.86 stabilized f64::next_up() and f32::next_up() methods, along with their next_down() counter

Re: [I] Release DataFusion `49.0.0` (July 2025) [datafusion]

2025-07-18 Thread via GitHub
alamb commented on issue #16235: URL: https://github.com/apache/datafusion/issues/16235#issuecomment-3090483735 > Skeleton of a blog post started with what I think are the highlights. feel free to edit/etc > > https://github.com/Omega359/datafusion-site/blob/df_49/content/blog/2025-0

Re: [PR] Update extending-operators.md [datafusion]

2025-07-18 Thread via GitHub
Adez017 commented on PR #15832: URL: https://github.com/apache/datafusion/pull/15832#issuecomment-3091662660 hi @alamb please have a look on this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] General framework to decorrelate the subqueries [datafusion]

2025-07-18 Thread via GitHub
duongcongtoai commented on issue #5492: URL: https://github.com/apache/datafusion/issues/5492#issuecomment-3091907474 in case anyone interested, I an @irenjj are working on this on a [fork](https://github.com/duongcongtoai/arrow-datafusion) -- This is an automated message from the Apache

[PR] Snowflake: CREATE DATABASE/SCHEMA ... CLONE [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
yoavcloud opened a new pull request, #1958: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1958 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] chore: use `equals_datatype` for `BinaryExpr` [datafusion]

2025-07-18 Thread via GitHub
comphead commented on PR #16813: URL: https://github.com/apache/datafusion/pull/16813#issuecomment-3090822322 > Makes sense. Could you add a test? Added -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] chore: use `equals_datatype` for `BinaryExpr` [datafusion]

2025-07-18 Thread via GitHub
comphead merged PR #16813: URL: https://github.com/apache/datafusion/pull/16813 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

Re: [PR] Add reproducing test cases for stackoverflows [datafusion]

2025-07-18 Thread via GitHub
comphead commented on PR #16787: URL: https://github.com/apache/datafusion/pull/16787#issuecomment-3091241093 I'm just wondering should it be Gist instead? 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] fix: clean up iceberg integration APIs [datafusion-comet]

2025-07-18 Thread via GitHub
parthchandra commented on code in PR #2032: URL: https://github.com/apache/datafusion-comet/pull/2032#discussion_r2217038689 ## common/src/main/java/org/apache/comet/parquet/BatchReader.java: ## @@ -183,9 +183,7 @@ public BatchReader( this.taskContext = TaskContext$.MODULE$

Re: [PR] fix: clean up iceberg integration APIs [datafusion-comet]

2025-07-18 Thread via GitHub
huaxingao commented on code in PR #2032: URL: https://github.com/apache/datafusion-comet/pull/2032#discussion_r2217013989 ## common/src/main/java/org/apache/comet/parquet/BatchReader.java: ## @@ -183,9 +183,7 @@ public BatchReader( this.taskContext = TaskContext$.MODULE$.ge

Re: [I] Move code in `user_defined_plan.rs` to the `extending-operators` doc [datafusion]

2025-07-18 Thread via GitHub
xudong963 commented on issue #15774: URL: https://github.com/apache/datafusion/issues/15774#issuecomment-3091410744 > [@xudong963](https://github.com/xudong963) I see not activity on this issue or either of the PR opened. Is this issue available, can i work on this? Yes, it's availabl

Re: [I] Replace π-related bound constants with `next_up` / `next_down` [datafusion]

2025-07-18 Thread via GitHub
rthummaluru commented on issue #16712: URL: https://github.com/apache/datafusion/issues/16712#issuecomment-3090726399 Just submitted the PR, please let me know if there's anything that needs to be changed or corrected. Thank you! -- This is an automated message from the Apache Git Servic

Re: [PR] Benchmark: Add micro-benchmark for Nested Loop Join operator [datafusion]

2025-07-18 Thread via GitHub
2010YOUY01 commented on code in PR #16819: URL: https://github.com/apache/datafusion/pull/16819#discussion_r2217129149 ## benchmarks/src/nlj.rs: ## @@ -0,0 +1,244 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See t

Re: [PR] Benchmark: Add micro-benchmark for Nested Loop Join operator [datafusion]

2025-07-18 Thread via GitHub
2010YOUY01 commented on code in PR #16819: URL: https://github.com/apache/datafusion/pull/16819#discussion_r2217129111 ## benchmarks/src/nlj.rs: ## @@ -0,0 +1,244 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See t

Re: [PR] Benchmark: Add micro-benchmark for Nested Loop Join operator [datafusion]

2025-07-18 Thread via GitHub
2010YOUY01 commented on PR #16819: URL: https://github.com/apache/datafusion/pull/16819#issuecomment-3091474891 > @2010YOUY01 Thanks for providing such a comprehensive set of benchmark cases. It would be even better if it could also output the memory consumption for each SQL query, just lik

Re: [PR] Benchmark: Add micro-benchmark for Nested Loop Join operator [datafusion]

2025-07-18 Thread via GitHub
2010YOUY01 commented on PR #16819: URL: https://github.com/apache/datafusion/pull/16819#issuecomment-3091475027 Thank you for the review @UBarney @jonathanc-n -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] Add Semi/Anti/Mark join types to Nested Loop Join Benchmark [datafusion]

2025-07-18 Thread via GitHub
2010YOUY01 commented on issue #16820: URL: https://github.com/apache/datafusion/issues/16820#issuecomment-3091476899 Thank you for filing the ticket! > [@2010YOUY01](https://github.com/2010YOUY01) This can be marked as a good first issue, lets new contributors for joins to learn more

Re: [PR] fix: The inconsistency between scalar and array on the cast decimal to timestamp [datafusion]

2025-07-18 Thread via GitHub
findepi merged PR #16539: URL: https://github.com/apache/datafusion/pull/16539 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafu

Re: [I] Different result of decimal to timestamp cast when source value is constant [datafusion]

2025-07-18 Thread via GitHub
findepi closed issue #16531: Different result of decimal to timestamp cast when source value is constant URL: https://github.com/apache/datafusion/issues/16531 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] Df 49 blog post [datafusion-site]

2025-07-18 Thread via GitHub
Omega359 opened a new pull request, #91: URL: https://github.com/apache/datafusion-site/pull/91 Posting as a draft as per @alamb's request @ https://github.com/apache/datafusion/issues/16235#issuecomment-3090483735 -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] feat: randn expression support [datafusion-comet]

2025-07-18 Thread via GitHub
mbutrovich merged PR #2010: URL: https://github.com/apache/datafusion-comet/pull/2010 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...

Re: [I] Support `map_entries` function [datafusion-comet]

2025-07-18 Thread via GitHub
comphead commented on issue #1916: URL: https://github.com/apache/datafusion-comet/issues/1916#issuecomment-3090691128 Depends on https://github.com/apache/datafusion-comet/issues/1993 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] Release DataFusion `49.0.0` (July 2025) [datafusion]

2025-07-18 Thread via GitHub
alamb commented on issue #16235: URL: https://github.com/apache/datafusion/issues/16235#issuecomment-3090505894 Current status: 1. I created https://github.com/apache/datafusion/tree/branch-49 2. There is a PR to update the version and changelog here: https://github.com/apache/datafus

Re: [I] Improve performance on ClickBench [datafusion-comet]

2025-07-18 Thread via GitHub
parthchandra commented on issue #2035: URL: https://github.com/apache/datafusion-comet/issues/2035#issuecomment-3090665794 With the EventDate clause we may have a second problem. ``` scala> spark.sql("SELECT TraficSourceID, SearchEngineID, AdvEngineID, CASE WHEN (SearchEngineID =

Re: [I] Improve performance on ClickBench [datafusion-comet]

2025-07-18 Thread via GitHub
parthchandra commented on issue #2035: URL: https://github.com/apache/datafusion-comet/issues/2035#issuecomment-3090666597 The query succeeds with `native_iceberg_compat` and `--conf spark.comet.scan.allowIncompatible=true` -- This is an automated message from the Apache Git Service

Re: [PR] Support multiple ordered array_agg aggregations [datafusion]

2025-07-18 Thread via GitHub
alamb commented on PR #16625: URL: https://github.com/apache/datafusion/pull/16625#issuecomment-3090664781 > @ozankabak @alamb can you please help me understand where you would want to go with this? I think supporting multiple ordered array_agg aggregations makes sense to me; I have

Re: [I] Improve performance on ClickBench [datafusion-comet]

2025-07-18 Thread via GitHub
parthchandra commented on issue #2035: URL: https://github.com/apache/datafusion-comet/issues/2035#issuecomment-3090665542 I can reproduce the above problem only after removing this part`EventDate >= '2013-07-01' AND EventDate <= '2013-07-31'` from the query. (With the clause all records

Re: [PR] feat: improve LiteralGuarantee for the case like `(a=1 AND b=1) OR (a=2 AND b=3)` [datafusion]

2025-07-18 Thread via GitHub
alamb commented on PR #16762: URL: https://github.com/apache/datafusion/pull/16762#issuecomment-3090672449 I am sorry @haohuaijin -- I will review this more carefully soon. I just need to sit down and think through the details to make sure it doesn't have any correctness problems -- Thi

[PR] Address memory over-accounting in array_agg [datafusion]

2025-07-18 Thread via GitHub
gabotechs opened a new pull request, #16816: URL: https://github.com/apache/datafusion/pull/16816 ## Which issue does this PR close? - Closes #. ## Rationale for this change Follow up on: - https://github.com/apache/datafusion/pull/16346 - https://github.com

[PR] chore(deps): bump aws-credential-types from 1.2.3 to 1.2.4 [datafusion]

2025-07-18 Thread via GitHub
dependabot[bot] opened a new pull request, #16815: URL: https://github.com/apache/datafusion/pull/16815 Bumps [aws-credential-types](https://github.com/smithy-lang/smithy-rs) from 1.2.3 to 1.2.4. Commits See full diff in https://github.com/smithy-lang/smithy-rs/commits";>compar

Re: [PR] Chore: refactor Comparison out of QueryPlanSerde [datafusion-comet]

2025-07-18 Thread via GitHub
codecov-commenter commented on PR #2028: URL: https://github.com/apache/datafusion-comet/pull/2028#issuecomment-3088718229 ## [Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/2028?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_ca

Re: [I] Release DataFusion `49.0.0` (July 2025) [datafusion]

2025-07-18 Thread via GitHub
xudong963 commented on issue #16235: URL: https://github.com/apache/datafusion/issues/16235#issuecomment-3088863934 @alamb I'm busy recently, if you don't get a chance to do it on Friday, I can start doing it on Saturday. -- This is an automated message from the Apache Git Service. To res

Re: [I] ScalarUDFImpl::equals default implementation is error-prone [datafusion]

2025-07-18 Thread via GitHub
findepi commented on issue #16677: URL: https://github.com/apache/datafusion/issues/16677#issuecomment-3088759771 Worth noting - #16781 That PR adds missing equality implementation to many functions and improves default implementation to consider type_id and aliases. It well

Re: [PR] Enhance `ScalarUDFImpl` Equality Handling with Pointer-Based Default and Customizable Logic [datafusion]

2025-07-18 Thread via GitHub
findepi commented on PR #16681: URL: https://github.com/apache/datafusion/pull/16681#issuecomment-3088768792 @alamb https://github.com/apache/datafusion/pull/16781 aims to solve immediate obvious problem - incorrect implementation of equality for stateful functions managed in this repo. Th

Re: [PR] fix: The inconsistency between scalar and array on the cast decimal to timestamp [datafusion]

2025-07-18 Thread via GitHub
findepi commented on PR #16539: URL: https://github.com/apache/datafusion/pull/16539#issuecomment-3088779085 @alamb PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Implement equals for stateful functions [datafusion]

2025-07-18 Thread via GitHub
findepi merged PR #16781: URL: https://github.com/apache/datafusion/pull/16781 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafu

[I] Minor documentation fix in parquet_index.rs [datafusion]

2025-07-18 Thread via GitHub
m09526 opened a new issue, #16817: URL: https://github.com/apache/datafusion/issues/16817 The parquet_index.rs example mentions "the (forthcoming) `advanced_parquet_index` example". This example now exists! -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] feat: randn expression support [datafusion-comet]

2025-07-18 Thread via GitHub
andygrove commented on code in PR #2010: URL: https://github.com/apache/datafusion-comet/pull/2010#discussion_r2215684316 ## native/proto/src/proto/expr.proto: ## @@ -80,7 +80,8 @@ message Expr { ArrayInsert array_insert = 58; MathExpr integral_divide = 59; ToPret

Re: [PR] feat: randn expression support [datafusion-comet]

2025-07-18 Thread via GitHub
akupchinskiy commented on code in PR #2010: URL: https://github.com/apache/datafusion-comet/pull/2010#discussion_r2215762841 ## native/proto/src/proto/expr.proto: ## @@ -80,7 +80,8 @@ message Expr { ArrayInsert array_insert = 58; MathExpr integral_divide = 59; ToP

Re: [I] Feature is not implemeneted: Unsupported cast with list of structs [datafusion]

2025-07-18 Thread via GitHub
alamb commented on issue #15338: URL: https://github.com/apache/datafusion/issues/15338#issuecomment-3089248741 @kosiew filed a ticket that I think describes a solution to this problem: - https://github.com/apache/datafusion/issues/16579 -- This is an automated message from the Apache

Re: [PR] chore(deps): Update sqlparser to 0.56 [datafusion]

2025-07-18 Thread via GitHub
crepererum commented on code in PR #16456: URL: https://github.com/apache/datafusion/pull/16456#discussion_r2216026026 ## Cargo.toml: ## @@ -167,7 +167,10 @@ recursive = "0.1.1" regex = "1.8" rstest = "0.25.0" serde_json = "1" -sqlparser = { version = "0.55.0", default-featur

Re: [I] Release sqlparser-rs version `0.58.0` around 2025-07-18 (was 2024-08-15) [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb commented on issue #1886: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1886#issuecomment-3089472631 We have a release candidate out for voting: - https://lists.apache.org/thread/88ptk90hkch48x7bx2ny87zggxokgxxp Thanks to @crepererum for approving the changelo

Re: [PR] DuckDB, Postgres, SQLite: NOT NULL and NOTNULL expressions [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
iffyio commented on code in PR #1927: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1927#discussion_r2216110543 ## tests/sqlparser_common.rs: ## @@ -16031,6 +16031,30 @@ fn parse_create_procedure_with_parameter_modes() { } } +#[test] +fn parse_not_null_sup

Re: [PR] Benchmark: Add micro-benchmark for Nested Loop Join operator [datafusion]

2025-07-18 Thread via GitHub
jonathanc-n commented on PR #16819: URL: https://github.com/apache/datafusion/pull/16819#issuecomment-3089640810 Recorded existence join work at #16820 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Add Semi/Anti/Mark join types to Nested Loop Join Benchmark [datafusion]

2025-07-18 Thread via GitHub
jonathanc-n commented on issue #16820: URL: https://github.com/apache/datafusion/issues/16820#issuecomment-3089643159 @2010YOUY01 This can be marked as a good first issue, lets new contributors for joins to learn more about the different types. -- This is an automated message from the Apa

[I] Add Semi/Anti/Mark join types to Nested Loop Join Benchmark [datafusion]

2025-07-18 Thread via GitHub
jonathanc-n opened a new issue, #16820: URL: https://github.com/apache/datafusion/issues/16820 ### Is your feature request related to a problem or challenge? @2010YOUY01 adds a benchmark for #16819. We want to use this to test existence join performance as well. ### Desc

Re: [PR] Address memory over-accounting in array_agg [datafusion]

2025-07-18 Thread via GitHub
fmonjalet commented on code in PR #16816: URL: https://github.com/apache/datafusion/pull/16816#discussion_r2216291759 ## datafusion/functions-aggregate/src/array_agg.rs: ## @@ -1008,8 +1002,7 @@ mod tests { acc2.update_batch(&[data(["b", "c", "a"])])?; acc1 = m

Re: [PR] Address memory over-accounting in array_agg [datafusion]

2025-07-18 Thread via GitHub
comphead commented on code in PR #16816: URL: https://github.com/apache/datafusion/pull/16816#discussion_r2216297987 ## datafusion/functions-aggregate/src/array_agg.rs: ## @@ -315,11 +313,7 @@ impl Accumulator for ArrayAggAccumulator { }; if !val.is_empty() {

Re: [PR] Benchmark: Add micro-benchmark for Nested Loop Join operator [datafusion]

2025-07-18 Thread via GitHub
UBarney commented on code in PR #16819: URL: https://github.com/apache/datafusion/pull/16819#discussion_r2216202985 ## benchmarks/src/nlj.rs: ## @@ -0,0 +1,244 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the

[PR] Add benchmark utility to profile peak memory usage [datafusion]

2025-07-18 Thread via GitHub
ding-young opened a new pull request, #16814: URL: https://github.com/apache/datafusion/pull/16814 ## Which issue does this PR close? - Closes #16720 . ## Rationale for this change ## What changes are included in this PR? ## Are these change

Re: [PR] DuckDB, Postgres, SQLite: NOT NULL and NOTNULL expressions [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
ryanschneider commented on PR #1927: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1927#issuecomment-3089851700 Sorry I should've mentioned that I did check the other ColumnOptions and confirmed that they don't _need_ to use `parse_column_option_expr` but after thinking about

Re: [PR] DuckDB, Postgres, SQLite: NOT NULL and NOTNULL expressions [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
ryanschneider commented on PR #1927: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1927#issuecomment-3089916338 Ok, I consolidated the tests, fixed the `cargo doc` error and use `parse_column_option_expr` in all spots for consistency. -- This is an automated message from th

Re: [D] Best practices for memory-efficient deduplication of pre-sorted Parquet files [datafusion]

2025-07-18 Thread via GitHub
GitHub user alamb added a comment to the discussion: Best practices for memory-efficient deduplication of pre-sorted Parquet files > The query plan for the original query: So this query is ordered like ```sql WITH ORDER (col_1 ASC, col_2 ASC) ``` But the grouping is on all columns ```sql

Re: [I] Improve performance on ClickBench [datafusion-comet]

2025-07-18 Thread via GitHub
parthchandra commented on issue #2035: URL: https://github.com/apache/datafusion-comet/issues/2035#issuecomment-3090174632 Thank you for trying to add Comet to Clickbench! Your configuration looks ok. The distribution of memory between heap and off-heap is a little tricky and really

[PR] Benchmark: Add micro-benchmark for Nested Loop Join operator [datafusion]

2025-07-18 Thread via GitHub
2010YOUY01 opened a new pull request, #16819: URL: https://github.com/apache/datafusion/pull/16819 ## Which issue does this PR close? - NA ## Rationale for this change Now, NLJ operator still has some room to improve performance and efficiency (less memor

Re: [PR] feat(datafusion-proto): allow TableSource to be serialized [datafusion]

2025-07-18 Thread via GitHub
colinmarc commented on PR #16750: URL: https://github.com/apache/datafusion/pull/16750#issuecomment-3089114038 The doc tests were fiddly, sorry about that. I got them to pass locally now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] chore(deps): Update sqlparser to 0.56 [datafusion]

2025-07-18 Thread via GitHub
Dimchikkk commented on code in PR #16456: URL: https://github.com/apache/datafusion/pull/16456#discussion_r2215987842 ## Cargo.toml: ## @@ -167,7 +167,10 @@ recursive = "0.1.1" regex = "1.8" rstest = "0.25.0" serde_json = "1" -sqlparser = { version = "0.55.0", default-feature

Re: [PR] Enhance `ScalarUDFImpl` Equality Handling with Pointer-Based Default and Customizable Logic [datafusion]

2025-07-18 Thread via GitHub
alamb commented on PR #16681: URL: https://github.com/apache/datafusion/pull/16681#issuecomment-3089418586 > remove default implementation of equality -- this is a breaking change As long as we sufficiently document how to fix it (with an example in the upgrade guide) I I think that m

Re: [PR] Prepare 0.58.0 release: update version + Changelog [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb merged PR #1955: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1955 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr.

Re: [PR] benchmark: Add parquet h2o support [datafusion]

2025-07-18 Thread via GitHub
alamb merged PR #16804: URL: https://github.com/apache/datafusion/pull/16804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

[PR] docs: Remove reference to forthcoming example (#16817) [datafusion]

2025-07-18 Thread via GitHub
m09526 opened a new pull request, #16818: URL: https://github.com/apache/datafusion/pull/16818 ## Which issue does this PR close? - Closes #16817 ## Rationale for this change Minor documentation clean-up. ## Are there any user-facing changes? No

Re: [PR] fix: skip predicates on struct unnest in PushDownFilter [datafusion]

2025-07-18 Thread via GitHub
akoshchiy commented on PR #16790: URL: https://github.com/apache/datafusion/pull/16790#issuecomment-3089036776 I've added some comments to the docs. btw, I've checked behaviour on duckdb, and it looks more clearly - there is no prefixes at all. Maybe we can do the same? ```

Re: [PR] [branch-0.56] Fix `limit` in subqueries (#1899) [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb commented on code in PR #1954: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1954#discussion_r2215910483 ## src/ast/mod.rs: ## @@ -729,7 +729,7 @@ pub enum Expr { /// `[ NOT ] IN (SELECT ...)` InSubquery { expr: Box, -subquery: Box

Re: [PR] [branch-0.56] Fix `limit` in subqueries (#1899) [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb closed pull request #1954: [branch-0.56] Fix `limit` in subqueries (#1899) URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Fix `limit` in subqueries [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb commented on PR #1899: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1899#issuecomment-3089305218 I tried to backport this code to 0.56.0 and I think it is not "semver" compatible (we need to release this as 0.57.0). I will do so See more details on - https://

Re: [PR] [branch-0.56] Fix `limit` in subqueries (#1899) [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb commented on code in PR #1954: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1954#discussion_r2215910483 ## src/ast/mod.rs: ## @@ -729,7 +729,7 @@ pub enum Expr { /// `[ NOT ] IN (SELECT ...)` InSubquery { expr: Box, -subquery: Box

Re: [I] Release sqlparser 0.56.1 (backport/fix release) [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb closed issue #1952: Release sqlparser 0.56.1 (backport/fix release) URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [I] Release sqlparser 0.56.1 (backport/fix release) [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb commented on issue #1952: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1952#issuecomment-3089318514 It turns out backporting https://github.com/apache/datafusion-sqlparser-rs/issues/1898 would break other users (it changes the AST and this is not SemVer compatible wi

[PR] [branch-0.56] Fix `limit` in subqueries (#1899) [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb opened a new pull request, #1954: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1954 - Part of https://github.com/apache/datafusion-sqlparser-rs/issues/1952 - Backport of https://github.com/apache/datafusion-sqlparser-rs/pull/1899 to 0.56.0 branch -- This is an

Re: [PR] chore(deps): Update sqlparser to 0.56 [datafusion]

2025-07-18 Thread via GitHub
alamb commented on code in PR #16456: URL: https://github.com/apache/datafusion/pull/16456#discussion_r2215925032 ## Cargo.toml: ## @@ -167,7 +167,10 @@ recursive = "0.1.1" regex = "1.8" rstest = "0.25.0" serde_json = "1" -sqlparser = { version = "0.55.0", default-features =

Re: [I] Release sqlparser-rs version `0.58.0` around 2024-08-15 [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb commented on issue #1886: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1886#issuecomment-3089328424 I am working on creating a 0.58 release now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[PR] Prepare 0.58.0 release: update version + Changelog [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb opened a new pull request, #1955: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1955 - Part of https://github.com/apache/datafusion-sqlparser-rs/issues/1886 You can see the rendered changelog here: - https://github.com/alamb/sqlparser-rs/blob/alamb/prepare_0.58

Re: [I] Release sqlparser-rs version `0.58.0` around 2025-07-18 (was 2024-08-15) [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb commented on issue #1886: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1886#issuecomment-3089344281 I have prepared a PR with version update and changelog: - https://github.com/apache/datafusion-sqlparser-rs/pull/1955 -- This is an automated message from the Apac

[I] Release sqlparser-rs version `0.59.0` around 2025-09-15 [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
alamb opened a new issue, #1956: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1956 Follow on to - https://github.com/apache/datafusion-sqlparser-rs/issues/1886 This ticket tracks creating the next sqlparser release (mostly so others can follow along) **Targ

Re: [PR] MySQL: Support `EXPLAIN ANALYZE` format variants [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
iffyio merged PR #1945: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1945 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

[PR] Snowflake: Support IDENTIFIER for GRANT ROLE [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
yoavcloud opened a new pull request, #1957: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1957 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Postgres: ALTER TABLE SET ( storage_parameters ) [datafusion-sqlparser-rs]

2025-07-18 Thread via GitHub
iffyio commented on code in PR #1947: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1947#discussion_r2216085532 ## src/ast/ddl.rs: ## @@ -351,6 +351,10 @@ pub enum AlterTableOperation { ValidateConstraint { name: Ident, }, +/// `SET ( storag

Re: [PR] docs: Remove reference to forthcoming example (#16817) [datafusion]

2025-07-18 Thread via GitHub
alamb merged PR #16818: URL: https://github.com/apache/datafusion/pull/16818 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [I] Minor documentation fix in parquet_index.rs [datafusion]

2025-07-18 Thread via GitHub
alamb closed issue #16817: Minor documentation fix in parquet_index.rs URL: https://github.com/apache/datafusion/issues/16817 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Release DataFusion `49.0.0` (July 2025) [datafusion]

2025-07-18 Thread via GitHub
Omega359 commented on issue #16235: URL: https://github.com/apache/datafusion/issues/16235#issuecomment-3090345846 Skeleton of a blog post started with what I think are the highlights. feel free to edit/etc https://github.com/Omega359/datafusion-site/blob/df_49/content/blog/2025-07-2

Re: [I] Release DataFusion `49.0.0` (July 2025) [datafusion]

2025-07-18 Thread via GitHub
alamb commented on issue #16235: URL: https://github.com/apache/datafusion/issues/16235#issuecomment-3090337698 I created a branch and will now create a PR there to update the version and changelog - https://github.com/apache/datafusion/tree/branch-49 -- This is an automated message fr

Re: [I] Release DataFusion `49.0.0` (July 2025) [datafusion]

2025-07-18 Thread via GitHub
alamb commented on issue #16235: URL: https://github.com/apache/datafusion/issues/16235#issuecomment-3090333269 Thanks @xudong963 -- I am beginning the process now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[I] RFC: What table provider features would be helpful in an example? [datafusion]

2025-07-18 Thread via GitHub
timsaucer opened a new issue, #16821: URL: https://github.com/apache/datafusion/issues/16821 ### Is your feature request related to a problem or challenge? During my own creation of table providers for a custom data source, I ran into cases where I could gain significant performance b

[PR] [branch-49] Update version to `49.0.0`, add changelog [datafusion]

2025-07-18 Thread via GitHub
alamb opened a new pull request, #16822: URL: https://github.com/apache/datafusion/pull/16822 ## Which issue does this PR close? - Part of https://github.com/apache/datafusion/issues/16235 ## Rationale for this change We are preparing for a release so we need to update th

Re: [PR] fix: The inconsistency between scalar and array on the cast decimal to timestamp [datafusion]

2025-07-18 Thread via GitHub
alamb commented on code in PR #16539: URL: https://github.com/apache/datafusion/pull/16539#discussion_r2216741569 ## datafusion/common/src/scalar/mod.rs: ## @@ -3075,38 +3075,7 @@ impl ScalarValue { target_type: &DataType, cast_options: &CastOptions<'static>,

Re: [PR] WIP Blog post for Datafusion 47.0.0 [datafusion-site]

2025-07-18 Thread via GitHub
alamb closed pull request #70: WIP Blog post for Datafusion 47.0.0 URL: https://github.com/apache/datafusion-site/pull/70 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] WIP Blog post for Datafusion 47.0.0 [datafusion-site]

2025-07-18 Thread via GitHub
alamb commented on PR #70: URL: https://github.com/apache/datafusion-site/pull/70#issuecomment-3090472238 @Omega359 did it for real here: https://github.com/apache/datafusion-site/pull/83 -- This is an automated message from the Apache Git Service. To respond to the message, please log o