Re: [PR] Support marking columns as system columns via Field's metadata [datafusion]

2025-02-08 Thread via GitHub
chenkovsky commented on PR #14362: URL: https://github.com/apache/datafusion/pull/14362#issuecomment-2645665768 1. could you please add more dataframe tests? e.g. ``` assert!(ctx.table("test").await.unwrap().union(ctx.table("test").await.unwrap()).unwrap().select(vec![col("_rowi

[PR] datafusion-python-44 announcement [datafusion-site]

2025-02-08 Thread via GitHub
timsaucer opened a new pull request, #54: URL: https://github.com/apache/datafusion-site/pull/54 This PR adds in a blog post announcing `datafusion-python` 44.0.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Update ClickBench benchmarks with DataFusion `45.0.0` (When Published) [datafusion]

2025-02-08 Thread via GitHub
pmcgleenon commented on issue #14246: URL: https://github.com/apache/datafusion/issues/14246#issuecomment-2645694266 Makes sense 👍 I've created a PR https://github.com/ClickHouse/ClickBench/pull/304 -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] Hide boilerplate in documentation examples [datafusion]

2025-02-08 Thread via GitHub
bhavesh26 commented on issue #14557: URL: https://github.com/apache/datafusion/issues/14557#issuecomment-2645713479 @alamb assign this to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[PR] BigQuery: Add support for `BEGIN` [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
iffyio opened a new pull request, #1718: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1718 Adds support for the `BEGIN ... EXCEPTION ... END` syntax in BigQuery: ```sql BEGIN SELECT 1; SELECT 2; EXCEPTION WHEN ERROR THEN SELECT 3; SELECT 4; E

Re: [PR] some dependency removals and setup for refactor of `FileScanConfig` [datafusion]

2025-02-08 Thread via GitHub
alamb commented on PR #14543: URL: https://github.com/apache/datafusion/pull/14543#issuecomment-2645733546 Thanks again @logan-keede -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-08 Thread via GitHub
alamb commented on PR #14483: URL: https://github.com/apache/datafusion/pull/14483#issuecomment-2645735179 I merged up from main to resolve a conflict and applied the suggestion. I plan to merge this once we get a clean CI run -- This is an automated message from the Apache Git Service. T

Re: [I] Hide boilerplate in documentation examples [datafusion]

2025-02-08 Thread via GitHub
alamb commented on issue #14557: URL: https://github.com/apache/datafusion/issues/14557#issuecomment-2645737409 > [@alamb](https://github.com/alamb) assign this to me Thanks @bhavesh26 Note in the future you can put the comment `take` and the bot will automatically assign it t

Re: [PR] minor: Move file compression [datafusion]

2025-02-08 Thread via GitHub
alamb commented on PR #14555: URL: https://github.com/apache/datafusion/pull/14555#issuecomment-2645741135 > can I get some help with wasm ci? I have hit a wall there. Looks like that has now been resolved, but there are some other issues that remain -- This is an automated message

Re: [PR] Test all examples from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on code in PR #14544: URL: https://github.com/apache/datafusion/pull/14544#discussion_r1947837198 ## docs/source/library-user-guide/adding-udfs.md: ## @@ -55,62 +55,138 @@ of arguments. This a lower level API with more functionality but is more complex, also doc

Re: [I] [EPIC] DuckDB-Inspired Feature Enhancements [datafusion]

2025-02-08 Thread via GitHub
berkaysynnada commented on issue #14514: URL: https://github.com/apache/datafusion/issues/14514#issuecomment-2645748627 > I noticed [#7622](https://github.com/apache/datafusion/pull/7622), if the syntax was originally not supported by [sqlparser](https://github.com/apache/datafusion-sqlpars

[PR] Make it easier to create a ScalarValure representing typed null (#14548) [datafusion]

2025-02-08 Thread via GitHub
cj-zhukov opened a new pull request, #14558: URL: https://github.com/apache/datafusion/pull/14558 ## Which issue does this PR close? - Closes #14548. ## Rationale for this change ## What changes are included in this PR? ## Are these changes

Re: [PR] Test all examples from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on PR #14544: URL: https://github.com/apache/datafusion/pull/14544#issuecomment-2645752534 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [I] Create UNION plan node with correct schema [datafusion]

2025-02-08 Thread via GitHub
jonahgao commented on issue #14380: URL: https://github.com/apache/datafusion/issues/14380#issuecomment-2645778218 > The original discussion about wildcard expansion is [#11639 (comment)](https://github.com/apache/datafusion/issues/11639#issuecomment-2250895868). It's used to delay the wild

Re: [I] Create UNION plan node with correct schema [datafusion]

2025-02-08 Thread via GitHub
jonahgao commented on issue #14380: URL: https://github.com/apache/datafusion/issues/14380#issuecomment-2645780946 > Hmm, my immediate thought here is that if we cannot guarantee correctly coerced inputs during the first pass, then it's probably not worth attempting to coerce in the builder

Re: [PR] Signature::Coercible with user defined implicit casting [datafusion]

2025-02-08 Thread via GitHub
alamb commented on code in PR #14440: URL: https://github.com/apache/datafusion/pull/14440#discussion_r1947837053 ## datafusion/expr-common/src/signature.rs: ## @@ -127,12 +132,11 @@ pub enum TypeSignature { Exact(Vec), /// One or more arguments belonging to the [`Type

[PR] chore: Refactor aggregate expression serde [datafusion-comet]

2025-02-08 Thread via GitHub
andygrove opened a new pull request, #1380: URL: https://github.com/apache/datafusion-comet/pull/1380 ## Which issue does this PR close? Part of https://github.com/apache/datafusion-comet/issues/1345 ## Rationale for this change Refactor in preparation for

Re: [I] Extended tests are failing on main [datafusion]

2025-02-08 Thread via GitHub
alamb closed issue #14549: Extended tests are failing on main URL: https://github.com/apache/datafusion/issues/14549 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] chore: update datafusion-testing pin to fix extended tests [datafusion]

2025-02-08 Thread via GitHub
alamb commented on PR #14556: URL: https://github.com/apache/datafusion/pull/14556#issuecomment-2645809640 Thank you @jonahgao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] chore: update datafusion-testing pin to fix extended tests [datafusion]

2025-02-08 Thread via GitHub
alamb merged PR #14556: URL: https://github.com/apache/datafusion/pull/14556 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [I] Discuss: Check in Cargo.lock file? [datafusion]

2025-02-08 Thread via GitHub
alamb closed issue #14135: Discuss: Check in Cargo.lock file? URL: https://github.com/apache/datafusion/issues/14135 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-08 Thread via GitHub
alamb commented on PR #14483: URL: https://github.com/apache/datafusion/pull/14483#issuecomment-2645809871 Thanks @mbrobbel and @findepi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-08 Thread via GitHub
alamb merged PR #14483: URL: https://github.com/apache/datafusion/pull/14483 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

[PR] chore(deps): bump nix from 0.28.0 to 0.29.0 [datafusion]

2025-02-08 Thread via GitHub
dependabot[bot] opened a new pull request, #14559: URL: https://github.com/apache/datafusion/pull/14559 Bumps [nix](https://github.com/nix-rust/nix) from 0.28.0 to 0.29.0. Changelog Sourced from https://github.com/nix-rust/nix/blob/master/CHANGELOG.md";>nix's changelog. [0.2

Re: [PR] Add `Cargo.lock` [datafusion]

2025-02-08 Thread via GitHub
alamb commented on PR #14483: URL: https://github.com/apache/datafusion/pull/14483#issuecomment-2645809803 🚀 let's give it a try -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Feat: support array_except function [datafusion-comet]

2025-02-08 Thread via GitHub
kazantsev-maksim commented on code in PR #1343: URL: https://github.com/apache/datafusion-comet/pull/1343#discussion_r1947891630 ## spark/src/test/scala/org/apache/comet/CometArrayExpressionSuite.scala: ## @@ -292,4 +292,89 @@ class CometArrayExpressionSuite extends CometTestBas

Re: [I] Support "Tracing" / Spans [datafusion]

2025-02-08 Thread via GitHub
geoffreyclaude commented on issue #9415: URL: https://github.com/apache/datafusion/issues/9415#issuecomment-2645830852 I've opened https://github.com/apache/datafusion/pull/14547 to launch the discussion. PR was simpler than expected as all new tasks spawned do so in the context of a tokio

Re: [PR] Feat: support array_except function [datafusion-comet]

2025-02-08 Thread via GitHub
kazantsev-maksim commented on code in PR #1343: URL: https://github.com/apache/datafusion-comet/pull/1343#discussion_r1947892464 ## spark/src/main/scala/org/apache/comet/serde/arrays.scala: ## @@ -126,6 +126,21 @@ object CometArraysOverlap extends CometExpressionSerde with Inco

Re: [PR] Feat: support array_except function [datafusion-comet]

2025-02-08 Thread via GitHub
kazantsev-maksim commented on code in PR #1343: URL: https://github.com/apache/datafusion-comet/pull/1343#discussion_r1947893539 ## spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala: ## @@ -2387,6 +2387,8 @@ object QueryPlanSerde extends Logging with ShimQueryPla

Re: [PR] Test all examples from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on PR #14544: URL: https://github.com/apache/datafusion/pull/14544#issuecomment-2645835876 So I managed to strip off the setup code with some regex trick in Python `docs/rustdoc_trim.py`, and now the doc shows clean snippets while the doc-test still able to pass. A few poi

Re: [PR] chore: remove partition_keys from (Bounded)WindowAggExec [datafusion]

2025-02-08 Thread via GitHub
irenjj commented on PR #14526: URL: https://github.com/apache/datafusion/pull/14526#issuecomment-2645837400 > LGTM @irenjj, thank you. I've sent a minor commit for further improvements. Do they make sense to you? Sure, thank you! @berkaysynnada -- This is an automated message from

Re: [PR] Test all examples from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on code in PR #14544: URL: https://github.com/apache/datafusion/pull/14544#discussion_r1947898504 ## docs/source/library-user-guide/query-optimizer.md: ## @@ -187,94 +235,101 @@ name to be used in a schema, `display_name` should be used. ### Utilities -There

Re: [PR] Test all examples from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on code in PR #14544: URL: https://github.com/apache/datafusion/pull/14544#discussion_r1947898504 ## docs/source/library-user-guide/query-optimizer.md: ## @@ -187,94 +235,101 @@ name to be used in a schema, `display_name` should be used. ### Utilities -There

Re: [PR] Test all examples from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on code in PR #14544: URL: https://github.com/apache/datafusion/pull/14544#discussion_r1947900039 ## docs/source/library-user-guide/query-optimizer.md: ## @@ -187,94 +235,101 @@ name to be used in a schema, `display_name` should be used. ### Utilities -There

Re: [I] Hide boilerplate in documentation examples [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on issue #14557: URL: https://github.com/apache/datafusion/issues/14557#issuecomment-2645846496 Please help review workaround solution in https://github.com/apache/datafusion/pull/14544/commits/997504f85e37ef6269e5ee660a1136031d6d2a2 in https://github.com/apache/datafusion/pu

[PR] chore: update datafusion to v.f45 [datafusion-ballista]

2025-02-08 Thread via GitHub
milenkovicm opened a new pull request, #1176: URL: https://github.com/apache/datafusion-ballista/pull/1176 # Which issue does this PR close? Closes #. # Rationale for this change keep up to date with datafusion releases # What changes are included in this

Re: [PR] chore: update datafusion to v.f45 [datafusion-ballista]

2025-02-08 Thread via GitHub
milenkovicm commented on PR #1176: URL: https://github.com/apache/datafusion-ballista/pull/1176#issuecomment-2645856481 converting to draft until #1173 and #1172 are closed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] perf: improve performance of update metrics [datafusion-comet]

2025-02-08 Thread via GitHub
andygrove commented on code in PR #1329: URL: https://github.com/apache/datafusion-comet/pull/1329#discussion_r1947907141 ## native/core/src/execution/jni_api.rs: ## @@ -233,11 +242,12 @@ pub unsafe extern "system" fn Java_org_apache_comet_Native_createPlan( stream

Re: [PR] function: Allow more expressive array signatures [datafusion]

2025-02-08 Thread via GitHub
jkosh44 commented on code in PR #14532: URL: https://github.com/apache/datafusion/pull/14532#discussion_r1947908046 ## datafusion/expr-common/src/signature.rs: ## @@ -227,25 +226,11 @@ impl Display for TypeSignatureClass { #[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Has

Re: [PR] minor: Move file compression [datafusion]

2025-02-08 Thread via GitHub
logan-keede commented on PR #14555: URL: https://github.com/apache/datafusion/pull/14555#issuecomment-2645871731 apologies, there were some major fumbles, this should not have taken 11+ commits, I can open a separate PR if this would look too ugly. -- This is an automated message from

Re: [PR] minor: Move file compression [datafusion]

2025-02-08 Thread via GitHub
alamb commented on PR #14555: URL: https://github.com/apache/datafusion/pull/14555#issuecomment-2645872676 > apologies, there were some major fumbles, this should not have taken 11+ commits, I can open a separate PR if this would look too ugly. No worries at all - when we merge to mai

Re: [PR] feat: Add `array_min` function support [datafusion]

2025-02-08 Thread via GitHub
erenavsarogullari commented on PR #14417: URL: https://github.com/apache/datafusion/pull/14417#issuecomment-2645880016 Thanks @jayzhan211 and @findepi for the reviews. Updated this PR in terms of previous feedback from `array_max` PR: https://github.com/apache/datafusion/pull/14470 Pl

Re: [PR] chore: remove partition_keys from (Bounded)WindowAggExec [datafusion]

2025-02-08 Thread via GitHub
berkaysynnada merged PR #14526: URL: https://github.com/apache/datafusion/pull/14526 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [I] Do We Need partition_by Field in (Bounded)WindowAggExec ? [datafusion]

2025-02-08 Thread via GitHub
berkaysynnada closed issue #14505: Do We Need partition_by Field in (Bounded)WindowAggExec ? URL: https://github.com/apache/datafusion/issues/14505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [I] Improve performance of update metrics [datafusion-comet]

2025-02-08 Thread via GitHub
andygrove closed issue #1328: Improve performance of update metrics URL: https://github.com/apache/datafusion-comet/issues/1328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] perf: improve performance of update metrics [datafusion-comet]

2025-02-08 Thread via GitHub
andygrove merged PR #1329: URL: https://github.com/apache/datafusion-comet/pull/1329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [PR] perf: improve performance of update metrics [datafusion-comet]

2025-02-08 Thread via GitHub
andygrove commented on code in PR #1329: URL: https://github.com/apache/datafusion-comet/pull/1329#discussion_r1947927557 ## native/core/src/execution/jni_api.rs: ## @@ -233,11 +242,12 @@ pub unsafe extern "system" fn Java_org_apache_comet_Native_createPlan( stream

[I] Use coarse time when calculating metrics refresh interval [datafusion-comet]

2025-02-08 Thread via GitHub
andygrove opened a new issue, #1381: URL: https://github.com/apache/datafusion-comet/issues/1381 ### What is the problem the feature request solves? See comment at https://github.com/apache/datafusion-comet/pull/1329#discussion_r1944106936 ### Describe the potential solution

Re: [PR] feat: Add fair unified memory pool [datafusion-comet]

2025-02-08 Thread via GitHub
andygrove commented on PR #1369: URL: https://github.com/apache/datafusion-comet/pull/1369#issuecomment-2645907012 I tried running TPC-H locally with this PR but there is excessive spilling and the benchmark is extremely slow. q3 took 3.4 minutes compared to 16 seconds on main branch.

[I] Update proto to support to/from json with an extension codec [datafusion]

2025-02-08 Thread via GitHub
Omega359 opened a new issue, #14560: URL: https://github.com/apache/datafusion/issues/14560 ### Is your feature request related to a problem or challenge? Currently there is support for logical plan -> to/from bytes with an extension codec but there is support for the same for json. T

Re: [I] Update proto to support to/from json with an extension codec [datafusion]

2025-02-08 Thread via GitHub
Omega359 commented on issue #14560: URL: https://github.com/apache/datafusion/issues/14560#issuecomment-2645910034 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[PR] Update proto to support to/from json with an extension codec [datafusion]

2025-02-08 Thread via GitHub
Omega359 opened a new pull request, #14561: URL: https://github.com/apache/datafusion/pull/14561 ## Which issue does this PR close? - Closes #14560 ## Rationale for this change json parity with bytes impl ## What changes are included in this PR? Code

Re: [PR] Extending support for INDEX parsing [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
LucaCappelletti94 commented on code in PR #1707: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1707#discussion_r1947550781 ## src/ast/ddl.rs: ## @@ -1030,20 +1030,29 @@ impl fmt::Display for KeyOrIndexDisplay { /// [1]: https://dev.mysql.com/doc/refman/8.0/en/cre

Re: [I] Update ClickBench benchmarks with DataFusion `45.0.0` (When Published) [datafusion]

2025-02-08 Thread via GitHub
pmcgleenon commented on issue #14246: URL: https://github.com/apache/datafusion/issues/14246#issuecomment-2644746877 Here are the initial ClickBench results for datafusion `45.0.0` [clickbench-latest 3.html.zip](https://github.com/user-attachments/files/18717824/clickbench-latest.3.ht

Re: [I] Test DataFusion 45.0.0 with Sail [datafusion]

2025-02-08 Thread via GitHub
shehabgamin commented on issue #14408: URL: https://github.com/apache/datafusion/issues/14408#issuecomment-2644716584 > > [@findepi](https://github.com/findepi) do you mean we should relax the check to ignore nullable / non nullable annotations? -- I think that would probably be ok too.

Re: [PR] feat: Add `array_max` function support [datafusion]

2025-02-08 Thread via GitHub
erenavsarogullari commented on PR #14470: URL: https://github.com/apache/datafusion/pull/14470#issuecomment-2644719497 > I have the same question for `array_min`, but if this function is highly interested from many people then adding it to datafusion core is not a bad idea. [#14417 (commen

Re: [PR] Implement SnowFlake ALTER SESSION [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
yoavcloud commented on code in PR #1712: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1712#discussion_r1947545495 ## tests/sqlparser_snowflake.rs: ## @@ -3404,3 +3404,29 @@ fn test_grant_database_role_to() { snowflake_and_generic().verified_stmt("GRANT DATAB

Re: [PR] Signature::Coercible with user defined implicit casting [datafusion]

2025-02-08 Thread via GitHub
shehabgamin commented on PR #14440: URL: https://github.com/apache/datafusion/pull/14440#issuecomment-2644720408 > I will try and review this carefully over the weekend > > Maybe @shehabgamin has some time to take a look too I will review over the weekend as well! -- This is

[PR] minor: Move file compression [datafusion]

2025-02-08 Thread via GitHub
logan-keede opened a new pull request, #14555: URL: https://github.com/apache/datafusion/pull/14555 ## Which issue does this PR close? - part of https://github.com/apache/datafusion/issues/14462 - Part of https://github.com/apache/datafusion/issues/1. ## Rationale

Re: [PR] minor: Move file compression [datafusion]

2025-02-08 Thread via GitHub
logan-keede commented on PR #14555: URL: https://github.com/apache/datafusion/pull/14555#issuecomment-2645220572 can I get some help with wasm ci? I have hit a wall there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Implement SnowFlake ALTER SESSION [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
yoavcloud commented on code in PR #1712: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1712#discussion_r1947544971 ## tests/sqlparser_snowflake.rs: ## @@ -3404,3 +3404,29 @@ fn test_grant_database_role_to() { snowflake_and_generic().verified_stmt("GRANT DATAB

Re: [PR] Implement SnowFlake ALTER SESSION [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
yoavcloud commented on code in PR #1712: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1712#discussion_r1947544999 ## tests/sqlparser_snowflake.rs: ## @@ -3404,3 +3404,29 @@ fn test_grant_database_role_to() { snowflake_and_generic().verified_stmt("GRANT DATAB

Re: [I] [EPIC] DuckDB-Inspired Feature Enhancements [datafusion]

2025-02-08 Thread via GitHub
PokIsemaine commented on issue #14514: URL: https://github.com/apache/datafusion/issues/14514#issuecomment-2644742294 > Hello [@PokIsemaine](https://github.com/PokIsemaine). I have an idea that hasn't been issued yet: implementing "ORDER BY ALL." Although this feature is not part of standar

Re: [PR] Add union_extract scalar function [datafusion]

2025-02-08 Thread via GitHub
gstvg commented on code in PR #12116: URL: https://github.com/apache/datafusion/pull/12116#discussion_r1947937272 ## datafusion/functions/src/core/union_extract.rs: ## @@ -0,0 +1,260 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

Re: [PR] Implement FROM-first selects [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
mitsuhiko commented on code in PR #1713: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1713#discussion_r1947940772 ## tests/sqlparser_duckdb.rs: ## @@ -246,6 +246,67 @@ fn test_create_table_macro() { assert_eq!(expected, macro_); } +#[test] +fn test_select

Re: [PR] Add union_extract scalar function [datafusion]

2025-02-08 Thread via GitHub
gstvg commented on code in PR #12116: URL: https://github.com/apache/datafusion/pull/12116#discussion_r1947939409 ## datafusion/expr/src/udf.rs: ## @@ -951,4 +952,10 @@ The following regular expression functions are supported:"#, label: "Other Functions", desc

Re: [PR] minor: Move file compression [datafusion]

2025-02-08 Thread via GitHub
logan-keede commented on PR #14555: URL: https://github.com/apache/datafusion/pull/14555#issuecomment-2645932154 > No worries at all - when we merge to main everything in each PR is squashed into a single commit. Thanks. I know but still, it feels off. This is ready for review,

Re: [I] Parser rejects column aliases that Snowflake accepts [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
iffyio closed issue #1607: Parser rejects column aliases that Snowflake accepts URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1607 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Parser rejects column aliases that Snowflake accepts [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
iffyio commented on issue #1607: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1607#issuecomment-2646091587 Done in https://github.com/apache/datafusion-sqlparser-rs/pull/1632 -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [I] Decorrelate scalar subqueries with more complex filter expressions [datafusion]

2025-02-08 Thread via GitHub
duongcongtoai commented on issue #14554: URL: https://github.com/apache/datafusion/issues/14554#issuecomment-2646013487 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Implement FROM-first selects [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
iffyio merged PR #1713: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1713 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

Re: [PR] [POC] Fuse operations in `equal_rows_arr` [datafusion]

2025-02-08 Thread via GitHub
github-actions[bot] closed pull request #13607: [POC] Fuse operations in `equal_rows_arr` URL: https://github.com/apache/datafusion/pull/13607 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Add supports for Hive's `SELECT ... GROUP BY .. GROUPING SETS` syntax [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
iffyio commented on code in PR #1653: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1653#discussion_r1948012742 ## src/parser/mod.rs: ## @@ -9092,6 +9092,16 @@ impl<'a> Parser<'a> { }); } } +if self.di

Re: [PR] Signature::Coercible with user defined implicit casting [datafusion]

2025-02-08 Thread via GitHub
shehabgamin commented on code in PR #14440: URL: https://github.com/apache/datafusion/pull/14440#discussion_r1948008922 ## datafusion/expr-common/src/signature.rs: ## @@ -225,6 +229,45 @@ impl Display for TypeSignatureClass { } } +impl TypeSignatureClass { +/// Retur

Re: [PR] Always use `StringViewArray` as output of `substr` [datafusion]

2025-02-08 Thread via GitHub
2010YOUY01 commented on PR #14498: URL: https://github.com/apache/datafusion/pull/14498#issuecomment-2644818671 > My benchmark results show no change in Q22 (but also clearly the data is quite noisy 🤔 ) > > ``` > > Benchmark tpch_sf1.json > --

Re: [I] Use `StringViewArray` as output of `substr` when input was `StringArray` [datafusion]

2025-02-08 Thread via GitHub
2010YOUY01 closed issue #12338: Use `StringViewArray` as output of `substr` when input was `StringArray` URL: https://github.com/apache/datafusion/issues/12338 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Always use `StringViewArray` as output of `substr` [datafusion]

2025-02-08 Thread via GitHub
2010YOUY01 merged PR #14498: URL: https://github.com/apache/datafusion/pull/14498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dat

Re: [PR] [WIP] Run test from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on PR #14544: URL: https://github.com/apache/datafusion/pull/14544#issuecomment-2645043123 Hi @alamb I have passed all the doc-test in `library-user-guide` folder, the one part left is in the `query-optimizer.md`, the previous code snippets are outdated and I cannot find alte

Re: [PR] Run test from `adding-udfs.md` documentation [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on PR #14485: URL: https://github.com/apache/datafusion/pull/14485#issuecomment-2645007165 Closing this PR as I have moved the changes to [PR 14544](https://github.com/apache/datafusion/pull/14544) -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] Run test from `adding-udfs.md` documentation [datafusion]

2025-02-08 Thread via GitHub
ugoa closed pull request #14485: Run test from `adding-udfs.md` documentation URL: https://github.com/apache/datafusion/pull/14485 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [I] Update ClickBench benchmarks with DataFusion `45.0.0` (When Published) [datafusion]

2025-02-08 Thread via GitHub
alamb commented on issue #14246: URL: https://github.com/apache/datafusion/issues/14246#issuecomment-2645571982 > If it looks reasonable, I can create a ClickBench PR to update the results It does look reasonable to me. Thank you @pmcgleenon 🙏 -- This is an automated messa

Re: [PR] Run test from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
alamb commented on code in PR #14544: URL: https://github.com/apache/datafusion/pull/14544#discussion_r1947793559 ## docs/source/library-user-guide/query-optimizer.md: ## @@ -187,94 +235,101 @@ name to be used in a schema, `display_name` should be used. ### Utilities -Ther

Re: [PR] minor: Move file compression [datafusion]

2025-02-08 Thread via GitHub
logan-keede commented on PR #14555: URL: https://github.com/apache/datafusion/pull/14555#issuecomment-2645589511 cc @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Test all examples from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on code in PR #14544: URL: https://github.com/apache/datafusion/pull/14544#discussion_r1947806611 ## docs/source/library-user-guide/adding-udfs.md: ## @@ -55,62 +55,138 @@ of arguments. This a lower level API with more functionality but is more complex, also doc

Re: [PR] Test all examples from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
ugoa commented on code in PR #14544: URL: https://github.com/apache/datafusion/pull/14544#discussion_r1947806611 ## docs/source/library-user-guide/adding-udfs.md: ## @@ -55,62 +55,138 @@ of arguments. This a lower level API with more functionality but is more complex, also doc

Re: [PR] Improve documentation about extended tests [datafusion]

2025-02-08 Thread via GitHub
alamb commented on PR #14320: URL: https://github.com/apache/datafusion/pull/14320#issuecomment-2645597947 Thanks @comphead and @Omega359 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] refactor: remove remaining uses of `arrow_array` and use reexport in `arrow` instead [datafusion]

2025-02-08 Thread via GitHub
alamb merged PR #14528: URL: https://github.com/apache/datafusion/pull/14528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] refactor: remove remaining uses of `arrow_array` and use reexport in `arrow` instead [datafusion]

2025-02-08 Thread via GitHub
alamb commented on PR #14528: URL: https://github.com/apache/datafusion/pull/14528#issuecomment-2645596984 Thanks again @Chen-Yuan-Lai -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Implement SnowFlake ALTER SESSION [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
alamb commented on PR #1712: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1712#issuecomment-2645599573 > See my comments inline. Generally speaking, I support parsing of arbitrary options using DataLoadingOptions but would rename and move this struct to a more central locati

Re: [PR] Improve documentation about extended tests [datafusion]

2025-02-08 Thread via GitHub
alamb merged PR #14320: URL: https://github.com/apache/datafusion/pull/14320 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Extending support for INDEX parsing [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
alamb commented on code in PR #1707: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1707#discussion_r1947816903 ## src/ast/ddl.rs: ## @@ -1030,20 +1030,29 @@ impl fmt::Display for KeyOrIndexDisplay { /// [1]: https://dev.mysql.com/doc/refman/8.0/en/create-table.ht

Re: [PR] Extending support for INDEX parsing [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
alamb commented on code in PR #1707: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1707#discussion_r1947815817 ## src/ast/ddl.rs: ## @@ -1030,20 +1030,29 @@ impl fmt::Display for KeyOrIndexDisplay { /// [1]: https://dev.mysql.com/doc/refman/8.0/en/create-table.ht

Re: [PR] Extending support for INDEX parsing [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
LucaCappelletti94 commented on code in PR #1707: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1707#discussion_r1947817780 ## src/ast/ddl.rs: ## @@ -1030,20 +1030,29 @@ impl fmt::Display for KeyOrIndexDisplay { /// [1]: https://dev.mysql.com/doc/refman/8.0/en/cre

Re: [PR] Test all examples from library user guide docs [datafusion]

2025-02-08 Thread via GitHub
alamb commented on code in PR #14544: URL: https://github.com/apache/datafusion/pull/14544#discussion_r1947818087 ## docs/source/library-user-guide/adding-udfs.md: ## @@ -55,62 +55,138 @@ of arguments. This a lower level API with more functionality but is more complex, also do

Re: [PR] Update expected error messages after upstream change [datafusion-testing]

2025-02-08 Thread via GitHub
alamb commented on PR #6: URL: https://github.com/apache/datafusion-testing/pull/6#issuecomment-2645291208 Thank you @comphead -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[PR] chore: update datafusion-testing pin to fix extended tests [datafusion]

2025-02-08 Thread via GitHub
alamb opened a new pull request, #14556: URL: https://github.com/apache/datafusion/pull/14556 ## Which issue does this PR close? - Closes https://github.com/apache/datafusion/issues/14549 ## Rationale for this change This currently fails on main: ``` INCLUDE_SQLITE=

[PR] Add support for `EXECUTE IMMEDIATE` [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
iffyio opened a new pull request, #1717: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1717 Adds support for the `EXECUTE IMMEDIATE` statement ```sql EXECUTE IMMEDIATE 'SELECT 1' USING 2 AS foo; ``` [BigQuery](https://cloud.google.com/bigquery/docs/reference

Re: [PR] feat: metadata columns [datafusion]

2025-02-08 Thread via GitHub
chenkovsky commented on PR #14057: URL: https://github.com/apache/datafusion/pull/14057#issuecomment-2644943280 for #14362 ,if we want to make it feasible, I think at least it should check metadata dict loaded from file for every format. (e.g. parquet), otherwise the whole system may be bro

Re: [PR] Replace `Method` and `CompositeAccess` with `CompoundFieldAccess` [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
iffyio commented on PR #1716: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1716#issuecomment-2644891227 FYI @goldmedal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[PR] Replace `Method` and `CompositeAccess` with `CompoundFieldAccess` [datafusion-sqlparser-rs]

2025-02-08 Thread via GitHub
iffyio opened a new pull request, #1716: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1716 Continuation of #1551 This moves over two of the special case compound expressions delimited by period to use the new `CompoundFieldAccess` representation. It moves the log

  1   2   >