Re: [I] Use uv for dependency management [datafusion-python]

2024-12-28 Thread via GitHub
ion-elgreco commented on issue #977: URL: https://github.com/apache/datafusion-python/issues/977#issuecomment-2564288557 UV is a blast to use -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Default to ZSTD compression when writing Parquet [datafusion-python]

2024-12-28 Thread via GitHub
ion-elgreco commented on PR #981: URL: https://github.com/apache/datafusion-python/pull/981#issuecomment-2564288959 In delta-rs we have the default to use "snappy" compression, except our optimize operation which uses ZSTD(4) -- This is an automated message from the Apache Git Service. T

[PR] feat: reads using global ctx [datafusion-python]

2024-12-28 Thread via GitHub
ion-elgreco opened a new pull request, #982: URL: https://github.com/apache/datafusion-python/pull/982 # Which issue does this PR close? - closes https://github.com/apache/datafusion-python/issues/918 -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] SQLite: Allow dollar signs in placeholder names [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
iffyio merged PR #1620: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1620 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

Re: [PR] Improve error for an unexpected token after DROP [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
iffyio merged PR #1623: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

Re: [PR] Fix `sqlparser_bench` benchmark compilation [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on PR #1625: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1625#issuecomment-2564331263 It does occur to me that this is not covered by CI. Maybe we should move the benchmark binary into the normal sqlparser crate (which I think would be more standard) rather

[PR] Fix `sqlparser_bench` benchmark compilation [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb opened a new pull request, #1625: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1625 If you try to run the sqlparser benchmarks it fails to compile: ```shell cd sqlparser_bench cargo clippy --all-targets ``` ``` error[E0277]: can't compare `std::s

Re: [PR] Fix `sqlparser_bench` benchmark compilation [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on code in PR #1625: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1625#discussion_r1898909295 ## sqlparser_bench/benches/sqlparser_bench.rs: ## @@ -78,8 +78,7 @@ fn basic_queries(c: &mut Criterion) { group.bench_function("format_large_state

Re: [PR] Update sqllogictest requirement from 0.24.0 to 0.25.0 [datafusion]

2024-12-28 Thread via GitHub
jonahgao commented on code in PR #13917: URL: https://github.com/apache/datafusion/pull/13917#discussion_r1898912465 ## datafusion/sqllogictest/test_files/group_by.slt: ## @@ -80,7 +80,7 @@ SELECT col1 * cor0.col1 * 56 AS col1 FROM tab2 AS cor0 GROUP BY cor0.col1 208376 94136

Re: [I] Release DataFusion `44.0.0` [datafusion]

2024-12-28 Thread via GitHub
alamb commented on issue #13334: URL: https://github.com/apache/datafusion/issues/13334#issuecomment-2564353410 As @timsaucer mentions above (and on the mailing list): https://lists.apache.org/thread/34xy1nm47tkrr8lszxzwcrlf5b47h333 There was a regression introduced into initcap that

Re: [I] Wrong result for parsing special double number value [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
goldmedal commented on issue #1619: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1619#issuecomment-2564364957 > Do we need to make a patch release for sqlparser 53.0.0? > I think we don't need to do it. >By my reading this is an issue that was introduced after

[PR] Improve parsing speed by Avoid clone in parse_identifier [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb opened a new pull request, #1624: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1624 I made a flamegraph of the sqlparser_bench and found a few unecessary clones ![flamegraph](https://github.com/user-attachments/assets/070959e1-6f3a-4572-8815-dd594d1b11c3) T

Re: [PR] Refactor advancing token to avoid duplication, avoid borrow checker issues [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on PR #1618: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1618#issuecomment-2564333072 > Not sure I followed the intent of the comment, did it mean potentially replacing this api with something similar called advance or would that have a different behavior?

Re: [PR] Support arbitrary composite access expressions [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on PR #1600: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1600#issuecomment-2564362929 - BTW this seems to have caused a regression: https://github.com/apache/datafusion-sqlparser-rs/pull/1621. Thanks to @goldmedal and @iffyio for fixing -- This is an auto

Re: [PR] Fix BigQuery hyphenated ObjectName with numbers [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on PR #1598: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1598#issuecomment-2564362968 - BTW this seems to have caused a regression: https://github.com/apache/datafusion-sqlparser-rs/pull/1621. Thanks to @goldmedal and @iffyio for fixing -- This is an auto

Re: [I] Wrong result for parsing special double number value [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on issue #1619: URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1619#issuecomment-2564362812 Thanks @goldmedal and @iffyio Do we need to make a patch release for sqlparser 53.0.0? By my reading this is an issue that was introduced after that rel

Re: [I] Release DataFusion `44.0.0` [datafusion]

2024-12-28 Thread via GitHub
alamb commented on issue #13334: URL: https://github.com/apache/datafusion/issues/13334#issuecomment-2564366465 have created a new RC and started a new vote thread: https://lists.apache.org/thread/p0hyr78zs03f3snj0mh3vqm6c3kjwshn -- This is an automated message from the Apache Git Servic

Re: [PR] doc-gen: migrate scalar functions (array) documentation 2/3 [datafusion]

2024-12-28 Thread via GitHub
comphead commented on PR #13929: URL: https://github.com/apache/datafusion/pull/13929#issuecomment-2564414852 Thanks @Chen-Yuan-Lai I feel its needed to add `datafusion-macros` into the crate `Cargo.toml` -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] chore: Upgrade to DataFusion 44.0.0-rc2 [datafusion-comet]

2024-12-28 Thread via GitHub
viirya commented on code in PR #1154: URL: https://github.com/apache/datafusion-comet/pull/1154#discussion_r1898976737 ## native/spark-expr/src/strings.rs: ## @@ -141,18 +137,43 @@ make_predicate_function!(EndsWith, ends_with_dyn, ends_with_utf8_scalar_dyn); make_predicate_f

Re: [PR] chore: Upgrade to DataFusion 44.0.0-rc2 [datafusion-comet]

2024-12-28 Thread via GitHub
viirya commented on code in PR #1154: URL: https://github.com/apache/datafusion-comet/pull/1154#discussion_r1898976555 ## native/spark-expr/src/strings.rs: ## @@ -141,18 +137,43 @@ make_predicate_function!(EndsWith, ends_with_dyn, ends_with_utf8_scalar_dyn); make_predicate_f

Re: [I] doc-gen: Migrate builtin scalar functions from code based documentation to attribute based [datafusion]

2024-12-28 Thread via GitHub
comphead commented on issue #13671: URL: https://github.com/apache/datafusion/issues/13671#issuecomment-2564416664 Supposedly it was closed automatically, I think we can close it once all the PRs merged. -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [I] SELECT * FROM subquery ignores ordering [datafusion]

2024-12-28 Thread via GitHub
comphead closed issue #13904: SELECT * FROM subquery ignores ordering URL: https://github.com/apache/datafusion/issues/13904 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] SELECT * FROM subquery ignores ordering [datafusion]

2024-12-28 Thread via GitHub
comphead commented on issue #13904: URL: https://github.com/apache/datafusion/issues/13904#issuecomment-2564417913 @TheBuilderJR thanks for opening the issue and please let us know if anything still needed. as @ozankabak correctly mentioned the if there is no top level ordering the engine c

[PR] build(deps): bump object_store from 0.11.1 to 0.11.2 [datafusion-python]

2024-12-28 Thread via GitHub
dependabot[bot] opened a new pull request, #984: URL: https://github.com/apache/datafusion-python/pull/984 Bumps [object_store](https://github.com/apache/arrow-rs) from 0.11.1 to 0.11.2. Changelog Sourced from https://github.com/apache/arrow-rs/blob/main/CHANGELOG-old.md";>object_s

Re: [PR] feat: Move shuffle block decompression and decoding to native code and add LZ4 & Snappy support [datafusion-comet]

2024-12-28 Thread via GitHub
andygrove commented on PR #1192: URL: https://github.com/apache/datafusion-comet/pull/1192#issuecomment-2564419774 > Hmm, the difference between ZSTD and LZ4 looks not significant? Does the benchmark of TPC-H shown in the description include native shuffle block decompression and decoding?

Re: [I] Wrong result for parsing special double number value [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
iffyio closed issue #1619: Wrong result for parsing special double number value URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1619 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Improve parsing speed by Avoid clone in parse_identifier [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on code in PR #1624: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1624#discussion_r1898909440 ## src/parser/mod.rs: ## @@ -970,15 +970,15 @@ impl<'a> Parser<'a> { t @ (Token::Word(_) | Token::SingleQuotedString(_)) => {

Re: [PR] Update sqllogictest requirement from 0.24.0 to 0.25.0 [datafusion]

2024-12-28 Thread via GitHub
alamb merged PR #13917: URL: https://github.com/apache/datafusion/pull/13917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] doc-gen: migrate scalar functions (encoding & regex) documentation [datafusion]

2024-12-28 Thread via GitHub
alamb commented on code in PR #13919: URL: https://github.com/apache/datafusion/pull/13919#discussion_r1898915999 ## datafusion/functions/src/encoding/inner.rs: ## @@ -126,10 +124,21 @@ impl ScalarUDFImpl for EncodeFunc { } fn documentation(&self) -> Option<&Document

Re: [PR] Consolidate Examples: memtable.rs and parquet_multiple_files.rs [datafusion]

2024-12-28 Thread via GitHub
alamb merged PR #13913: URL: https://github.com/apache/datafusion/pull/13913 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Consolidate Examples: memtable.rs and parquet_multiple_files.rs [datafusion]

2024-12-28 Thread via GitHub
alamb commented on PR #13913: URL: https://github.com/apache/datafusion/pull/13913#issuecomment-2564340863 Thanks @xudong963 and @zhuqi-lucas ❤️ for the reviews -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Consolidate Examples: memtable.rs and parquet_multiple_files.rs [datafusion]

2024-12-28 Thread via GitHub
alamb closed issue #13912: Consolidate Examples: memtable.rs and parquet_multiple_files.rs URL: https://github.com/apache/datafusion/issues/13912 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] doc-gen: migrate scalar functions (string) documentation 2/4 [datafusion]

2024-12-28 Thread via GitHub
Omega359 commented on PR #13925: URL: https://github.com/apache/datafusion/pull/13925#issuecomment-2564361309 Did you run the doc generation script prior to pushing this change? The doc diff seems ... odd -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] SQLite: Allow dollar signs in placeholder names [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on PR #1620: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1620#issuecomment-2564361341 > @iffyio Addressed all your comments, great feedback 🥇 @iffyio ++ -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] doc-gen: migrate scalar functions (string) documentation 1/4 [datafusion]

2024-12-28 Thread via GitHub
Chen-Yuan-Lai commented on PR #13924: URL: https://github.com/apache/datafusion/pull/13924#issuecomment-2564361748 @Omega359 Yes 😢 I'll run doc generation script for other PRs altogether. Thanks for reviewing 🙏 -- This is an automated message from the Apache Git Service. To respond to th

Re: [I] Automatically check "invariants" [datafusion]

2024-12-28 Thread via GitHub
alamb commented on issue #13652: URL: https://github.com/apache/datafusion/issues/13652#issuecomment-2564362086 > My impression was that the plan construction occurred with the LP (as we do), and not by constructing their own de novo physical plan. Is this correct? I am sorry -- I don

Re: [PR] doc-gen: migrate scalar functions (string) documentation 4/4 [datafusion]

2024-12-28 Thread via GitHub
Omega359 commented on code in PR #13927: URL: https://github.com/apache/datafusion/pull/13927#discussion_r1898942215 ## datafusion/functions/src/unicode/substrindex.rs: ## @@ -26,12 +26,42 @@ use arrow::datatypes::{DataType, Int32Type, Int64Type}; use crate::utils::{make_scal

Re: [PR] feat: Move shuffle block decompression and decoding to native code and add LZ4 & Snappy support [datafusion-comet]

2024-12-28 Thread via GitHub
viirya commented on PR #1192: URL: https://github.com/apache/datafusion-comet/pull/1192#issuecomment-2564394569 Hmm, the difference between ZSTD and LZ4 looks not significant? Does the benchmark of TPC-H shown in the description include native shuffle block decompression and decoding? Do w

Re: [PR] Implement predicate pruning for `like` expressions (prefix matching) [datafusion]

2024-12-28 Thread via GitHub
alamb commented on PR #12978: URL: https://github.com/apache/datafusion/pull/12978#issuecomment-2564348858 > Yeah this is what I was hinting at in https://github.com/apache/datafusion/pull/12978#issuecomment-2542335627. > > I'm happy to throw threads at it for a start, and restricting

Re: [PR] chore: Upgrade to DataFusion 44.0.0-rc2 [datafusion-comet]

2024-12-28 Thread via GitHub
andygrove commented on PR #1154: URL: https://github.com/apache/datafusion-comet/pull/1154#issuecomment-2564408017 @viirya @kazuyukitanimura This is ready for review now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Add sqlite test files into sqllogictests [datafusion]

2024-12-28 Thread via GitHub
Omega359 closed pull request #13935: Add sqlite test files into sqllogictests URL: https://github.com/apache/datafusion/pull/13935 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Fix the parsing result for the special double number [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
goldmedal commented on PR #1621: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1621#issuecomment-2564323807 Thanks @iffyio 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [I] Fix the special double number for the dialect supporting unquoted_hyphenated_identifiers [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
iffyio closed issue #1622: Fix the special double number for the dialect supporting unquoted_hyphenated_identifiers URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Fix the parsing result for the special double number [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
iffyio merged PR #1621: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

Re: [PR] Update sqllogictest requirement from 0.24.0 to 0.25.0 [datafusion]

2024-12-28 Thread via GitHub
jonahgao commented on code in PR #13917: URL: https://github.com/apache/datafusion/pull/13917#discussion_r1898912465 ## datafusion/sqllogictest/test_files/group_by.slt: ## @@ -80,7 +80,7 @@ SELECT col1 * cor0.col1 * 56 AS col1 FROM tab2 AS cor0 GROUP BY cor0.col1 208376 94136

Re: [PR] doc-gen: migrate scalar functions (string) documentation 3/4 [datafusion]

2024-12-28 Thread via GitHub
jonahgao merged PR #13926: URL: https://github.com/apache/datafusion/pull/13926 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

Re: [I] doc-gen: Migrate builtin scalar functions from code based documentation to attribute based [datafusion]

2024-12-28 Thread via GitHub
jonahgao closed issue #13671: doc-gen: Migrate builtin scalar functions from code based documentation to attribute based URL: https://github.com/apache/datafusion/issues/13671 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[PR] Correct return type for initcap scalar function with utf8view (#13909) [datafusion]

2024-12-28 Thread via GitHub
alamb opened a new pull request, #13934: URL: https://github.com/apache/datafusion/pull/13934 - Part of https://github.com/apache/datafusion/issues/13334 - Backport https://github.com/apache/datafusion/pull/13909 to `branch-44` -- This is an automated message from the Apache Gi

Re: [PR] BACKPORT: Correct return type for initcap scalar function with utf8view (#13909) [datafusion]

2024-12-28 Thread via GitHub
alamb merged PR #13934: URL: https://github.com/apache/datafusion/pull/13934 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] doc-gen: migrate scalar functions (array) documentation 1/3 [datafusion]

2024-12-28 Thread via GitHub
Omega359 commented on PR #13928: URL: https://github.com/apache/datafusion/pull/13928#issuecomment-2564359963 Can you take a look at the Clippy failure? Seems to be just an unused import -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [I] doc-gen: Migrate builtin scalar functions from code based documentation to attribute based [datafusion]

2024-12-28 Thread via GitHub
alamb commented on issue #13671: URL: https://github.com/apache/datafusion/issues/13671#issuecomment-2564360376 Thank you @Chen-Yuan-Lai and @Omega359 for driving this forward -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [I] Downloading IMDB dataset for benchmarks gives 404 Not Found [datafusion]

2024-12-28 Thread via GitHub
alamb commented on issue #13896: URL: https://github.com/apache/datafusion/issues/13896#issuecomment-2564360776 > I was wondering if there might me similar instances in other parts of our code? Should I make a PR for making this action atomic? Making downloads atomic in bench.sh seems

Re: [PR] doc-gen: migrate scalar functions (array) documentation 3/3 [datafusion]

2024-12-28 Thread via GitHub
Omega359 commented on PR #13930: URL: https://github.com/apache/datafusion/pull/13930#issuecomment-2564360684 There seems to be an issue with importing the doc macro here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] doc-gen: migrate scalar functions (array) documentation 2/3 [datafusion]

2024-12-28 Thread via GitHub
Omega359 commented on PR #13929: URL: https://github.com/apache/datafusion/pull/13929#issuecomment-2564360503 Can you fix the compilation failure please :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] doc-gen: migrate scalar functions (string) documentation 1/4 [datafusion]

2024-12-28 Thread via GitHub
Omega359 commented on PR #13924: URL: https://github.com/apache/datafusion/pull/13924#issuecomment-2564361046 I believe you may have forgotten to run the doc generation script prior to pushing this change -- This is an automated message from the Apache Git Service. To respond to the messa

[PR] Simplify `parse_keyword_apis` more [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb opened a new pull request, #1626: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1626 - Follow on to https://github.com/apache/datafusion-sqlparser-rs/pull/1618 - Part of https://github.com/apache/datafusion-sqlparser-rs/issues/1558 Rationale: @davisp made

Re: [PR] Simplify `parse_keyword_apis` more [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on code in PR #1626: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1626#discussion_r1898911096 ## src/parser/mod.rs: ## @@ -3729,27 +3729,6 @@ impl<'a> Parser<'a> { } } -/// If the current token is the `expected` keyword, consum

Re: [PR] Improve parsing speed by Avoid clone in parse_identifier [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on code in PR #1624: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1624#discussion_r1898910328 ## src/parser/mod.rs: ## @@ -13475,13 +13477,23 @@ impl<'a> Parser<'a> { } impl Word { +#[deprecated(since = "0.54.0", note = "please use `into_i

Re: [PR] Improve parsing speed by Avoid clone in parse_identifier [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on code in PR #1624: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1624#discussion_r1898910191 ## src/parser/mod.rs: ## @@ -13475,13 +13477,23 @@ impl<'a> Parser<'a> { } impl Word { +#[deprecated(since = "0.55.0", note = "please use `into_i

Re: [PR] doc-gen: migrate scalar functions (string) documentation 3/4 [datafusion]

2024-12-28 Thread via GitHub
jonahgao commented on PR #13926: URL: https://github.com/apache/datafusion/pull/13926#issuecomment-2564336634 I merged it, and maybe other related PRs can automatically run CI , which was previously blocked by “first-time contributor”. -- This is an automated message from the Apache Git

Re: [PR] Fix `sqlparser_bench` benchmark compilation [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on PR #1625: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1625#issuecomment-2564336230 > It does occur to me that this is not covered by CI. Maybe we should move the benchmark binary into the normal sqlparser crate (which I think would be more standard) rathe

Re: [PR] Update sqllogictest requirement from 0.24.0 to 0.25.0 [datafusion]

2024-12-28 Thread via GitHub
jonahgao commented on code in PR #13917: URL: https://github.com/apache/datafusion/pull/13917#discussion_r1898912465 ## datafusion/sqllogictest/test_files/group_by.slt: ## @@ -80,7 +80,7 @@ SELECT col1 * cor0.col1 * 56 AS col1 FROM tab2 AS cor0 GROUP BY cor0.col1 208376 94136

Re: [PR] doc-gen: migrate scalar functions (datetime) documentation 1/2 [datafusion]

2024-12-28 Thread via GitHub
jonahgao commented on PR #13920: URL: https://github.com/apache/datafusion/pull/13920#issuecomment-2564337685 This PR has 1 failing CI check that needs to be fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Avoid cloning during token creation [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb closed pull request #1603: Avoid cloning during token creation URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1603 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Avoid cloning during token creation [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on PR #1603: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1603#issuecomment-2564337737 We have begin work on the real thing, so closing this draft -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] feat(substrait): modular substrait producer [datafusion]

2024-12-28 Thread via GitHub
Blizzara commented on code in PR #13931: URL: https://github.com/apache/datafusion/pull/13931#discussion_r1898902943 ## datafusion/substrait/src/logical_plan/producer.rs: ## @@ -185,257 +501,290 @@ pub fn to_substrait_extended_expr( })) } -/// Convert DataFusion LogicalP

Re: [PR] doc-gen: migrate scalar functions (datetime) documentation 1/2 [datafusion]

2024-12-28 Thread via GitHub
Chen-Yuan-Lai commented on PR #13920: URL: https://github.com/apache/datafusion/pull/13920#issuecomment-2564339225 @jonahgao I found similar CI check failures in other PRs - I'll fix them all together. Thanks for reviewing 🙏 -- This is an automated message from the Apache Git Service. To

Re: [PR] Fix the parsing result for the special double number [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
alamb commented on PR #1621: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1621#issuecomment-2564364083 🚀 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] feat: Add ConfigOptions to ScalarFunctionArgs [datafusion]

2024-12-28 Thread via GitHub
Omega359 commented on PR #13527: URL: https://github.com/apache/datafusion/pull/13527#issuecomment-2564560762 Examples check failure is transient I believe. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] chore: Upgrade to DataFusion 44.0.0-rc2 [datafusion-comet]

2024-12-28 Thread via GitHub
codecov-commenter commented on PR #1154: URL: https://github.com/apache/datafusion-comet/pull/1154#issuecomment-2564426418 ## [Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/1154?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_ca

Re: [PR] chore: Upgrade to DataFusion 44.0.0-rc2 [datafusion-comet]

2024-12-28 Thread via GitHub
andygrove commented on PR #1154: URL: https://github.com/apache/datafusion-comet/pull/1154#issuecomment-2564428865 > Looks like most changes are around `PartialEq`, `Eq` and `Hash`. Is any particular change in DataFusion for this? Yes, this is due to https://github.com/apache/datafus

Re: [PR] Fix DISTINCT ON expressions match ORDER BY expressions check - normal… [datafusion]

2024-12-28 Thread via GitHub
github-actions[bot] closed pull request #13039: Fix DISTINCT ON expressions match ORDER BY expressions check - normal… URL: https://github.com/apache/datafusion/pull/13039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Fix the parsing result for the special double number [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
ayman-sigma commented on PR #1621: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1621#issuecomment-2564430894 Thanks @goldmedal , @iffyio for the quick fix! 🙏 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] feat: support `RightAnti` for `SortMergeJoin` [datafusion]

2024-12-28 Thread via GitHub
Omega359 commented on PR #13680: URL: https://github.com/apache/datafusion/pull/13680#issuecomment-2564554474 @comphead - are you able to review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] feat: Move shuffle block decompression and decoding to native code and add LZ4 & Snappy support [datafusion-comet]

2024-12-28 Thread via GitHub
andygrove commented on PR #1192: URL: https://github.com/apache/datafusion-comet/pull/1192#issuecomment-2564548965 I enabled the unsafe version of lz4_flex, and the time reduced from 340s to 336s. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[PR] refactor: Only create one native plan for a query on an executor [datafusion-comet]

2024-12-28 Thread via GitHub
viirya opened a new pull request, #1203: URL: https://github.com/apache/datafusion-comet/pull/1203 ## Which issue does this PR close? Closes #. ## Rationale for this change ## What changes are included in this PR? ## How are these changes te

Re: [PR] chore: Upgrade to DataFusion 44.0.0-rc2 [datafusion-comet]

2024-12-28 Thread via GitHub
viirya commented on PR #1154: URL: https://github.com/apache/datafusion-comet/pull/1154#issuecomment-2564476512 Looks good. I re-triggered a failed Spark Hive test. It should be just flaky one. -- This is an automated message from the Apache Git Service. To respond to the message, please

[PR] Add sqlite test files into sqllogictests [datafusion]

2024-12-28 Thread via GitHub
Omega359 opened a new pull request, #13935: URL: https://github.com/apache/datafusion/pull/13935 ## Which issue does this PR close? Closes #13812 ## Rationale for this change Add most of the sqlite test suite to Datafusion sqllogictests. Note: **THESE TESTS DO NO

[PR] Add sqlite test files into sqllogictests [datafusion]

2024-12-28 Thread via GitHub
Omega359 opened a new pull request, #13936: URL: https://github.com/apache/datafusion/pull/13936 ## Which issue does this PR close? Closes #13812 ## Rationale for this change Add most of the sqlite test suite to Datafusion sqllogictests. Note: **THESE TESTS DO NO

[I] Only create one native plan for a query on an executor [datafusion-comet]

2024-12-28 Thread via GitHub
viirya opened a new issue, #1204: URL: https://github.com/apache/datafusion-comet/issues/1204 ### What is the problem the feature request solves? Currently we create a native plan per task for a query. Even those tasks are on same executor, they still have separate native plans. For t

Re: [PR] chore: Upgrade to DataFusion 44.0.0-rc2 [datafusion-comet]

2024-12-28 Thread via GitHub
andygrove commented on PR #1154: URL: https://github.com/apache/datafusion-comet/pull/1154#issuecomment-2564513812 Thanks for the review @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] chore: Upgrade to DataFusion 44.0.0-rc2 [datafusion-comet]

2024-12-28 Thread via GitHub
andygrove merged PR #1154: URL: https://github.com/apache/datafusion-comet/pull/1154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [PR] Add sqlite test files into sqllogictests [datafusion]

2024-12-28 Thread via GitHub
Omega359 closed pull request #13932: Add sqlite test files into sqllogictests URL: https://github.com/apache/datafusion/pull/13932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Add sqlite test files into sqllogictests [datafusion]

2024-12-28 Thread via GitHub
Omega359 commented on PR #13932: URL: https://github.com/apache/datafusion/pull/13932#issuecomment-2564519315 Closing this PR to create another using a clean branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Add sqlite test files into sqllogictests [datafusion]

2024-12-28 Thread via GitHub
Omega359 commented on PR #13936: URL: https://github.com/apache/datafusion/pull/13936#issuecomment-2564545013 Related: https://github.com/apache/datafusion-testing/pull/2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[PR] support for `array_repeat` array function [datafusion-comet]

2024-12-28 Thread via GitHub
jatin510 opened a new pull request, #1205: URL: https://github.com/apache/datafusion-comet/pull/1205 ## Which issue does this PR close? Closes #. ## Rationale for this change ## What changes are included in this PR? ## How are these changes

[PR] Find a way to communicate the ordering of a file back with the existi… [datafusion]

2024-12-28 Thread via GitHub
zhuqi-lucas opened a new pull request, #13933: URL: https://github.com/apache/datafusion/pull/13933 …ng listing table implementation ## Which issue does this PR close? close [issue_13891](https://github.com/apache/datafusion/issues/13891) ## Rationale for this cha

Re: [PR] feat: add support for array_remove expression [datafusion-comet]

2024-12-28 Thread via GitHub
jatin510 commented on PR #1179: URL: https://github.com/apache/datafusion-comet/pull/1179#issuecomment-2564300648 @andygrove PR is ready -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Fix the parsing result for the special double number [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
goldmedal commented on code in PR #1621: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1621#discussion_r1898893767 ## src/tokenizer.rs: ## @@ -1141,33 +1141,44 @@ impl<'a> Tokenizer<'a> { let s2 = peeking_take_while(chars, |ch| ch.is_asci

Re: [PR] Fix the parsing result for the special double number [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
goldmedal commented on code in PR #1621: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1621#discussion_r1898894836 ## src/parser/mod.rs: ## @@ -9002,29 +9021,32 @@ impl<'a> Parser<'a> { } /// Parse a simple one-word identifier (possibly quoted, possibl

[PR] chore: set validation and type hint for ffi tableprovider [datafusion-python]

2024-12-28 Thread via GitHub
ion-elgreco opened a new pull request, #983: URL: https://github.com/apache/datafusion-python/pull/983 # Which issue does this PR close? - closes https://github.com/apache/datafusion-python/issues/950 -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] Add support for MYSQL's `RENAME TABLE` expr [datafusion-sqlparser-rs]

2024-12-28 Thread via GitHub
iffyio commented on code in PR #1616: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1616#discussion_r1898902087 ## src/ast/mod.rs: ## @@ -3401,6 +3401,13 @@ pub enum Statement { partitioned: Option>, table_format: Option, }, +/// ```sql