Weijun-H commented on PR #13683:
URL: https://github.com/apache/datafusion/pull/13683#issuecomment-2525400134
> We follow DuckDB for array function mostly, the best I can think of is
implementing spark function in
[datafusion-contrib/datafusion-functions-extra](https://github.com/datafusion
jayzhan211 commented on issue #13519:
URL: https://github.com/apache/datafusion/issues/13519#issuecomment-2525424413
I guess we can also add `nullable` info to `ScalarFunctionArgs`
https://github.com/apache/datafusion/issues/11923
--
This is an automated message from the Apache Git Se
Kimahriman commented on PR #13683:
URL: https://github.com/apache/datafusion/pull/13683#issuecomment-2525401068
Interesting thing is that DuckDB says they based it on the PrestoDB
behavior: https://github.com/duckdb/duckdb/pull/3065
But a quick look at the PrestoDB implementation sugg
jayzhan211 commented on issue #12905:
URL: https://github.com/apache/datafusion/issues/12905#issuecomment-2525406814
I'm thinking about make `ValuesExec` partitioned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
jayzhan211 commented on PR #13683:
URL: https://github.com/apache/datafusion/pull/13683#issuecomment-2525402657
> Interesting thing is that DuckDB says they based it on the PrestoDB
behavior: [duckdb/duckdb#3065](https://github.com/duckdb/duckdb/pull/3065)
>
> But a quick look at the
jayzhan211 merged PR #13394:
URL: https://github.com/apache/datafusion/pull/13394
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@dat
jayzhan211 commented on issue #13099:
URL: https://github.com/apache/datafusion/issues/13099#issuecomment-2525483850
> > > @alamb I would really appreciate any advice you could give when you
have a moment.
> >
> >
> > I think we would have to get some detailed profiling to really
jayzhan211 commented on PR #13394:
URL: https://github.com/apache/datafusion/pull/13394#issuecomment-2525480746
Thanks @jiashenC @alamb @Omega359
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
Kimahriman commented on code in PR #13683:
URL: https://github.com/apache/datafusion/pull/13683#discussion_r1874582555
##
datafusion/sqllogictest/test_files/array.slt:
##
@@ -5260,6 +5270,13 @@ select array_has([], null),
NULL NULL NULL
+# If lhs is has any Nulls, we re
sunchao commented on code in PR #1143:
URL: https://github.com/apache/datafusion-comet/pull/1143#discussion_r1874543948
##
native/core/src/data_type.rs:
##
@@ -1,241 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements
andygrove merged PR #1151:
URL: https://github.com/apache/datafusion-comet/pull/1151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@
Kimahriman opened a new pull request, #13683:
URL: https://github.com/apache/datafusion/pull/13683
## Which issue does this PR close?
Closes #13682
## Rationale for this change
Makes null handling for `array_has` consistent across scalars and arrays,
and mak
andygrove commented on issue #1133:
URL:
https://github.com/apache/datafusion-ballista/issues/1133#issuecomment-2525226778
It looks like it is between 3 & 4.
@pinarbayata would it be possible to see one more variation - option 3 but
using the colors from option 4?
--
This is an au
jayzhan211 commented on PR #13683:
URL: https://github.com/apache/datafusion/pull/13683#issuecomment-2525371434
We follow DuckDB for array function, I think the better approach is having a
easy way to customize the function
--
This is an automated message from the Apache Git Service.
To r
alamb commented on code in PR #13637:
URL: https://github.com/apache/datafusion/pull/13637#discussion_r1874553258
##
datafusion-examples/examples/advanced_udf.rs:
##
@@ -191,6 +199,51 @@ impl ScalarUDFImpl for PowUdf {
}
}
+/// Evaluate `base ^ exp` *without* allocating
alamb commented on code in PR #13683:
URL: https://github.com/apache/datafusion/pull/13683#discussion_r1874558578
##
datafusion/functions-nested/src/array_has.rs:
##
@@ -215,7 +215,11 @@ fn array_has_dispatch_for_array(
let needle_row = Scalar::new(needle.slice(i, 1));
viirya commented on code in PR #1143:
URL: https://github.com/apache/datafusion-comet/pull/1143#discussion_r1874555050
##
native/core/src/data_type.rs:
##
@@ -1,241 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements.
dependabot[bot] opened a new pull request, #966:
URL: https://github.com/apache/datafusion-python/pull/966
Bumps [pyo3-build-config](https://github.com/pyo3/pyo3) from 0.22.6 to
0.23.3.
Release notes
Sourced from https://github.com/pyo3/pyo3/releases";>pyo3-build-config's
releases
dependabot[bot] opened a new pull request, #968:
URL: https://github.com/apache/datafusion-python/pull/968
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.41.1 to 1.42.0.
Release notes
Sourced from https://github.com/tokio-rs/tokio/releases";>tokio's releases.
Tokio v
dependabot[bot] opened a new pull request, #967:
URL: https://github.com/apache/datafusion-python/pull/967
Bumps [prost](https://github.com/tokio-rs/prost) from 0.13.3 to 0.13.4.
Changelog
Sourced from https://github.com/tokio-rs/prost/blob/master/CHANGELOG.md";>prost's
changelog.
findepi merged PR #13685:
URL: https://github.com/apache/datafusion/pull/13685
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafu
milenkovicm opened a new pull request, #1148:
URL: https://github.com/apache/datafusion-ballista/pull/1148
# Which issue does this PR close?
Closes #1107.
# Rationale for this change
# What changes are included in this PR?
- added basic classes for scheduler & exe
andygrove merged PR #1147:
URL: https://github.com/apache/datafusion-ballista/pull/1147
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr.
davisp commented on issue #1557:
URL:
https://github.com/apache/datafusion-sqlparser-rs/issues/1557#issuecomment-2525239854
@alamb How are you generating your flamegraphs?
Locally, `cargo flamegraph --bench sqlparser_bench` does not appear to be
doing the trick as the flamegraph appe
findepi commented on issue #13661:
URL: https://github.com/apache/datafusion/issues/13661#issuecomment-2525246516
> It is my understanding that the apache voting / approval process prevents
automated builds
That's my understanding too, but i hope this process isn't nonnegotiable.
mbutrovich commented on PR #1150:
URL:
https://github.com/apache/datafusion-comet/pull/1150#issuecomment-2525245173
I am curious what's going on in this scenario in particular:
```
Sorted Lg Str Filter Scan (50.0% zeros): Best Time(ms) Avg Time(ms)
Stdev(ms)Rate(M/s) Per Ro
findepi commented on code in PR #13672:
URL: https://github.com/apache/datafusion/pull/13672#discussion_r1874533180
##
datafusion-cli/tests/integration_setup.bash:
##
@@ -0,0 +1,16 @@
+# you should have localstack up, e.g by
+#$
LOCALSTACK_VERSION=sha256:a0b79cb2430f1818de2c66c
findepi commented on code in PR #13576:
URL: https://github.com/apache/datafusion/pull/13576#discussion_r1874535271
##
datafusion/common/src/config.rs:
##
@@ -973,16 +980,24 @@ impl ConfigField for Option {
#[macro_export]
macro_rules! config_field {
-($t:ty) => {
+(
andygrove commented on code in PR #1152:
URL: https://github.com/apache/datafusion-comet/pull/1152#discussion_r1874535391
##
native/spark-expr/src/sum_decimal.rs:
##
@@ -286,18 +284,16 @@ struct SumDecimalGroupsAccumulator {
sum: Vec,
result_type: DataType,
precis
andygrove opened a new pull request, #1152:
URL: https://github.com/apache/datafusion-comet/pull/1152
## Which issue does this PR close?
Part of https://github.com/apache/datafusion-comet/issues/659
## Rationale for this change
We do not publish the core `
findepi opened a new pull request, #13685:
URL: https://github.com/apache/datafusion/pull/13685
## Which issue does this PR close?
none
## Rationale for this change
- remove unused dependencies
- move user_doc macro from top-level lib.rs to module (in prep for more
ma
davisp opened a new pull request, #1581:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1581
I mostly did this as an exercise to get a general feel of how the Parser
implementation is organized. The basics here are that for every top level
keyword in Parser::parse_statement I c
davisp commented on PR #1581:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1581#issuecomment-2525327041
I originally had intentions on spending today trying to see if I couldn't
figure out how to help move #1561 forward, but my usual attempt at commenting
out the non-clone A
comphead commented on code in PR #13683:
URL: https://github.com/apache/datafusion/pull/13683#discussion_r1874573459
##
datafusion/sqllogictest/test_files/array.slt:
##
@@ -5260,6 +5270,13 @@ select array_has([], null),
NULL NULL NULL
+# If lhs is has any Nulls, we retu
2010YOUY01 commented on PR #13581:
URL: https://github.com/apache/datafusion/pull/13581#issuecomment-2525258213
> Hi, I think this one is pretty close, do you have time to look at the
review comments @2010YOUY01 ?
Yes, I will be back and finish this PR in next 2 days, I'm traveling an
Kimahriman commented on code in PR #13683:
URL: https://github.com/apache/datafusion/pull/13683#discussion_r1874569927
##
datafusion/functions-nested/src/array_has.rs:
##
@@ -215,7 +215,11 @@ fn array_has_dispatch_for_array(
let needle_row = Scalar::new(needle.slice(i,
jonahgao opened a new pull request, #13684:
URL: https://github.com/apache/datafusion/pull/13684
## Which issue does this PR close?
Part of https://github.com/apache/datafusion/issues/11687
## Rationale for this change
Replace `OnceLock` with `LazyLock`.
##
findepi commented on code in PR #13651:
URL: https://github.com/apache/datafusion/pull/13651#discussion_r1874536239
##
datafusion/optimizer/src/optimizer.rs:
##
@@ -451,6 +468,33 @@ impl Optimizer {
}
}
+/// These are invariants to hold true for each logical plan.
+/// D
findepi commented on issue #13679:
URL: https://github.com/apache/datafusion/issues/13679#issuecomment-2525252358
> ParquetSink (used for `COPY TO`) encodes bytes to parquet and writes to
the sink (e.g. object store). It currently does not include retry logic for
failed multipart PUTs to ob
rluvaton commented on PR #13520:
URL: https://github.com/apache/datafusion/pull/13520#issuecomment-2525342446
@Omega359 do you wanna take this as you have better configuration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
findepi commented on issue #7548:
URL: https://github.com/apache/datafusion/issues/7548#issuecomment-2525243892
@AdamGS +1 from me.
The average data size sounds most logical from optimizer's perspective
(i was involved in the introduction of ColumnStatistics.dataSize of
Presto/Trino,
alamb commented on PR #13581:
URL: https://github.com/apache/datafusion/pull/13581#issuecomment-2525285569
I also harbor hopes of contributing a benchmark for `corr`, hopefully
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
comphead commented on code in PR #13684:
URL: https://github.com/apache/datafusion/pull/13684#discussion_r1874552141
##
datafusion/macros/src/lib.rs:
##
@@ -235,27 +248,23 @@ pub fn user_doc(args: TokenStream, input: TokenStream) ->
TokenStream {
}
});
-let
comphead commented on code in PR #13684:
URL: https://github.com/apache/datafusion/pull/13684#discussion_r1874552306
##
datafusion/macros/src/lib.rs:
##
@@ -26,16 +26,19 @@ use syn::{parse_macro_input, DeriveInput, LitStr};
/// declared on `AggregateUDF`, `WindowUDFImpl`, `Scal
alamb commented on code in PR #13637:
URL: https://github.com/apache/datafusion/pull/13637#discussion_r1868550990
##
datafusion/physical-expr/src/scalar_function.rs:
##
@@ -134,20 +134,20 @@ impl PhysicalExpr for ScalarFunctionExpr {
}
fn evaluate(&self, batch: &Reco
dependabot[bot] opened a new pull request, #969:
URL: https://github.com/apache/datafusion-python/pull/969
Bumps [prost-types](https://github.com/tokio-rs/prost) from 0.13.3 to 0.13.4.
Changelog
Sourced from https://github.com/tokio-rs/prost/blob/master/CHANGELOG.md";>prost-types's
dependabot[bot] closed pull request #961: build(deps): bump pyo3-build-config
from 0.22.6 to 0.23.2
URL: https://github.com/apache/datafusion-python/pull/961
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
dependabot[bot] commented on PR #961:
URL:
https://github.com/apache/datafusion-python/pull/961#issuecomment-2525294073
Superseded by #966.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the sp
charmfocus opened a new issue, #1582:
URL: https://github.com/apache/datafusion-sqlparser-rs/issues/1582
```rust
let sql = "RENAME TABLE `test`.`test` TO `test`.`test2`";
// parse to a Statement
let dialect = GenericDialect {};
let ast = Parser::parse_sql(&dial
timsaucer commented on PR #48:
URL: https://github.com/apache/datafusion-site/pull/48#issuecomment-2525385983
@alamb I believe this is ready to go. I resolved the issue with the infra
team. We had to make an adjustment to the .asf.yaml file within the `asf-site`
branch. I had to make some u
7phs commented on code in PR #1576:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1576#discussion_r1874399142
##
tests/sqlparser_redshift.rs:
##
@@ -353,3 +380,23 @@ fn test_parse_json_path_from() {
_ => panic!(),
}
}
+
+#[test]
+fn test_parse_selec
alamb commented on issue #13334:
URL: https://github.com/apache/datafusion/issues/13334#issuecomment-2525081316
I would personally love to see DataFusion 44.0.0 be lauded as "super stable"
and have few upgrade issues (we would largely achieve this by testing upgrades
with other projects pri
alamb opened a new pull request, #46:
URL: https://github.com/apache/datafusion-site/pull/46
I am testing the instructions on
https://github.com/apache/datafusion-site/pull/44
Also I aspire to write a post about 44.0.0's release
--
This is an automated message from the Apache Git S
alamb opened a new pull request, #45:
URL: https://github.com/apache/datafusion-site/pull/45
This is an aspirational PR with a DataFusion 44.0.0 release announcement
It is mostly to test out https://github.com/apache/datafusion-site/pull/44
but I figured I would at least pretend I wou
alamb commented on PR #45:
URL: https://github.com/apache/datafusion-site/pull/45#issuecomment-2525083301
I didn't name the branch correctly, closing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
alamb closed pull request #45: Add DataFusion 44.0.0 blog post
URL: https://github.com/apache/datafusion-site/pull/45
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubsc
alamb opened a new pull request, #47:
URL: https://github.com/apache/datafusion-site/pull/47
I am testing the instructions on
https://github.com/apache/datafusion-site/pull/44
Also I aspire to write a post about 44.0.0's release
--
This is an automated message from the Apache Git S
alamb closed pull request #46: Add DataFusion 44.0.0 blog post
URL: https://github.com/apache/datafusion-site/pull/46
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubsc
alamb commented on PR #46:
URL: https://github.com/apache/datafusion-site/pull/46#issuecomment-2525084254
Staging push failed:
-
https://github.com/apache/datafusion-site/actions/runs/12212614636/job/34071269677?pr=46
--
This is an automated message from the Apache Git Service.
To res
haohuaijin commented on code in PR #13560:
URL: https://github.com/apache/datafusion/pull/13560#discussion_r1874368912
##
datafusion/sqllogictest/test_files/joins.slt:
##
@@ -4313,3 +4313,86 @@ physical_plan
04)--HashJoinExec: mode=CollectLeft, join_type=Inner, on=[(binary_
iffyio commented on code in PR #1576:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1576#discussion_r1874372151
##
tests/sqlparser_redshift.rs:
##
@@ -353,3 +380,23 @@ fn test_parse_json_path_from() {
_ => panic!(),
}
}
+
+#[test]
+fn test_parse_sel
irenjj opened a new pull request, #13680:
URL: https://github.com/apache/datafusion/pull/13680
## Which issue does this PR close?
Closes #13472
## Rationale for this change
## What changes are included in this PR?
## Are these changes tested
alamb commented on code in PR #44:
URL: https://github.com/apache/datafusion-site/pull/44#discussion_r1874404718
##
pelicanconf.py:
##
@@ -37,9 +37,10 @@
ARCHIVES_SAVE_AS = ''
# Disable articles by pointing to a (should-be-absent) subdir
ARTICLE_PATHS = [ 'blog' ]
-# needed t
alamb commented on code in PR #13663:
URL: https://github.com/apache/datafusion/pull/13663#discussion_r1874406962
##
datafusion/common/src/pyarrow.rs:
##
@@ -17,6 +17,10 @@
//! Conversions between PyArrow and DataFusion types
+// TODO update to pyo3 new APIs
Review Comment
Rachelint opened a new pull request, #13681:
URL: https://github.com/apache/datafusion/pull/13681
## Which issue does this PR close?
Closes #.
## Rationale for this change
## What changes are included in this PR?
## Are these changes tested?
timsaucer merged PR #44:
URL: https://github.com/apache/datafusion-site/pull/44
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@dataf
timsaucer closed issue #13: Automate deploying the website
URL: https://github.com/apache/datafusion-site/issues/13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscri
alamb commented on issue #1133:
URL:
https://github.com/apache/datafusion-ballista/issues/1133#issuecomment-2525104848
I also like option 4:
--
This is an automated message from the Apache G
timsaucer commented on PR #48:
URL: https://github.com/apache/datafusion-site/pull/48#issuecomment-2525128812
There is still a problem with the new site serving up when I attempt to
remove the old site content. I have opened a ticket with the infra team:
https://issues.apache.org/jira/brows
Dandandan commented on PR #13581:
URL: https://github.com/apache/datafusion/pull/13581#issuecomment-2525138937
Hi, I think this one is pretty close, do you have time to look at the review
comments @2010YOUY01 ?
--
This is an automated message from the Apache Git Service.
To respond to the
Kimahriman opened a new issue, #13682:
URL: https://github.com/apache/datafusion/issues/13682
### Describe the bug
`array_has` handles nulls differently for scalars and arrays. Scalars return
null if all the left hand side values are null, while arrays don't care what
values in the l
korowa closed issue #12328: The file with non-standard newline character can't
be read when sqllogictests testing
URL: https://github.com/apache/datafusion/issues/12328
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
korowa merged PR #13677:
URL: https://github.com/apache/datafusion/pull/13677
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafus
73 matches
Mail list logo
