richox commented on code in PR #13886:
URL: https://github.com/apache/datafusion/pull/13886#discussion_r1896512603
##
datafusion/sqllogictest/test_files/case.slt:
##
@@ -50,7 +50,7 @@ NULL
6
NULL
NULL
-7
+NULL
Review Comment:
done
--
This is an automated message from
kosiew opened a new pull request, #981:
URL: https://github.com/apache/datafusion-python/pull/981
# Which issue does this PR close?
Closes #978.
# Rationale for this change
Currently, the write_parquet method defaults to "uncompressed" Parquet
files, whi
jatin510 commented on issue #13872:
URL: https://github.com/apache/datafusion/issues/13872#issuecomment-2560679722
> I think we can return null for this case
This will be a breaking change.
@jayzhan211
cc: @alamb
--
This is an automated message from the Apache
kosiew commented on issue #978:
URL:
https://github.com/apache/datafusion-python/issues/978#issuecomment-2560675582
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
wiedld commented on code in PR #13651:
URL: https://github.com/apache/datafusion/pull/13651#discussion_r1896088892
##
datafusion/optimizer/src/optimizer.rs:
##
@@ -384,9 +394,16 @@ impl Optimizer {
// rule handles recursion itself
None =
wiedld commented on PR #13651:
URL: https://github.com/apache/datafusion/pull/13651#issuecomment-2560616923
Fixed the performance regression. It wasn't where we thought it was.
The problems was a recursive check (down the LP) of the check_fields within
the `assert_unique_field_names()
wiedld commented on code in PR #13651:
URL: https://github.com/apache/datafusion/pull/13651#discussion_r1896088892
##
datafusion/optimizer/src/optimizer.rs:
##
@@ -384,9 +394,16 @@ impl Optimizer {
// rule handles recursion itself
None =
zhuqi-lucas commented on PR #13874:
URL: https://github.com/apache/datafusion/pull/13874#issuecomment-2560602968
Thank you @alamb @Dandandan for review, it makes sense we continue
investigating the solution for communicating the ordering of a file back with
the existing listing table imple
zhuqi-lucas opened a new issue, #13891:
URL: https://github.com/apache/datafusion/issues/13891
### Is your feature request related to a problem or challenge?
We add support (order by / sort) for DataFrameWriteOptions, but when a user
try to query the table which the file already order
jonahgao commented on issue #13784:
URL: https://github.com/apache/datafusion/issues/13784#issuecomment-2560545696
If
[fail_on_overflow](https://github.com/apache/datafusion/blob/e718c1a5c5770c071c9c2e14a7681a7f1a2f3f23/datafusion/physical-expr/src/expressions/binary.rs#L88)
is enabled, Dat
goldmedal commented on PR #13856:
URL: https://github.com/apache/datafusion/pull/13856#issuecomment-2560542467
Thanks @UBarney
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific commen
goldmedal closed issue #13615: Support 1 or 3 arg in `generate_series()` UDTF
URL: https://github.com/apache/datafusion/issues/13615
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comme
goldmedal merged PR #13856:
URL: https://github.com/apache/datafusion/pull/13856
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@data
matthewmturner closed pull request #13890: Branch 43 downgraded tonic
URL: https://github.com/apache/datafusion/pull/13890
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To un
matthewmturner opened a new pull request, #13890:
URL: https://github.com/apache/datafusion/pull/13890
## Which issue does this PR close?
Closes #.
## Rationale for this change
## What changes are included in this PR?
## Are these changes te
Omega359 commented on code in PR #13876:
URL: https://github.com/apache/datafusion/pull/13876#discussion_r1896196677
##
.github/workflows/rust.yml:
##
@@ -288,17 +318,20 @@ jobs:
mv *.tbl ../datafusion/sqllogictest/test_files/tpch/data
- name: Verify that bench
Omega359 commented on code in PR #13876:
URL: https://github.com/apache/datafusion/pull/13876#discussion_r1896192661
##
.github/actions/setup-builder/action.yaml:
##
@@ -42,6 +42,8 @@ runs:
"${RETRY[@]}" rustup component add rustfmt
- name: Configure rust runtime e
Omega359 commented on code in PR #13876:
URL: https://github.com/apache/datafusion/pull/13876#discussion_r1896187659
##
.github/actions/setup-builder/action.yaml:
##
@@ -42,6 +42,8 @@ runs:
"${RETRY[@]}" rustup component add rustfmt
- name: Configure rust runtime e
alamb commented on issue #13845:
URL: https://github.com/apache/datafusion/issues/13845#issuecomment-2560403938
Thanks for the ideas @edmondop -- FWIW the ASF doens't have its own hosted
workers (individual projects could do that -- arrow did for a time -- but it
requires dedicated time /
Omega359 commented on code in PR #13876:
URL: https://github.com/apache/datafusion/pull/13876#discussion_r1896184558
##
.github/actions/setup-rust-runtime/action.yaml:
##
@@ -34,5 +34,6 @@ runs:
echo "RUSTC_WRAPPER=sccache" >> $GITHUB_ENV
echo "SCCACHE_GHA_EN
alamb commented on code in PR #13876:
URL: https://github.com/apache/datafusion/pull/13876#discussion_r1896174869
##
.github/actions/setup-rust-runtime/action.yaml:
##
@@ -34,5 +34,6 @@ runs:
echo "RUSTC_WRAPPER=sccache" >> $GITHUB_ENV
echo "SCCACHE_GHA_ENABL
alamb opened a new pull request, #13889:
URL: https://github.com/apache/datafusion/pull/13889
This is an experiment on top of
- https://github.com/apache/datafusion/pull/13876
To test how much a difference the rust cache crate actually makes
--
This is an automated messa
alamb commented on code in PR #13874:
URL: https://github.com/apache/datafusion/pull/13874#discussion_r1896169061
##
datafusion/core/src/dataframe/parquet.rs:
##
@@ -74,8 +74,16 @@ impl DataFrame {
let file_type = format_as_file_type(format);
+let plan = if
alamb commented on PR #13878:
URL: https://github.com/apache/datafusion/pull/13878#issuecomment-2560387220
Marking as draft as I think this PR is no longer waiting on feedback. Please
ping us on the other PRs 🙏
Thanks aain @Chen-Yuan-Lai
--
This is an automated message from the A
jwimberl closed issue #10020: Panic when querying a hive-partitioned parquet
dataset created with wrong column name
URL: https://github.com/apache/datafusion/issues/10020
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use th
jwimberl commented on issue #10020:
URL: https://github.com/apache/datafusion/issues/10020#issuecomment-2560386636
Belatedly confirmed that this issue is now resolved and using the wrong
partition column name yields an error `Exception: DataFusion error:
Plan("Inferred partitions to be ..."
alamb merged PR #13879:
URL: https://github.com/apache/datafusion/pull/13879
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb commented on PR #13879:
URL: https://github.com/apache/datafusion/pull/13879#issuecomment-2560378852
Thanks again @zhuliquan and @berkaysynnada 🚀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
alamb commented on code in PR #13886:
URL: https://github.com/apache/datafusion/pull/13886#discussion_r1896165275
##
datafusion/sqllogictest/test_files/case.slt:
##
@@ -50,7 +50,7 @@ NULL
6
NULL
NULL
-7
+NULL
Review Comment:
Can you also please add a test to this file whe
Omega359 commented on issue #13845:
URL: https://github.com/apache/datafusion/issues/13845#issuecomment-2560352170
> > Maintaining an extended workflow shouldn't be too bad tbh. I think
having a workflow that runs outside of PR's (iow runs nightly) could be useful
as well for expensive test
Omega359 commented on issue #13816:
URL: https://github.com/apache/datafusion/issues/13816#issuecomment-2560327632
```
[profile.release]
codegen-units = 1
strip = true
panic = "abort"
opt-level = "s"
```
Expanding on @comphead's idea adding opt-level = "s" reduced the
jwimberl commented on issue #9797:
URL: https://github.com/apache/datafusion/issues/9797#issuecomment-2560326293
Apologies for the delay -- this issue no longer reproduces using the
datafusion version 43! I did not attempt to find where between [37,43] the
resolution happened but could if y
alamb commented on PR #13778:
URL: https://github.com/apache/datafusion/pull/13778#issuecomment-2560323839
Here is a PR to try and fix this:
https://github.com/apache/datafusion/pull/13887
--
This is an automated message from the Apache Git Service.
To respond to the message, please log o
edmondop commented on issue #13845:
URL: https://github.com/apache/datafusion/issues/13845#issuecomment-2560320481
Other alternatives:
# Split test executions in multiple jobs
We can generate a docker image with the code and the tests compiled, and
then having separate jobs that pu
alamb commented on code in PR #13887:
URL: https://github.com/apache/datafusion/pull/13887#discussion_r1895897853
##
datafusion/common/Cargo.toml:
##
@@ -36,12 +36,11 @@ name = "datafusion_common"
path = "src/lib.rs"
[features]
-default = ["recursive-protection"]
avro = ["a
alamb commented on PR #13883:
URL: https://github.com/apache/datafusion/pull/13883#issuecomment-2560316188
Thanks again @xarus01
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comm
alamb merged PR #13883:
URL: https://github.com/apache/datafusion/pull/13883
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb closed issue #13843: Consolidate Example: simplify_udwf_expression.rs
into advanced_udwf.rs #13842
URL: https://github.com/apache/datafusion/issues/13843
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
Omega359 commented on PR #13527:
URL: https://github.com/apache/datafusion/pull/13527#issuecomment-2560262193
I think this may be ready for review again. For this round I refactored the
code to use &ConfigOptions everywhere except for ScalarFunctionExpr so the cost
for cloning ConfigOptions
wiedld commented on PR #13651:
URL: https://github.com/apache/datafusion/pull/13651#issuecomment-2560247024
We no longer have any per-optimizer-pass validation in release mode.
Instead, the checks run before and after ALL optimizer passes are completed.
When running in debug mode, the
robtandy commented on PR #13888:
URL: https://github.com/apache/datafusion/pull/13888#issuecomment-2560251488
Thank you @vbarua and @Blizzara for your review and comments!
Yes, I think the existing tests do not go far enough, and I encountered bugs
not covered by these tests already,
wiedld commented on code in PR #13651:
URL: https://github.com/apache/datafusion/pull/13651#discussion_r1896089589
##
datafusion/optimizer/src/optimizer.rs:
##
@@ -445,35 +462,38 @@ impl Optimizer {
}
i += 1;
}
+
+// verify LP is valid,
wiedld commented on code in PR #13651:
URL: https://github.com/apache/datafusion/pull/13651#discussion_r1896088892
##
datafusion/optimizer/src/optimizer.rs:
##
@@ -384,9 +394,16 @@ impl Optimizer {
// rule handles recursion itself
None =
timsaucer commented on PR #12754:
URL: https://github.com/apache/datafusion/pull/12754#issuecomment-2560224119
I'm planning on taking a look at this over the next week or so, but it will
take a little time for me to get up to speed on the details of what you're
trying to do here. Can you ad
Blizzara commented on PR #13888:
URL: https://github.com/apache/datafusion/pull/13888#issuecomment-2560223118
Haha, looks like @vbarua commented pretty pretty much the same thing while I
was writing my own reply! 😄
--
This is an automated message from the Apache Git Service.
To respond t
Blizzara commented on PR #13888:
URL: https://github.com/apache/datafusion/pull/13888#issuecomment-2560214683
I like the idea, more testing the better! We already have some Substrait TCP
testing, but I think that's from "known Substrait" -> DF, so it only tests the
consumer, while this woul
vbarua commented on code in PR #13888:
URL: https://github.com/apache/datafusion/pull/13888#discussion_r1896067593
##
datafusion/substrait/tests/testdata/tpch_queries/query_01.sql:
##
@@ -0,0 +1,22 @@
+SELECT
+l_returnflag,
+l_linestatus,
+sum(l_quantity) AS sum_qty,
vbarua commented on PR #13888:
URL: https://github.com/apache/datafusion/pull/13888#issuecomment-2560209539
There are already some tests for TPCH functionality in
https://github.com/apache/datafusion/blob/main/datafusion/substrait/tests/cases/consumer_integration.rs,
but IMO those are weake
vbarua commented on code in PR #13888:
URL: https://github.com/apache/datafusion/pull/13888#discussion_r1896058000
##
datafusion/substrait/tests/testdata/tpch_queries/query_01.sql:
##
@@ -0,0 +1,22 @@
+SELECT
+l_returnflag,
+l_linestatus,
+sum(l_quantity) AS sum_qty,
Dandandan commented on code in PR #1192:
URL: https://github.com/apache/datafusion-comet/pull/1192#discussion_r1896016551
##
native/core/benches/shuffle_writer.rs:
##
@@ -35,23 +35,42 @@ fn criterion_benchmark(c: &mut Criterion) {
group.bench_function("shuffle_writer: encod
Dandandan commented on code in PR #1192:
URL: https://github.com/apache/datafusion-comet/pull/1192#discussion_r1896016551
##
native/core/benches/shuffle_writer.rs:
##
@@ -35,23 +35,42 @@ fn criterion_benchmark(c: &mut Criterion) {
group.bench_function("shuffle_writer: encod
mvzink commented on code in PR #1538:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1538#discussion_r1896016157
##
src/parser/mod.rs:
##
@@ -11375,7 +11466,11 @@ impl<'a> Parser<'a> {
} else {
let object_type =
self.parse_one
andygrove commented on PR #1192:
URL:
https://github.com/apache/datafusion-comet/pull/1192#issuecomment-2560087256
@Dandandan you may be interested in the benchmark results
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
andygrove commented on PR #1181:
URL:
https://github.com/apache/datafusion-comet/pull/1181#issuecomment-2560034413
LZ4 support is now part of
https://github.com/apache/datafusion-comet/pull/1192
--
This is an automated message from the Apache Git Service.
To respond to the message, pleas
andygrove closed pull request #1181: feat: Add support for LZ4 compression
URL: https://github.com/apache/datafusion-comet/pull/1181
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comme
Omega359 commented on issue #13812:
URL: https://github.com/apache/datafusion/issues/13812#issuecomment-2559955748
Apparently there isn't a way to fork an empty repository in github, or at
least I can't seem to find a way so I'm unable to push a PR there for the slt
files. Can someone with
robtandy opened a new pull request, #13888:
URL: https://github.com/apache/datafusion/pull/13888
## Which issue does this PR close?
I've been investigating and experimenting with federating tpch query plans
and sending the federated portion downstream encoded as substrait protos (Side
Omega359 commented on issue #13885:
URL: https://github.com/apache/datafusion/issues/13885#issuecomment-2559893186
possibly related: #13779
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the spe
alamb commented on PR #13887:
URL: https://github.com/apache/datafusion/pull/13887#issuecomment-2559860749
Also before I merge this I want to rename the flag to `recursive_protection`
(underscore) to be consistent with the rest
--
This is an automated message from the Apache Git Service.
alamb commented on code in PR #13887:
URL: https://github.com/apache/datafusion/pull/13887#discussion_r1895840813
##
datafusion/core/Cargo.toml:
##
@@ -69,6 +69,13 @@ pyarrow = ["datafusion-common/pyarrow", "parquet"]
regex_expressions = [
"datafusion-functions/regex_expre
Omega359 commented on PR #13876:
URL: https://github.com/apache/datafusion/pull/13876#issuecomment-2559855782
Thanks for the update - no rush on my side
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
alamb commented on PR #13887:
URL: https://github.com/apache/datafusion/pull/13887#issuecomment-2559851448
Given this experience, we really need a test for the `recursive-protection`
feature flag / compiling datafusion with various feature flags. Maybe someone
can figure out how to do so, f
alamb commented on PR #13876:
URL: https://github.com/apache/datafusion/pull/13876#issuecomment-2559850885
I can not wait to test / review this but I think I will run out of time
today -- hopefully either later today or tomorrow
--
This is an automated message from the Apache Git Service.
alamb opened a new pull request, #13887:
URL: https://github.com/apache/datafusion/pull/13887
## Which issue does this PR close?
- Closes https://github.com/apache/datafusion/issues/13766
## Rationale for this change
The recursive-protect flag was added in
https://github
alamb commented on issue #13766:
URL: https://github.com/apache/datafusion/issues/13766#issuecomment-2559842765
I think this issue is not quite closed by
https://github.com/apache/datafusion/pull/13778, reopening
See
https://github.com/apache/datafusion/issues/13334#issuecomment-25585769
alamb merged PR #13881:
URL: https://github.com/apache/datafusion/pull/13881
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb closed issue #13763: Make migration to `Signature::nullary` in 44.0.0
easier / less confusing
URL: https://github.com/apache/datafusion/issues/13763
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
alamb commented on PR #13881:
URL: https://github.com/apache/datafusion/pull/13881#issuecomment-2559813476
Thank you for the review @jayzhan211
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th
zhuliquan commented on PR #13789:
URL: https://github.com/apache/datafusion/pull/13789#issuecomment-2559812675
> I ran your benchmark and it looks good. I am unsure though the benefit of
this benchmark over the existing 'regx' benchmark which targets the udf
versions of the Postgresql symbo
alamb commented on PR #13881:
URL: https://github.com/apache/datafusion/pull/13881#issuecomment-2559813030
here is a PR to just update the signatures:
- https://github.com/apache/datafusion/pull/13881
--
This is an automated message from the Apache Git Service.
To respond to the message
alamb commented on issue #13334:
URL: https://github.com/apache/datafusion/issues/13334#issuecomment-2559811428
> Since @buraksenn did the work to make this optional in [minor] make
recursive package dependency optional #13778, perhaps they have some idea?
Update I think I know what
alamb commented on PR #1154:
URL:
https://github.com/apache/datafusion-comet/pull/1154#issuecomment-2559808577
I found the issue:
- https://github.com/apache/datafusion/pull/13778#issuecomment-2559808094
Working on a fix
--
This is an automated message from the Apache Git Servi
alamb commented on PR #13778:
URL: https://github.com/apache/datafusion/pull/13778#issuecomment-2559808094
While testing this in comet, I am pretty sure this PR didn't quite fix the
problem
-
https://github.com/apache/datafusion-comet/pull/1154#issuecomment-2559777474
When someone
alamb commented on PR #1154:
URL:
https://github.com/apache/datafusion-comet/pull/1154#issuecomment-2559777474
> I have upgraded to latest DF and reinstated the miri checks. I'll keep
this in draft until the RC is available later this week.
I see the MIRI CI check fails:
http
alamb commented on PR #12978:
URL: https://github.com/apache/datafusion/pull/12978#issuecomment-2559758701
This is my top priority after DF 44 is released:
- #13334
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use t
alamb commented on PR #12978:
URL: https://github.com/apache/datafusion/pull/12978#issuecomment-2559758189
This is still on my list, hopefully other people can check it out too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
alamb commented on PR #13310:
URL: https://github.com/apache/datafusion/pull/13310#issuecomment-2559754026
BTW we have been having problems downstream wiht the new recursive / stacker
dependency. See for example:
- https://github.com/apache/datafusion/issues/13513
- https://github.com/
phillipleblanc commented on code in PR #13880:
URL: https://github.com/apache/datafusion/pull/13880#discussion_r1895780031
##
datafusion/sql/src/unparser/extension_unparser.rs:
##
@@ -0,0 +1,66 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contrib
freshtonic commented on code in PR #1614:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1614#discussion_r1895681055
##
src/parser/mod.rs:
##
@@ -9604,7 +9604,13 @@ impl<'a> Parser<'a> {
top = Some(self.parse_top()?);
}
-let projecti
freshtonic commented on code in PR #1614:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1614#discussion_r1895681055
##
src/parser/mod.rs:
##
@@ -9604,7 +9604,13 @@ impl<'a> Parser<'a> {
top = Some(self.parse_top()?);
}
-let projecti
richox opened a new pull request, #13886:
URL: https://github.com/apache/datafusion/pull/13886
## Which issue does this PR close?
Closes #13885 .
## Rationale for this change
## What changes are included in this PR?
## Are these changes test
iffyio commented on code in PR #1538:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1538#discussion_r1895651779
##
src/parser/mod.rs:
##
@@ -3869,11 +3873,16 @@ impl<'a> Parser<'a> {
.is_some();
let persistent = dialect_of!(self is DuckDbDial
yuval-illumex commented on code in PR #1604:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1604#discussion_r1895655809
##
src/keywords.rs:
##
@@ -941,6 +941,7 @@ pub const RESERVED_FOR_TABLE_ALIAS: &[Keyword] = &[
// Reserved for Snowflake table sample
K
iffyio commented on code in PR #1604:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1604#discussion_r1895636641
##
src/ast/query.rs:
##
@@ -2790,3 +2790,14 @@ impl fmt::Display for ValueTableMode {
}
}
}
+
+/// The update table from options
Review
richox opened a new issue, #13885:
URL: https://github.com/apache/datafusion/issues/13885
### Describe the bug
some queries containing `case when .. ELSE NULL end` produces incorrect
results.
### To Reproduce
preparing test table:
```sql
$ datafusion-cli
DataFus
berkaysynnada commented on code in PR #13879:
URL: https://github.com/apache/datafusion/pull/13879#discussion_r1895488724
##
datafusion/physical-plan/src/repartition/mod.rs:
##
@@ -170,7 +170,7 @@ impl RepartitionExecState {
/// which is commonly set to the number of CPU cores
dependabot[bot] opened a new pull request, #13884:
URL: https://github.com/apache/datafusion/pull/13884
Updates the requirements on
[sqllogictest](https://github.com/risinglightdb/sqllogictest-rs) to permit the
latest version.
Release notes
Sourced from https://github.com/risingli
xarus01 commented on PR #13883:
URL: https://github.com/apache/datafusion/pull/13883#issuecomment-2559110139
@alamb - mentioning per [contribution
guide](https://datafusion.apache.org/contributor-guide/index.html#pull-request-overview).
Please trigger the CI tasks.
--
This is an automat
88 matches
Mail list logo
