Messages by Thread
-
Re: [I] Missing Redshift constructs [datafusion-sqlparser-rs]
via GitHub
-
Re: [I] Discussion: guidelines for LLM-generated PR reviews [datafusion]
via GitHub
-
[PR] [branch-55] fix: preserve compound expression equivalences through projection (#25922) [datafusion]
via GitHub
-
Re: [I] `AND` chain pre-selection tests its threshold against the accumulated prefix, not the next conjunct [datafusion]
via GitHub
-
[I] Support OneRowRelation as a native Comet source [datafusion-comet]
via GitHub
-
Re: [PR] feat: Use DataFusion date_trunc for scalar-format timestamp truncation [datafusion-comet]
via GitHub
-
[PR] feat: add an experimental Comet version of RangeExec [datafusion-comet]
via GitHub
-
[PR] fix: [branch-1.1] dispatch StaticInvoke and Invoke only into Spark's own classes (#6542) [datafusion-comet]
via GitHub
-
Re: [PR] Update contributor guidelines regarding AI spam [datafusion]
via GitHub
-
Re: [PR] The great `Box`-ification experiment [datafusion-sqlparser-rs]
via GitHub
-
Re: [PR] feat: support SortAggregateExec [datafusion-comet]
via GitHub
-
[PR] fix: stop struct outputs of the codegen dispatcher from leaking Arrow memory [datafusion-comet]
via GitHub
-
Re: [PR] feat: add build-gated Hudi read integration [datafusion-ballista]
via GitHub
-
Re: [PR] feat: add Hudi contrib build gate [datafusion-ballista]
via GitHub
-
[PR] docs: explain which Spark operators Comet leaves in place [datafusion-comet]
via GitHub
-
[PR] improve Spark from_utc_timestamp compatibility [datafusion]
via GitHub
-
[I] Comet 1.2.0 Release [datafusion-comet]
via GitHub
-
[I] Native map construction doesn't match Spark 4.0+ float key normalization (-0.0 keys, missing DUPLICATED_MAP_KEY) [datafusion-comet]
via GitHub
-
[PR] feat: support native wide-decimal hashing [datafusion-comet]
via GitHub
-
[PR] perf: read the shuffle natively in operators AQE reuses from the initial plan [datafusion-comet]
via GitHub
-
[I] Internal error: GROUP BY over UNION ALL fails when a branch projects `coalesce(<nullable bool>, FALSE)` (CSE rewrite is physically nullable) [datafusion]
via GitHub
-
[I] Native Iceberg scan returns NULL for a nested field renamed after its data file was written [datafusion-comet]
via GitHub
-
[PR] feat: [branch-1.1] reuse S3 credentials until shortly before their reported expiry (#6509) [datafusion-comet]
via GitHub
-
[PR] fix: let a spilled final aggregate read its spill files back past its memory share [datafusion-comet]
via GitHub
-
Re: [I] Decide keep-or-lift for each remaining native Iceberg write eligibility restriction [datafusion-comet]
via GitHub
-
Re: [I] Call for Presentations: Community Showcase: Regular series for sharing what you're building with DataFusion [datafusion]
via GitHub
-
Re: [PR] fix: align wide-decimal ANSI overflow value with Spark [datafusion-comet]
via GitHub
-
Re: [PR] test: enable `DurationSecond` fuzzing [datafusion]
via GitHub
-
Re: [PR] Parse the DEFERRABLE transaction mode [datafusion-sqlparser-rs]
via GitHub
-
[PR] fix: fall back from the native Iceberg scan when a nested field was added or renamed [datafusion-comet]
via GitHub
-
Re: [PR] `PostgreSQL`: Support the `VARIADIC` function call argument marker [datafusion-sqlparser-rs]
via GitHub
-
Re: [PR] Added derive for arbitrary [datafusion-sqlparser-rs]
via GitHub
-
[PR] Add skewness metric to nodes that execute in partitioned mode [datafusion]
via GitHub
-
[I] Pre-select before cheap conjuncts when the undecided rows form a few long runs [datafusion]
via GitHub
-
[PR] chore: bump Ballista version to 55.0.0 [datafusion-ballista]
via GitHub
-
[PR] fix: dispatch StaticInvoke and Invoke only into Spark's own classes [datafusion-comet]
via GitHub
-
Re: [PR] fix: support struct-typed scalar subquery results [datafusion-comet]
via GitHub
-
Re: [PR] fix: preserve Spark errors for Parquet timestamp overflow [datafusion-comet]
via GitHub
-
Re: [PR] Add ExpressionAnalyzer for pluggable expression-level statistics estimation [datafusion]
via GitHub
-
Re: [PR] feat: support map inputs for explode [datafusion-comet]
via GitHub
-
Re: [I] Improve performance of first/last aggregates [datafusion-comet]
via GitHub
-
[PR] oss: add benches remaining scalars [datafusion-comet]
via GitHub
-
[PR] refactor: deprecate statistics providers that duplicate operator estimates [datafusion]
via GitHub
-
[PR] fix: [branch-1.1] defer Parquet conversion errors until a row group is decoded, as Spark does (#6515) [datafusion-comet]
via GitHub
-
[PR] fix: keep decimal fractions in modulo by one [datafusion]
via GitHub
-
Re: [I] Fuse operations in `equal_rows_arr` [datafusion]
via GitHub
-
Re: [PR] Add a fuzz target that flags superlinear parsing and printing [datafusion-sqlparser-rs]
via GitHub
-
[I] Final aggregate under AQE reads its shuffle through the JVM decoder instead of native direct read [datafusion-comet]
via GitHub
-
[PR] Js/store partitioning in dynamic filters [datafusion]
via GitHub
-
Re: [PR] fix: mark avg, bit_and/or/xor, stddev and variance as order-insensitive [datafusion]
via GitHub
-
Re: [I] Spark `xxhash64` ignores the running hash for dictionary-encoded values inside structs and lists [datafusion]
via GitHub
-
[PR] Show decimal add bug [datafusion]
via GitHub
-
Re: [PR] perf: bounded distinct count optimization [datafusion]
via GitHub
-
[PR] fix: hash every NaN like the canonical NaN in Spark xxhash64 [datafusion]
via GitHub
-
[PR] fix: array_concat panics or fails on non-default nested lists [datafusion]
via GitHub
-
Re: [PR] Capture the `COPY ... FROM STDIN` payload verbatim [datafusion-sqlparser-rs]
via GitHub
-
Re: [PR] ci: shard the Iceberg extensions test task [datafusion-comet]
via GitHub
-
Re: [PR] Add ordered bind placeholder dialect metadata [datafusion-sqlparser-rs]
via GitHub
-
Re: [PR] Support PostgreSQL `ALTER DEFAULT PRIVILEGES` [datafusion-sqlparser-rs]
via GitHub
-
Re: [PR] PostgreSQL: parse `CREATE GROUP` and `DROP GROUP` [datafusion-sqlparser-rs]
via GitHub
-
Re: [PR] feat: stage ORDER BY expression evaluation [datafusion]
via GitHub
-
[PR] perf: PartitionedTopKRank on the shared store with decide-then-gather [datafusion]
via GitHub
-
Re: [I] TPC-H SF1000 results for Ballista main (DataFusion 55.1.0): 0.72x of Spark 3.5, 1.45x of Spark + Comet, 15% of time in planning (#2497) [datafusion-ballista]
via GitHub
-
Re: [I] Proposal for more efficient disk-based shuffle mechanism [datafusion-ballista]
via GitHub
-
[I] PostgreSQL DO statement [datafusion-sqlparser-rs]
via GitHub
-
[PR] ci: Enforce PR limit to 3 for non-committers [datafusion]
via GitHub
-
[PR] fix: keep Spark's cache format when Kryo would reject Comet's or Comet disables itself [datafusion-comet]
via GitHub
-
[PR] perf: normalize float hash join build keys once [datafusion]
via GitHub
-
Re: [I] CI should run Ballista integration tests [datafusion-ballista]
via GitHub
-
Re: [I] Implement dynamic discrete pruning through a join [datafusion]
via GitHub
-
Re: [PR] fix: align nested collection buffer nullability before spilling [datafusion-comet]
via GitHub
-
Re: [PR] Plan struct colon access as get_field [datafusion]
via GitHub
-
Re: [I] fix: string-to-timestamp does not trim ISO control characters, and leading '+' returns null under ANSI [datafusion-comet]
via GitHub
-
Re: [I] panic: GROUP BY or SELECT DISTINCT over a zero-field (empty) struct key [datafusion]
via GitHub
-
Re: [PR] fix: gate the power and log rewrites on the base's value, not its nullability [datafusion]
via GitHub
-
Re: [PR] fix: skip the null-aware join when a NOT IN correlation duplicates the IN equality [datafusion]
via GitHub
-
Re: [PR] ci: drop non-existent suites from the Linux and macOS test matrices [datafusion-comet]
via GitHub
-
Re: [PR] fix: avoid query failures from inferred numeric cast bounds [datafusion]
via GitHub
-
Re: [PR] fix: preserve sorts for wrapping integer negation [datafusion]
via GitHub
-
[I] Support rolling upgrades without an outage window [datafusion-ballista]
via GitHub
-
Re: [I] Consolidate built-in statistics estimation in one place [datafusion]
via GitHub
-
[PR] feat: record why Spark scans a relation cached in Comet's format when Comet is off [datafusion-comet]
via GitHub
-
Re: [I] Preserve primitive Iceberg pruning beside unsupported complex null conjuncts [datafusion-comet]
via GitHub
-
Re: [I] Q10 SF1000 regression after #2315: SortPreservingMergeExec exhausts fair memory pool under AQE default-on [datafusion-ballista]
via GitHub
-
Re: [I] Avoid single-task execution for null-aware anti joins (NOT IN subqueries) [datafusion-ballista]
via GitHub
-
Re: [I] Implement fast path for QueryStageExec when writing 1 shuffle partition [datafusion-ballista]
via GitHub
-
Re: [I] Implement hash partitioned aggregation in Ballista [datafusion-ballista]
via GitHub
-
Re: [I] The type of Timestamp(Nanosecond, None) >= Date32 of binary physical should be same [datafusion-ballista]
via GitHub
-
Re: [I] The type of Float64 >= Int64 of binary physical should be same [datafusion-ballista]
via GitHub
-
Re: [I] Integration test script should test the Python bindings [datafusion-ballista]
via GitHub
-
[PR] fix: plan and run repeated SUM(x + literal) on decimals and intervals [datafusion]
via GitHub
-
Re: [I] Move ballista-proto to separate module/crate [datafusion-ballista]
via GitHub
-
Re: [I] Scheduler infinite loop after failed/canceled job [datafusion-ballista]
via GitHub
-
Re: [I] Add python crate to workspace [datafusion-ballista]
via GitHub
-
Re: [I] Stop showing very verbose streaming errors [datafusion-ballista]
via GitHub
-
Re: [I] Implement memory management in shuffle writer [datafusion-ballista]
via GitHub
-
Re: [I] Configuration guide should be generated from code [datafusion-ballista]
via GitHub
-
Re: [I] Ballista: Executor must return statistics in CompletedTask / CompletedJob [datafusion-ballista]
via GitHub
-
Re: [I] Broadcast join promotion drops a sort order required by a parent SortMergeJoinExec, silently producing wrong results [datafusion-ballista]
via GitHub
-
Re: [I] Enable benchmark data validation for distributed execution [datafusion-ballista]
via GitHub
-
Re: [I] Optimize shuffle before coalesce [datafusion-ballista]
via GitHub
-
Re: [I] Provide integration with Grafana / Prometheus [datafusion-ballista]
via GitHub
-
Re: [I] [Python] Allow configuration options to be set when creating BallistaContext [datafusion-ballista]
via GitHub
-
Re: [I] Add "Getting Started" docs to repo [datafusion-ballista]
via GitHub
-
Re: [PR] feat: enable Comet's in-memory cache by default [datafusion-comet]
via GitHub
-
Re: [I] Ballista: Fix hacks around concurrency=2 to force hash-partitioned joins [datafusion-ballista]
via GitHub
-
Re: [I] Ballista should serialize Parquet statistics [datafusion-ballista]
via GitHub
-
Re: [I] Remove `datafusion.proto` and use the `datafusion-proto` api for logical plan serde [datafusion-ballista]
via GitHub
-
Re: [I] Improve new top-level README for this project [datafusion-ballista]
via GitHub
-
[I] S3 credential SPI: follow-ups from #6478 (hostless Iceberg metadata locations, docs, tests) [datafusion-comet]
via GitHub
-
Re: [I] Account for partitions in local TopK statistics [datafusion]
via GitHub
-
Re: [I] date_trunc rejects scalar timestamps that array evaluation accepts [datafusion]
via GitHub
-
[PR] refactor: move shuffle planning into dedicated module [datafusion-comet]
via GitHub
-
[PR] test: strengthen scalar UDF type recovery regression coverage [datafusion]
via GitHub