github  

Messages by Thread

• [I] Report parquet decompression time as a scan metric, split by dictionary and data pages [datafusion] via GitHub
• [PR] [branch-55] Preserve field metadata when coercing INT96 timestamps (#24790) [datafusion] via GitHub
  • Re: [PR] [branch-55] Preserve field metadata when coercing INT96 timestamps (#24790) [datafusion] via GitHub
  • Re: [PR] [branch-55] Preserve field metadata when coercing INT96 timestamps (#24790) [datafusion] via GitHub
  • Re: [PR] [branch-55] Preserve field metadata when coercing INT96 timestamps (#24790) [datafusion] via GitHub
• [PR] feat: support whole-value Variant scan pushdown [datafusion-comet] via GitHub
• [I] Broadcast/hash join fallback reasons are lost from the AQE-final plan [datafusion-comet] via GitHub
• [I] Add explicit phase boundaries to optimizer rules for simplicity and safety [datafusion] via GitHub
• [PR] fix: gate unsupported Iceberg S3 write settings [datafusion-comet] via GitHub
  • Re: [PR] fix: gate unsupported Iceberg S3 write settings [datafusion-comet] via GitHub
• [I] Recursive CTE silently casts recursive term to the static term's type, producing wrong results or a non-terminating query [datafusion] via GitHub
• [PR] fix: preserve functional dependencies across unnest [datafusion] via GitHub
  • Re: [PR] fix: preserve functional dependencies across unnest [datafusion] via GitHub
  • Re: [PR] fix: preserve functional dependencies across unnest [datafusion] via GitHub
• [PR] fix: free and account for native shuffle spill metadata [datafusion-comet] via GitHub
  • Re: [PR] fix: free and account for native shuffle spill metadata [datafusion-comet] via GitHub
  • Re: [PR] fix: free and account for native shuffle spill metadata [datafusion-comet] via GitHub
• Re: [PR] feat(physical-plan): expose finalized Accumulator state on BoundedWindowAggExec [datafusion] via GitHub
• Re: [PR] feat: memoize for FFI_PartitionEvaluator [datafusion] via GitHub
• [PR] fix: preserve grouping set member names during type coercion [datafusion] via GitHub
  • Re: [PR] fix: preserve grouping set member names during type coercion [datafusion] via GitHub
• [I] Group by rollup/cube/etc.. fails with new type_coercion [datafusion] via GitHub
• [I] CatalogProvider Limitations: Query Scoped Resolution [datafusion-iceberg] via GitHub
• Re: [I] Avoid temporary heap allocation when copying off-heap UTF8String into Arrow [datafusion-comet] via GitHub
• [PR] fix: remove nested planner placeholders after recursive conversion [datafusion-comet] via GitHub
  • Re: [PR] fix: remove nested planner placeholders after recursive conversion [datafusion-comet] via GitHub
  • Re: [PR] fix: remove nested planner placeholders after recursive conversion [datafusion-comet] via GitHub
  • Re: [PR] fix: remove nested planner placeholders after recursive conversion [datafusion-comet] via GitHub
  • Re: [PR] fix: remove nested planner placeholders after recursive conversion [datafusion-comet] via GitHub
• [I] Nested planner placeholders survive cleanup after aggregate fallback [datafusion-comet] via GitHub
  • Re: [I] Nested planner placeholders survive cleanup after aggregate fallback [datafusion-comet] via GitHub
• [PR] perf: retain batched build storage for large hash joins [datafusion] via GitHub
  • Re: [PR] perf: retain batched build storage for large hash joins [datafusion] via GitHub
• [PR] fix: avoid divide-by-zero in `avg` for negative-scale decimals [datafusion] via GitHub
• Re: [PR] feat: log drift between MemoryPool reservations and allocated bytes in SLT [datafusion] via GitHub
• [PR] feat: carry completed join filters across Union inputs [datafusion-comet] via GitHub
• [PR] fix: [branch-1.0] format NativeMemoryConsumer id in toString (#5398) [datafusion-comet] via GitHub
  • Re: [PR] fix: [branch-1.0] format NativeMemoryConsumer id in toString (#5398) [datafusion-comet] via GitHub
  • Re: [PR] fix: [branch-1.0] format NativeMemoryConsumer id in toString (#5398) [datafusion-comet] via GitHub
• [PR] refactor: separate JNI-free core logic and error classification at the native boundary [datafusion-comet] via GitHub
  • Re: [PR] refactor: separate JNI-free core logic and error classification at the native boundary [datafusion-comet] via GitHub
  • Re: [PR] refactor: separate JNI-free core logic and error classification at the native boundary [datafusion-comet] via GitHub
• [I] Separate JNI-free core logic and error classification from native JNI entry points [datafusion-comet] via GitHub
• [PR] feat: filter decoded probe batches before intermediate joins [datafusion-comet] via GitHub
• [PR] feat: Extend adaptive partial aggregation to more aggregate functions [datafusion-comet] via GitHub
• Re: [I] Projected IN subqueries plan two extra nested-loop mark joins per subquery and become quadratic [datafusion] via GitHub
• [PR] feat: error when the client and scheduler have different major versions [datafusion-ballista] via GitHub
• [PR] feat: opt in to join filter pushdown through deterministic residuals [datafusion-comet] via GitHub
• [I] [EPIC] Distributed execution with Ballista [datafusion-iceberg] via GitHub
  • Re: [I] [EPIC] Distributed Execution for Reads/Writes [datafusion-iceberg] via GitHub
  • Re: [I] [EPIC] Distributed Execution for Reads/Writes [datafusion-iceberg] via GitHub
  • Re: [I] [EPIC] Distributed Execution for Reads/Writes [datafusion-iceberg] via GitHub
  • Re: [I] [EPIC] Distributed Execution for Reads/Writes [datafusion-iceberg] via GitHub
  • Re: [I] [EPIC] Distributed Execution for Reads/Writes [datafusion-iceberg] via GitHub
• [PR] Information schema extension [datafusion] via GitHub
  • Re: [PR] Information schema extension [datafusion] via GitHub
• [I] Observed metrics (Dataset.observe) on a cached DataFrame are lost with Comet's native cache scan [datafusion-comet] via GitHub
• Re: [I] Match Spark ordering and rank semantics for floating values nested in arrays and structs [datafusion-comet] via GitHub
  • Re: [I] Match Spark ordering and rank semantics for floating values nested in arrays and structs [datafusion-comet] via GitHub
• [I] Error when the Python client and the cluster have different major versions [datafusion-ballista] via GitHub
• Re: [PR] feat(parquet): eager pruning during planning to refine scan statistics [datafusion] via GitHub
• [PR] fix: bump DataFusion to 54.1.0 and adapt to RecursiveQuery API change (#1645) [datafusion-python] via GitHub
  • Re: [PR] fix: bump DataFusion to 54.1.0 and adapt to RecursiveQuery API change (#1645) [datafusion-python] via GitHub
• [PR] chore(release): release the Python client separately from the Rust crates [datafusion-ballista] via GitHub
• Re: [I] Build fails against DataFusion 54.1.0 due to new `schema` field on `RecursiveQuery` [datafusion-python] via GitHub
• [PR] perf: specialize scattered Boolean shuffle gathers [datafusion-comet] via GitHub
  • Re: [PR] perf: specialize scattered Boolean shuffle gathers [datafusion-comet] via GitHub
• Re: [PR] WIP: Extension type casts using extension registry [datafusion] via GitHub
  • Re: [PR] WIP: Extension type casts using extension registry [datafusion] via GitHub
  • Re: [PR] WIP: Extension type casts using extension registry [datafusion] via GitHub
  • Re: [PR] WIP: Extension type casts using extension registry [datafusion] via GitHub
• [PR] feat: support inputless aggregate UDFs [datafusion] via GitHub
  • Re: [PR] feat: support inputless aggregate UDFs [datafusion] via GitHub
  • Re: [PR] feat: add row-count-aware accumulator interfaces [datafusion] via GitHub
  • Re: [PR] feat: add row-count-aware accumulator interfaces [datafusion] via GitHub
• [I] Async UDF returning a scalar fails with "all columns in a record batch must have the same length" [datafusion] via GitHub
  • Re: [I] Async UDF returning a scalar fails with "all columns in a record batch must have the same length" [datafusion] via GitHub
  • Re: [I] Async UDF returning a scalar fails with "all columns in a record batch must have the same length" [datafusion] via GitHub
• [PR] feat: expose per-scan S3 HTTP attempts and retries [datafusion-comet] via GitHub
• [PR] fix: preserve positional nested types and named struct nullability [datafusion-comet] via GitHub
  • Re: [PR] fix: preserve positional nested types and named struct nullability [datafusion-comet] via GitHub
• [PR] [branch-54] chore: repair CI issues on branch-54 [datafusion-python] via GitHub
  • Re: [PR] [branch-54] chore: repair CI issues on branch-54 [datafusion-python] via GitHub
• [PR] fix: summarize Bloom literals in filter plan displays [datafusion-comet] via GitHub
  • Re: [PR] fix: summarize Bloom literals in filter plan displays [datafusion-comet] via GitHub
  • Re: [PR] fix: summarize Bloom literals in filter plan displays [datafusion-comet] via GitHub
• Re: [I] Memory leak in error cases in JVM-Rust (and vice versa) transition for C2R [datafusion-comet] via GitHub
• [I] Support inputless native Rust aggregate UDFs with explicit row counts [datafusion] via GitHub
  • Re: [I] Support inputless native Rust aggregate UDFs with explicit row counts [datafusion] via GitHub
• Re: [PR] Feat/map sql extension types [datafusion] via GitHub
• Re: [PR] fix: surface SparkArithmeticException(DIVIDE_BY_ZERO) for divide-by-zero in dispatched ScalaUDF path [datafusion-comet] via GitHub
• Re: [I] Ballista 55.0.0 Release [datafusion-ballista] via GitHub
• [PR] test: cover integer bounds in cache pruning tests [datafusion-comet] via GitHub
  • Re: [PR] test: cover integer bounds in cache pruning tests [datafusion-comet] via GitHub
• [I] Discussion: release the Python client separately from the Rust crates [datafusion-ballista] via GitHub
  • Re: [I] Discussion: release the Python client separately from the Rust crates [datafusion-ballista] via GitHub
  • Re: [I] Discussion: release the Python client separately from the Rust crates [datafusion-ballista] via GitHub
  • Re: [I] Discussion: release the Python client separately from the Rust crates [datafusion-ballista] via GitHub
  • Re: [I] Discussion: release the Python client separately from the Rust crates [datafusion-ballista] via GitHub
  • Re: [I] Discussion: release the Python client separately from the Rust crates [datafusion-ballista] via GitHub
• [PR] feat(scheduler): add a setting to disable the shared file statistics cache [datafusion-ballista] via GitHub
  • Re: [PR] feat(scheduler): add a setting to disable the shared file statistics cache [datafusion-ballista] via GitHub
• [I] Typed Dataset.filter returns wrong rows over a native aggregate [datafusion-comet] via GitHub
  • Re: [I] Typed Dataset.filter returns wrong rows over a native aggregate [datafusion-comet] via GitHub
  • Re: [I] Typed Dataset.filter returns wrong rows over a native aggregate [datafusion-comet] via GitHub
• [PR] chore(deps): bump taiki-e/install-action from 2.87.16 to 2.87.21 [datafusion] via GitHub
  • Re: [PR] chore(deps): bump taiki-e/install-action from 2.87.16 to 2.87.21 [datafusion] via GitHub
• Re: [PR] feat: support `WITH ... AS [NOT] MATERIALIZED` CTEs [datafusion] via GitHub
  • Re: [PR] feat: support `WITH ... AS [NOT] MATERIALIZED` CTEs [datafusion] via GitHub
  • Re: [PR] feat: support `WITH ... AS [NOT] MATERIALIZED` CTEs [datafusion] via GitHub
  • Re: [PR] feat: support `WITH ... AS [NOT] MATERIALIZED` CTEs [datafusion] via GitHub
  • Re: [PR] feat: support `WITH ... AS [NOT] MATERIALIZED` CTEs [datafusion] via GitHub
  • Re: [PR] feat: support `WITH ... AS [NOT] MATERIALIZED` CTEs [datafusion] via GitHub
  • Re: [PR] feat: support `WITH ... AS [NOT] MATERIALIZED` CTEs [datafusion] via GitHub
• [I] Decimal-returning DSv2 functions lose their scale through the codegen dispatcher [datafusion-comet] via GitHub
• [I] Native regr_* aggregates return wrong results for a constant variable merged from several partitions [datafusion-comet] via GitHub
• [I] Native Iceberg years/months/days/hours differ from Iceberg for some pre-1970 timestamps [datafusion-comet] via GitHub
• [PR] fix: Remove `serde_json/preserve_order` feature from library crates [datafusion] via GitHub
  • Re: [PR] fix: Remove `serde_json/preserve_order` feature from library crates [datafusion] via GitHub
  • Re: [PR] fix: Remove `serde_json/preserve_order` feature from library crates [datafusion] via GitHub
  • Re: [PR] fix: Remove `serde_json/preserve_order` feature from library crates [datafusion] via GitHub
  • Re: [PR] fix: Remove `serde_json/preserve_order` feature from library crates [datafusion] via GitHub
  • Re: [PR] fix: Remove `serde_json/preserve_order` feature from library crates [datafusion] via GitHub
• [PR] fix: [branch-1.0] revert unsafe partial aggregates after final fallback (#5421) [datafusion-comet] via GitHub
  • Re: [PR] fix: [branch-1.0] revert unsafe partial aggregates after final fallback (#5421) [datafusion-comet] via GitHub
• [I] ClickHouse: tuple element access followed by a field changes meaning after round-trip [datafusion-sqlparser-rs] via GitHub
• [PR] ClickHouse: Support GLOBAL IN / GLOBAL NOT IN [datafusion-sqlparser-rs] via GitHub
  • Re: [PR] ClickHouse: Support GLOBAL IN / GLOBAL NOT IN [datafusion-sqlparser-rs] via GitHub
• [I] Side effects from `serde_json/preserve_order` feature impact user applications [datafusion] via GitHub
  • Re: [I] Side effects from `serde_json/preserve_order` feature impact user applications [datafusion] via GitHub
  • Re: [I] Side effects from `serde_json/preserve_order` feature impact user applications [datafusion] via GitHub
  • Re: [I] Side effects from `serde_json/preserve_order` feature impact user applications [datafusion] via GitHub
  • Re: [I] Side effects from `serde_json/preserve_order` feature impact user applications [datafusion] via GitHub
• [I] ClickHouse: Support `GLOBAL IN` / `GLOBAL NOT IN` [datafusion-sqlparser-rs] via GitHub
• [PR] fix: keep Spark's cache scan for a relation whose cached plan records observed metrics [datafusion-comet] via GitHub
  • Re: [PR] fix: keep Spark's cache scan for a relation whose cached plan records observed metrics [datafusion-comet] via GitHub
  • Re: [PR] fix: keep Spark's cache scan for a relation whose cached plan records observed metrics [datafusion-comet] via GitHub
• [PR] Update changelog and version number to preprare for patch release [datafusion-python] via GitHub
  • Re: [PR] [branch-54] Update changelog and version number to preprare for patch release [datafusion-python] via GitHub
  • Re: [PR] [branch-54] Update changelog and version number to preprare for patch release [datafusion-python] via GitHub
• [PR] fix: [branch-1.0] log partial memory grants at DEBUG and drop the memory usage dump (#6269) [datafusion-comet] via GitHub
  • Re: [PR] fix: [branch-1.0] log partial memory grants at DEBUG and drop the memory usage dump (#6269) [datafusion-comet] via GitHub
  • Re: [PR] fix: [branch-1.0] log partial memory grants at DEBUG and drop the memory usage dump (#6269) [datafusion-comet] via GitHub
  • Re: [PR] fix: [branch-1.0] log partial memory grants at DEBUG and drop the memory usage dump (#6269) [datafusion-comet] via GitHub
• [I] Investigate TPC-DS q54 and q68 slowdowns in the 1.1.0 benchmark on Spark 4.2 [datafusion-comet] via GitHub
  • Re: [I] Investigate TPC-DS q54 and q68 slowdowns in the 1.1.0 benchmark on Spark 4.2 [datafusion-comet] via GitHub
  • Re: [I] Investigate TPC-DS q54 and q68 slowdowns in the 1.1.0 benchmark on Spark 4.2 [datafusion-comet] via GitHub
• [PR] docs: promote Spark 4.2 from experimental to supported [datafusion-comet] via GitHub
• [I] ci: raise the merge queue's build concurrency from 2 to 4 [datafusion-comet] via GitHub
  • Re: [I] ci: raise the merge queue's build concurrency from 2 to 4 [datafusion-comet] via GitHub
• Re: [I] CometHashAggregateExec doesn't participate in Spark's AQEPropagateEmptyRelation optimization [datafusion-comet] via GitHub
• Re: [PR] fix: scope information schema to catalog [datafusion] via GitHub
  • Re: [PR] fix: scope information schema to catalog [datafusion] via GitHub
  • Re: [PR] fix: scope information schema to catalog [datafusion] via GitHub
  • Re: [PR] fix: scope information schema to catalog [datafusion] via GitHub
• Re: [PR] chore: Fix Scala code warnings [datafusion-comet] via GitHub
• Re: [I] Unparsing of filter depending on projection generates invalid SQL [datafusion] via GitHub
• Re: [PR] docs: add version picker and manual release snapshots [datafusion] via GitHub
• [PR] fix: add native metrics from every plan a task runs instead of keeping the last [datafusion-comet] via GitHub
  • Re: [PR] fix: add native metrics from every plan a task runs instead of keeping the last [datafusion-comet] via GitHub
• Re: [I] Dictionary performance improvement tickets [datafusion] via GitHub
• Re: [PR] Allow the `HLLAccumulator` used by `approx_distinct()` to consume state produced by `HllGroupsAccumulator` [datafusion] via GitHub
  • Re: [PR] Allow the `HLLAccumulator` used by `approx_distinct()` to consume state produced by `HllGroupsAccumulator` [datafusion] via GitHub
  • Re: [PR] Allow the `HLLAccumulator` used by `approx_distinct()` to consume state produced by `HllGroupsAccumulator` [datafusion] via GitHub
  • Re: [PR] Allow the `HLLAccumulator` used by `approx_distinct()` to consume state produced by `HllGroupsAccumulator` [datafusion] via GitHub
  • Re: [PR] Allow the `HLLAccumulator` used by `approx_distinct()` to consume state produced by `HllGroupsAccumulator` [datafusion] via GitHub
• [PR] test: run Comet in Spark SQL and Iceberg suites that build their own sessions [datafusion-comet] via GitHub
• [PR] fix: handle scalar results from async UDFs [datafusion] via GitHub
  • Re: [PR] fix: handle scalar results from async UDFs [datafusion] via GitHub
  • Re: [PR] fix: handle scalar results from async UDFs [datafusion] via GitHub
• [I] Comet shuffle does not report Spark 4.1's order-independent shuffle checksum [datafusion-comet] via GitHub
• [PR] ci: raise the merge queue's build concurrency from 2 to 4 [datafusion-comet] via GitHub
  • Re: [PR] ci: raise the merge queue's build concurrency from 2 to 4 [datafusion-comet] via GitHub
  • Re: [PR] ci: raise the merge queue's build concurrency from 2 to 4 [datafusion-comet] via GitHub
• Re: [I] Epic: Improve TPC-H and TPC-DS cardinality estimates [datafusion] via GitHub
• [PR] docs: publish 55.0.0 and 55.1.0 documentation snapshots [datafusion] via GitHub
• [PR] docs: preserve released documentation during deployment [datafusion] via GitHub
• [PR] perf: add dict fast path to make_scalar_function [datafusion] via GitHub
  • Re: [PR] perf: add dict fast path to make_scalar_function [datafusion] via GitHub
• [PR] fix: hash NaN the way Spark does in hash, xxhash64 and native shuffle [datafusion-comet] via GitHub
  • Re: [PR] fix: hash NaN the way Spark does in hash, xxhash64 and native shuffle [datafusion-comet] via GitHub
  • Re: [PR] fix: hash NaN the way Spark does in hash, xxhash64 and native shuffle [datafusion-comet] via GitHub
  • Re: [PR] fix: hash NaN the way Spark does in hash, xxhash64 and native shuffle [datafusion-comet] via GitHub
  • Re: [PR] fix: hash NaN the way Spark does in hash, xxhash64 and native shuffle [datafusion-comet] via GitHub
• [PR] fix: report a cached relation's decoded size to the planner, not its compressed size [datafusion-comet] via GitHub
  • Re: [PR] fix: report a cached relation's decoded size to the planner, not its compressed size [datafusion-comet] via GitHub
  • Re: [PR] fix: report a cached relation's decoded size to the planner, not its compressed size [datafusion-comet] via GitHub
• [I] Comet's in-memory cache reports its compressed size to the planner, which changes join planning [datafusion-comet] via GitHub
• [PR] chore(deps): bump webpack-dev-middleware from 8.0.4 to 8.3.0 in /datafusion/wasmtest/datafusion-wasm-app [datafusion] via GitHub
  • Re: [PR] chore(deps): bump webpack-dev-middleware from 8.0.4 to 8.3.0 in /datafusion/wasmtest/datafusion-wasm-app [datafusion] via GitHub
• Re: [I] beautify default column names [datafusion] via GitHub
  • Re: [I] beautify default column names [datafusion] via GitHub
  • Re: [I] beautify default column names [datafusion] via GitHub
• Re: [PR] fix: do not prune ORDER BY keys using nullable UNIQUE constraints [datafusion] via GitHub
  • Re: [PR] fix: do not prune ORDER BY keys using nullable UNIQUE constraints [datafusion] via GitHub
• Re: [PR] IN LIST: treat signed zeros as equal [datafusion] via GitHub
  • Re: [PR] IN LIST: treat signed zeros as equal [datafusion] via GitHub
  • Re: [PR] IN LIST: treat signed zeros as equal [datafusion] via GitHub
  • Re: [PR] IN LIST: treat signed zeros as equal [datafusion] via GitHub
  • Re: [PR] IN LIST: treat signed zeros as equal [datafusion] via GitHub

• Earlier messages
• Later messages