Messages by Thread
-
-
[jira] [Created] (SPARK-59322) Add an optional maximum nesting depth for XML parsing
Hyukjin Kwon (Jira)
-
[jira] [Created] (SPARK-59320) Optionally require an expiration claim in JWSFilter
Hyukjin Kwon (Jira)
-
[jira] [Created] (SPARK-59318) Add opt-in restrictions for data source client options, schema URLs, and class loading
Hyukjin Kwon (Jira)
-
[jira] [Created] (SPARK-59319) Add an opt-in restricted SQL execution mode
Hyukjin Kwon (Jira)
-
[jira] [Updated] (SPARK-59306) Redact all JAAS credential styles in Kafka parameter logging
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-59308) Avoid integer overflow when computing ML feature and image sizes
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-59309) Add spark.sql.ml.maxNumFeatures to bound feature counts read from data
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-59307) Detect cyclic node ids when loading decision tree and bisecting k-means models
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-59312) Add spark.yarn.am.trustProxyUserCookie option to AmIpFilter
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-59311) Add spark.sql.avro.catalystTypeParsingMaxDepth to bound Catalyst type parsing
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-57990) Extend shared get_json_object parsing to array-index paths and object-or-array fallbacks
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-59187) SPJ across differently named struct partition keys: a join throws and a union drops rows
Peter Toth (Jira)
-
[jira] [Updated] (SPARK-59315) New resolution logic for dropDuplicates(WithinWatermark) introduced correctness issue with metadata column
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-58265) Reuse projected broadcast values for dynamic partition pruning
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-59317) Unwrap the cast of a DSv2 scalar subquery runtime filter before pushing it down
Cheng Pan (Jira)
-
[jira] [Created] (SPARK-59317) Unwrap the cast of a DSv2 scalar subquery runtime filter before pushing it down
Cheng Pan (Jira)
-
[jira] [Updated] (SPARK-57175) Extend nested column pruning to exists and forall over arrays of structs
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58272) Enable runtime Bloom filters for safely materialized cached join inputs
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-36082) when the right side is small enough to use SingleColumn Null Aware Anti Join
Dongjoon Hyun (Jira)
-
[jira] [Assigned] (SPARK-59314) Add `Heterogeneous Executor Management` section to `running-on-kubernetes.md`
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-59314) Add `Heterogeneous Executor Management` section to `running-on-kubernetes.md`
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58834) Upgrade docker action version to follow ASF allowlist
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-35158) Add some guides for authors to retrigger the workflow run
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-56903) Spread NULL outer join keys across shuffle partitions
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-56874) Fast-path DateTimeUtils.daysToMicros for ZoneOffset.UTC
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-57043) Collapse grouped SUM(COUNT(1)) rollups in RemoveRedundantAggregates
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-57108) Skip core/utils build matrix entry for unrelated changes
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-37019) Add Codegen support to array higher-order-functions
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-56868) Refactor runtime partition filtering into a helper method
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58202) Skip driver containerPort declaration when the port is 0
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-57642) Require predicateSql to be present for the DSv2 CHECK constraint
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-57176) Extend nested column pruning through array-returning functions
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-57626) Extend shared get_json_object parsing to nested named paths
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-57282) Spread NULL left anti join keys across shuffle partitions
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58378) Fuse compatible approximate percentile sketches
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-57485) Exclude package-private Scala types from the generated Javadoc
Dongjoon Hyun (Jira)
-
[jira] [Assigned] (SPARK-59316) Remove the obsolete `-Pvolcano` build instruction from K8s docs
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-59310) Report driver-side grouping metrics for GroupPartitionsExec
ASF GitHub Bot (Jira)
-
[jira] [Resolved] (SPARK-59316) Remove the obsolete `-Pvolcano` build instruction from K8s docs
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-59316) Remove the obsolete `-Pvolcano` build instruction from K8s docs
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59316) Remove the obsolete `-Pvolcano` build instruction from K8s docs
Dongjoon Hyun (Jira)
-
[jira] [Commented] (SPARK-59315) New resolution logic for dropDuplicates(WithinWatermark) introduced correctness issue with metadata column
Jungtaek Lim (Jira)
-
[jira] [Created] (SPARK-59315) New resolution logic for dropDuplicates(WithinWatermark) introduced correctness issue with metadata column
Jungtaek Lim (Jira)
-
[jira] [Updated] (SPARK-59314) Add `Heterogeneous Executor Management` section to `running-on-kubernetes.md`
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59314) Add `Heterogeneous Executor Management` section to `running-on-kubernetes.md`
Dongjoon Hyun (Jira)
-
[jira] [Closed] (SPARK-33276) Fix K8s IT Flakiness
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-33276) Fix K8s IT Flakiness
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-53509) Publish Apache Spark 4.0.1 to docker registry
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-54994) Publish Apache Spark `4.1.1` to docker registry
Dongjoon Hyun (Jira)
-
[jira] [Closed] (SPARK-47126) Re-enable Spark 3.4 test in HiveExternalCatalogVersionsSuite
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-47126) Re-enable Spark 3.4 test in HiveExternalCatalogVersionsSuite
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-51336) Upgrade `upload-artifact` to v4 in `spark-docker` repository
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-51345) Remove all EOL versions from `spark-docker` repository
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-51345) Remove all EOL versions from `spark-docker` repository
Dongjoon Hyun (Jira)
-
[jira] [Assigned] (SPARK-59303) Support `spark.kubernetes.executor.resizeMaxMemory` in `ExecutorResizePlugin`
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-59303) Support `spark.kubernetes.executor.resizeMaxMemory` in `ExecutorResizePlugin`
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-59304) Support `spark.kubernetes.executor.pvc.resizeMaxStorage` in `ExecutorPVCResizePlugin`
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-59304) Support `spark.kubernetes.executor.pvc.resizeMaxStorageinExecutorPVCResizePlugin`
Dongjoon Hyun (Jira)
-
[jira] [Assigned] (SPARK-59304) Support `spark.kubernetes.executor.pvc.resizeMaxStorageinExecutorPVCResizePlugin`
Dongjoon Hyun (Jira)
-
[jira] [Created] (SPARK-59313) Eagerly initialize the ANN model in MultilayerPerceptronClassificationModel
Ruifeng Zheng (Jira)
-
[jira] [Created] (SPARK-59312) Add spark.yarn.am.trustProxyUserCookie option to AmIpFilter
Hyukjin Kwon (Jira)
-
[jira] [Created] (SPARK-59311) Add spark.sql.avro.catalystTypeParsingMaxDepth to bound Catalyst type parsing
Hyukjin Kwon (Jira)
-
[jira] [Created] (SPARK-59310) Report driver-side grouping metrics for GroupPartitionsExec
XiDuo You (Jira)
-
[jira] [Created] (SPARK-59309) Add spark.sql.ml.maxNumFeatures to bound feature counts read from data
Hyukjin Kwon (Jira)
-
[jira] [Created] (SPARK-59308) Avoid integer overflow when computing ML feature and image sizes
Hyukjin Kwon (Jira)
-
[jira] [Created] (SPARK-59307) Detect cyclic node ids when loading decision tree and bisecting k-means models
Hyukjin Kwon (Jira)
-
[jira] [Created] (SPARK-59306) Redact all JAAS credential styles in Kafka parameter logging
Hyukjin Kwon (Jira)
-
[jira] [Resolved] (SPARK-59270) Run the JWT auth end-to-end walkthrough in CI
L. C. Hsieh (Jira)
-
[jira] [Resolved] (SPARK-59248) Keep storage-partitioned join when partition keys are pruned from the scan output
XiDuo You (Jira)
-
[jira] [Assigned] (SPARK-59248) Keep storage-partitioned join when partition keys are pruned from the scan output
XiDuo You (Jira)
-
[jira] [Resolved] (SPARK-59302) Include the LICENSE file in the maturin sdist so PyPI accepts it
Hyukjin Kwon (Jira)
-
[jira] [Updated] (SPARK-59302) Include the LICENSE file in the maturin sdist so PyPI accepts it
ASF GitHub Bot (Jira)
-
[jira] [Assigned] (SPARK-59302) Include the LICENSE file in the maturin sdist so PyPI accepts it
Hyukjin Kwon (Jira)
-
[jira] [Updated] (SPARK-59277) [SQL][HIVE] Support first-class CHAR/VARCHAR in Hive extension surfaces
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-59305) Track Apache Spark 4.3.0: retarget CI to the RC, re-vendor upstream, and close 4.3.0 client gaps
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59305) Track Apache Spark 4.3.0: retarget CI to the RC, re-vendor upstream, and close 4.3.0 client gaps
Hyukjin Kwon (Jira)
-
[jira] [Updated] (SPARK-59304) Support `spark.kubernetes.executor.pvc.resizeMaxStorageinExecutorPVCResizePlugin`
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59304) Support `spark.kubernetes.executor.pvc.resizeMaxStorageinExecutorPVCResizePlugin`
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-59303) Support `spark.kubernetes.executor.resizeMaxMemory` in `ExecutorResizePlugin`
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59303) Support `spark.kubernetes.executor.resizeMaxMemory` in `ExecutorResizePlugin`
Dongjoon Hyun (Jira)
-
[jira] [Created] (SPARK-59302) Include the LICENSE file in the maturin sdist so PyPI accepts it
Hyukjin Kwon (Jira)
-
[jira] [Updated] (SPARK-58772) SelectedField drops column metadata during nested column pruning
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58864) Suggest customSchema / custom JdbcDialect in the UNRECOGNIZED_SQL_TYPE error
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-59276) [CONNECT] Preserve CHAR/VARCHAR collations and define client compatibility
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-51356) FilterExec incorrectly reorders IsNotNull predicates for nested access
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58865) Include an example in the JDBC hint option validation error message
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58916) Handle infinite relativeError in approxQuantile
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58420) COUNT(DISTINCT ...) does not sort maps in the argument trees
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58625) Support pa.ChunkedArray columns in PandasToArrowConversion
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-47397) count_distinct ignores null values
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58672) Reject CHAR/VARCHAR map keys in variant casts
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58615) Avoid temporary column conflicts in FPGrowthModel transform
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-58696) [SQL] CTE predicate push-down discards filters inferred by other optimizer rules
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-41246) When RddId exceeds Integer.Max_VALUE, RddId becomes negative
Dongjoon Hyun (Jira)
-
[jira] [Updated] (SPARK-27052) Using PySpark udf in transform yields NULL values
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-59263) Upgrade `compress-lzf` to 1.2.1
Dongjoon Hyun (Jira)
-
[jira] [Assigned] (SPARK-59263) Upgrade `compress-lzf` to 1.2.1
Dongjoon Hyun (Jira)
-
[jira] [Resolved] (SPARK-57938) Print active SKIP_ flags when Ruby doc build script runs
Nicholas Chammas (Jira)
-
[jira] [Updated] (SPARK-59135) Spark Connect command streams fail on valid non-dictionary ExecutePlan responses
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-59301) Unwrap the cast of a DSv2 runtime IN filter before pushing it down
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59301) Unwrap the cast of a DSv2 runtime IN filter before pushing it down
Cheng Pan (Jira)
-
[jira] [Updated] (SPARK-59293) Fix update_build_status.yml to map non-standard workflow run conclusions
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-59299) Add missing ASOF JOIN parser unit tests
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-59300) Support nanosecond-precision timestamps in from_utc_timestamp and to_utc_timestamp
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59300) Support nanosecond-precision timestamps in from_utc_timestamp and to_utc_timestamp
Stevo Mitric (Jira)
-
[jira] [Created] (SPARK-59299) Add missing ASOF JOIN parser unit tests
Luka Zdravic (Jira)
-
[jira] [Created] (SPARK-59298) ASOF JOIN production-readiness: improve test coverage and QA
Luka Zdravic (Jira)
-
[jira] [Updated] (SPARK-59297) Cover nanosecond precisions 7 and 8 in RowToColumnConverter round-trip tests
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59297) Cover nanosecond precisions 7 and 8 in RowToColumnConverter round-trip tests
Stevo Mitric (Jira)
-
[jira] [Updated] (SPARK-59296) Apply OIDC provider-declared properties on the driver by separating provider selection from credential resolution
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59296) Apply OIDC provider-declared properties on the driver by separating provider selection from credential resolution
Kousuke Saruta (Jira)
-
[jira] [Assigned] (SPARK-57938) Print active SKIP_ flags when Ruby doc build script runs
Nicholas Chammas (Jira)
-
[jira] [Updated] (SPARK-58789) Add CredentialProvider.additionalSparkProperties() SPI and auto-config wiring
Kousuke Saruta (Jira)
-
[jira] [Assigned] (SPARK-59163) Reuse projected table state options across table loading and cache lookup
Wenchen Fan (Jira)
-
[jira] [Resolved] (SPARK-59163) Reuse projected table state options across table loading and cache lookup
Wenchen Fan (Jira)
-
[jira] [Created] (SPARK-59295) Emit a common expression's definition once per scope under whole-stage codegen
Yang Jie (Jira)
-
[jira] [Resolved] (SPARK-59286) Optimize NaiveBayesModel transform closures
Ruifeng Zheng (Jira)
-
[jira] [Assigned] (SPARK-59286) Optimize NaiveBayesModel transform closures
Ruifeng Zheng (Jira)
-
[jira] [Assigned] (SPARK-59261) Memoize outputPartitioning on the projection and SPJ grouping nodes
Peter Toth (Jira)
-
[jira] [Assigned] (SPARK-58999) Allow using only main error class when subclasses exist
Jira
-
[jira] [Assigned] (SPARK-59050) SPJ one-side shuffle with out-of-set keys produces wrong results in multi-joins
XiDuo You (Jira)
-
[jira] [Assigned] (SPARK-59171) [SQL] SchemaPruning is not idempotent when a query reads _metadata and a VARIANT column is only referenced below a nondeterministic filter
Jira
-
[jira] [Resolved] (SPARK-58999) Allow using only main error class when subclasses exist
Jira
-
[jira] [Updated] (SPARK-59294) Support partially clustered distribution and skew join split for LeftSingle and ExistenceJoin
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59294) Support partially clustered distribution and skew join split for LeftSingle and ExistenceJoin
Cheng Pan (Jira)
-
[jira] [Created] (SPARK-59293) Fix update_build_status.yml to map non-standard workflow run conclusions
Tornike Gurgenidze (Jira)
-
[jira] [Resolved] (SPARK-59261) Memoize outputPartitioning on the projection and SPJ grouping nodes
Peter Toth (Jira)
-
[jira] [Resolved] (SPARK-59050) SPJ one-side shuffle with out-of-set keys produces wrong results in multi-joins
XiDuo You (Jira)
-
[jira] [Updated] (SPARK-59292) Add a client-side deadline to Spark Connect ML cache cleanup RPCs
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59292) Add a client-side deadline to Spark Connect ML cache cleanup RPCs
Ruifeng Zheng (Jira)
-
[jira] [Created] (SPARK-59291) Add @return tags to the public StateSpec methods
Jira
-
[jira] [Commented] (SPARK-59291) Add @return tags to the public StateSpec methods
Jira
-
[jira] [Updated] (SPARK-59291) Add @return tags to the public StateSpec methods
ASF GitHub Bot (Jira)
-
[jira] [Resolved] (SPARK-59252) SPJ scan reports a different partitioning at execution than it did at planning
Peter Toth (Jira)
-
[jira] [Updated] (SPARK-59290) The operation "stream" EXCEPT "static" is allowed but does not work as expected
ASF GitHub Bot (Jira)
-
[jira] [Resolved] (SPARK-59284) Read INT64 TIMESTAMP(MICROS) columns as nanosecond timestamps
Jira
-
[jira] [Assigned] (SPARK-59284) Read INT64 TIMESTAMP(MICROS) columns as nanosecond timestamps
Jira
-
[jira] [Created] (SPARK-59290) The operation "stream" EXCEPT "static" is allowed but does not work as expected
Jungtaek Lim (Jira)
-
[jira] [Assigned] (SPARK-58985) Fix HistoryServerDiskManager double-counting store size on concurrent release and makeRoom
Cheng Pan (Jira)
-
[jira] [Resolved] (SPARK-58985) Fix HistoryServerDiskManager double-counting store size on concurrent release and makeRoom
Cheng Pan (Jira)
-
[jira] [Assigned] (SPARK-59252) SPJ scan reports a different partitioning at execution than it did at planning
Peter Toth (Jira)
-
[jira] [Assigned] (SPARK-59080) Pick one ShuffleSpecCollection member for the SPJ pushdown and the re-shuffle
Peter Toth (Jira)
-
[jira] [Resolved] (SPARK-59080) Pick one ShuffleSpecCollection member for the SPJ pushdown and the re-shuffle
Peter Toth (Jira)
-
[jira] [Created] (SPARK-59287) Optimize MultilayerPerceptronClassificationModel transform closures
Ruifeng Zheng (Jira)
-
[jira] [Commented] (SPARK-59289) Insert a co-partitioned child's GroupPartitionsExec once, where the join's merged keys are known
Peter Toth (Jira)
-
[jira] [Updated] (SPARK-59289) Insert a co-partitioned child's GroupPartitionsExec once, where the join's merged keys are known
Peter Toth (Jira)
-
[jira] [Created] (SPARK-59289) Insert a co-partitioned child's GroupPartitionsExec once, where the join's merged keys are known
Peter Toth (Jira)
-
[jira] [Assigned] (SPARK-58996) Partially clustered storage-partitioned join duplicates rows when EnsureRequirements re-runs
XiDuo You (Jira)
-
[jira] [Resolved] (SPARK-58996) Partially clustered storage-partitioned join duplicates rows when EnsureRequirements re-runs
XiDuo You (Jira)
-
[jira] [Updated] (SPARK-59288) Optimize FMClassificationModel transform closures
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59288) Optimize FMClassificationModel transform closures
Ruifeng Zheng (Jira)
-
[jira] [Updated] (SPARK-59287) Optimize MultilayerPerceptronClassificationModel transform closures
ASF GitHub Bot (Jira)
-
[jira] [Updated] (SPARK-59286) Optimize NaiveBayesModel transform closures
ASF GitHub Bot (Jira)
-
[jira] [Created] (SPARK-59286) Optimize NaiveBayesModel transform closures
Ruifeng Zheng (Jira)
-
[jira] [Resolved] (SPARK-59199) SPJ: Support runtime partition filtering for more join types
Cheng Pan (Jira)
-
[jira] [Assigned] (SPARK-59199) SPJ: Support runtime partition filtering for more join types
Cheng Pan (Jira)
-
[jira] [Assigned] (SPARK-59250) Translate an empty runtime IN filter to AlwaysFalse in DSv2 runtime filtering
Cheng Pan (Jira)
-
[jira] [Resolved] (SPARK-59250) Translate an empty runtime IN filter to AlwaysFalse in DSv2 runtime filtering
Cheng Pan (Jira)
-
[jira] [Resolved] (SPARK-59247) Optimize tree ensemble classification model transform closures
Ruifeng Zheng (Jira)
-
[jira] [Assigned] (SPARK-59247) Optimize tree ensemble classification model transform closures
Ruifeng Zheng (Jira)