[jira] [Updated] (SPARK-51658) SPIP: Add geospatial types in Spark

2025-04-17 Thread Menelaos Karavelas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Menelaos Karavelas updated SPARK-51658: --- Description: *Q1. What are you trying to do? Articulate your objectives using absolu

[jira] [Updated] (SPARK-51829) Client side should update `client.thread_local.ml_caches` after deletion

2025-04-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-51829: -- Fix Version/s: 4.1.0 (was: 4.0.0) > Client side should update `client.t

[jira] [Updated] (SPARK-51838) Add a test to check function wildcard import

2025-04-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51838: --- Labels: pull-request-available (was: ) > Add a test to check function wildcard import > ---

[jira] [Updated] (SPARK-51772) Use of `copy()` instead of `withNewChildren` to create a copy of `TreeNode` with new children

2025-04-17 Thread Jihoon Son (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jihoon Son updated SPARK-51772: --- Description: The `TreeNode` class provides `withNewChildren()` function to create a copy of the nod

[jira] [Updated] (SPARK-51772) Use of `copy()` instead of `withNewChildren` to create a copy of `TreeNode` with new children

2025-04-17 Thread Jihoon Son (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jihoon Son updated SPARK-51772: --- Description: The `TreeNode` class provides `withNewChildren()` function to create a copy of the nod

[jira] [Updated] (SPARK-51830) Spark SQL: Error handling for partition datatype conversion

2025-04-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51830: --- Labels: pull-request-available (was: ) > Spark SQL: Error handling for partition datatype c

[jira] [Updated] (SPARK-51836) Avoid per-test function session setup

2025-04-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51836: --- Labels: pull-request-available (was: ) > Avoid per-test function session setup > --

[jira] [Resolved] (SPARK-51803) Store the original JDBC type in each SparkFields metadata

2025-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51803. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50591 [https://gith

[jira] [Resolved] (SPARK-51820) Move `UnresolvedOrdinal` construction before analysis to avoid issue with group by ordinal

2025-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51820. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50606 [https://gith

[jira] [Created] (SPARK-51831) No Column Pruning while using ExistJoin and datasource v2

2025-04-17 Thread Junqing Li (Jira)
Junqing Li created SPARK-51831: -- Summary: No Column Pruning while using ExistJoin and datasource v2 Key: SPARK-51831 URL: https://issues.apache.org/jira/browse/SPARK-51831 Project: Spark Issue

[jira] [Commented] (SPARK-51820) Move `UnresolvedOrdinal` construction before analysis to avoid issue with group by ordinal

2025-04-17 Thread Mihailo Timotic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17945259#comment-17945259 ] Mihailo Timotic commented on SPARK-51820: - Tests to be added back after we align

[jira] [Resolved] (SPARK-51839) Support `except(All)?/intersect(All)?/union(All)?/unionByName` in `DataFrame`

2025-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51839. --- Fix Version/s: connect-swift-0.1.0 Resolution: Fixed Issue resolved by pull request 6

[jira] [Resolved] (SPARK-51837) Support `inputFiles` for `DataFrame`

2025-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51837. --- Fix Version/s: connect-swift-0.1.0 Resolution: Fixed Issue resolved by pull request 6

[jira] [Resolved] (SPARK-51838) Add a test to check function wildcard import

2025-04-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-51838. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50634 [https://

[jira] [Assigned] (SPARK-51837) Support `inputFiles` for `DataFrame`

2025-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51837: - Assignee: Dongjoon Hyun > Support `inputFiles` for `DataFrame` > --

[jira] [Assigned] (SPARK-51838) Add a test to check function wildcard import

2025-04-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-51838: - Assignee: Ruifeng Zheng > Add a test to check function wildcard import > --

[jira] [Resolved] (SPARK-51836) Avoid per-test function session setup

2025-04-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-51836. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50632 [https://

[jira] [Assigned] (SPARK-51836) Avoid per-test function session setup

2025-04-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-51836: - Assignee: Ruifeng Zheng > Avoid per-test function session setup > -

[jira] [Updated] (SPARK-51839) Support `except(All)?/intersect(All)?/union(All)?/unionByName` in `DataFrame`

2025-04-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51839: --- Labels: pull-request-available (was: ) > Support `except(All)?/intersect(All)?/union(All)?/

[jira] [Created] (SPARK-51839) Support `except(All)?/intersect(All)?/union(All)?/unionByName` in `DataFrame`

2025-04-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-51839: - Summary: Support `except(All)?/intersect(All)?/union(All)?/unionByName` in `DataFrame` Key: SPARK-51839 URL: https://issues.apache.org/jira/browse/SPARK-51839 Proje

[jira] [Commented] (SPARK-51549) Bump Parquet 1.15.1

2025-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17945595#comment-17945595 ] Dongjoon Hyun commented on SPARK-51549: --- This is reverted from `branch-3.5` via [

[jira] [Updated] (SPARK-51549) Bump Parquet 1.15.1

2025-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-51549: -- Fix Version/s: (was: 3.5.6) > Bump Parquet 1.15.1 > --- > >

[jira] [Resolved] (SPARK-51829) Client side should update `client.thread_local.ml_caches` after deletion

2025-04-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-51829. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50618 [https://

[jira] [Assigned] (SPARK-51829) Client side should update `client.thread_local.ml_caches` after deletion

2025-04-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-51829: - Assignee: Ruifeng Zheng > Client side should update `client.thread_local.ml_caches` aft

[jira] [Updated] (SPARK-51836) Avoid per-test function session setup

2025-04-17 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-51836: -- Summary: Avoid per-test function session setup (was: Avoid per-test function session initiali

[jira] [Created] (SPARK-51837) Support `inputFiles` for `DataFrame`

2025-04-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-51837: - Summary: Support `inputFiles` for `DataFrame` Key: SPARK-51837 URL: https://issues.apache.org/jira/browse/SPARK-51837 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-51825) Add `SparkFileUtils`

2025-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51825: - Assignee: Dongjoon Hyun > Add `SparkFileUtils` > > >

[jira] [Resolved] (SPARK-51825) Add `SparkFileUtils`

2025-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51825. --- Fix Version/s: connect-swift-0.1.0 Resolution: Fixed Issue resolved by pull request 6

[jira] [Created] (SPARK-51833) Normalize relations in NormalizePlan

2025-04-17 Thread Vladimir Golubev (Jira)
Vladimir Golubev created SPARK-51833: Summary: Normalize relations in NormalizePlan Key: SPARK-51833 URL: https://issues.apache.org/jira/browse/SPARK-51833 Project: Spark Issue Type: Impr

[jira] [Updated] (SPARK-51833) Normalize relations in NormalizePlan

2025-04-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51833: --- Labels: pull-request-available (was: ) > Normalize relations in NormalizePlan > ---

[jira] [Assigned] (SPARK-51820) Move `UnresolvedOrdinal` construction before analysis to avoid issue with group by ordinal

2025-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-51820: --- Assignee: Mihailo Timotic > Move `UnresolvedOrdinal` construction before analysis to avoid

[jira] [Created] (SPARK-51838) Add a test to check function wildcard import

2025-04-17 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-51838: - Summary: Add a test to check function wildcard import Key: SPARK-51838 URL: https://issues.apache.org/jira/browse/SPARK-51838 Project: Spark Issue Type: Te

[jira] [Created] (SPARK-51836) Avoid per-test function session initializtion

2025-04-17 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-51836: - Summary: Avoid per-test function session initializtion Key: SPARK-51836 URL: https://issues.apache.org/jira/browse/SPARK-51836 Project: Spark Issue Type: T

[jira] [Created] (SPARK-51834) Support end-to-end table constraint management

2025-04-17 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-51834: -- Summary: Support end-to-end table constraint management Key: SPARK-51834 URL: https://issues.apache.org/jira/browse/SPARK-51834 Project: Spark Issue Type

[jira] [Created] (SPARK-51835) Extend InMemoryTableCatalog to support table versions

2025-04-17 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-51835: -- Summary: Extend InMemoryTableCatalog to support table versions Key: SPARK-51835 URL: https://issues.apache.org/jira/browse/SPARK-51835 Project: Spark Iss

[jira] [Created] (SPARK-51840) HiveExternalCatalog alterTable misses partition columns

2025-04-17 Thread Kent Yao (Jira)
Kent Yao created SPARK-51840: Summary: HiveExternalCatalog alterTable misses partition columns Key: SPARK-51840 URL: https://issues.apache.org/jira/browse/SPARK-51840 Project: Spark Issue Type: B

[jira] [Assigned] (SPARK-51839) Support `except(All)?/intersect(All)?/union(All)?/unionByName` in `DataFrame`

2025-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51839: - Assignee: Dongjoon Hyun > Support `except(All)?/intersect(All)?/union(All)?/unionByName

[jira] [Assigned] (SPARK-51824) Force to clean up the ML cache after each test

2025-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-51824: Assignee: Ruifeng Zheng > Force to clean up the ML cache after each test > --

[jira] [Resolved] (SPARK-51826) Update `YuniKorn` docs with `1.6.2`

2025-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-51826. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50617 [https://gi

[jira] [Resolved] (SPARK-51824) Force to clean up the ML cache after each test

2025-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-51824. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50619 [https://gi

[jira] [Updated] (SPARK-51831) No Column Pruning while using ExistJoin and datasource v2

2025-04-17 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-51831: - Affects Version/s: 4.1.0 > No Column Pruning while using ExistJoin and datasource v2 >

[jira] [Updated] (SPARK-51826) Update `YuniKorn` docs with `1.6.2`

2025-04-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51826: --- Labels: pull-request-available (was: ) > Update `YuniKorn` docs with `1.6.2` >

[jira] [Assigned] (SPARK-51826) Update `YuniKorn` docs with `1.6.2`

2025-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-51826: Assignee: Dongjoon Hyun > Update `YuniKorn` docs with `1.6.2` > -

[jira] [Updated] (SPARK-51832) Simplify maven version evaluation in the build tool

2025-04-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51832: --- Labels: pull-request-available (was: ) > Simplify maven version evaluation in the build too

[jira] [Updated] (SPARK-51832) Simplify maven version evaluation in the build tool

2025-04-17 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-51832: - Summary: Simplify maven version evaluation in the build tool (was: Simply maven version evaluation in t

[jira] [Created] (SPARK-51832) Simply maven version evaluation in the build tool

2025-04-17 Thread Kent Yao (Jira)
Kent Yao created SPARK-51832: Summary: Simply maven version evaluation in the build tool Key: SPARK-51832 URL: https://issues.apache.org/jira/browse/SPARK-51832 Project: Spark Issue Type: Improve

[jira] [Assigned] (SPARK-51819) Update pyspark-errors test module to include missing tests

2025-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-51819: Assignee: Yihong He > Update pyspark-errors test module to include missing tests > --

[jira] [Updated] (SPARK-51824) Force to clean up the ML cache after each test

2025-04-17 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-51824: - Fix Version/s: (was: 4.1.0) > Force to clean up the ML cache after each test > -

[jira] [Assigned] (SPARK-51803) Store the original JDBC type in each SparkFields metadata

2025-04-17 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-51803: --- Assignee: Alek Jarmov > Store the original JDBC type in each SparkFields metadata > ---

[jira] [Assigned] (SPARK-51832) Simplify maven version evaluation in the build tool

2025-04-17 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-51832: Assignee: Kent Yao > Simplify maven version evaluation in the build tool > --