[jira] [Assigned] (SPARK-51816) Simplify `StatFunctions.multipleApproxQuantiles` with dataframe APIs

2025-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-51816: Assignee: Ruifeng Zheng > Simplify `StatFunctions.multipleApproxQuantiles` with dataframe

[jira] [Resolved] (SPARK-51816) Simplify `StatFunctions.multipleApproxQuantiles` with dataframe APIs

2025-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-51816. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50602 [https://gi

[jira] [Created] (SPARK-51823) Add option to not persist state stores on executors

2025-04-16 Thread Adam Binford (Jira)
Adam Binford created SPARK-51823: Summary: Add option to not persist state stores on executors Key: SPARK-51823 URL: https://issues.apache.org/jira/browse/SPARK-51823 Project: Spark Issue Typ

[jira] [Resolved] (SPARK-51819) Update pyspark-errors test module to include missing tests

2025-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-51819. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50608 [https://gi

[jira] [Updated] (SPARK-51818) An AnalyzePlanRequest should not eagerly execute analyzed SQL commands

2025-04-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51818: --- Labels: pull-request-available (was: ) > An AnalyzePlanRequest should not eagerly execute a

[jira] [Updated] (SPARK-51823) Add option to not persist state stores on executors

2025-04-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51823: --- Labels: pull-request-available (was: ) > Add option to not persist state stores on executor

[jira] [Updated] (SPARK-51824) Force to clean up the ML cache after each test

2025-04-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51824: --- Labels: pull-request-available (was: ) > Force to clean up the ML cache after each test > -

[jira] [Updated] (SPARK-51806) Upgrade kryo-shaded to 4.0.3

2025-04-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-51806: - Fix Version/s: 4.0.0 (was: 4.1.0) > Upgrade kryo-shaded to 4.0.3 > --

[jira] [Reopened] (SPARK-51824) Force to clean up the ML cache after each test

2025-04-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-51824: -- Assignee: (was: Ruifeng Zheng) Reverted in https://github.com/apache/spark/commit/e9b50

[jira] [Assigned] (SPARK-51828) Update `README.md` with YuniKorn 1.6.2

2025-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51828: - Assignee: Dongjoon Hyun > Update `README.md` with YuniKorn 1.6.2 >

[jira] [Created] (SPARK-51828) Update `README.md` with YuniKorn 1.6.2

2025-04-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-51828: - Summary: Update `README.md` with YuniKorn 1.6.2 Key: SPARK-51828 URL: https://issues.apache.org/jira/browse/SPARK-51828 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-51828) Update `README.md` with YuniKorn 1.6.2

2025-04-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51828: --- Labels: pull-request-available (was: ) > Update `README.md` with YuniKorn 1.6.2 > -

[jira] [Created] (SPARK-51820) Move `UnresolvedOrdinal` construction before analysis to avoid issue with group by ordinal

2025-04-16 Thread Mihailo Timotic (Jira)
Mihailo Timotic created SPARK-51820: --- Summary: Move `UnresolvedOrdinal` construction before analysis to avoid issue with group by ordinal Key: SPARK-51820 URL: https://issues.apache.org/jira/browse/SPARK-51820

[jira] [Commented] (SPARK-51710) Using Dataframe.dropDuplicates with an empty array as argument behaves unexpectedly

2025-04-16 Thread Afrin Jaman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17945137#comment-17945137 ] Afrin Jaman commented on SPARK-51710: - Hello [~siriousjoke], I am willing to work to

[jira] [Commented] (SPARK-51392) Improve PythonException for unexpected UDF usage

2025-04-16 Thread Afrin Jaman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17945139#comment-17945139 ] Afrin Jaman commented on SPARK-51392: - Hello [~itholic], I am willing to work to res

[jira] [Updated] (SPARK-51805) Get function with improper argument should throw proper exception instead of an internal one

2025-04-16 Thread Mihailo Aleksic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Aleksic updated SPARK-51805: Description: Following query throw _Cannot cast NullType to Arraytype_: {code:java} SELEC

[jira] [Updated] (SPARK-51805) Get function with improper argument should throw proper exception instead of an internal one

2025-04-16 Thread Mihailo Aleksic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Aleksic updated SPARK-51805: Summary: Get function with improper argument should throw proper exception instead of an i

[jira] [Created] (SPARK-51821) Possible deadlock in UninterruptibleThread

2025-04-16 Thread Vlad Rozov (Jira)
Vlad Rozov created SPARK-51821: -- Summary: Possible deadlock in UninterruptibleThread Key: SPARK-51821 URL: https://issues.apache.org/jira/browse/SPARK-51821 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-51824) Force to clean up the ML cache after each test

2025-04-16 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-51824. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50614 [https://

[jira] [Created] (SPARK-51827) Support Spark Connect in new API `transformWithState` in PySpark

2025-04-16 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-51827: Summary: Support Spark Connect in new API `transformWithState` in PySpark Key: SPARK-51827 URL: https://issues.apache.org/jira/browse/SPARK-51827 Project: Spark

[jira] [Updated] (SPARK-51829) Client side should update `client.thread_local.ml_caches` after deletion

2025-04-16 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-51829: -- Summary: Client side should update `client.thread_local.ml_caches` after deletion (was: Updat

[jira] [Created] (SPARK-51829) Update `client.thread_local.ml_caches` after deletion

2025-04-16 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-51829: - Summary: Update `client.thread_local.ml_caches` after deletion Key: SPARK-51829 URL: https://issues.apache.org/jira/browse/SPARK-51829 Project: Spark Issue

[jira] [Created] (SPARK-51830) Spark SQL: Error handling for partition datatype conversion

2025-04-16 Thread Madhukar (Jira)
Madhukar created SPARK-51830: Summary: Spark SQL: Error handling for partition datatype conversion Key: SPARK-51830 URL: https://issues.apache.org/jira/browse/SPARK-51830 Project: Spark Issue Ty

[jira] [Updated] (SPARK-51829) Client side should update `client.thread_local.ml_caches` after deletion

2025-04-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51829: --- Labels: pull-request-available (was: ) > Client side should update `client.thread_local.ml_

[jira] [Updated] (SPARK-51830) Spark SQL: Error handling for partition datatype conversion

2025-04-16 Thread Madhukar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Madhukar updated SPARK-51830: - Description: A partition is of int datatype and a partition in legacy table was created as partition_nam

[jira] [Assigned] (SPARK-51414) Add the make_time() function

2025-04-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-51414: Assignee: Rob > Add the make_time() function > > > K

[jira] [Updated] (SPARK-51830) Spark SQL: Error handling for partition datatype conversion

2025-04-16 Thread Madhukar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Madhukar updated SPARK-51830: - Component/s: SQL > Spark SQL: Error handling for partition datatype conversion > ---

[jira] [Updated] (SPARK-51806) Upgrade kryo-shaded to 4.0.3

2025-04-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-51806: - Affects Version/s: 4.0.0 > Upgrade kryo-shaded to 4.0.3 > > >

[jira] [Resolved] (SPARK-51828) Update `README.md` with YuniKorn 1.6.2

2025-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51828. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Resolved] (SPARK-51414) Add the make_time() function

2025-04-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-51414. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50269 [https://github.com

[jira] [Created] (SPARK-51825) Add `SparkFileUtils`

2025-04-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-51825: - Summary: Add `SparkFileUtils` Key: SPARK-51825 URL: https://issues.apache.org/jira/browse/SPARK-51825 Project: Spark Issue Type: Sub-task Compone

[jira] [Updated] (SPARK-51825) Add `SparkFileUtils`

2025-04-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51825: --- Labels: pull-request-available (was: ) > Add `SparkFileUtils` > > >

[jira] [Resolved] (SPARK-51797) Add table name to TableConstraint and remove parser dependencies

2025-04-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-51797. Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50584 [https:

[jira] [Created] (SPARK-51826) Update `YuniKorn` docs with `1.6.2`

2025-04-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-51826: - Summary: Update `YuniKorn` docs with `1.6.2` Key: SPARK-51826 URL: https://issues.apache.org/jira/browse/SPARK-51826 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-51817) [CONNECT][SCALA] Removing ansiConfig parameter in error-class causes client to be incompatible

2025-04-16 Thread Niranjan Jayakar (Jira)
Niranjan Jayakar created SPARK-51817: Summary: [CONNECT][SCALA] Removing ansiConfig parameter in error-class causes client to be incompatible Key: SPARK-51817 URL: https://issues.apache.org/jira/browse/SPARK-5

[jira] [Updated] (SPARK-51817) [CONNECT][SCALA] Removing ansiConfig parameter in error-class causes client to be incompatible

2025-04-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51817: --- Labels: pull-request-available (was: ) > [CONNECT][SCALA] Removing ansiConfig parameter in

[jira] [Updated] (SPARK-51513) Fix RewriteMergeIntoTable rule produces unresolved plan

2025-04-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-51513: Description: org.apache.spark.SparkException: [INTERNAL_ERROR] Found the unresolved operator: 'App

[jira] [Assigned] (SPARK-51815) Add `Row` struct

2025-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51815: - Assignee: Dongjoon Hyun > Add `Row` struct > > > Key:

[jira] [Updated] (SPARK-51554) Add the time_trunc() function

2025-04-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51554: --- Labels: pull-request-available (was: ) > Add the time_trunc() function > --

[jira] [Created] (SPARK-51819) Update pyspark-errors test module to include missing tests

2025-04-16 Thread Yihong He (Jira)
Yihong He created SPARK-51819: - Summary: Update pyspark-errors test module to include missing tests Key: SPARK-51819 URL: https://issues.apache.org/jira/browse/SPARK-51819 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-51812) Remove redundant parameters of method `QueryExecution#createSparkPlan`

2025-04-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-51812. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50598 [https://github.com

[jira] [Assigned] (SPARK-51812) Remove redundant parameters of method `QueryExecution#createSparkPlan`

2025-04-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-51812: Assignee: Pan Bingkun > Remove redundant parameters of method `QueryExecution#createSparkPlan` >

[jira] [Updated] (SPARK-51819) Update pyspark-errors test module to include missing tests

2025-04-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51819: --- Labels: pull-request-available (was: ) > Update pyspark-errors test module to include missi

[jira] [Created] (SPARK-51818) An AnalyzePlanRequest should not eagerly execute analyzed SQL commands

2025-04-16 Thread Robert Dillitz (Jira)
Robert Dillitz created SPARK-51818: -- Summary: An AnalyzePlanRequest should not eagerly execute analyzed SQL commands Key: SPARK-51818 URL: https://issues.apache.org/jira/browse/SPARK-51818 Project: S

[jira] [Resolved] (SPARK-51815) Add `Row` struct

2025-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51815. --- Fix Version/s: connect-swift-0.1.0 Resolution: Fixed Issue resolved by pull request 6

[jira] [Updated] (SPARK-51820) Move `UnresolvedOrdinal` construction before analysis to avoid issue with group by ordinal

2025-04-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51820: --- Labels: pull-request-available (was: ) > Move `UnresolvedOrdinal` construction before analy

[jira] [Created] (SPARK-51822) Throwing classified error when disallowed functions are called during StatefulProcessor.init()

2025-04-16 Thread Eric Marnadi (Jira)
Eric Marnadi created SPARK-51822: Summary: Throwing classified error when disallowed functions are called during StatefulProcessor.init() Key: SPARK-51822 URL: https://issues.apache.org/jira/browse/SPARK-51822