[jira] [Updated] (SPARK-45673) Enhancing clarity and usability of PySpark error messages

2025-04-23 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-45673: Labels: releasenotes (was: ) > Enhancing clarity and usability of PySpark error messages > --

[jira] [Created] (SPARK-51874) Add Enumration TypedConfigBuilder

2025-04-23 Thread Kent Yao (Jira)
Kent Yao created SPARK-51874: Summary: Add Enumration TypedConfigBuilder Key: SPARK-51874 URL: https://issues.apache.org/jira/browse/SPARK-51874 Project: Spark Issue Type: Improvement C

[jira] [Resolved] (SPARK-44076) SPIP: Python Data Source API

2025-04-23 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-44076. - Fix Version/s: 4.0.0 Resolution: Fixed > SPIP: Python Data Source API > -

[jira] [Assigned] (SPARK-44076) SPIP: Python Data Source API

2025-04-23 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-44076: --- Assignee: Allison Wang > SPIP: Python Data Source API > > >

[jira] [Resolved] (SPARK-46685) PySpark UDF Unified Profiling

2025-04-23 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-46685. - Fix Version/s: 4.0.0 Assignee: Xinrong Meng Resolution: Fixed > PySpark UDF Unified Prof

[jira] [Resolved] (SPARK-45673) Enhancing clarity and usability of PySpark error messages

2025-04-23 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-45673. - Fix Version/s: 4.0.0 Assignee: Haejoon Lee Resolution: Fixed > Enhancing clarity and usa

[jira] [Resolved] (SPARK-43797) Python User-defined Table Functions

2025-04-23 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-43797. - Fix Version/s: 4.0.0 Assignee: Allison Wang Resolution: Fixed > Python User-defined Tabl

[jira] [Created] (SPARK-51884) Subquery Definition Changes For Adding Support For Nested Correlated Subqueries

2025-04-23 Thread Avery Qi (Jira)
Avery Qi created SPARK-51884: Summary: Subquery Definition Changes For Adding Support For Nested Correlated Subqueries Key: SPARK-51884 URL: https://issues.apache.org/jira/browse/SPARK-51884 Project: Spar

[jira] [Resolved] (SPARK-51874) Add Enumration TypedConfigBuilder

2025-04-23 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-51874. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50674 [https://github.com

[jira] [Assigned] (SPARK-51874) Add Enumration TypedConfigBuilder

2025-04-23 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-51874: Assignee: Kent Yao > Add Enumration TypedConfigBuilder > - > >

[jira] [Resolved] (SPARK-51865) Print stacktrace message when RootAllocator fails to close

2025-04-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51865. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50663 [https://gith

[jira] [Updated] (SPARK-51884) Subquery Definition Changes For Adding Support For Nested Correlated Subqueries

2025-04-23 Thread Avery Qi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avery Qi updated SPARK-51884: - Description: Newly added argument for SubqueryExpression: OuterScopeAttrs. `OuterScopeAttrs` contains a

[jira] [Created] (SPARK-51889) Fix MapState bug on clear() for TWS Python

2025-04-23 Thread Bo Gao (Jira)
Bo Gao created SPARK-51889: -- Summary: Fix MapState bug on clear() for TWS Python Key: SPARK-51889 URL: https://issues.apache.org/jira/browse/SPARK-51889 Project: Spark Issue Type: Task Com

[jira] [Updated] (SPARK-51889) Fix MapState bug on clear() for TWS Python

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51889: --- Labels: pull-request-available (was: ) > Fix MapState bug on clear() for TWS Python > -

[jira] [Resolved] (SPARK-51439) Support default arguments in SQL UDFs

2025-04-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51439. - Fix Version/s: 4.0.0 Resolution: Fixed > Support default arguments in SQL UDFs >

[jira] [Updated] (SPARK-51874) Add Enumration TypedConfigBuilder

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51874: --- Labels: pull-request-available (was: ) > Add Enumration TypedConfigBuilder > --

[jira] [Updated] (SPARK-51867) Make scala model supporting save / load against local filesystem path

2025-04-23 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-51867: -- Parent Issue: SPARK-51236 (was: SPARK-50812) > Make scala model supporting save / load agains

[jira] [Created] (SPARK-51875) Support `repartition(ByExpression)?` and `coalesce`

2025-04-23 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-51875: - Summary: Support `repartition(ByExpression)?` and `coalesce` Key: SPARK-51875 URL: https://issues.apache.org/jira/browse/SPARK-51875 Project: Spark Issue T

[jira] [Updated] (SPARK-51875) Support `repartition(ByExpression)?` and `coalesce`

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51875: --- Labels: pull-request-available (was: ) > Support `repartition(ByExpression)?` and `coalesce

[jira] [Updated] (SPARK-51876) Add configuration for `log4j.configurationFile` in the PySpark submission args of `run_individual_python_test`

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51876: --- Labels: pull-request-available (was: ) > Add configuration for `log4j.configurationFile` in

[jira] [Resolved] (SPARK-51876) Add configuration for `log4j.configurationFile` in the PySpark submission args of `run_individual_python_test`

2025-04-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-51876. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50675 [https://github.com

[jira] [Created] (SPARK-51882) Skip the test test_transform_with_state_with_wmark_and_non_event_time

2025-04-23 Thread Yang Jie (Jira)
Yang Jie created SPARK-51882: Summary: Skip the test test_transform_with_state_with_wmark_and_non_event_time Key: SPARK-51882 URL: https://issues.apache.org/jira/browse/SPARK-51882 Project: Spark

[jira] [Updated] (SPARK-51883) Python Data Source filter pushdown docs

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51883: --- Labels: pull-request-available (was: ) > Python Data Source filter pushdown docs >

[jira] [Created] (SPARK-51878) Improve fillDefaultValue by exec the foldable default expression.

2025-04-23 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-51878: -- Summary: Improve fillDefaultValue by exec the foldable default expression. Key: SPARK-51878 URL: https://issues.apache.org/jira/browse/SPARK-51878 Project: Spark

[jira] [Updated] (SPARK-51880) Avoid eager model removal in meta algorithms when collectSubModel is true

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51880: --- Labels: pull-request-available (was: ) > Avoid eager model removal in meta algorithms when

[jira] [Resolved] (SPARK-51879) Support `groupBy/rollup/cube` in `DataFrame`

2025-04-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51879. --- Fix Version/s: connect-swift-0.1.0 Resolution: Fixed Issue resolved by pull request 8

[jira] [Commented] (SPARK-45265) Support Hive 4.0 metastore

2025-04-23 Thread Madhukar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946702#comment-17946702 ] Madhukar commented on SPARK-45265: -- Hi [~dongjoon] [~zhangbutao] [~yao]  For addressin

[jira] [Assigned] (SPARK-51875) Support `repartition(ByExpression)?` and `coalesce`

2025-04-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51875: - Assignee: Dongjoon Hyun > Support `repartition(ByExpression)?` and `coalesce` > ---

[jira] [Resolved] (SPARK-51875) Support `repartition(ByExpression)?` and `coalesce`

2025-04-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-51875. --- Fix Version/s: connect-swift-0.1.0 Resolution: Fixed Issue resolved by pull request 8

[jira] [Created] (SPARK-51881) Make AvroOptions comparable

2025-04-23 Thread Vladimir Golubev (Jira)
Vladimir Golubev created SPARK-51881: Summary: Make AvroOptions comparable Key: SPARK-51881 URL: https://issues.apache.org/jira/browse/SPARK-51881 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-51881) Make AvroOptions comparable

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51881: --- Labels: pull-request-available (was: ) > Make AvroOptions comparable >

[jira] [Assigned] (SPARK-51865) Print stacktrace message when RootAllocator fails to close

2025-04-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-51865: --- Assignee: Yang Jie > Print stacktrace message when RootAllocator fails to close > -

[jira] [Assigned] (SPARK-51876) Add configuration for `log4j.configurationFile` in the PySpark submission args of `run_individual_python_test`

2025-04-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-51876: Assignee: Yang Jie > Add configuration for `log4j.configurationFile` in the PySpark submission >

[jira] [Updated] (SPARK-51877) Add functions 'chr', 'random' and 'uuid'

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51877: --- Labels: pull-request-available (was: ) > Add functions 'chr', 'random' and 'uuid' > ---

[jira] [Created] (SPARK-51877) Add functions 'chr', 'random' and 'uuid'

2025-04-23 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-51877: - Summary: Add functions 'chr', 'random' and 'uuid' Key: SPARK-51877 URL: https://issues.apache.org/jira/browse/SPARK-51877 Project: Spark Issue Type: New Fe

[jira] [Assigned] (SPARK-51868) Move type coercion error path to a separate object

2025-04-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-51868: Assignee: Vladimir Golubev > Move type coercion error path to a separate object > ---

[jira] [Resolved] (SPARK-51868) Move type coercion error path to a separate object

2025-04-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-51868. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50666 [https://github.com

[jira] [Updated] (SPARK-51879) Support `groupBy/rollup/cube` in `DataFrame`

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51879: --- Labels: pull-request-available (was: ) > Support `groupBy/rollup/cube` in `DataFrame` > ---

[jira] [Created] (SPARK-51879) Support `groupBy/rollup/cube` in `DataFrame`

2025-04-23 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-51879: - Summary: Support `groupBy/rollup/cube` in `DataFrame` Key: SPARK-51879 URL: https://issues.apache.org/jira/browse/SPARK-51879 Project: Spark Issue Type: Su

[jira] [Assigned] (SPARK-51879) Support `groupBy/rollup/cube` in `DataFrame`

2025-04-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51879: - Assignee: Dongjoon Hyun > Support `groupBy/rollup/cube` in `DataFrame` > --

[jira] [Resolved] (SPARK-51873) For OneVsRest algorithm, allow using save / load to replace cache

2025-04-23 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu resolved SPARK-51873. Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50672 [https://github

[jira] [Assigned] (SPARK-51873) For OneVsRest algorithm, allow using save / load to replace cache

2025-04-23 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-51873: -- Assignee: Weichen Xu > For OneVsRest algorithm, allow using save / load to replace cache > --

[jira] [Created] (SPARK-51880) Avoid eager model removal in meta algorithms when collectSubModel is true

2025-04-23 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-51880: - Summary: Avoid eager model removal in meta algorithms when collectSubModel is true Key: SPARK-51880 URL: https://issues.apache.org/jira/browse/SPARK-51880 Project:

[jira] [Created] (SPARK-51883) Python Data Source filter pushdown docs

2025-04-23 Thread Haoyu Weng (Jira)
Haoyu Weng created SPARK-51883: -- Summary: Python Data Source filter pushdown docs Key: SPARK-51883 URL: https://issues.apache.org/jira/browse/SPARK-51883 Project: Spark Issue Type: Documentation

[jira] [Resolved] (SPARK-51889) Fix MapState bug on clear() for TWS Python

2025-04-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-51889. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50686 [https://gi

[jira] [Commented] (SPARK-51555) Add the timediff() function

2025-04-23 Thread Senthil Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946648#comment-17946648 ] Senthil Kumar commented on SPARK-51555: --- [~maxgekk] We do not have functions for M

[jira] [Created] (SPARK-51876) Add configuration for `log4j.configurationFile` in the PySpark submission args of `run_individual_python_test`

2025-04-23 Thread Yang Jie (Jira)
Yang Jie created SPARK-51876: Summary: Add configuration for `log4j.configurationFile` in the PySpark submission args of `run_individual_python_test` Key: SPARK-51876 URL: https://issues.apache.org/jira/browse/SPARK-5

[jira] [Created] (SPARK-51894) Update `setup-minikube` to v0.0.19

2025-04-23 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-51894: - Summary: Update `setup-minikube` to v0.0.19 Key: SPARK-51894 URL: https://issues.apache.org/jira/browse/SPARK-51894 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-51877) Add functions 'chr', 'random' and 'uuid'

2025-04-23 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-51877. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50678 [https://

[jira] [Created] (SPARK-51888) Add optimizer support for nested correlated In/Exists subqueries

2025-04-23 Thread Avery Qi (Jira)
Avery Qi created SPARK-51888: Summary: Add optimizer support for nested correlated In/Exists subqueries Key: SPARK-51888 URL: https://issues.apache.org/jira/browse/SPARK-51888 Project: Spark Iss

[jira] [Updated] (SPARK-50775) Allow char/varchar in string expressions

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50775: --- Labels: pull-request-available (was: ) > Allow char/varchar in string expressions > ---

[jira] [Assigned] (SPARK-51882) Skip the test test_transform_with_state_with_wmark_and_non_event_time

2025-04-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-51882: Assignee: Yang Jie > Skip the test test_transform_with_state_with_wmark_and_non_event_time >

[jira] [Updated] (SPARK-51878) Improve fillDefaultValue by exec the foldable default expression.

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51878: --- Labels: pull-request-available (was: ) > Improve fillDefaultValue by exec the foldable defa

[jira] [Resolved] (SPARK-48334) NettyServer doesn't shutdown if SparkContext initialize failed

2025-04-23 Thread IsisPolei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] IsisPolei resolved SPARK-48334. --- Fix Version/s: 4.0.0 Resolution: Fixed This issue does't exist in the master branch. The Spar

[jira] [Resolved] (SPARK-51882) Skip the test test_transform_with_state_with_wmark_and_non_event_time

2025-04-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-51882. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50681 [https://github.com

[jira] [Updated] (SPARK-51882) Skip the test test_transform_with_state_with_wmark_and_non_event_time

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51882: --- Labels: pull-request-available (was: ) > Skip the test test_transform_with_state_with_wmark

[jira] [Created] (SPARK-51890) Store user-specified options when reading views

2025-04-23 Thread Yumingxuan Guo (Jira)
Yumingxuan Guo created SPARK-51890: -- Summary: Store user-specified options when reading views Key: SPARK-51890 URL: https://issues.apache.org/jira/browse/SPARK-51890 Project: Spark Issue Typ

[jira] [Updated] (SPARK-51890) Store user-specified options when reading views

2025-04-23 Thread Yumingxuan Guo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yumingxuan Guo updated SPARK-51890: --- Description: User specified options when reading views can be stored for later use. I.e., {

[jira] [Created] (SPARK-51887) Add analyzer++ support for nested correlated subqueries

2025-04-23 Thread Avery Qi (Jira)
Avery Qi created SPARK-51887: Summary: Add analyzer++ support for nested correlated subqueries Key: SPARK-51887 URL: https://issues.apache.org/jira/browse/SPARK-51887 Project: Spark Issue Type: S

[jira] [Created] (SPARK-51886) Add optimizer support for nested correlated scalar subquery

2025-04-23 Thread Avery Qi (Jira)
Avery Qi created SPARK-51886: Summary: Add optimizer support for nested correlated scalar subquery Key: SPARK-51886 URL: https://issues.apache.org/jira/browse/SPARK-51886 Project: Spark Issue Ty

[jira] [Updated] (SPARK-51885) Add analyzer support for nested correlated subqueries

2025-04-23 Thread Avery Qi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avery Qi updated SPARK-51885: - Description: h2. The task includes: * Add support for queries containing nested correlations in multi-p

[jira] [Created] (SPARK-51885) Add analyzer support for nested correlated subqueries

2025-04-23 Thread Avery Qi (Jira)
Avery Qi created SPARK-51885: Summary: Add analyzer support for nested correlated subqueries Key: SPARK-51885 URL: https://issues.apache.org/jira/browse/SPARK-51885 Project: Spark Issue Type: Sub

[jira] [Updated] (SPARK-51884) Subquery Definition Changes For Adding Support For Nested Correlated Subqueries

2025-04-23 Thread Avery Qi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avery Qi updated SPARK-51884: - Description: Newly added argument for SubqueryExpression: OuterScopeAttrs. `OuterScopeAttrs` contains a

[jira] [Updated] (SPARK-51884) Subquery Definition Changes For Adding Support For Nested Correlated Subqueries

2025-04-23 Thread Avery Qi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avery Qi updated SPARK-51884: - Description: Newly added argument for SubqueryExpression: OuterScopeAttrs. `OuterScopeAttrs` contains a

[jira] [Updated] (SPARK-51891) Squeeze the protocol of ListState GET / PUT / APPENDLIST for transformWithState in PySpark

2025-04-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51891: --- Labels: pull-request-available (was: ) > Squeeze the protocol of ListState GET / PUT / APPE

[jira] [Created] (SPARK-51891) Squeeze the protocol of ListState GET / PUT / APPENDLIST for transformWithState in PySpark

2025-04-23 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-51891: Summary: Squeeze the protocol of ListState GET / PUT / APPENDLIST for transformWithState in PySpark Key: SPARK-51891 URL: https://issues.apache.org/jira/browse/SPARK-51891

[jira] [Updated] (SPARK-51892) Reading JSON file with schema array[array[struct]] fails with ClassCastException in Spark 3.5.1

2025-04-23 Thread Praneet Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praneet Sharma updated SPARK-51892: --- Attachment: b.json > Reading JSON file with schema array[array[struct]] fails with > ClassC

[jira] [Created] (SPARK-51892) Reading JSON file with schema array[array[struct]] fails with ClassCastException in Spark 3.5.1

2025-04-23 Thread Praneet Sharma (Jira)
Praneet Sharma created SPARK-51892: -- Summary: Reading JSON file with schema array[array[struct]] fails with ClassCastException in Spark 3.5.1 Key: SPARK-51892 URL: https://issues.apache.org/jira/browse/SPARK-5189

[jira] [Updated] (SPARK-51892) Reading JSON file with schema array[array[struct]] fails with ClassCastException in Spark 3.5.1

2025-04-23 Thread Praneet Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praneet Sharma updated SPARK-51892: --- Description: Hi, we have a JSON with 1 column of type: array[array[struct]]. In Spark 3.5.1,