[jira] [Resolved] (SPARK-46988) proto message abbreviation should support map fields

[ https://issues.apache.org/jira/browse/SPARK-46988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46988. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45413 [https://

[jira] [Assigned] (SPARK-46988) proto message abbreviation should support map fields

[ https://issues.apache.org/jira/browse/SPARK-46988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-46988: - Assignee: Ruifeng Zheng > proto message abbreviation should support map fields > --

[jira] [Created] (SPARK-47322) Make `withColumnsRenamed` duplicated column name handling consisten with `withColumnRenamed`

Ruifeng Zheng created SPARK-47322: - Summary: Make `withColumnsRenamed` duplicated column name handling consisten with `withColumnRenamed` Key: SPARK-47322 URL: https://issues.apache.org/jira/browse/SPARK-47322

[jira] [Created] (SPARK-47334) Make `withColumnRenamed` reuse the implementation of `withColumnsRenamed`

Ruifeng Zheng created SPARK-47334: - Summary: Make `withColumnRenamed` reuse the implementation of `withColumnsRenamed` Key: SPARK-47334 URL: https://issues.apache.org/jira/browse/SPARK-47334 Project:

[jira] [Created] (SPARK-47347) Factor session-related tests out of test_connect_basic

Ruifeng Zheng created SPARK-47347: - Summary: Factor session-related tests out of test_connect_basic Key: SPARK-47347 URL: https://issues.apache.org/jira/browse/SPARK-47347 Project: Spark Issu

[jira] [Created] (SPARK-47377) Factor out tests from `SparkConnectSQLTestCase`

Ruifeng Zheng created SPARK-47377: - Summary: Factor out tests from `SparkConnectSQLTestCase` Key: SPARK-47377 URL: https://issues.apache.org/jira/browse/SPARK-47377 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-41762) Make `Column.__neg__` return the same column name as PySpark

[ https://issues.apache.org/jira/browse/SPARK-41762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41762. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45493 [https://

[jira] [Assigned] (SPARK-41762) Make `Column.__neg__` return the same column name as PySpark

[ https://issues.apache.org/jira/browse/SPARK-41762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41762: - Assignee: Ruifeng Zheng > Make `Column.__neg__` return the same column name as PySpark

[jira] [Created] (SPARK-47437) Correct the error class for `DataFrame.sort`

Ruifeng Zheng created SPARK-47437: - Summary: Correct the error class for `DataFrame.sort` Key: SPARK-47437 URL: https://issues.apache.org/jira/browse/SPARK-47437 Project: Spark Issue Type: Bu

[jira] [Resolved] (SPARK-47436) Fix docstring links and type hints in Python Data Source

[ https://issues.apache.org/jira/browse/SPARK-47436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47436. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45557 [https://

[jira] [Assigned] (SPARK-47436) Fix docstring links and type hints in Python Data Source

[ https://issues.apache.org/jira/browse/SPARK-47436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47436: - Assignee: Hyukjin Kwon > Fix docstring links and type hints in Python Data Source > ---

[jira] [Created] (SPARK-47499) Reuse `test_help_command` in Connect

Ruifeng Zheng created SPARK-47499: - Summary: Reuse `test_help_command` in Connect Key: SPARK-47499 URL: https://issues.apache.org/jira/browse/SPARK-47499 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-47500) Factor column name handling out of `plan.py`

Ruifeng Zheng created SPARK-47500: - Summary: Factor column name handling out of `plan.py` Key: SPARK-47500 URL: https://issues.apache.org/jira/browse/SPARK-47500 Project: Spark Issue Type: Im

[jira] [Created] (SPARK-47560) Avoid RPC to validate column name with cached schema

Ruifeng Zheng created SPARK-47560: - Summary: Avoid RPC to validate column name with cached schema Key: SPARK-47560 URL: https://issues.apache.org/jira/browse/SPARK-47560 Project: Spark Issue

[jira] [Assigned] (SPARK-47560) Avoid RPC to validate column name with cached schema

[ https://issues.apache.org/jira/browse/SPARK-47560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47560: - Assignee: Ruifeng Zheng > Avoid RPC to validate column name with cached schema > --

[jira] [Resolved] (SPARK-47560) Avoid RPC to validate column name with cached schema

[ https://issues.apache.org/jira/browse/SPARK-47560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47560. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45717 [https://

[jira] [Created] (SPARK-47562) Factor literal handling out of `plan.py`

Ruifeng Zheng created SPARK-47562: - Summary: Factor literal handling out of `plan.py` Key: SPARK-47562 URL: https://issues.apache.org/jira/browse/SPARK-47562 Project: Spark Issue Type: Improv

[jira] [Assigned] (SPARK-47562) Factor literal handling out of `plan.py`

[ https://issues.apache.org/jira/browse/SPARK-47562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47562: - Assignee: Ruifeng Zheng > Factor literal handling out of `plan.py` > --

[jira] [Resolved] (SPARK-47562) Factor literal handling out of `plan.py`

[ https://issues.apache.org/jira/browse/SPARK-47562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47562. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45719 [https://

[jira] [Resolved] (SPARK-47621) Refine docstring of `try_sum`, `try_avg`, `avg`, `sum`, `mean`

[ https://issues.apache.org/jira/browse/SPARK-47621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47621. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45745 [https://

[jira] [Created] (SPARK-47638) Skip column name validation in PS

Ruifeng Zheng created SPARK-47638: - Summary: Skip column name validation in PS Key: SPARK-47638 URL: https://issues.apache.org/jira/browse/SPARK-47638 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-47638) Skip column name validation in PS

[ https://issues.apache.org/jira/browse/SPARK-47638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47638: - Assignee: Ruifeng Zheng > Skip column name validation in PS > -

[jira] [Resolved] (SPARK-47638) Skip column name validation in PS

[ https://issues.apache.org/jira/browse/SPARK-47638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47638. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45752 [https://

[jira] [Created] (SPARK-47664) Validate the column name with cached schema

Ruifeng Zheng created SPARK-47664: - Summary: Validate the column name with cached schema Key: SPARK-47664 URL: https://issues.apache.org/jira/browse/SPARK-47664 Project: Spark Issue Type: Imp

[jira] [Resolved] (SPARK-47669) Add `try_cast` function in DataFrame

[ https://issues.apache.org/jira/browse/SPARK-47669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47669. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45796 [https://

[jira] [Created] (SPARK-47749) Dataframe.collect should accept duplicated column names

Ruifeng Zheng created SPARK-47749: - Summary: Dataframe.collect should accept duplicated column names Key: SPARK-47749 URL: https://issues.apache.org/jira/browse/SPARK-47749 Project: Spark Iss

[jira] [Created] (SPARK-47771) Make max_by, min_by doctests deterministic

Ruifeng Zheng created SPARK-47771: - Summary: Make max_by, min_by doctests deterministic Key: SPARK-47771 URL: https://issues.apache.org/jira/browse/SPARK-47771 Project: Spark Issue Type: Impr

[jira] [Resolved] (SPARK-47771) Make max_by, min_by doctests deterministic

[ https://issues.apache.org/jira/browse/SPARK-47771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47771. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45939 [https://

[jira] [Resolved] (SPARK-47772) Fix the doctest of mode function

[ https://issues.apache.org/jira/browse/SPARK-47772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47772. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45940 [https://

[jira] [Updated] (SPARK-47779) Add a helper function to sort PS Frame/Series

[ https://issues.apache.org/jira/browse/SPARK-47779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47779: -- Component/s: PS Tests (was: PySpark) > Add a helper func

[jira] [Created] (SPARK-47779) Add a helper function to sort PS Frame/Series

Ruifeng Zheng created SPARK-47779: - Summary: Add a helper function to sort PS Frame/Series Key: SPARK-47779 URL: https://issues.apache.org/jira/browse/SPARK-47779 Project: Spark Issue Type: I

[jira] [Resolved] (SPARK-47779) Add a helper function to sort PS Frame/Series

[ https://issues.apache.org/jira/browse/SPARK-47779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47779. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45952 [https://

[jira] [Assigned] (SPARK-47779) Add a helper function to sort PS Frame/Series

[ https://issues.apache.org/jira/browse/SPARK-47779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47779: - Assignee: Ruifeng Zheng > Add a helper function to sort PS Frame/Series > -

[jira] [Updated] (SPARK-47815) Unify the user agent with json

[ https://issues.apache.org/jira/browse/SPARK-47815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47815: -- Summary: Unify the user agent with json (was: Unify the user agent string with json) > Unify

[jira] [Created] (SPARK-47815) Unify the user agent string representation with json

Ruifeng Zheng created SPARK-47815: - Summary: Unify the user agent string representation with json Key: SPARK-47815 URL: https://issues.apache.org/jira/browse/SPARK-47815 Project: Spark Issue

[jira] [Updated] (SPARK-47815) Unify the user agent string with json

[ https://issues.apache.org/jira/browse/SPARK-47815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47815: -- Summary: Unify the user agent string with json (was: Unify the user agent string representati

[jira] [Created] (SPARK-47816) Document the lazy evaluation of views in spark.{sql, table}

Ruifeng Zheng created SPARK-47816: - Summary: Document the lazy evaluation of views in spark.{sql, table} Key: SPARK-47816 URL: https://issues.apache.org/jira/browse/SPARK-47816 Project: Spark

[jira] [Resolved] (SPARK-47815) Unify the user agent with json

[ https://issues.apache.org/jira/browse/SPARK-47815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47815. --- Resolution: Not A Problem > Unify the user agent with json > --

[jira] [Created] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

Ruifeng Zheng created SPARK-47828: - Summary: DataFrameWriterV2.overwrite fails with invalid plan Key: SPARK-47828 URL: https://issues.apache.org/jira/browse/SPARK-47828 Project: Spark Issue T

[jira] [Updated] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47828: -- Issue Type: Bug (was: Improvement) > DataFrameWriterV2.overwrite fails with invalid plan > --

[jira] [Updated] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47828: -- Affects Version/s: 3.4.2 > DataFrameWriterV2.overwrite fails with invalid plan > -

[jira] [Resolved] (SPARK-47816) Document the lazy evaluation of views in spark.{sql, table}

[ https://issues.apache.org/jira/browse/SPARK-47816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47816. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46007 [https://

[jira] [Assigned] (SPARK-47816) Document the lazy evaluation of views in spark.{sql, table}

[ https://issues.apache.org/jira/browse/SPARK-47816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47816: - Assignee: Ruifeng Zheng > Document the lazy evaluation of views in spark.{sql, table} >

[jira] [Assigned] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47828: - Assignee: Ruifeng Zheng > DataFrameWriterV2.overwrite fails with invalid plan > ---

[jira] [Resolved] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47828. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46023 [https://

[jira] [Created] (SPARK-47855) Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect

Ruifeng Zheng created SPARK-47855: - Summary: Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect Key: SPARK-47855 URL: https://issues.apache.org/jira/browse/SPARK-47855 Project: Spark

[jira] [Resolved] (SPARK-47868) Recursion Limit Error in SparkSession and SparkConnectPlanner

[ https://issues.apache.org/jira/browse/SPARK-47868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47868. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46075 [https://

[jira] [Created] (SPARK-47883) Make CollectTailExec lazily execute

Ruifeng Zheng created SPARK-47883: - Summary: Make CollectTailExec lazily execute Key: SPARK-47883 URL: https://issues.apache.org/jira/browse/SPARK-47883 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-47883) Make CollectTailExec execute lazily

[ https://issues.apache.org/jira/browse/SPARK-47883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47883: -- Summary: Make CollectTailExec execute lazily (was: Make CollectTailExec lazily execute) > M

[jira] [Resolved] (SPARK-47883) Make CollectTailExec execute lazily

[ https://issues.apache.org/jira/browse/SPARK-47883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47883. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46101 [https://

[jira] [Assigned] (SPARK-47883) Make CollectTailExec execute lazily

[ https://issues.apache.org/jira/browse/SPARK-47883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47883: - Assignee: Ruifeng Zheng > Make CollectTailExec execute lazily > --

[jira] [Resolved] (SPARK-47906) Fix docstring and type hint of `hll_union_agg`

[ https://issues.apache.org/jira/browse/SPARK-47906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47906. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46128 [https://

[jira] [Assigned] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47845: - Assignee: Liu Cao > Support column type in split function in scala and python > ---

[jira] [Resolved] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47845. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46045 [https://

[jira] [Created] (SPARK-47935) Pin pandas==2.0.3 for pypy3.8

Ruifeng Zheng created SPARK-47935: - Summary: Pin pandas==2.0.3 for pypy3.8 Key: SPARK-47935 URL: https://issues.apache.org/jira/browse/SPARK-47935 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-47937) Fix docstring of `hll_sketch_agg`

Ruifeng Zheng created SPARK-47937: - Summary: Fix docstring of `hll_sketch_agg` Key: SPARK-47937 URL: https://issues.apache.org/jira/browse/SPARK-47937 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-47970) Revisit skipped parity tests for PySpark

Ruifeng Zheng created SPARK-47970: - Summary: Revisit skipped parity tests for PySpark Key: SPARK-47970 URL: https://issues.apache.org/jira/browse/SPARK-47970 Project: Spark Issue Type: Umbrel

[jira] [Updated] (SPARK-47499) Reuse `test_help_command` in Connect

[ https://issues.apache.org/jira/browse/SPARK-47499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47499: -- Parent: SPARK-47970 Issue Type: Sub-task (was: Test) > Reuse `test_help_command` in C

[jira] [Created] (SPARK-47985) Simplify functions with `lit`

Ruifeng Zheng created SPARK-47985: - Summary: Simplify functions with `lit` Key: SPARK-47985 URL: https://issues.apache.org/jira/browse/SPARK-47985 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-47970) Revisit skipped parity tests for PySpark Connect

[ https://issues.apache.org/jira/browse/SPARK-47970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47970: -- Summary: Revisit skipped parity tests for PySpark Connect (was: Revisit skipped parity tests

[jira] [Created] (SPARK-47987) Reenable `ArrowParityTests.test_createDataFrame_empty_partition`

Ruifeng Zheng created SPARK-47987: - Summary: Reenable `ArrowParityTests.test_createDataFrame_empty_partition` Key: SPARK-47987 URL: https://issues.apache.org/jira/browse/SPARK-47987 Project: Spark

[jira] [Updated] (SPARK-47623) Enable `QuietTest` in parity tests

[ https://issues.apache.org/jira/browse/SPARK-47623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47623: -- Summary: Enable `QuietTest` in parity tests (was: Use `QuietTest` in parity tests) > Enable

[jira] [Assigned] (SPARK-47985) Simplify functions with `lit`

[ https://issues.apache.org/jira/browse/SPARK-47985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47985: - Assignee: Ruifeng Zheng > Simplify functions with `lit` > -

[jira] [Resolved] (SPARK-47985) Simplify functions with `lit`

[ https://issues.apache.org/jira/browse/SPARK-47985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47985. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46219 [https://

[jira] [Resolved] (SPARK-47986) [CONNECT][PYTHON] Unable to create a new session when the default session is closed by the server

[ https://issues.apache.org/jira/browse/SPARK-47986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47986. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46221 [https://

[jira] [Created] (SPARK-48005) Enable `DefaultIndexParityTests. test_index_distributed_sequence_cleanup`

Ruifeng Zheng created SPARK-48005: - Summary: Enable `DefaultIndexParityTests. test_index_distributed_sequence_cleanup` Key: SPARK-48005 URL: https://issues.apache.org/jira/browse/SPARK-48005 Project:

[jira] [Created] (SPARK-48020) Pin 'pandas==2.2.2'

Ruifeng Zheng created SPARK-48020: - Summary: Pin 'pandas==2.2.2' Key: SPARK-48020 URL: https://issues.apache.org/jira/browse/SPARK-48020 Project: Spark Issue Type: Bug Components: P

[jira] [Updated] (SPARK-47129) Make ResolveRelations cache connect plan properly

[ https://issues.apache.org/jira/browse/SPARK-47129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47129: -- Affects Version/s: 3.4.3 3.5.1 > Make ResolveRelations cache connect pl

[jira] [Updated] (SPARK-47129) Make ResolveRelations cache connect plan properly

[ https://issues.apache.org/jira/browse/SPARK-47129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47129: -- Issue Type: Bug (was: Improvement) > Make ResolveRelations cache connect plan properly >

[jira] [Resolved] (SPARK-48055) Enable PandasUDFScalarParityTests.{test_vectorized_udf_empty_partition, test_vectorized_udf_struct_with_empty_partition}

[ https://issues.apache.org/jira/browse/SPARK-48055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48055. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46296 [https://

[jira] [Created] (SPARK-48058) `UserDefinedFunction.returnType` parse the DDL string

Ruifeng Zheng created SPARK-48058: - Summary: `UserDefinedFunction.returnType` parse the DDL string Key: SPARK-48058 URL: https://issues.apache.org/jira/browse/SPARK-48058 Project: Spark Issue

[jira] [Created] (SPARK-48142) Enable `CogroupedApplyInPandasTests.test_wrong_args`

Ruifeng Zheng created SPARK-48142: - Summary: Enable `CogroupedApplyInPandasTests.test_wrong_args` Key: SPARK-48142 URL: https://issues.apache.org/jira/browse/SPARK-48142 Project: Spark Issue

[jira] [Assigned] (SPARK-48142) Enable `CogroupedApplyInPandasTests.test_wrong_args`

[ https://issues.apache.org/jira/browse/SPARK-48142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48142: - Assignee: Ruifeng Zheng > Enable `CogroupedApplyInPandasTests.test_wrong_args` > --

[jira] [Resolved] (SPARK-48142) Enable `CogroupedApplyInPandasTests.test_wrong_args`

[ https://issues.apache.org/jira/browse/SPARK-48142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48142. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46397 [https://

[jira] [Updated] (SPARK-48184) Always set the seed of dataframe.sample in Client side

[ https://issues.apache.org/jira/browse/SPARK-48184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48184: -- Description: In Spark Classic: In [1]: df = spark.range(1).sample(0.1) In [2]: [df.count(

[jira] [Resolved] (SPARK-48190) Introduce a helper function to drop metadata

[ https://issues.apache.org/jira/browse/SPARK-48190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48190. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46466 [https://

[jira] [Created] (SPARK-48227) Document the requirement of seed in protos

Ruifeng Zheng created SPARK-48227: - Summary: Document the requirement of seed in protos Key: SPARK-48227 URL: https://issues.apache.org/jira/browse/SPARK-48227 Project: Spark Issue Type: Impr

[jira] [Created] (SPARK-48228) Implement the missing function validation in ApplyInXXX

Ruifeng Zheng created SPARK-48228: - Summary: Implement the missing function validation in ApplyInXXX Key: SPARK-48228 URL: https://issues.apache.org/jira/browse/SPARK-48228 Project: Spark Iss

[jira] [Created] (SPARK-48259) Add 3 missing methods in dsl

Ruifeng Zheng created SPARK-48259: - Summary: Add 3 missing methods in dsl Key: SPARK-48259 URL: https://issues.apache.org/jira/browse/SPARK-48259 Project: Spark Issue Type: Test Com

[jira] [Resolved] (SPARK-48259) Add 3 missing methods in dsl

[ https://issues.apache.org/jira/browse/SPARK-48259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48259. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46559 [https://

[jira] [Created] (SPARK-48272) Add function `timestamp_diff`

Ruifeng Zheng created SPARK-48272: - Summary: Add function `timestamp_diff` Key: SPARK-48272 URL: https://issues.apache.org/jira/browse/SPARK-48272 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-48278) Refine the string representation of `Cast`

Ruifeng Zheng created SPARK-48278: - Summary: Refine the string representation of `Cast` Key: SPARK-48278 URL: https://issues.apache.org/jira/browse/SPARK-48278 Project: Spark Issue Type: Impr

[jira] [Updated] (SPARK-48276) Add the missing __repr__ method for SQLExpression

[ https://issues.apache.org/jira/browse/SPARK-48276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48276: -- Fix Version/s: 4.0.0 > Add the missing __repr__ method for SQLExpression > ---

[jira] [Resolved] (SPARK-48272) Add function `timestamp_diff`

[ https://issues.apache.org/jira/browse/SPARK-48272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48272. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46576 [https://

[jira] [Assigned] (SPARK-48272) Add function `timestamp_diff`

[ https://issues.apache.org/jira/browse/SPARK-48272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48272: - Assignee: Ruifeng Zheng > Add function `timestamp_diff` > -

[jira] [Resolved] (SPARK-48278) Refine the string representation of `Cast`

[ https://issues.apache.org/jira/browse/SPARK-48278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48278. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46585 [https://

[jira] [Created] (SPARK-48287) Apply the builtin `timestamp_diff` method

Ruifeng Zheng created SPARK-48287: - Summary: Apply the builtin `timestamp_diff` method Key: SPARK-48287 URL: https://issues.apache.org/jira/browse/SPARK-48287 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-48295) Turn on compute.ops_on_diff_frames by default

Ruifeng Zheng created SPARK-48295: - Summary: Turn on compute.ops_on_diff_frames by default Key: SPARK-48295 URL: https://issues.apache.org/jira/browse/SPARK-48295 Project: Spark Issue Type: I

[jira] [Assigned] (SPARK-48287) Apply the builtin `timestamp_diff` method

[ https://issues.apache.org/jira/browse/SPARK-48287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48287: - Assignee: Ruifeng Zheng > Apply the builtin `timestamp_diff` method > -

[jira] [Resolved] (SPARK-48287) Apply the builtin `timestamp_diff` method

[ https://issues.apache.org/jira/browse/SPARK-48287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48287. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46595 [https://

[jira] [Resolved] (SPARK-48301) Rename CREATE_FUNC_WITH_IF_NOT_EXISTS_AND_REPLACE to CREATE_ROUTINE_WITH_IF_NOT_EXISTS_AND_REPLACE

[ https://issues.apache.org/jira/browse/SPARK-48301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48301. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46608 [https://

[jira] [Updated] (SPARK-48319) Test `assert_true` and `raise_error` with the same error class as Spark Classic

[ https://issues.apache.org/jira/browse/SPARK-48319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48319: -- Summary: Test `assert_true` and `raise_error` with the same error class as Spark Classic (was

[jira] [Created] (SPARK-48319) Test `assert_true` and `raise_error` with more specific error class

Ruifeng Zheng created SPARK-48319: - Summary: Test `assert_true` and `raise_error` with more specific error class Key: SPARK-48319 URL: https://issues.apache.org/jira/browse/SPARK-48319 Project: Spark

[jira] [Created] (SPARK-48321) Avoid using deprecated methods in dsl

Ruifeng Zheng created SPARK-48321: - Summary: Avoid using deprecated methods in dsl Key: SPARK-48321 URL: https://issues.apache.org/jira/browse/SPARK-48321 Project: Spark Issue Type: Improveme

[jira] [Created] (SPARK-48322) Drop internal metadata in `DataFrame.schema`

Ruifeng Zheng created SPARK-48322: - Summary: Drop internal metadata in `DataFrame.schema` Key: SPARK-48322 URL: https://issues.apache.org/jira/browse/SPARK-48322 Project: Spark Issue Type: Im

[jira] [Resolved] (SPARK-48321) Avoid using deprecated methods in dsl

[ https://issues.apache.org/jira/browse/SPARK-48321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48321. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46635 [https://

[jira] [Created] (SPARK-48333) Test `test_sorting_functions_with_column` with same `Column`

Ruifeng Zheng created SPARK-48333: - Summary: Test `test_sorting_functions_with_column` with same `Column` Key: SPARK-48333 URL: https://issues.apache.org/jira/browse/SPARK-48333 Project: Spark

[jira] [Updated] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-48335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48335: -- Issue Type: Improvement (was: Bug) > Make `_parse_datatype_string` compatible with Spark Conn

[jira] [Created] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

Ruifeng Zheng created SPARK-48335: - Summary: Make `_parse_datatype_string` compatible with Spark Connect Key: SPARK-48335 URL: https://issues.apache.org/jira/browse/SPARK-48335 Project: Spark

[jira] [Assigned] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-48335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48335: - Assignee: Ruifeng Zheng > Make `_parse_datatype_string` compatible with Spark Connect >

  1   2   3   4   5   6   7   8   9   10   >