[jira] [Resolved] (SPARK-51315) Enable object level collation flag

[ https://issues.apache.org/jira/browse/SPARK-51315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51315. - Resolution: Fixed Issue resolved by pull request 50082 [https://github.com/apache/spark/pull/500

[jira] [Resolved] (SPARK-51309) Upgrade rocksdbjni to 9.10.0

[ https://issues.apache.org/jira/browse/SPARK-51309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-51309. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50076 [https://github.com

[jira] [Created] (SPARK-51325) Check in source code for `smallJar.jar`

Venkata Sai Akhil Gudesa created SPARK-51325: Summary: Check in source code for `smallJar.jar` Key: SPARK-51325 URL: https://issues.apache.org/jira/browse/SPARK-51325 Project: Spark

[jira] [Updated] (SPARK-51325) Check in source code for `smallJar.jar`

[ https://issues.apache.org/jira/browse/SPARK-51325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51325: --- Labels: pull-request-available (was: ) > Check in source code for `smallJar.jar` >

[jira] (SPARK-51321) Add support for LPAD and RPAD pushdown in MsSQL Server JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-51321 ] Milos Stojanovic deleted comment on SPARK-51321: -- was (Author: JIRAUSER308844): PR: https://github.com/apache/spark/pull/50060 > Add support for LPAD and RPAD pushdown in MsSQL Server J

[jira] [Commented] (SPARK-51321) Add support for LPAD and RPAD pushdown in MsSQL Server JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-51321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930693#comment-17930693 ] Milos Stojanovic commented on SPARK-51321: -- PR: https://github.com/apache/spark

[jira] [Updated] (SPARK-51327) [SPARK-CONNECT] unresolved_star in DataFrame select Mishandles Column Scoping After Join

[ https://issues.apache.org/jira/browse/SPARK-51327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Reddy Kumbham updated SPARK-51327: --- Description: Spark Connect misinterprets df["*"] in a DataFrame select after

[jira] [Created] (SPARK-51327) [SPARK-CONNECT] unresolved_star in DataFrame select Mishandles Column Scoping After Join

Srikanth Reddy Kumbham created SPARK-51327: -- Summary: [SPARK-CONNECT] unresolved_star in DataFrame select Mishandles Column Scoping After Join Key: SPARK-51327 URL: https://issues.apache.org/jira/browse/S

[jira] [Updated] (SPARK-51327) [SPARK-CONNECT] unresolved_star in DataFrame select Mishandles Column Scoping After Join

[ https://issues.apache.org/jira/browse/SPARK-51327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Reddy Kumbham updated SPARK-51327: --- Description: Spark Connect misinterprets df["*"] in a DataFrame select after

[jira] [Resolved] (SPARK-51277) Implement 0-arg implementation in Arrow-optimized Python UDF

[ https://issues.apache.org/jira/browse/SPARK-51277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-51277. -- Fix Version/s: 4.0.0 Assignee: Hyukjin Kwon Resolution: Fixed Fixed in https:

[jira] [Resolved] (SPARK-51273) Spark Connect Call Procedure runs the procedure twice

[ https://issues.apache.org/jira/browse/SPARK-51273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51273. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50031 [https://gith

[jira] [Assigned] (SPARK-51273) Spark Connect Call Procedure runs the procedure twice

[ https://issues.apache.org/jira/browse/SPARK-51273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-51273: --- Assignee: Szehon Ho > Spark Connect Call Procedure runs the procedure twice > -

[jira] [Resolved] (SPARK-51324) FOR empty results throws error if nested and only statement in body.

[ https://issues.apache.org/jira/browse/SPARK-51324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51324. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50090 [https://gith

[jira] [Assigned] (SPARK-51324) FOR empty results throws error if nested and only statement in body.

[ https://issues.apache.org/jira/browse/SPARK-51324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-51324: --- Assignee: Dusan Tisma > FOR empty results throws error if nested and only statement in body

[jira] [Created] (SPARK-51328) Add Spark Connect Overview link into PySpark documentation

Haejoon Lee created SPARK-51328: --- Summary: Add Spark Connect Overview link into PySpark documentation Key: SPARK-51328 URL: https://issues.apache.org/jira/browse/SPARK-51328 Project: Spark Issu

[jira] [Commented] (SPARK-51320) Failed to run spark ml on connect with pyspark-connect==4.0.0.dev2 installation

[ https://issues.apache.org/jira/browse/SPARK-51320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930906#comment-17930906 ] Bobby Wang commented on SPARK-51320: Close this task due to pyspark-connect==4.0.0.d

[jira] [Resolved] (SPARK-51320) Failed to run spark ml on connect with pyspark-connect==4.0.0.dev2 installation

[ https://issues.apache.org/jira/browse/SPARK-51320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bobby Wang resolved SPARK-51320. Resolution: Won't Fix > Failed to run spark ml on connect with pyspark-connect==4.0.0.dev2 > inst

[jira] [Updated] (SPARK-49618) Union ( & UnionExec) nodes equality not take into account unaligned positions of branches causing NO ( reuse of exchange and cached plans)

[ https://issues.apache.org/jira/browse/SPARK-49618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-49618: - Target Version/s: 4.1 Affects Version/s: 4.1 > Union ( & UnionExec) nodes equality not take into account un

[jira] [Updated] (SPARK-49789) org.apache.spark.SparkUnsupportedOperationException: [ENCODER_NOT_FOUND] Not found an encoder of the type T

[ https://issues.apache.org/jira/browse/SPARK-49789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-49789: - Target Version/s: 4.1 (was: 4.0.0) Affects Version/s: 4.1 > org.apache.spark.SparkUnsupportedOperationExce

[jira] [Updated] (SPARK-49881) SPIP : Improving analyzer performance by skipping DeduplicateRelations rule conditionally

[ https://issues.apache.org/jira/browse/SPARK-49881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-49881: - Target Version/s: 4.1 Affects Version/s: 4.1 > SPIP : Improving analyzer performance by skipping Deduplicat

[jira] [Updated] (SPARK-45959) SPIP: Abusing DataSet.withColumn can cause huge tree with severe perf degradation

[ https://issues.apache.org/jira/browse/SPARK-45959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-45959: - Target Version/s: 4.1 (was: 3.5.1) Affects Version/s: 4.1 > SPIP: Abusing DataSet.withColumn can cause hug

[jira] [Updated] (SPARK-51329) Add `numFeatures` for clustering models

[ https://issues.apache.org/jira/browse/SPARK-51329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51329: --- Labels: pull-request-available (was: ) > Add `numFeatures` for clustering models >

[jira] [Updated] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-33152: - Target Version/s: 4.1 Affects Version/s: 4.1 > SPIP: Constraint Propagation code causes OOM issues or incre

[jira] [Updated] (SPARK-51330) Enable spark.sql.execution.pythonUDTF.arrow.enabled by default

[ https://issues.apache.org/jira/browse/SPARK-51330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-51330: - Summary: Enable spark.sql.execution.pythonUDTF.arrow.enabled by default (was: Enable spark.sql.

[jira] [Updated] (SPARK-51330) Enable spark.sql.execution.pythonUDTF.arrow.enabled by default

[ https://issues.apache.org/jira/browse/SPARK-51330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51330: --- Labels: pull-request-available (was: ) > Enable spark.sql.execution.pythonUDTF.arrow.enable

[jira] [Updated] (SPARK-51330) Enable spark.sql.execution.pythonUDTF.arrow.enabled by default

[ https://issues.apache.org/jira/browse/SPARK-51330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-51330: - Labels: pull-request-available release-notes (was: pull-request-available) > Enable spark.sql.e

[jira] [Created] (SPARK-51330) Enable spark.sql.execution.pythonUDTF.arrow.enabled by defalt

Hyukjin Kwon created SPARK-51330: Summary: Enable spark.sql.execution.pythonUDTF.arrow.enabled by defalt Key: SPARK-51330 URL: https://issues.apache.org/jira/browse/SPARK-51330 Project: Spark

[jira] [Assigned] (SPARK-51206) Python Data Sources incorrectly imports from Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-51206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-51206: Assignee: Haoyu Weng > Python Data Sources incorrectly imports from Spark Connect > -

[jira] [Created] (SPARK-51331) Structured streaming batch with fixed interval trigger is committed only when next batch is about to start

Alex created SPARK-51331: Summary: Structured streaming batch with fixed interval trigger is committed only when next batch is about to start Key: SPARK-51331 URL: https://issues.apache.org/jira/browse/SPARK-51331

[jira] [Resolved] (SPARK-51206) Python Data Sources incorrectly imports from Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-51206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-51206. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 49941 [https://gi

[jira] [Updated] (SPARK-51323) "total" is mentioned twice in SparkUI for Python SQL metrics

[ https://issues.apache.org/jira/browse/SPARK-51323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51323: --- Labels: pull-request-available (was: ) > "total" is mentioned twice in SparkUI for Python S

[jira] [Created] (SPARK-51326) Remove LazyExpression proto message.

Takuya Ueshin created SPARK-51326: - Summary: Remove LazyExpression proto message. Key: SPARK-51326 URL: https://issues.apache.org/jira/browse/SPARK-51326 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-51326) Remove LazyExpression proto message.

[ https://issues.apache.org/jira/browse/SPARK-51326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51326: --- Labels: pull-request-available (was: ) > Remove LazyExpression proto message. > ---

[jira] [Commented] (SPARK-51331) Structured streaming batch with fixed interval trigger is committed only when next batch is about to start

[ https://issues.apache.org/jira/browse/SPARK-51331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930934#comment-17930934 ] Jungtaek Lim commented on SPARK-51331: -- I guess there is misunderstanding here. Co

[jira] [Resolved] (SPARK-51331) Structured streaming batch with fixed interval trigger is committed only when next batch is about to start

[ https://issues.apache.org/jira/browse/SPARK-51331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-51331. -- Resolution: Invalid > Structured streaming batch with fixed interval trigger is committed only

[jira] [Updated] (SPARK-50792) Format binary data as a binary literal in JDBC.

[ https://issues.apache.org/jira/browse/SPARK-50792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-50792: --- Parent: SPARK-38852 Issue Type: Sub-task (was: Bug) > Format binary data as a binary litera

[jira] [Created] (SPARK-51332) DS V2 supports push down BIT_AND, BIT_OR, BIT_XOR, BIT_COUNT and BIT_GET

Jiaan Geng created SPARK-51332: -- Summary: DS V2 supports push down BIT_AND, BIT_OR, BIT_XOR, BIT_COUNT and BIT_GET Key: SPARK-51332 URL: https://issues.apache.org/jira/browse/SPARK-51332 Project: Spark

[jira] [Updated] (SPARK-51314) Add proper note for distributed-sequence about indeterministic case

[ https://issues.apache.org/jira/browse/SPARK-51314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51314: --- Labels: pull-request-available (was: ) > Add proper note for distributed-sequence about ind

[jira] [Assigned] (SPARK-51265) StackOverflowError or Internal Error have raised when executing eagerlyExecuteCommands containing streaming source

[ https://issues.apache.org/jira/browse/SPARK-51265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-51265: --- Assignee: Jungtaek Lim > StackOverflowError or Internal Error have raised when executing >

[jira] [Commented] (SPARK-44157) Outdated JARs in PySpark package

[ https://issues.apache.org/jira/browse/SPARK-44157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930648#comment-17930648 ] Sakthi commented on SPARK-44157: This is mostly because probably Spark historically depe

[jira] [Created] (SPARK-51321) Add support for LPAD and RPAD pushdown in MsSQL Server JDBC connector

Uros Stankovic created SPARK-51321: -- Summary: Add support for LPAD and RPAD pushdown in MsSQL Server JDBC connector Key: SPARK-51321 URL: https://issues.apache.org/jira/browse/SPARK-51321 Project: Sp

[jira] [Resolved] (SPARK-51265) StackOverflowError or Internal Error have raised when executing eagerlyExecuteCommands containing streaming source

[ https://issues.apache.org/jira/browse/SPARK-51265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-51265. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50037 [https://gith

[jira] [Updated] (SPARK-51321) Add support for LPAD and RPAD pushdown in MsSQL Server JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-51321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51321: --- Labels: pull-request-available (was: ) > Add support for LPAD and RPAD pushdown in MsSQL Se

[jira] [Created] (SPARK-51320) Failed to run spark ml on connect with pyspark-connect==4.0.0.dev2 installation

Bobby Wang created SPARK-51320: -- Summary: Failed to run spark ml on connect with pyspark-connect==4.0.0.dev2 installation Key: SPARK-51320 URL: https://issues.apache.org/jira/browse/SPARK-51320 Project:

[jira] [Created] (SPARK-51319) array_contains functions return null when contains value

Yu Xu created SPARK-51319: - Summary: array_contains functions return null when contains value Key: SPARK-51319 URL: https://issues.apache.org/jira/browse/SPARK-51319 Project: Spark Issue Type: Improv

[jira] [Resolved] (SPARK-51312) Fix createDataFrame from RDD[Row]

[ https://issues.apache.org/jira/browse/SPARK-51312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-51312. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50079 [https://github.com

[jira] [Assigned] (SPARK-51312) Fix createDataFrame from RDD[Row]

[ https://issues.apache.org/jira/browse/SPARK-51312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-51312: Assignee: Mihailo Milosevic > Fix createDataFrame from RDD[Row] > ---

[jira] [Updated] (SPARK-51319) array_contains functions return null when contains value

[ https://issues.apache.org/jira/browse/SPARK-51319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Solimando updated SPARK-51319: - Issue Type: Bug (was: Improvement) > array_contains functions return null when

[jira] [Updated] (SPARK-51319) array_contains functions return null when contains value

[ https://issues.apache.org/jira/browse/SPARK-51319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Xu updated SPARK-51319: -- Description: Function array_contains(array(1,null), 2) return null and array_contains(array(1,null), 1) retur

[jira] [Created] (SPARK-51322) streaming subquery expression

Wenchen Fan created SPARK-51322: --- Summary: streaming subquery expression Key: SPARK-51322 URL: https://issues.apache.org/jira/browse/SPARK-51322 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-51323) "Time" is mentioned twice in SparkUI for Python SQL metrics

[ https://issues.apache.org/jira/browse/SPARK-51323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Hillig updated SPARK-51323: - Description: !image-2025-02-26-13-22-34-930.png! (was: !image-2025-02-26-13-21-31-252.p

[jira] [Created] (SPARK-51323) "Time" is mentioned twice in SparkUI for Python SQL metrics

Sebastian Hillig created SPARK-51323: Summary: "Time" is mentioned twice in SparkUI for Python SQL metrics Key: SPARK-51323 URL: https://issues.apache.org/jira/browse/SPARK-51323 Project: Spark

[jira] [Updated] (SPARK-51323) "Time" is mentioned twice in SparkUI for Python SQL metrics

[ https://issues.apache.org/jira/browse/SPARK-51323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Hillig updated SPARK-51323: - Attachment: image-2025-02-26-13-22-34-930.png > "Time" is mentioned twice in SparkUI for

[jira] [Updated] (SPARK-51323) "total" is mentioned twice in SparkUI for Python SQL metrics

[ https://issues.apache.org/jira/browse/SPARK-51323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Hillig updated SPARK-51323: - Summary: "total" is mentioned twice in SparkUI for Python SQL metrics (was: "Time" is m

[jira] [Created] (SPARK-51324) FOR empty results throws error if nested and only statement in body.

Dusan Tisma created SPARK-51324: --- Summary: FOR empty results throws error if nested and only statement in body. Key: SPARK-51324 URL: https://issues.apache.org/jira/browse/SPARK-51324 Project: Spark

[jira] [Updated] (SPARK-51324) FOR empty results throws error if nested and only statement in body.

[ https://issues.apache.org/jira/browse/SPARK-51324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dusan Tisma updated SPARK-51324: Description: FOR statement will currently fail if it is nested in a compound statement, is the on

[jira] [Updated] (SPARK-51324) FOR empty results throws error if nested and only statement in body.

[ https://issues.apache.org/jira/browse/SPARK-51324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51324: --- Labels: pull-request-available (was: ) > FOR empty results throws error if nested and only

[jira] [Updated] (SPARK-51280) The RESPONSE_ALREADY_RECEIVED error message is ambiguous

[ https://issues.apache.org/jira/browse/SPARK-51280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51280: --- Labels: pull-request-available (was: ) > The RESPONSE_ALREADY_RECEIVED error message is amb

[jira] [Updated] (SPARK-51323) "total" is mentioned twice in SparkUI for Python SQL metrics

[ https://issues.apache.org/jira/browse/SPARK-51323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Hillig updated SPARK-51323: - Description: !image-2025-02-26-13-22-34-930.png!   Should ideally just say "time to in

[jira] [Updated] (SPARK-51333) Unwrap `InvocationTargetException` thrown by `invokeMethod`

[ https://issues.apache.org/jira/browse/SPARK-51333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-51333: -- Summary: Unwrap `InvocationTargetException` thrown by `invokeMethod` (was: Unwrap `Invocation

[jira] [Created] (SPARK-51333) Unwrap `InvocationTargetException` in `invokeMethod`

Ruifeng Zheng created SPARK-51333: - Summary: Unwrap `InvocationTargetException` in `invokeMethod` Key: SPARK-51333 URL: https://issues.apache.org/jira/browse/SPARK-51333 Project: Spark Issue

[jira] [Updated] (SPARK-51333) Unwrap `InvocationTargetException` thrown by `invokeMethod`

[ https://issues.apache.org/jira/browse/SPARK-51333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51333: --- Labels: pull-request-available (was: ) > Unwrap `InvocationTargetException` thrown by `invo

[jira] [Commented] (SPARK-48091) Using `explode` together with `transform` in the same select statement causes aliases in the transformed column to be ignored

[ https://issues.apache.org/jira/browse/SPARK-48091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930987#comment-17930987 ] Sakthi commented on SPARK-48091: Worth noting that the issue is fixed in current main (m

[jira] [Updated] (SPARK-51332) DS V2 supports push down BIT_AND, BIT_OR, BIT_XOR, BIT_COUNT and BIT_GET

[ https://issues.apache.org/jira/browse/SPARK-51332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51332: --- Labels: pull-request-available (was: ) > DS V2 supports push down BIT_AND, BIT_OR, BIT_XOR,

[jira] [Resolved] (SPARK-51329) Add `numFeatures` for clustering models

[ https://issues.apache.org/jira/browse/SPARK-51329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-51329. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50095 [https://

[jira] [Commented] (SPARK-38983) Pyspark throws AnalysisException with incorrect error message when using .grouping() or .groupingId() (AnalysisException: grouping() can only be used with GroupingSets

[ https://issues.apache.org/jira/browse/SPARK-38983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930982#comment-17930982 ] Sakthi commented on SPARK-38983: It's worth noting that the error message issue is fixed

[jira] [Resolved] (SPARK-51323) "total" is mentioned twice in SparkUI for Python SQL metrics

[ https://issues.apache.org/jira/browse/SPARK-51323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-51323. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50089 [https://gi

[jira] [Assigned] (SPARK-51323) "total" is mentioned twice in SparkUI for Python SQL metrics

[ https://issues.apache.org/jira/browse/SPARK-51323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-51323: Assignee: Sebastian Hillig > "total" is mentioned twice in SparkUI for Python SQL metrics

[jira] [Resolved] (SPARK-51333) Unwrap `InvocationTargetException` thrown by `invokeMethod`

[ https://issues.apache.org/jira/browse/SPARK-51333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-51333. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50098 [https://

[jira] [Resolved] (SPARK-51302) Spark Connect supports JDBC should use the DataFrameReader API

[ https://issues.apache.org/jira/browse/SPARK-51302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-51302. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50059 [https://github

[jira] [Assigned] (SPARK-51333) Unwrap `InvocationTargetException` thrown by `invokeMethod`

[ https://issues.apache.org/jira/browse/SPARK-51333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-51333: - Assignee: Ruifeng Zheng > Unwrap `InvocationTargetException` thrown by `invokeMethod` >

[jira] [Updated] (SPARK-51271) Python Data Sources Filter Pushdown API

[ https://issues.apache.org/jira/browse/SPARK-51271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51271: --- Labels: pull-request-available (was: ) > Python Data Sources Filter Pushdown API >

[jira] [Updated] (SPARK-51332) DS V2 supports push down BIT_AND, BIT_OR, BIT_XOR, BIT_COUNT and BIT_GET

[ https://issues.apache.org/jira/browse/SPARK-51332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng updated SPARK-51332: --- Parent: SPARK-38852 Issue Type: Sub-task (was: Improvement) > DS V2 supports push down BIT_

[jira] [Resolved] (SPARK-51316) Allow Arrow batches in bytes instead of number of rows

[ https://issues.apache.org/jira/browse/SPARK-51316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-51316. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50080 [https://gi

[jira] [Assigned] (SPARK-51316) Allow Arrow batches in bytes instead of number of rows

[ https://issues.apache.org/jira/browse/SPARK-51316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-51316: Assignee: Hyukjin Kwon > Allow Arrow batches in bytes instead of number of rows > ---

[jira] [Updated] (SPARK-44856) Improve Python UDTF arrow serializer performance

[ https://issues.apache.org/jira/browse/SPARK-44856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44856: --- Labels: pull-request-available (was: ) > Improve Python UDTF arrow serializer performance >