Re: [PR] [SPARK-48353][SQL] Introduction of Error Handling mechanism in SQL Scripting [spark]

srielau commented on code in PR #49427: URL: https://github.com/apache/spark/pull/49427#discussion_r1920376339 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala: ## @@ -159,15 +159,104 @@ class AstBuilder extends DataTypeAstBuilder script

Re: [PR] [SPARK-48353][SQL] Introduction of Error Handling mechanism in SQL Scripting [spark]

srielau commented on code in PR #49427: URL: https://github.com/apache/spark/pull/49427#discussion_r1920376339 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala: ## @@ -159,15 +159,104 @@ class AstBuilder extends DataTypeAstBuilder script

Re: [PR] [SPARK-50871][BUILD] Upgrade `scala-parallel-collections` to 1.2.0 [spark]

LuciferYang commented on PR #49548: URL: https://github.com/apache/spark/pull/49548#issuecomment-2597669816 Test first -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[PR] [SPARK-50871][BUILD] Upgrade `scala-parallel-collections` to 1.2.0 [spark]

LuciferYang opened a new pull request, #49548: URL: https://github.com/apache/spark/pull/49548 ### What changes were proposed in this pull request? This pr aims to upgrade `scala-parallel-collections` from 1.0.4 to 1.2.0 ### Why are the changes needed? The new version includes a

Re: [PR] [SPARK-50793][SQL] Fix MySQL cast function for DOUBLE, LONGTEXT, BIGINT and BLOB types [spark]

sunxiaoguang commented on code in PR #49453: URL: https://github.com/apache/spark/pull/49453#discussion_r1919787981 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/MySQLIntegrationSuite.scala: ## @@ -241,6 +241,72 @@ class MySQLIntegrationSuite

Re: [PR] Remove the explicit dependency on `Guava` from `plugins.sbt` [spark]

LuciferYang commented on code in PR #49550: URL: https://github.com/apache/spark/pull/49550#discussion_r1919782249 ## project/plugins.sbt: ## @@ -21,9 +21,6 @@ addSbtPlugin("software.purpledragon" % "sbt-checkstyle-plugin" % "4.0.1") // please check pom.xml in the root of the

Re: [PR] [SPARK-50869][ML][CONNECT][PYTHON] Support evaluators on ML Connet [spark]

zhengruifeng commented on code in PR #49547: URL: https://github.com/apache/spark/pull/49547#discussion_r1919749377 ## python/pyspark/ml/tests/connect/test_parity_evaluation.py: ## @@ -0,0 +1,49 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contr

Re: [PR] [SPARK-50793][SQL] Fix MySQL cast function for DOUBLE, LONGTEXT, BIGINT and BLOB types [spark]

sunxiaoguang commented on code in PR #49453: URL: https://github.com/apache/spark/pull/49453#discussion_r1919787981 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/MySQLIntegrationSuite.scala: ## @@ -241,6 +241,72 @@ class MySQLIntegrationSuite

Re: [PR] [SPARK-50793][SQL] Fix MySQL cast function for DOUBLE, LONGTEXT, BIGINT and BLOB types [spark]

sunxiaoguang commented on code in PR #49453: URL: https://github.com/apache/spark/pull/49453#discussion_r1919787981 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/MySQLIntegrationSuite.scala: ## @@ -241,6 +241,72 @@ class MySQLIntegrationSuite

Re: [PR] [SPARK-50739][SQL][FOLLOW] Simplify ResolveRecursiveCTESuite with dsl [spark]

cloud-fan commented on PR #49557: URL: https://github.com/apache/spark/pull/49557#issuecomment-2598677019 cc @nemanjapetr-db @MaxGekk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[PR] [SPARK-50739][SQL][FOLLOW] Simplify ResolveRecursiveCTESuite with dsl [spark]

cloud-fan opened a new pull request, #49557: URL: https://github.com/apache/spark/pull/49557 ### What changes were proposed in this pull request? A followup of https://github.com/apache/spark/pull/49351 to simplify the test via dsl. ### Why are the changes needed?

Re: [PR] [SPARK-50847] [SQL] Deny ApplyCharTypePadding from applying on specific In expressions [spark]

mihailoale-db commented on code in PR #49526: URL: https://github.com/apache/spark/pull/49526#discussion_r1920400881 ## sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala: ## @@ -1174,4 +1175,22 @@ class DSV2CharVarcharTestSuite extends CharVarcharTestSuite

Re: [PR] [SPARK-48353][SQL] Introduction of Error Handling mechanism in SQL Scripting [spark]

srielau commented on code in PR #49427: URL: https://github.com/apache/spark/pull/49427#discussion_r1920517706 ## sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingExecutionContext.scala: ## @@ -81,12 +107,79 @@ class SqlScriptingExecutionFrame( scopes.re

Re: [PR] [SPARK-50853][CORE] Close temp shuffle file writable channel [spark]

ChenMichael commented on PR #49531: URL: https://github.com/apache/spark/pull/49531#issuecomment-2598839045 Yea, I was finding it difficult to write a unit test for this, so the way I tested was by launching spark shell, running some queries that would force shuffles and looking at the numb

Re: [PR] [SPARK-50874][ML][PYTHON][CONNECT] Support `LinearRegression` on connect [spark]

grundprinzip commented on code in PR #49553: URL: https://github.com/apache/spark/pull/49553#discussion_r1920467658 ## mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala: ## @@ -903,7 +903,7 @@ class LinearRegressionSummary private[regression] ( val

Re: [PR] [SPARK-48353][SQL] Introduction of Error Handling mechanism in SQL Scripting [spark]

miland-db commented on code in PR #49427: URL: https://github.com/apache/spark/pull/49427#discussion_r1920400108 ## sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4: ## @@ -79,6 +81,29 @@ setStatementWithOptionalVarKeyword LEFT_PAREN query R

Re: [PR] [WIP][SPARK-50838][SQL]Add checkRecursion to check if all the rules about recursive queries are fulfilled. Adjust optimizer with UnionLoop cases. [spark]

dtenedor commented on code in PR #49518: URL: https://github.com/apache/spark/pull/49518#discussion_r1920578691 ## common/utils/src/main/resources/error/error-conditions.json: ## @@ -3099,6 +3099,29 @@ ], "sqlState" : "42602" }, + "INVALID_RECURSIVE_REFERENCE" : {

Re: [PR] [SPARK-50847] [SQL] Deny ApplyCharTypePadding from applying on specific In expressions [spark]

mihailoale-db commented on code in PR #49526: URL: https://github.com/apache/spark/pull/49526#discussion_r1919805518 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ApplyCharTypePaddingHelper.scala: ## @@ -99,7 +99,9 @@ object ApplyCharTypePaddingHelper {

Re: [PR] [WIP][SPARK-24815] [CORE] Trigger Interval based DRA for Structured Streaming [spark]

EdoardoVarani commented on PR #42352: URL: https://github.com/apache/spark/pull/42352#issuecomment-2597911581 Guys can you reopen this? I've found the streaming.dynamicAllocation setting to work beautifully in the normal streaming jobs and it's sad to see it's not available in SSS. Is there

Re: [PR] Remove the explicit dependency on `Guava` from `plugins.sbt` [spark]

LuciferYang commented on code in PR #49550: URL: https://github.com/apache/spark/pull/49550#discussion_r1919834335 ## project/plugins.sbt: ## @@ -21,9 +21,6 @@ addSbtPlugin("software.purpledragon" % "sbt-checkstyle-plugin" % "4.0.1") // please check pom.xml in the root of the

[PR] [SPARK-50872][SQL][UI] Makes ToPrettyString expression not affect UI presentation [spark]

yaooqinn opened a new pull request, #49551: URL: https://github.com/apache/spark/pull/49551 ### What changes were proposed in this pull request? The Plan graph and plan detail are affected by the injected ToPrettyString expression. They are not consistent with the output of our `expla

Re: [PR] [SPARK-48353][SQL] Introduction of Error Handling mechanism in SQL Scripting [spark]

miland-db commented on code in PR #49427: URL: https://github.com/apache/spark/pull/49427#discussion_r1919870849 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/SqlScriptingLogicalPlans.scala: ## @@ -298,3 +305,45 @@ case class ForStatement( For

Re: [PR] [SPARK-48353][SQL] Introduction of Error Handling mechanism in SQL Scripting [spark]

miland-db commented on code in PR #49427: URL: https://github.com/apache/spark/pull/49427#discussion_r1919874160 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala: ## @@ -159,15 +159,104 @@ class AstBuilder extends DataTypeAstBuilder scrip

Re: [PR] [SPARK-48353][SQL] Introduction of Error Handling mechanism in SQL Scripting [spark]

miland-db commented on code in PR #49427: URL: https://github.com/apache/spark/pull/49427#discussion_r1919873267 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala: ## @@ -159,15 +159,104 @@ class AstBuilder extends DataTypeAstBuilder scrip

Re: [PR] [SPARK-50633][INFRA][FOLLOWUP][4.0] Let CODECOV_TOKEN transfer to build_and_test.yml [spark]

panbingkun commented on PR #49541: URL: https://github.com/apache/spark/pull/49541#issuecomment-2598006543 cc @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] [SPARK-49014][BUILD][DOCS] Synchronize the Avro version in `SparkBuild.scala/docs/comments to` 1.12.0. [spark]

LuciferYang opened a new pull request, #49552: URL: https://github.com/apache/spark/pull/49552 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How

Re: [PR] [SPARK-50875][SQL] Add RTRIM collations to TVF [spark]

stevomitric commented on PR #49554: URL: https://github.com/apache/spark/pull/49554#issuecomment-2598164875 cc @stefankandic and @dejankrak-db to take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] [WIP][SPARK-50838][SQL]Add checkRecursion to check if all the rules about recursive queries are fulfilled. Adjust optimizer with UnionLoop cases. [spark]

milanisvet commented on code in PR #49518: URL: https://github.com/apache/spark/pull/49518#discussion_r1920073718 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala: ## @@ -1042,6 +1043,75 @@ trait CheckAnalysis extends PredicateHelper with

[PR] [SPARK-50875][SQL] Add RTRIM collations to TVF [spark]

stevomitric opened a new pull request, #49554: URL: https://github.com/apache/spark/pull/49554 ### What changes were proposed in this pull request? Added all RTRIM collations to TVF when using `Collations` generator. ### Why are the changes needed? As per the collation projec

<    1   2