Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-20 Thread via GitHub
dongjoon-hyun commented on PR #49445: URL: https://github.com/apache/spark/pull/49445#issuecomment-2671929034 Thank you, @dusantism-db and @cloud-fan . +1 for the backporting decision for Apache Spark 4.0.0. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-20 Thread via GitHub
cloud-fan commented on PR #49445: URL: https://github.com/apache/spark/pull/49445#issuecomment-2671483674 thanks, merging to master/4.0! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-20 Thread via GitHub
cloud-fan closed pull request #49445: [SPARK-48530][SQL] Support for local variables in SQL Scripting URL: https://github.com/apache/spark/pull/49445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-20 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1963389892 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -47,21 +50,45 @@ case class SetVariableExec(variables: Seq[Varia

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-19 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1962975996 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -47,21 +50,45 @@ case class SetVariableExec(variables: Seq[Variable

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-18 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1960801086 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,41 @@ trait ColumnResolutionHelper extends L

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-18 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1960093997 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/executeImmediate.scala: ## @@ -152,9 +154,16 @@ class SubstituteExecuteImmediate(val catalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-14 Thread via GitHub
vladimirg-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1956472572 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/executeImmediate.scala: ## @@ -152,9 +154,16 @@ class SubstituteExecuteImmediate(val catalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-13 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1954770223 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,41 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-13 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1954762992 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala: ## @@ -350,3 +357,12 @@ class SqlScriptingLabelContext { } } } + +obje

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-13 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1954447510 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,41 @@ trait ColumnResolutionHelper extends L

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-13 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1954282034 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala: ## @@ -350,3 +357,12 @@ class SqlScriptingLabelContext { } } } + +o

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-13 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1954078549 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala: ## @@ -350,3 +357,12 @@ class SqlScriptingLabelContext { } } } + +obje

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-13 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1954045545 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,41 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-12 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1953075312 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,41 @@ trait ColumnResolutionHelper extends L

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-12 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1952799496 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,41 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-12 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1952787519 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +109,34 @@ class ResolveCatalogs(val catalogManager: Catalog

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-12 Thread via GitHub
dusantism-db commented on PR #49445: URL: https://github.com/apache/spark/pull/49445#issuecomment-2653856575 @cloud-fan I've made changes according to your comments, forbade use of DROP TEMPORARY VARIABLE in scripts per offline discussion, updated API of VariableManager and updated FOR to u

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-12 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1952747979 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -35,10 +38,29 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-12 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1952747169 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-11 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1950444526 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-11 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1950444526 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-11 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1950436510 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -35,10 +38,29 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-11 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1950432423 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -35,10 +38,29 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-10 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1948984762 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/VariableManager.scala: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-10 Thread via GitHub
dusantism-db commented on PR #49445: URL: https://github.com/apache/spark/pull/49445#issuecomment-2647865354 Considering the size of this PR, and its complexity, I propose that we split some further improvements into followup PRs. Handling `session` and `system` variable names could be don

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-09 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1948367347 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -17,26 +17,29 @@ package org.apache.spark.sql.execution.comman

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-09 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1948366905 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -47,21 +50,51 @@ case class SetVariableExec(variables: Seq[Varia

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-09 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1948357547 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -47,21 +50,51 @@ case class SetVariableExec(variables: Seq[Variable

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-09 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1948357356 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -17,26 +17,29 @@ package org.apache.spark.sql.execution.command.v

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-09 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1948357083 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/VariableManager.scala: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-09 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1948356321 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-09 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1948356233 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-09 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1948354168 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,42 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946918322 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/VariableManager.scala: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946919312 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/CreateVariableExec.scala: ## @@ -19,29 +19,47 @@ package org.apache.spark.sql.execution.comman

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946916971 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends L

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on PR #49445: URL: https://github.com/apache/spark/pull/49445#issuecomment-2643554156 > @dusantism-db we have the same problem in column resolution: is `x.x` a column `x` of table `x`, or an inner field `x` inside column `x`? The solution is to define priority: prefer

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on PR #49445: URL: https://github.com/apache/spark/pull/49445#issuecomment-2643541976 @dusantism-db we have the same problem in column resolution: is `x.x` a column `x` of table `x`, or an inner field `x` inside column `x`? The solution is to define priority: prefer the

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946879302 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/V2CommandStrategy.scala: ## @@ -28,13 +28,15 @@ object V2CommandStrategy extends Strategy { //

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946878523 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -47,21 +50,45 @@ case class SetVariableExec(variables: Seq[Variable

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946879402 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/VariableManager.scala: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946867299 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946877129 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -47,21 +50,45 @@ case class SetVariableExec(variables: Seq[Varia

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946874343 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -47,21 +50,45 @@ case class SetVariableExec(variables: Seq[Variable

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946873936 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946871954 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on PR #49445: URL: https://github.com/apache/spark/pull/49445#issuecomment-2643491679 @cloud-fan Another question regarding ambiguity. Consider the following situation: ``` BEGIN x: BEGIN DECLARE x = named_struct('x', 1); SELECT x.x; E

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946867827 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946864387 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946863023 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/CreateVariableExec.scala: ## @@ -19,29 +19,47 @@ package org.apache.spark.sql.execution.command.v

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946860345 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala: ## @@ -269,6 +269,94 @@ class SqlScriptingParserSuite extends Sp

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946859026 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/VariableManager.scala: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946860345 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala: ## @@ -269,6 +269,94 @@ class SqlScriptingParserSuite extends Sp

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946857357 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +95,49 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946851423 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946721199 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala: ## @@ -269,6 +269,94 @@ class SqlScriptingParserSuite extends Sp

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946701832 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends L

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946705264 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +93,39 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946703595 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -34,11 +37,29 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946702999 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -34,11 +37,29 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946696933 ## sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingInterpreter.scala: ## @@ -17,10 +17,8 @@ package org.apache.spark.sql.scripting -import

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946693407 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -34,11 +37,29 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946691800 ## sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingLocalVariableManager.scala: ## @@ -60,42 +60,52 @@ class SqlScriptingLocalVariableManager(con

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
davidm-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946681720 ## sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingLocalVariableManager.scala: ## @@ -60,42 +60,52 @@ class SqlScriptingLocalVariableManager(contex

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
davidm-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946624121 ## sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingLocalVariableManager.scala: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946424294 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/VariableManager.scala: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946120645 ## sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingLocalVariableManager.scala: ## @@ -0,0 +1,124 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-07 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946116964 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala: ## @@ -269,6 +269,94 @@ class SqlScriptingParserSuite extends Spark

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946101345 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/VariableManager.scala: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946098020 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala: ## @@ -269,6 +269,94 @@ class SqlScriptingParserSuite extends Spark

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946077466 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946078177 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946078177 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946077466 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1946077466 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1945743963 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -34,11 +37,29 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1944733178 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala: ## @@ -269,6 +269,94 @@ class SqlScriptingParserSuite extends Sp

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1944727379 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +94,39 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1944736691 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -47,21 +50,46 @@ case class SetVariableExec(variables: Seq[Varia

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1944679337 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends L

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1944679337 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends L

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-29 Thread via GitHub
MaxGekk commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1933782530 ## sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingInterpreter.scala: ## @@ -17,10 +17,8 @@ package org.apache.spark.sql.scripting -import org.a

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931463024 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -34,11 +37,29 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931463233 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -34,11 +37,29 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931446944 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -47,21 +50,46 @@ case class SetVariableExec(variables: Seq[Variable

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931445591 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala: ## @@ -269,6 +269,94 @@ class SqlScriptingParserSuite extends Spark

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931444990 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala: ## @@ -269,6 +269,94 @@ class SqlScriptingParserSuite extends Spark

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931442079 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -34,11 +37,29 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931442784 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +94,39 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931437532 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -34,11 +37,29 @@ class ResolveCatalogs(val catalogManager: CatalogM

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931436154 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
cloud-fan commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931434864 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends Logg

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1931001798 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveCatalogs.scala: ## @@ -73,28 +94,39 @@ class ResolveCatalogs(val catalogManager: Catal

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
srielau commented on PR #49445: URL: https://github.com/apache/spark/pull/49445#issuecomment-2615922926 We still need to know that the label is not user generated. So we do not expose the name in error messages. Sent from my iPhone On Jan 27, 2025, at 4:24 AM, Dušan Tišma ***@*

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
dusantism-db commented on PR #49445: URL: https://github.com/apache/spark/pull/49445#issuecomment-2615827477 @cloud-fan I've resolved the comments, please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1930537381 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala: ## @@ -1591,7 +1591,10 @@ case class TableSpec( case class CreateVari

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1930538996 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +293,54 @@ trait ColumnResolutionHelper extends L

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1930538139 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlScriptingVariableManager.scala: ## @@ -0,0 +1,25 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-01-27 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1930446620 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlScriptingVariableManager.scala: ## @@ -0,0 +1,25 @@ +/* + * Licensed to the Apache Software Foundat

  1   2   3   >