Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
beliefer commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912016999 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Shared

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912037537 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50403][SQL] Fix parameterized `EXECUTE IMMEDIATE` [spark]

2025-01-11 Thread via GitHub
srielau commented on PR #49442: URL: https://github.com/apache/spark/pull/49442#issuecomment-2585286568 General comment: ensure that the parameter markers inside the execute immediate string are NOT bound by anything except the USING clause. So spark.sql("execute immediate 'select ?'", se

[PR] [SPARK-50793][SQL] Fix MySQL cast function for LONGTEXT, SMALLINT, INTEGER, BIGINT and BLOB types [spark]

2025-01-11 Thread via GitHub
sunxiaoguang opened a new pull request, #49453: URL: https://github.com/apache/spark/pull/49453 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### Ho

Re: [PR] [SPARK-50704][SQL] Support more pushdown functions for MySQL connector [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49335: URL: https://github.com/apache/spark/pull/49335#discussion_r1912063118 ## sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala: ## @@ -374,6 +374,7 @@ abstract class JdbcDialect extends Serializable with Logging {

Re: [PR] [SPARK-50403][SQL] Fix parameterized `EXECUTE IMMEDIATE` [spark]

2025-01-11 Thread via GitHub
srielau commented on PR #49442: URL: https://github.com/apache/spark/pull/49442#issuecomment-2585328459 Because it’s a question mark in a string. It is the job of execute immediate using to bind parameter markers to the plan generated by that string. The outside client has no business bin

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912092465 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

[PR] Test ivy 2.5.3 [spark]

2025-01-11 Thread via GitHub
LuciferYang opened a new pull request, #49454: URL: https://github.com/apache/spark/pull/49454 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How

Re: [PR] [SPARK-50403][SQL] Fix parameterized `EXECUTE IMMEDIATE` [spark]

2025-01-11 Thread via GitHub
srielau commented on PR #49442: URL: https://github.com/apache/spark/pull/49442#issuecomment-2585359079 Let me add some more color: "EXECUTE IMMEDIATE 'SELECT :var1, :var2' USING :var2 AS var1, :var1 AS var2", map("var1" -> 5, "var2" -> 6) The EXECUTE IMMEDIATE binds in map("var1" ->

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1911960764 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50791] Fix NPE in State Store error handling [spark]

2025-01-11 Thread via GitHub
LuciferYang commented on code in PR #49451: URL: https://github.com/apache/spark/pull/49451#discussion_r1912021158 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala: ## @@ -291,7 +291,8 @@ private[sql] class HDFSBackedSt

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912024105 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912024105 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50791] Fix NPE in State Store error handling [spark]

2025-01-11 Thread via GitHub
LuciferYang commented on code in PR #49451: URL: https://github.com/apache/spark/pull/49451#discussion_r1912021158 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala: ## @@ -291,7 +291,8 @@ private[sql] class HDFSBackedSt

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
beliefer commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912016999 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Shared

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912024105 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
beliefer commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912022086 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Shared

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912024105 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912024105 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50791] Fix NPE in State Store error handling [spark]

2025-01-11 Thread via GitHub
LuciferYang commented on PR #49451: URL: https://github.com/apache/spark/pull/49451#issuecomment-2585254674 ``` [info] - get, put, remove, commit, and all data iterator - with codec lz4 - with colFamiliesEnabled=true *** FAILED *** (157 milliseconds) [info] "[CANNOT_LOAD_STATE_STORE.

Re: [PR] [SPARK-50791] Fix NPE in State Store error handling [spark]

2025-01-11 Thread via GitHub
LuciferYang commented on code in PR #49451: URL: https://github.com/apache/spark/pull/49451#discussion_r1912021158 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala: ## @@ -291,7 +291,8 @@ private[sql] class HDFSBackedSt

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
beliefer commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912016999 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Shared

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912037537 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912038868 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912038868 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1911960764 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1911960764 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50787][DOCS] Fix typos and add missing semicolons in sql examples [spark]

2025-01-11 Thread via GitHub
the-sakthi commented on PR #49448: URL: https://github.com/apache/spark/pull/49448#issuecomment-2585311127 Thanks for all the help here, @MaxGekk and @dongjoon-hyun ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [SPARK-50403][SQL] Fix parameterized `EXECUTE IMMEDIATE` [spark]

2025-01-11 Thread via GitHub
MaxGekk commented on PR #49442: URL: https://github.com/apache/spark/pull/49442#issuecomment-2585325909 > General comment: ensure that the parameter markers inside the execute immediate string are NOT bound by anything except the USING clause. @srielau Is it restricted by the SQL stan

Re: [PR] [MINOR][BUILD] Pom compiler source target [spark]

2025-01-11 Thread via GitHub
github-actions[bot] closed pull request #48312: [MINOR][BUILD] Pom compiler source target URL: https://github.com/apache/spark/pull/48312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [SPARK-49604][SQL] GROUP_BY_POS_OUT_OF_RANGE should not appear if group by Literal [spark]

2025-01-11 Thread via GitHub
github-actions[bot] commented on PR #48340: URL: https://github.com/apache/spark/pull/48340#issuecomment-2585493911 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

Re: [PR] [SPARK-49683][SQL]Block trim collations on all collation code paths. [spark]

2025-01-11 Thread via GitHub
github-actions[bot] commented on PR #48311: URL: https://github.com/apache/spark/pull/48311#issuecomment-2585493919 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912343465 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912343465 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50704][SQL] Support more pushdown functions for MySQL connector [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49335: URL: https://github.com/apache/spark/pull/49335#discussion_r1912344126 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/MySQLIntegrationSuite.scala: ## @@ -241,6 +241,323 @@ class MySQLIntegrationSuite

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
beliefer commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1911957461 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Shared

Re: [PR] [SPARK-50704][SQL] Support more pushdown functions for MySQL connector [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49335: URL: https://github.com/apache/spark/pull/49335#discussion_r1911941444 ## sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala: ## @@ -374,6 +374,7 @@ abstract class JdbcDialect extends Serializable with Logging {

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912002030 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912002030 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

[PR] [SPARK-50792] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang opened a new pull request, #49452: URL: https://github.com/apache/spark/pull/49452 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### Ho

Re: [PR] [SPARK-50704][SQL] Support more pushdown functions for MySQL connector [spark]

2025-01-11 Thread via GitHub
beliefer commented on code in PR #49335: URL: https://github.com/apache/spark/pull/49335#discussion_r1911955010 ## sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala: ## @@ -374,6 +374,7 @@ abstract class JdbcDialect extends Serializable with Logging { ca

Re: [PR] [SPARK-50704][SQL] Support more pushdown functions for MySQL connector [spark]

2025-01-11 Thread via GitHub
beliefer commented on code in PR #49335: URL: https://github.com/apache/spark/pull/49335#discussion_r1911955010 ## sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala: ## @@ -374,6 +374,7 @@ abstract class JdbcDialect extends Serializable with Logging { ca

Re: [PR] [SPARK-50780][SQL] Use `overrideStdFeatures` instead of `setFeatureMask` in `JacksonParser` [spark]

2025-01-11 Thread via GitHub
LuciferYang commented on PR #49434: URL: https://github.com/apache/spark/pull/49434#issuecomment-2585220909 Thanks @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] [SPARK-50768][CORE] Introduce TaskContext.createResourceUninterruptibly to avoid stream leak by task interruption [spark]

2025-01-11 Thread via GitHub
LuciferYang commented on code in PR #49413: URL: https://github.com/apache/spark/pull/49413#discussion_r1911977257 ## core/src/test/scala/org/apache/spark/JobCancellationSuite.scala: ## @@ -712,6 +713,140 @@ class JobCancellationSuite extends SparkFunSuite with Matchers with Be

Re: [PR] [SPARK-50773][Core] Disable structured logging by default [spark]

2025-01-11 Thread via GitHub
gengliangwang commented on PR #49421: URL: https://github.com/apache/spark/pull/49421#issuecomment-2585459964 @dongjoon-hyun Thank you for following up on this PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] [SPARK-50788][TESTS] Add Benchmark for Large-Row Dataframe [spark]

2025-01-11 Thread via GitHub
yhuang-db commented on code in PR #49447: URL: https://github.com/apache/spark/pull/49447#discussion_r1912371900 ## sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/LargeRowBenchmark.scala: ## @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-43225][BUILD][SQL] Remove jackson-core-asl and jackson-mapper-asl from pre-built distribution [spark]

2025-01-11 Thread via GitHub
Madhukar525722 commented on PR #40893: URL: https://github.com/apache/spark/pull/40893#issuecomment-2585621542 HI @pan3793 @dongjoon-hyun . Since, the spark 4.0.0 has been upgraded to hive 2.3.10 and hadoop 3.3.6. Cant we now implement this change back to remove jackson-mapper-asl and jacks

Re: [PR] [SPARK-50773][Core] Disable structured logging by default [spark]

2025-01-11 Thread via GitHub
beliefer commented on code in PR #49421: URL: https://github.com/apache/spark/pull/49421#discussion_r1912349422 ## core/src/main/scala/org/apache/spark/util/Utils.scala: ## @@ -2689,7 +2689,7 @@ private[spark] object Utils * loading SparkConf. */ def resetStructuredLo

Re: [PR] [SPARK-50704][SQL] Support more pushdown functions for MySQL connector [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49335: URL: https://github.com/apache/spark/pull/49335#discussion_r1911941444 ## sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala: ## @@ -374,6 +374,7 @@ abstract class JdbcDialect extends Serializable with Logging {

Re: [PR] [SPARK-50787][DOCS] Fix typos and add missing semicolons in sql examples [spark]

2025-01-11 Thread via GitHub
MaxGekk commented on PR #49448: URL: https://github.com/apache/spark/pull/49448#issuecomment-2585159334 +1, LGTM. Merging to master. Thank you, @the-sakthi and @dongjoon-hyun for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] [SPARK-50787][DOCS] Fix typos and add missing semicolons in sql examples [spark]

2025-01-11 Thread via GitHub
MaxGekk closed pull request #49448: [SPARK-50787][DOCS] Fix typos and add missing semicolons in sql examples URL: https://github.com/apache/spark/pull/49448 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [SPARK-50704][SQL] Support more pushdown functions for MySQL connector [spark]

2025-01-11 Thread via GitHub
beliefer commented on code in PR #49335: URL: https://github.com/apache/spark/pull/49335#discussion_r1910336670 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/MySQLIntegrationSuite.scala: ## @@ -241,6 +241,323 @@ class MySQLIntegrationSuite ext

Re: [PR] [SPARK-50787][DOCS] Fix typos and add missing semicolons in sql examples [spark]

2025-01-11 Thread via GitHub
MaxGekk commented on PR #49448: URL: https://github.com/apache/spark/pull/49448#issuecomment-2585159960 @the-sakthi Congratulations with your first contribution to Apache Spark! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [SPARK-50704][SQL] Support more pushdown functions for MySQL connector [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49335: URL: https://github.com/apache/spark/pull/49335#discussion_r1912067448 ## sql/core/src/main/scala/org/apache/spark/sql/jdbc/MySQLDialect.scala: ## @@ -112,6 +151,19 @@ private case class MySQLDialect() extends JdbcDialect with SQLCon

Re: [PR] [SPARK-50704][SQL] Support more pushdown functions for MySQL connector [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49335: URL: https://github.com/apache/spark/pull/49335#discussion_r1912067929 ## sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala: ## @@ -374,6 +374,7 @@ abstract class JdbcDialect extends Serializable with Logging {

Re: [PR] [SPARK-50704][SQL] Support more pushdown functions for MySQL connector [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49335: URL: https://github.com/apache/spark/pull/49335#discussion_r1911941444 ## sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala: ## @@ -374,6 +374,7 @@ abstract class JdbcDialect extends Serializable with Logging {

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912038868 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50792][SQL] Format binary data as a binary literal in JDBC. [spark]

2025-01-11 Thread via GitHub
sunxiaoguang commented on code in PR #49452: URL: https://github.com/apache/spark/pull/49452#discussion_r1912037537 ## connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2/V2JDBCTest.scala: ## @@ -986,4 +986,18 @@ private[v2] trait V2JDBCTest extends Sh

Re: [PR] [SPARK-50790][PYTHON] Implement parse json in pyspark [spark]

2025-01-11 Thread via GitHub
gene-db commented on code in PR #49450: URL: https://github.com/apache/spark/pull/49450#discussion_r1912204137 ## python/pyspark/sql/variant_utils.py: ## @@ -496,3 +525,297 @@ def _handle_array(cls, value: bytes, pos: int, func: Callable[[List[int]], Any]) element_

Re: [PR] [SPARK-50787][DOCS] Fix typos and add missing semicolons in sql examples [spark]

2025-01-11 Thread via GitHub
dongjoon-hyun commented on PR #49448: URL: https://github.com/apache/spark/pull/49448#issuecomment-2585402415 Thank YOU, @the-sakthi and @MaxGekk ! :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go