Re: [PR] [SPARK-51042][SQL] Read and write the month and days fields of intervals with one call in Unsafe* classes [spark]

2025-02-06 Thread via GitHub
MaxGekk closed pull request #49737: [SPARK-51042][SQL] Read and write the month and days fields of intervals with one call in Unsafe* classes URL: https://github.com/apache/spark/pull/49737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[PR] [SPARK-50557][CONNECT][SQL] Support RuntimeConfig.contains(..) in Scala SQL Interface [spark]

2025-02-06 Thread via GitHub
hvanhovell opened a new pull request, #49836: URL: https://github.com/apache/spark/pull/49836 ### What changes were proposed in this pull request? This PR restores support for the `contains(..)` method in the `org.apache.spark.sql.RuntimeConfig` interface. ### Why are the changes n

Re: [PR] [SPARK-51097] [SS] Adding partition-level metrics for last uploaded snapshot version in RocksDB [spark]

2025-02-06 Thread via GitHub
ericm-db commented on code in PR #49816: URL: https://github.com/apache/spark/pull/49816#discussion_r1945082257 ## sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala: ## @@ -321,6 +321,37 @@ case class StateStoreCustomTimingMetric(name: Strin

Re: [PR] [SPARK-51042][SQL] Read and write the month and days fields of intervals with one call in Unsafe* classes [spark]

2025-02-06 Thread via GitHub
MaxGekk commented on PR #49737: URL: https://github.com/apache/spark/pull/49737#issuecomment-2640466606 @jonathan-albrecht-ibm Congratulations with your first contribution to Apache Spark! -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] [SPARK-51093][SQL][TESTS] Fix minor endianness issues in tests. [spark]

2025-02-06 Thread via GitHub
jonathan-albrecht-ibm commented on PR #49812: URL: https://github.com/apache/spark/pull/49812#issuecomment-2640479692 The CI failed on a pyspark mllib test with: ``` RuntimeError: The server socket has failed to listen on any local network address. port: 38587, useIpv6: 0, code: -98, n

Re: [PR] [SPARK-50605][CONNECT] Support SQL API mode for easier migration to Spark Connect [spark]

2025-02-06 Thread via GitHub
LuciferYang commented on code in PR #49107: URL: https://github.com/apache/spark/pull/49107#discussion_r1945123620 ## resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala: ## @@ -237,6 +265,16 @@ class YarnClusterSuite extends BaseYarnCluster

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1944736691 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/v2/SetVariableExec.scala: ## @@ -47,21 +50,46 @@ case class SetVariableExec(variables: Seq[Varia

Re: [PR] [SPARK-49371][SPARK-51086][SQL][CONNECT][DOCS] Fix various doc SQL issues. [spark]

2025-02-06 Thread via GitHub
hvanhovell commented on PR #49800: URL: https://github.com/apache/spark/pull/49800#issuecomment-2639845374 Merging to 4.0/master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] [SPARK-49371][SPARK-51086][SQL][CONNECT][DOCS] Fix various doc SQL issues. [spark]

2025-02-06 Thread via GitHub
asfgit closed pull request #49800: [SPARK-49371][SPARK-51086][SQL][CONNECT][DOCS] Fix various doc SQL issues. URL: https://github.com/apache/spark/pull/49800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[PR] Proper error handling for file status when reading files [spark]

2025-02-06 Thread via GitHub
fusheng9399 opened a new pull request, #49833: URL: https://github.com/apache/spark/pull/49833 ### What changes were proposed in this pull request? Add clear error messages for reading a file ### Why are the changes needed? For these errors, we report ambiguous er

Re: [PR] [SPARK-51111][SS] Avoid consumer rebalancing stuck when starting a spark streaming job [spark]

2025-02-06 Thread via GitHub
yabola commented on PR #49831: URL: https://github.com/apache/spark/pull/49831#issuecomment-2640354253 It can also reproduce locally (my computer has 12 cores , so I create create 13 DirectStream) demo: [KafkaJobTest.txt](https://github.com/user-attachments/files/18693279/KafkaJobTest

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1944679337 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala: ## @@ -266,22 +275,46 @@ trait ColumnResolutionHelper extends L

Re: [PR] [SPARK-48530][SQL] Support for local variables in SQL Scripting [spark]

2025-02-06 Thread via GitHub
dusantism-db commented on code in PR #49445: URL: https://github.com/apache/spark/pull/49445#discussion_r1944733178 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/SqlScriptingParserSuite.scala: ## @@ -269,6 +269,94 @@ class SqlScriptingParserSuite extends Sp

Re: [PR] [SPARK-51107][CORE] Refactor CommandBuilderUtils#join to reuse the lines about strings join and reduce the redundant lines. [spark]

2025-02-06 Thread via GitHub
RocMarshal commented on code in PR #49826: URL: https://github.com/apache/spark/pull/49826#discussion_r1944752481 ## launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java: ## @@ -46,24 +47,15 @@ static boolean isEmpty(String s) { /** Joins a list of stri

Re: [PR] [SPARK-43415][CONNECT][SQL] Implement `KVGDS.agg` with custom `mapValues` function [spark]

2025-02-06 Thread via GitHub
hvanhovell commented on PR #49111: URL: https://github.com/apache/spark/pull/49111#issuecomment-2639875318 Merging to master/4.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] [SPARK-51093][SQL][TESTS] Fix minor endianness issues in tests. [spark]

2025-02-06 Thread via GitHub
jonathan-albrecht-ibm commented on code in PR #49812: URL: https://github.com/apache/spark/pull/49812#discussion_r1944888942 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/ArrayBasedMapBuilderSuite.scala: ## @@ -143,12 +145,15 @@ class ArrayBasedMapBuilderSuit

Re: [PR] [SPARK-51104][DOC] Self-host JavaScript and CSS in Spark website [spark]

2025-02-06 Thread via GitHub
gengliangwang commented on PR #49823: URL: https://github.com/apache/spark/pull/49823#issuecomment-2640685506 @LuciferYang @HyukjinKwon @dongjoon-hyun @nchammas thanks for the reviews. I am merging this one to master/branch-4.0/branch-3.5 -- This is an automated message from the Apache Gi

Re: [PR] [SPARK-51104][DOC] Self-host JavaScript and CSS in Spark website [spark]

2025-02-06 Thread via GitHub
gengliangwang closed pull request #49823: [SPARK-51104][DOC] Self-host JavaScript and CSS in Spark website URL: https://github.com/apache/spark/pull/49823 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

<    1   2   3