dongjoon-hyun commented on PR #50040:
URL: https://github.com/apache/spark/pull/50040#issuecomment-2702759012
For the record, `branch-3.5` is recovered.
https://github.com/user-attachments/assets/d88ab2ae-d817-4552-b7c4-edbf5bf41740";
/>
--
This is an automated message from the
dongjoon-hyun commented on PR #50040:
URL: https://github.com/apache/spark/pull/50040#issuecomment-2702640013
Now, this is reverted from branch-3.5 via
https://github.com/apache/spark/commit/c8c0f1feb0f2680ac7437eae926c17724af0d94e
due to the CI failure.
Please make a backporting PR
dongjoon-hyun commented on PR #50040:
URL: https://github.com/apache/spark/pull/50040#issuecomment-2702637256
I verified that the reverting recovers `unidoc`.
```
$ git log --oneline -n1
c8c0f1feb0f (HEAD -> branch-3.5) Revert "[SPARK-51281][SQL]
DataFrameWriterV2 should respect the
dongjoon-hyun commented on PR #50040:
URL: https://github.com/apache/spark/pull/50040#issuecomment-2702546080
@cloud-fan . It's a `unidoc` failure and it happens on all commit builder
and PR builder (on branch-3.5) after this PR.
```
[info] Main Scala API documentation to
/__w/spark/s
cloud-fan commented on PR #50040:
URL: https://github.com/apache/spark/pull/50040#issuecomment-2700284666
hmm how can it be? This PR doesn't touch any doc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abov
yaooqinn commented on PR #50040:
URL: https://github.com/apache/spark/pull/50040#issuecomment-268205
Hi, https://github.com/apache/spark/actions/runs/13567662565/job/37924455859
The doc build CI for 3.5 seems broken after this PR merged
--
This is an automated message from the A
cloud-fan closed pull request #50040: [SPARK-51281][SQL] DataFrameWriterV2
should respect the path option
URL: https://github.com/apache/spark/pull/50040
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
cloud-fan commented on PR #50040:
URL: https://github.com/apache/spark/pull/50040#issuecomment-2687956522
thanks for the review, merging to master/4.0/3.5!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
cloud-fan commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1973575824
##
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWriterV2Suite.scala:
##
@@ -841,20 +841,24 @@ class DataFrameWriterV2Suite extends QueryTest with
SharedSpar
cloud-fan commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1973251531
##
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##
@@ -5545,6 +5545,15 @@ object SQLConf {
.booleanConf
.createWithDefault(false
cloud-fan commented on PR #50040:
URL: https://github.com/apache/spark/pull/50040#issuecomment-2684585464
We only have the API doc:
https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/DataFrameWriterV2.html
--
This is an automated message from the Apache Git Service.
To re
cloud-fan commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1971357977
##
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWriterV2Suite.scala:
##
@@ -839,4 +839,30 @@ class DataFrameWriterV2Suite extends QueryTest with
SharedSpark
manuzhang commented on PR #50040:
URL: https://github.com/apache/spark/pull/50040#issuecomment-2681116334
BTW, do we have documentations on DataFrameWriterV2 API? I don't find any
from searching on the Spark website.
--
This is an automated message from the Apache Git Service.
To respond
beliefer commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1969247223
##
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWriterV2Suite.scala:
##
@@ -841,20 +841,24 @@ class DataFrameWriterV2Suite extends QueryTest with
SharedSpark
szehon-ho commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1969046534
##
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWriterV2Suite.scala:
##
@@ -839,4 +839,30 @@ class DataFrameWriterV2Suite extends QueryTest with
SharedSpark
dongjoon-hyun commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1969044495
##
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##
@@ -5545,6 +5545,15 @@ object SQLConf {
.booleanConf
.createWithDefault(f
cloud-fan commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1968959548
##
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWriterV2Suite.scala:
##
@@ -841,20 +841,24 @@ class DataFrameWriterV2Suite extends QueryTest with
SharedSpar
beliefer commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1967675305
##
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWriterV2Suite.scala:
##
@@ -841,20 +841,24 @@ class DataFrameWriterV2Suite extends QueryTest with
SharedSpark
cloud-fan commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1967634418
##
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##
@@ -5545,6 +5545,15 @@ object SQLConf {
.booleanConf
.createWithDefault(false
beliefer commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1967072225
##
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##
@@ -5545,6 +5545,15 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
beliefer commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1967061151
##
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##
@@ -5545,6 +5545,15 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
dongjoon-hyun commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1967023059
##
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWriterV2Suite.scala:
##
@@ -839,4 +839,26 @@ class DataFrameWriterV2Suite extends QueryTest with
SharedS
dongjoon-hyun commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1967023059
##
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWriterV2Suite.scala:
##
@@ -839,4 +839,26 @@ class DataFrameWriterV2Suite extends QueryTest with
SharedS
cloud-fan commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1966991532
##
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##
@@ -5545,6 +5545,15 @@ object SQLConf {
.booleanConf
.createWithDefault(false
dongjoon-hyun commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1965769460
##
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##
@@ -5545,6 +5545,15 @@ object SQLConf {
.booleanConf
.createWithDefault(f
cloud-fan commented on PR #50040:
URL: https://github.com/apache/spark/pull/50040#issuecomment-2673951588
@dongjoon-hyun shall we include it in 3.5.5? Also cc @aokolnychyi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and u
cloud-fan commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1965099273
##
sql/core/src/test/scala/org/apache/spark/sql/DataFrameWriterV2Suite.scala:
##
@@ -839,4 +839,26 @@ class DataFrameWriterV2Suite extends QueryTest with
SharedSpark
cloud-fan commented on code in PR #50040:
URL: https://github.com/apache/spark/pull/50040#discussion_r1965098875
##
sql/core/src/main/scala/org/apache/spark/sql/classic/DataFrameWriterV2.scala:
##
@@ -146,25 +148,30 @@ final class DataFrameWriterV2[T] private[sql](table:
String
cloud-fan opened a new pull request, #50040:
URL: https://github.com/apache/spark/pull/50040
### What changes were proposed in this pull request?
Unlike `DataFrameWriter.saveAsTable` where we explicitly get the "path"
option and treat it as table location, `DataFrameWriterV2`
29 matches
Mail list logo
