HeartSaVioR commented on PR #50195:
URL: https://github.com/apache/spark/pull/50195#issuecomment-2725857269
Thanks! Merging to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
wangyum commented on PR #50281:
URL: https://github.com/apache/spark/pull/50281#issuecomment-2726246165
cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To un
donan92 commented on PR #50283:
URL: https://github.com/apache/spark/pull/50283#issuecomment-2726175280
The related PR: https://github.com/apache/spark/pull/45817
I couldn't control & replace of the "sun.util.calendar.ZoneInfo" logic..
--
This is an automated message from the Apache Git
beliefer commented on PR #50277:
URL: https://github.com/apache/spark/pull/50277#issuecomment-2724700728
I think we can advance the check at report map status.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
beliefer commented on PR #50279:
URL: https://github.com/apache/spark/pull/50279#issuecomment-2724788720
ping @dongjoon-hyun @LuciferYang @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
donan92 opened a new pull request, #50283:
URL: https://github.com/apache/spark/pull/50283
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was
stefankandic commented on PR #50268:
URL: https://github.com/apache/spark/pull/50268#issuecomment-2725030373
@MaxGekk can you take a look at this when you get the chance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
donan92 commented on PR #50283:
URL: https://github.com/apache/spark/pull/50283#issuecomment-2725222433
Related Jira : https://issues.apache.org/jira/browse/SPARK-51515
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use t
cloud-fan closed pull request #50085: [SPARK-50759][SQL] Deprecate a few legacy
Catalog APIs
URL: https://github.com/apache/spark/pull/50085
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the speci
dtenedor commented on PR #50284:
URL: https://github.com/apache/spark/pull/50284#issuecomment-2725767439
cc @cloud-fan @gengliangwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
LuciferYang opened a new pull request, #50276:
URL: https://github.com/apache/spark/pull/50276
### What changes were proposed in this pull request?
This ps upgraded `apache-rat` used in the `dev/check-license` from 0.15 to
0.16.1
### Why are the changes needed?
The full release
wankunde opened a new pull request, #50277:
URL: https://github.com/apache/spark/pull/50277
### What changes were proposed in this pull request?
When the application crashes unexpectedly, the registered map statuses may
contain null values, therefore we should skip these null valu
HeartSaVioR commented on code in PR #50195:
URL: https://github.com/apache/spark/pull/50195#discussion_r1994940008
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala:
##
@@ -1465,6 +1470,7 @@ class RocksDB(
log"with uniqueId: ${MDC(L
cloud-fan commented on PR #50085:
URL: https://github.com/apache/spark/pull/50085#issuecomment-2725734260
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific c
cloud-fan commented on code in PR #50179:
URL: https://github.com/apache/spark/pull/50179#discussion_r1996429062
##
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriterV2.scala:
##
@@ -109,24 +111,30 @@ final class DataFrameWriterV2[T] private[sql](table:
String, ds: Da
cloud-fan commented on PR #50179:
URL: https://github.com/apache/spark/pull/50179#issuecomment-2726005255
It seems like a bug in sbt or sbtunidoc and it tries to generate API doc for
generated java classes in the `target` director. Unfortunately I don't know how
to fix this...
--
This is
MaxGekk commented on code in PR #50270:
URL: https://github.com/apache/spark/pull/50270#discussion_r1996172846
##
common/variant/src/main/java/org/apache/spark/types/variant/VariantUtil.java:
##
@@ -123,6 +123,12 @@ public class VariantUtil {
// Long string value. The content
cloud-fan commented on PR #50267:
URL: https://github.com/apache/spark/pull/50267#issuecomment-2726013226
thanks, merging to master/4.0!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specif
cloud-fan commented on code in PR #50192:
URL: https://github.com/apache/spark/pull/50192#discussion_r1996433418
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##
@@ -4082,3 +4084,31 @@ object RemoveTempResolvedColumn extends
Rule[LogicalP
stefankandic commented on code in PR #50192:
URL: https://github.com/apache/spark/pull/50192#discussion_r1996434982
##
sql/core/src/test/resources/sql-tests/analyzer-results/collations.sql.out:
##
@@ -821,15 +821,15 @@ InsertIntoHadoopFsRelationCommand file:[not included in
com
cloud-fan closed pull request #50267: [SPARK-51501][SQL] Disable
ObjectHashAggregate for group by on collated columns
URL: https://github.com/apache/spark/pull/50267
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
zhengruifeng commented on PR #50255:
URL: https://github.com/apache/spark/pull/50255#issuecomment-2726028127
Thanks, I am going to merge it only in master, in order to check whether it
also updates the dependencies in branch-4.0.
If branch-4.0 is not touched at all, that is great; otherwi
zhengruifeng commented on PR #50255:
URL: https://github.com/apache/spark/pull/50255#issuecomment-2726028853
merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
zhengruifeng closed pull request #50255: [SPARK-51487][PYTHON][INFRA] Refresh
testing images for pyarrow 19
URL: https://github.com/apache/spark/pull/50255
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
robreeves commented on PR #50269:
URL: https://github.com/apache/spark/pull/50269#issuecomment-2726042770
@MaxGekk this is ready for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the spe
github-actions[bot] closed pull request #48947: [SPARK-50404][PYTHON] PySpark
DataFrame Pipe Method
URL: https://github.com/apache/spark/pull/48947
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th
dongjoon-hyun closed pull request #50280: [MINOR][DOCS] Add Profile
-Pdocker-integration-tests to sbt docker IT tests
URL: https://github.com/apache/spark/pull/50280
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
calilisantos commented on PR #50194:
URL: https://github.com/apache/spark/pull/50194#issuecomment-2725418212
> @calilisantos Congratulations with your first contribution to Apache Spark!
Thanks @MaxGekk, see you next time!
--
This is an automated message from the Apache Git Service.
HeartSaVioR closed pull request #50195: [SPARK-51097][SS] Re-introduce RocksDB
state store's last uploaded snapshot version instance metrics
URL: https://github.com/apache/spark/pull/50195
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AveryQi115 opened a new pull request, #50285:
URL: https://github.com/apache/spark/pull/50285
### What changes were proposed in this pull request?
Add unresolved outer attributes as a SubqueryExpression parameter.
### Why are the changes needed?
For supporting
ueshin opened a new pull request, #50286:
URL: https://github.com/apache/spark/pull/50286
### What changes were proposed in this pull request?
Fixes `ExtractGenerator` to wait for `UnresolvedStarWithColumns` to be
resolved.
### Why are the changes needed?
`df.withColumn`
ahshahid commented on code in PR #50033:
URL: https://github.com/apache/spark/pull/50033#discussion_r1996488616
##
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapStage.scala:
##
@@ -90,8 +90,11 @@ private[spark] class ShuffleMapStage(
/** Returns the sequence of p
yaooqinn opened a new pull request, #50280:
URL: https://github.com/apache/spark/pull/50280
### What changes were proposed in this pull request?
Add Profile -Pdocker-integration-tests to sbt docker IT tests
### Why are the changes needed?
Fix build instructions
### Doe
vladimirg-db commented on code in PR #50192:
URL: https://github.com/apache/spark/pull/50192#discussion_r199540
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##
@@ -4082,3 +4084,31 @@ object RemoveTempResolvedColumn extends
Rule[Logic
WeichenXu123 opened a new pull request, #50278:
URL: https://github.com/apache/spark/pull/50278
### What changes were proposed in this pull request?
Implement model size estimation.
2 new interfaces are added:
1. `Estimator.estimateModelSize`: This method is used to
WeichenXu123 commented on PR #50106:
URL: https://github.com/apache/spark/pull/50106#issuecomment-2724183420
I created a new PR in https://github.com/apache/spark/pull/50278
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
WeichenXu123 closed pull request #50106: [SPARK-51340][ML][CONNECT] Model size
estimation for linear classification & regression models
URL: https://github.com/apache/spark/pull/50106
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitH
WeichenXu123 commented on code in PR #50278:
URL: https://github.com/apache/spark/pull/50278#discussion_r1995257146
##
mllib/src/main/scala/org/apache/spark/ml/Model.scala:
##
@@ -43,4 +44,18 @@ abstract class Model[M <: Model[M]] extends Transformer {
def hasParent: Boolean
WeichenXu123 commented on code in PR #50278:
URL: https://github.com/apache/spark/pull/50278#discussion_r1995269037
##
mllib/src/main/scala/org/apache/spark/ml/Estimator.scala:
##
@@ -81,4 +81,25 @@ abstract class Estimator[M <: Model[M]] extends
PipelineStage {
}
overr
pan3793 commented on PR #50275:
URL: https://github.com/apache/spark/pull/50275#issuecomment-2723809616
because one of `commandOptions` and `dsOptions` was canonicalized but
another wasn't? if so, should it be better to do canonicalization on the caller
side too?
--
This is an automated
dongjoon-hyun commented on code in PR #16:
URL:
https://github.com/apache/spark-connect-swift/pull/16#discussion_r1994369570
##
Tests/SparkConnectTests/DataFrameTests.swift:
##
@@ -68,6 +68,76 @@ struct DataFrameTests {
await spark.stop()
}
+ @Test
+ func selectNone
dongjoon-hyun closed pull request #50274: [SPARK-51509][CORE][UI] Make `Spark
Master` Environment page support filters
URL: https://github.com/apache/spark/pull/50274
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
UR
ahshahid commented on code in PR #50033:
URL: https://github.com/apache/spark/pull/50033#discussion_r1994975654
##
resource-managers/yarn/pom.xml:
##
@@ -37,6 +37,11 @@
spark-core_${scala.binary.version}
${project.version}
+
+ org.apache.spark
+
ahshahid commented on code in PR #50033:
URL: https://github.com/apache/spark/pull/50033#discussion_r1994979719
##
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapStage.scala:
##
@@ -90,8 +90,11 @@ private[spark] class ShuffleMapStage(
/** Returns the sequence of p
dongjoon-hyun commented on PR #18:
URL:
https://github.com/apache/spark-connect-swift/pull/18#issuecomment-2723811234
Merged to main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
vladimirg-db commented on code in PR #50192:
URL: https://github.com/apache/spark/pull/50192#discussion_r1995424455
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##
@@ -4082,3 +4084,31 @@ object RemoveTempResolvedColumn extends
Rule[Logic
zecookiez commented on code in PR #50195:
URL: https://github.com/apache/spark/pull/50195#discussion_r1995701396
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala:
##
@@ -203,20 +203,46 @@ trait StateStoreWriter
def operatorStateMet
MaxGekk commented on PR #50268:
URL: https://github.com/apache/spark/pull/50268#issuecomment-2725121155
+1, LGTM. Merging to master/4.0.
Thank you, @stefankandic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
MaxGekk closed pull request #50268: [SPARK-51502][SQL][TESTS] Move collations
test to collations package
URL: https://github.com/apache/spark/pull/50268
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
MaxGekk opened a new pull request, #50282:
URL: https://github.com/apache/spark/pull/50282
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was
beliefer commented on code in PR #50277:
URL: https://github.com/apache/spark/pull/50277#discussion_r1995277288
##
core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala:
##
@@ -418,7 +418,7 @@ class BlockManagerMasterEndpoint(
if (externalShuffleServ
beliefer commented on PR #50277:
URL: https://github.com/apache/spark/pull/50277#issuecomment-2724219738
Could we investigate the reason lead to the null value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL a
stefankandic commented on PR #50267:
URL: https://github.com/apache/spark/pull/50267#issuecomment-2724231215
@cloud-fan please take a look when you get the chance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
UR
WeichenXu123 commented on code in PR #50278:
URL: https://github.com/apache/spark/pull/50278#discussion_r1995269037
##
mllib/src/main/scala/org/apache/spark/ml/Estimator.scala:
##
@@ -81,4 +81,25 @@ abstract class Estimator[M <: Model[M]] extends
PipelineStage {
}
overr
ahshahid commented on code in PR #50033:
URL: https://github.com/apache/spark/pull/50033#discussion_r1996507549
##
core/src/main/scala/org/apache/spark/scheduler/ShuffleMapStage.scala:
##
@@ -90,8 +90,11 @@ private[spark] class ShuffleMapStage(
/** Returns the sequence of p
wankunde commented on PR #50277:
URL: https://github.com/apache/spark/pull/50277#issuecomment-2724377835
> Could we investigate the reason lead to the null value?
I manually cancelled the running query, the registered running shuffle had
no mapStatus for the running tasks.
The shu
donan92 closed pull request #50283: [WIP][SPARK-51515] refactor
fromJavaDate/toJavaDate
URL: https://github.com/apache/spark/pull/50283
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific c
robreeves commented on PR #50269:
URL: https://github.com/apache/spark/pull/50269#issuecomment-2726212080
These test failures don't look related to my changes. I'm rerunning them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
AveryQi115 commented on PR #50285:
URL: https://github.com/apache/spark/pull/50285#issuecomment-2726059950
cc @agubichev
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
szehon-ho commented on code in PR #50271:
URL: https://github.com/apache/spark/pull/50271#discussion_r1996621009
##
sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/InMemoryRowLevelOperationTableCatalog.scala:
##
@@ -43,4 +43,27 @@ class InMemoryRowLevelOperati
60 matches
Mail list logo
