dongjoon-hyun closed pull request #48740: [SPARK-50208][CORE] Support
`spark.master.useDriverIdAsAppName.enabled`
URL: https://github.com/apache/spark/pull/48740
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abo
dongjoon-hyun commented on code in PR #48740:
URL: https://github.com/apache/spark/pull/48740#discussion_r1826509525
##
core/src/main/scala/org/apache/spark/deploy/master/Master.scala:
##
@@ -1330,10 +1331,33 @@ private[deploy] class Master(
appId
}
+ /**
+ * Update
dongjoon-hyun commented on code in PR #48740:
URL: https://github.com/apache/spark/pull/48740#discussion_r1826509456
##
core/src/main/scala/org/apache/spark/deploy/master/Master.scala:
##
@@ -1330,10 +1331,33 @@ private[deploy] class Master(
appId
}
+ /**
+ * Update
dongjoon-hyun commented on code in PR #48740:
URL: https://github.com/apache/spark/pull/48740#discussion_r1826509242
##
core/src/main/scala/org/apache/spark/deploy/master/Master.scala:
##
@@ -1330,10 +1331,33 @@ private[deploy] class Master(
appId
}
+ /**
+ * Update
viirya commented on code in PR #48740:
URL: https://github.com/apache/spark/pull/48740#discussion_r1826508006
##
core/src/main/scala/org/apache/spark/deploy/master/Master.scala:
##
@@ -1330,10 +1331,33 @@ private[deploy] class Master(
appId
}
+ /**
+ * Update and ad
dongjoon-hyun commented on PR #48740:
URL: https://github.com/apache/spark/pull/48740#issuecomment-2452855528
Could you review this when you have some time, @viirya ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
Angryrou commented on code in PR #48649:
URL: https://github.com/apache/spark/pull/48649#discussion_r1826472117
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##
@@ -5874,53 +5886,72 @@ class AstBuilder extends DataTypeAstBuilder
if (!
Angryrou commented on code in PR #48649:
URL: https://github.com/apache/spark/pull/48649#discussion_r1826472117
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##
@@ -5874,53 +5886,72 @@ class AstBuilder extends DataTypeAstBuilder
if (!
github-actions[bot] commented on PR #46995:
URL: https://github.com/apache/spark/pull/46995#issuecomment-2452750535
We're closing this PR because it hasn't been updated in a while. This isn't
a judgement on the merit of the PR in any way. It's just a way of keeping the
PR queue manageable.
github-actions[bot] commented on PR #47456:
URL: https://github.com/apache/spark/pull/47456#issuecomment-2452750507
We're closing this PR because it hasn't been updated in a while. This isn't
a judgement on the merit of the PR in any way. It's just a way of keeping the
PR queue manageable.
github-actions[bot] closed pull request #47422: [SPARK-48948][SQL] Introduce
`SHOW VARIABLES LIKE ... ` SQL syntax to get variables
URL: https://github.com/apache/spark/pull/47422
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
github-actions[bot] closed pull request #47281: [SPARK-48856][SQL] Use isolated
JobArtifactSet for each spark session
URL: https://github.com/apache/spark/pull/47281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
cloud-fan commented on PR #47422:
URL: https://github.com/apache/spark/pull/47422#issuecomment-2452714305
let's move away from SHOW commands. I think UDTF is a better choice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
dongjoon-hyun opened a new pull request, #48740:
URL: https://github.com/apache/spark/pull/48740
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### H
ueshin commented on PR #48730:
URL: https://github.com/apache/spark/pull/48730#issuecomment-2452666013
I don't think the remaining failures are related to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
UR
stefankandic commented on code in PR #48737:
URL: https://github.com/apache/spark/pull/48737#discussion_r1826332674
##
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationFactory.java:
##
@@ -394,7 +395,11 @@ private static Collation fetchCollation(int collat
liviazhu-db commented on code in PR #48697:
URL: https://github.com/apache/spark/pull/48697#discussion_r1826336625
##
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBSuite.scala:
##
@@ -2240,6 +2240,200 @@ class RocksDBSuite extends
AlsoTestWithCha
liviazhu-db commented on code in PR #48697:
URL: https://github.com/apache/spark/pull/48697#discussion_r1826336442
##
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBSuite.scala:
##
@@ -2240,6 +2240,200 @@ class RocksDBSuite extends
AlsoTestWithCha
liviazhu-db commented on code in PR #48697:
URL: https://github.com/apache/spark/pull/48697#discussion_r1826335482
##
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBSuite.scala:
##
@@ -2240,6 +2240,200 @@ class RocksDBSuite extends
AlsoTestWithCha
liviazhu-db commented on code in PR #48697:
URL: https://github.com/apache/spark/pull/48697#discussion_r1826335367
##
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBSuite.scala:
##
@@ -2240,6 +2240,200 @@ class RocksDBSuite extends
AlsoTestWithCha
liviazhu-db commented on code in PR #48697:
URL: https://github.com/apache/spark/pull/48697#discussion_r1826334501
##
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBSuite.scala:
##
@@ -2240,6 +2240,200 @@ class RocksDBSuite extends
AlsoTestWithCha
liviazhu-db commented on code in PR #48697:
URL: https://github.com/apache/spark/pull/48697#discussion_r1826170578
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala:
##
@@ -965,12 +968,39 @@ class RocksDB(
/**
* Function to release
Angryrou commented on code in PR #48649:
URL: https://github.com/apache/spark/pull/48649#discussion_r1826225397
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##
@@ -5874,53 +5886,72 @@ class AstBuilder extends DataTypeAstBuilder
if (!
Angryrou commented on code in PR #48649:
URL: https://github.com/apache/spark/pull/48649#discussion_r1826225397
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##
@@ -5874,53 +5886,72 @@ class AstBuilder extends DataTypeAstBuilder
if (!
dtenedor commented on code in PR #48649:
URL: https://github.com/apache/spark/pull/48649#discussion_r1826211578
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##
@@ -5874,53 +5886,72 @@ class AstBuilder extends DataTypeAstBuilder
if (!
dtenedor commented on code in PR #48724:
URL: https://github.com/apache/spark/pull/48724#discussion_r1826204314
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##
@@ -495,7 +495,9 @@ class AstBuilder extends DataTypeAstBuilder
optio
Angryrou commented on code in PR #48649:
URL: https://github.com/apache/spark/pull/48649#discussion_r1825127207
##
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala:
##
@@ -5874,53 +5886,72 @@ class AstBuilder extends DataTypeAstBuilder
if (!
liviazhu-db commented on code in PR #48697:
URL: https://github.com/apache/spark/pull/48697#discussion_r1826184687
##
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala:
##
@@ -1001,6 +1031,10 @@ class RocksDB(
}
}
+ private[state] de
Madhukar525722 commented on PR #48337:
URL: https://github.com/apache/spark/pull/48337#issuecomment-2452396704
Gentle ping @mridulm @pan3793 @HyukjinKwon @shardulm94
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
MaxGekk closed pull request #48738: [SPARK-50204][SQL] Factor out
`HiveTableRelation` read path resolution
URL: https://github.com/apache/spark/pull/48738
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
uros-db commented on code in PR #48739:
URL: https://github.com/apache/spark/pull/48739#discussion_r1826137770
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
@@ -1270,6 +1442,7 @@ class CollationSQLExpressionsSuite
)
}
+ // TODO
uros-db commented on code in PR #48739:
URL: https://github.com/apache/spark/pull/48739#discussion_r1826137516
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
@@ -2760,6 +3076,7 @@ class CollationSQLExpressionsSuite
}
}
+ // TODO
MaxGekk commented on PR #48738:
URL: https://github.com/apache/spark/pull/48738#issuecomment-2452341121
+1, LGTM. Merging to master.
Thank you, @vladimirg-db.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
jovanpavl-db opened a new pull request, #48739:
URL: https://github.com/apache/spark/pull/48739
### What changes were proposed in this pull request?
Enabling usage of passthrough expressions for trim collation.
**NOTE: it looks like a tons of changes but only changes are:
for each e
cloud-fan commented on code in PR #48650:
URL: https://github.com/apache/spark/pull/48650#discussion_r1826072005
##
connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/CheckConnectJvmClientCompatibility.scala:
##
@@ -314,7 +314,48 @@ object CheckConne
cloud-fan commented on PR #48523:
URL: https://github.com/apache/spark/pull/48523#issuecomment-2452228091
The pyspark failure is unrelated, thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
cloud-fan closed pull request #48523: [SPARK-50033][SQL] Add a hint to
logical.Aggregate() node
URL: https://github.com/apache/spark/pull/48523
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the sp
xupefei commented on code in PR #48120:
URL: https://github.com/apache/spark/pull/48120#discussion_r1823014488
##
sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala:
##
@@ -120,93 +120,97 @@ object SQLExecution extends Logging {
val redactedConfigs
jiashenC commented on PR #48724:
URL: https://github.com/apache/spark/pull/48724#issuecomment-2452141499
> Mind filing a JIRA please?
This is the JIRA ticket https://issues.apache.org/jira/browse/SPARK-49565
--
This is an automated message from the Apache Git Service.
To respond to
xupefei commented on code in PR #48736:
URL: https://github.com/apache/spark/pull/48736#discussion_r1825969946
##
sql/core/src/test/scala/org/apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala:
##
@@ -112,7 +112,8 @@ class SparkSessionJobTaggingAndCancellationSuit
grundprinzip closed pull request #82: Add config support
URL: https://github.com/apache/spark-connect-go/pull/82
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe,
grundprinzip commented on PR #82:
URL: https://github.com/apache/spark-connect-go/pull/82#issuecomment-2452083824
Thanks for your contribution, approving.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above t
vladimirg-db opened a new pull request, #48738:
URL: https://github.com/apache/spark/pull/48738
### What changes were proposed in this pull request?
Factor out `HiveTableRelation` resolution into a separate
`RelationConversions` method to reuse it in the single-pass Analyzer.
#
grundprinzip commented on code in PR #84:
URL: https://github.com/apache/spark-connect-go/pull/84#discussion_r1825931136
##
spark/sql/dataframe.go:
##
@@ -148,6 +148,8 @@ type DataFrame interface {
Rollup(ctx context.Context, cols ...column.Convertible) *GroupedData
dongjoon-hyun commented on code in PR #48736:
URL: https://github.com/apache/spark/pull/48736#discussion_r1825925144
##
sql/core/src/test/scala/org/apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala:
##
@@ -112,7 +112,8 @@ class SparkSessionJobTaggingAndCancellati
vladanvasi-db commented on PR #48737:
URL: https://github.com/apache/spark/pull/48737#issuecomment-2452018108
@stevomitric @stefankandic @cloud-fan please take a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
vladanvasi-db opened a new pull request, #48737:
URL: https://github.com/apache/spark/pull/48737
### What changes were proposed in this pull request?
In this PR, I propose adding a separate collation id for `UTF8_BINARY`
collation and for non-collated string type. Furthermore,
xupefei commented on PR #48622:
URL: https://github.com/apache/spark/pull/48622#issuecomment-2451957729
PR created at https://github.com/apache/spark/pull/48736.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
xupefei commented on PR #48622:
URL: https://github.com/apache/spark/pull/48622#issuecomment-2451946231
It's weird that only the `interrupt tag` test is failing, as I believe they
are suffering from the same issue (thread pool only got 2 threads instead of
requested 4). If my change didn't
xupefei opened a new pull request, #48736:
URL: https://github.com/apache/spark/pull/48736
### What changes were proposed in this pull request?
This PR disables a flaky test
`SparkSessionJobTaggingAndCancellationSuite.Cancellation APIs in SparkSession
are isolated`. The flakiness is
xupefei commented on PR #48736:
URL: https://github.com/apache/spark/pull/48736#issuecomment-2451956214
@dongjoon-hyun Could you review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th
ivanjevtic-db commented on PR #48368:
URL: https://github.com/apache/spark/pull/48368#issuecomment-2451760148
> @ivanjevtic-db Could you fix the failed tests like:
>
> ```
> [info] - Error conditions are correctly formatted *** FAILED *** (112
milliseconds)
> [info] "... ]
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825695456
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
Review Comment:
- Let's use the following code to reproduce it (collation
beliefer commented on PR #48693:
URL: https://github.com/apache/spark/pull/48693#issuecomment-2451750798
> How much does it get improved?
The benchmark added into PR's description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
panbingkun commented on PR #48720:
URL: https://github.com/apache/spark/pull/48720#issuecomment-2451743711
> Generally, a useful change. Can you please update the Python client as
well.
Allow me to do it, thank you very much for the review! ❤️
--
This is an automated message from t
panbingkun commented on PR #48721:
URL: https://github.com/apache/spark/pull/48721#issuecomment-2451740948
@uros-db @dongjoon-hyun @stefankandic @MaxGekk
The detailed explanation has been updated, this PR is ready for review.
Thank you very much for the review, if you has free time. ❤
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825695456
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
Review Comment:
- Let's use the following code to reproduce it (collation
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825695456
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
Review Comment:
- Let's use the following code to reproduce it (collation
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825722866
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
@@ -200,10 +200,10 @@ class CollationSQLExpressionsSuite
Murmur3HashTes
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825722211
##
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollationExpressionSuite.scala:
##
@@ -168,6 +168,7 @@ class CollationExpressionSuite extend
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825695456
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
Review Comment:
- Let's use the following code to reproduce it (collation
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825695456
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
Review Comment:
- Let's use the following code to reproduce it (collation
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825695456
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
Review Comment:
- Let's use the following code to reproduce it (collation
HyukjinKwon closed pull request #48735: [SPARK-49530][PYTHON] Get active
session from dataframes
URL: https://github.com/apache/spark/pull/48735
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
HyukjinKwon commented on PR #48735:
URL: https://github.com/apache/spark/pull/48735#issuecomment-2451670463
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
grundprinzip commented on code in PR #48720:
URL: https://github.com/apache/spark/pull/48720#discussion_r1825674459
##
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/application/ConnectRepl.scala:
##
@@ -102,8 +102,11 @@ Spark session available as 'spark'.
PetarVasiljevic-DB commented on PR #48625:
URL: https://github.com/apache/spark/pull/48625#issuecomment-2451628205
What is your concern here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
xinrong-meng commented on PR #48734:
URL: https://github.com/apache/spark/pull/48734#issuecomment-2451427287
Thank you @dongjoon-hyun and @HyukjinKwon !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825593552
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
Review Comment:
`CollationKeys#writeSortKeyUpToQuaternary`
--
This is
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825543236
##
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollationExpressionSuite.scala:
##
@@ -168,6 +168,7 @@ class CollationExpressionSuite extend
zhengruifeng opened a new pull request, #48735:
URL: https://github.com/apache/spark/pull/48735
### What changes were proposed in this pull request?
Get active session from dataframes
### Why are the changes needed?
we can directly get session from dataframes
### D
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825543236
##
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollationExpressionSuite.scala:
##
@@ -168,6 +168,7 @@ class CollationExpressionSuite extend
xinrong-meng commented on code in PR #48677:
URL: https://github.com/apache/spark/pull/48677#discussion_r1825512645
##
python/pyspark/sql/connect/session.py:
##
@@ -602,9 +602,11 @@ def createDataFrame(
if not isinstance(schema, StructType):
schema
panbingkun commented on code in PR #48721:
URL: https://github.com/apache/spark/pull/48721#discussion_r1825539871
##
sql/core/src/test/scala/org/apache/spark/sql/CollationSQLExpressionsSuite.scala:
##
Review Comment:
The reason for this change is that the `CollationKey` ret
HeartSaVioR commented on code in PR #48685:
URL: https://github.com/apache/spark/pull/48685#discussion_r1825500634
##
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBSuite.scala:
##
Review Comment:
So the bug in the test is that file mapping do
HyukjinKwon closed pull request #48717: [SPARK-50184][SS][TESTS] Make test case
`SPARK-48997: maintenance threads with exceptions unload only themselves` use a
self-cleaning `rootLocation`
URL: https://github.com/apache/spark/pull/48717
--
This is an automated message from the Apache Git Ser
76 matches
Mail list logo
