Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-21 Thread via GitHub
HyukjinKwon commented on PR #50921: URL: https://github.com/apache/spark/pull/50921#issuecomment-3096177694 Can we enable ANSI in this suite? https://github.com/apache/spark/actions/runs/16406624611/job/46353606361 Seems it fails with non-ansi build. -- This is an automated message from

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-15 Thread via GitHub
cloud-fan closed pull request #50921: [SPARK-52187][SQL] Introduce Join pushdown for DSv2 URL: https://github.com/apache/spark/pull/50921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-15 Thread via GitHub
cloud-fan commented on PR #50921: URL: https://github.com/apache/spark/pull/50921#issuecomment-3073645814 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-15 Thread via GitHub
PetarVasiljevic-DB commented on PR #50921: URL: https://github.com/apache/spark/pull/50921#issuecomment-3073123234 @cloud-fan can we merge? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-15 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2207009972 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/join/JoinType.java: ## @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-15 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2206999839 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/join/JoinType.java: ## @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-15 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2206670819 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,146 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-15 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2206667234 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/join/JoinType.java: ## @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-14 Thread via GitHub
agubichev commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2206119767 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,146 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-14 Thread via GitHub
agubichev commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2206115863 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/join/JoinType.java: ## @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-14 Thread via GitHub
PetarVasiljevic-DB commented on PR #50921: URL: https://github.com/apache/spark/pull/50921#issuecomment-3068369142 @cloud-fan I will fix it. By the way, I think we should wait for a review from @agubichev. I suggest not merging this yet. -- This is an automated message from the Apache Gi

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-13 Thread via GitHub
cloud-fan commented on PR #50921: URL: https://github.com/apache/spark/pull/50921#issuecomment-3067906272 There is a linter failure ``` Checkstyle checks failed at following occurrences: Error: src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java:[50] (sizes

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-11 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2200796923 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-11 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2200205650 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-10 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2199267663 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-10 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2199265772 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-10 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2199264987 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-10 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2197953985 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-10 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2197727060 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,112 @@ case class JDBCScanBuilder(

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-10 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2197693056 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,112 @@ case class JDBCScanBuilder( }

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-10 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2197539031 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,112 @@ case class JDBCScanBuilder(

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-10 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2197547433 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown extends

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-10 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2197477011 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-10 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2197140112 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown extends

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2196775535 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195212137 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown extends

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195219122 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown extends

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195212137 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown extends

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195200593 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown extends

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195158550 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown extends

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195149030 ## common/utils/src/main/resources/error/error-conditions.json: ## @@ -6422,9 +6422,9 @@ }, "sqlState" : "42809" }, - "UNSUPPORTED_JOIN_TYPE" : { Rev

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195145421 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -194,4 +304,13 @@ case class JDBCScanBuilder( JDBCSc

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195141439 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown extends

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195143353 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown extends

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195140234 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown extends

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195054967 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -194,4 +304,13 @@ case class JDBCScanBuilder( JDBCScan

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195049422 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,112 @@ case class JDBCScanBuilder( } }

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195046554 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195041104 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2195039085 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,142 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2194748951 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1698,6 +1698,14 @@ object SQLConf { .booleanConf .createWithDefault

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2194693856 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1698,6 +1698,14 @@ object SQLConf { .booleanConf .createWith

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-09 Thread via GitHub
beliefer commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2194661556 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1698,6 +1698,14 @@ object SQLConf { .booleanConf .createWithDefault(!U

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192878985 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192902919 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,112 @@ case class JDBCScanBuilder( }

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192898858 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1698,6 +1698,14 @@ object SQLConf { .booleanConf .createWithDefault

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192876214 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1698,6 +1698,14 @@ object SQLConf { .booleanConf .createWith

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192875696 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,112 @@ case class JDBCScanBuilder(

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192843839 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +107,138 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192842396 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192735671 ## sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCV2Suite.scala: ## @@ -265,6 +266,288 @@ class JDBCV2Suite extends QueryTest with SharedSparkSession

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192732818 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java: ## @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192723418 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -194,4 +304,13 @@ case class JDBCScanBuilder(

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192715977 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,114 @@ case class JDBCScanBuilder(

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192708612 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192705888 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +100,132 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192691005 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1698,6 +1698,14 @@ object SQLConf { .booleanConf .createWith

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
urosstan-db commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192348720 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushedDownOperators.scala: ## @@ -30,6 +30,7 @@ case class PushedDownOperators( limit:

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
beliefer commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192451387 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1698,6 +1698,14 @@ object SQLConf { .booleanConf .createWithDefault(!U

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192413802 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +107,138 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192334112 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +107,138 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192283750 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +107,138 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192279075 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1698,6 +1698,14 @@ object SQLConf { .booleanConf .createWith

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192161308 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,112 @@ case class JDBCScanBuilder( } }

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192155725 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +107,138 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192152901 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +107,138 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192150902 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +107,138 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192122642 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java: ## @@ -0,0 +1,82 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192143070 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +107,138 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192137141 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +107,138 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192130423 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1698,6 +1698,14 @@ object SQLConf { .booleanConf .createWithDefault(!

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192124785 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java: ## @@ -0,0 +1,82 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-08 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2192120820 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/join/JoinType.java: ## @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-06 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2188515666 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-04 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2185417361 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,103 @@ case class JDBCScanBuilder(

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-04 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2185390225 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-04 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2185390225 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-04 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2185373386 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-04 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2185379163 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,103 @@ case class JDBCScanBuilder( } }

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-04 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2185373386 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-04 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2185005748 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +108,117 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184500117 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,103 @@ case class JDBCScanBuilder( } }

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184498663 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,103 @@ case class JDBCScanBuilder( } }

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184496685 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,103 @@ case class JDBCScanBuilder( } }

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184495403 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala: ## @@ -121,6 +123,103 @@ case class JDBCScanBuilder( } }

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184490651 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -441,8 +599,18 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184488031 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +108,117 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184485530 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +108,117 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184481233 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +108,117 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184476378 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -98,6 +108,117 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184470442 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -58,7 +60,15 @@ object V2ScanRelationPushDown extends Rul

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184467698 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-03 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2184467207 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownJoin.java: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-01 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2177959251 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -61,41 +63,232 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-01 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2177612709 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -61,41 +63,232 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-01 Thread via GitHub
PetarVasiljevic-DB commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2177959251 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -61,41 +63,232 @@ object V2ScanRelationPushDown

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-01 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2177671896 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -61,41 +63,232 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-01 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2177670452 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -61,41 +63,232 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-01 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2177667466 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -61,41 +63,232 @@ object V2ScanRelationPushDown extends R

Re: [PR] [SPARK-52187][SQL] Introduce Join pushdown for DSv2 [spark]

2025-07-01 Thread via GitHub
cloud-fan commented on code in PR #50921: URL: https://github.com/apache/spark/pull/50921#discussion_r2177667466 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala: ## @@ -61,41 +63,232 @@ object V2ScanRelationPushDown extends R

  1   2   >