[
https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16237973#comment-16237973
]
Mike Drob commented on HBASE-16179:
-----------------------------------
bq. Can you provide summary here ?
Sure, from the scope doc on HBASE-18405:
* Our integration works at runtime with Spark releases in the 2.y major release
line2. At this time, that includes Apache Spark 2.0 through 2.2.
* Our integration works at runtime with Spark deployments that rely on whatever
Scala version(s) a given Spark version is made to work with. At this time, that
includes Scala 2.10 and Scala 2.11.
* Our integration is present in one or more Maven modules, producing any needed
jars for the above supported combinations of Spark and Scala versions in a
single build. Such modules should ensure that no Scala dependencies are present
outside of those modules specific to the integration work.
After the writing of the doc, Spark 2.3 and Scala 2.12 have also been released.
I haven't been following their support matrix closely enough to know how this
affects us.
Since supporting all versions continues to be a moving target, I think the
easiest path forward is to get this patch committed so that we have support
against _some_ 2.x version and then worry about the pairwise combinations of
all the sparks and scalas in follow on issues. That way we will have more
patches that are smaller in size and easier to review (and hopefully faster to
commit), and we can get something out to users in a more timely manner. WDYT?
> Fix compilation errors when building hbase-spark against Spark 2.0
> ------------------------------------------------------------------
>
> Key: HBASE-16179
> URL: https://issues.apache.org/jira/browse/HBASE-16179
> Project: HBase
> Issue Type: Bug
> Components: spark
> Reporter: Ted Yu
> Assignee: Ted Yu
> Priority: Critical
> Labels: build
> Fix For: 3.0.0
>
> Attachments: 16179.v0.txt, 16179.v1.txt, 16179.v1.txt, 16179.v10.txt,
> 16179.v11.txt, 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt,
> 16179.v15.txt, 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt,
> 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt, 16179.v25.txt,
> 16179.v26.txt, 16179.v27.txt, 16179.v28.txt, 16179.v28.txt, 16179.v4.txt,
> 16179.v5.txt, 16179.v7.txt, 16179.v8.txt, 16179.v9.txt
>
>
> I tried building hbase-spark module against Spark-2.0 snapshot and got the
> following compilation errors:
> http://pastebin.com/bg3w247a
> Some Spark classes such as DataTypeParser and Logging are no longer
> accessible to downstream projects.
> hbase-spark module should not depend on such classes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
