[
https://issues.apache.org/jira/browse/SQOOP-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612402#comment-16612402
]
Daniel Voros commented on SQOOP-3381:
-------------------------------------
Hey [~fero], thanks for keeping that in mind. What I've seen during the hadoop3
upgrade, is that Avro is added to the MR classpath from under hadoop. So where
this could lead to issues is conflicting versions of Avro in hadoop and Parquet
shipped with Sqoop.
Could you try your patch (having new parquet jar in lib/) on a cluster with
current Hadoop versions? I don't think we should bother with testing with
Hadoop 3, we'll face that in the Hadoop 3 patch.
(One more thing to keep in mind, is that parquet-hadoop-bundle is also shaded
into the hive-exec artifact. However, I think the classes involved in
PARQUET-243 are not bundled there.)
> Upgrade the Parquet library from 1.6.0 to 1.9.0
> -----------------------------------------------
>
> Key: SQOOP-3381
> URL: https://issues.apache.org/jira/browse/SQOOP-3381
> Project: Sqoop
> Issue Type: Sub-task
> Affects Versions: 1.4.7
> Reporter: Fero Szabo
> Assignee: Fero Szabo
> Priority: Major
> Fix For: 3.0.0
>
>
> As we will need to register a data supplier in the fix for parquet decimal
> support, we will need a version that contains PARQUET-243.
> We need to upgrade the Parquet library to a version that contains this fix
> and is compatible with Hadoop. Most probably, the newest version will be
> adequate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
