[
https://issues.apache.org/jira/browse/HIVE-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13763308#comment-13763308
]
Sushanth Sowmyan commented on HIVE-4388:
----------------------------------------
Hi Brock,
>From looking through the patch, I'm +1 with your basic intent.
Your patch does not apply cleanly with HEAD, but that's largely because of the
hcatalog refactoring done recently with HIVE-4869, HIVE-5163 and HIVE-5197.
Going back to d0da259d02b546979b93a411493b1704f488b72d allows me to test most
of these (although still has a problem patching build.xml, but I was able to
resolve that manually) But I wanted to put in a tentative review before you put
in the effort to rebase it. As of right now, I think it looks good to rebase
and bring this patch up to date.
--
Additional thoughts:
I did, however, have had similar experiences trying to make hbase tests work
with hadoop 2.x, and faced a couple of further problems. When fetching from
maven-ant-tasks, I noticed that hbase was still bringing in hadoop1 transitive
dependencies. This doesn't hurt our unit tests, but when running HCatalog E2E
tests, caused a couple of situations where the presence of the hadoop1 client
jar in hive lib dir would cause hbase tests to start failing. To fix this, we
had to set a system property called hadoop.profile before it fetched the
correct transitive dependencies. (This is different from the mvn.hadoop.profile
= "hadoop23" that hive and hcat use.) (This was, however, 2 weeks back, and I
see a couple of pom.xml changes since then, so I'm unsure if the hbase pom.xml
has changed to make hadoop 2.x their default profile, and if so, that will fix
this, but if so, will break it for hadoop 1.x E2E tests - we need to set the
appropriate system property to be sure.) I will further test this today to see
if we still need it. (Another case where we'd be okay if we used mvn, but in
the maven-ant-tasks world, we need to do a bit of weird gluing.)
Also, one of the comments in your patch (to hcatalog/build.xml) about setting a
property build.dir.hive and triggering if not building from top level caught my
eye. I do not recommend building hcatalog by itself, outside of being called
from hive's build.xml. If need be, for testing, for eg., overriding
-Diterate.hive.full.tests from the root level works better. Having dealt with
build systems that try to build hive and hcatalog as separate components
post-merge, I can vouch that it's very painful, and some build fixes get very
kludgy very fast. I'm not against setting the build.dir property itself - it
makes the build more robust, but I'm not sure that promoting that style of
usage is something we should do.
> HBase tests fail against Hadoop 2
> ---------------------------------
>
> Key: HIVE-4388
> URL: https://issues.apache.org/jira/browse/HIVE-4388
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
> Reporter: Gunther Hagleitner
> Assignee: Brock Noland
> Attachments: HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch,
> HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch, HIVE-4388.patch,
> HIVE-4388.patch, HIVE-4388-wip.txt
>
>
> Currently we're building by default against 0.92. When you run against hadoop
> 2 (-Dhadoop.mr.rev=23) builds fail because of: HBASE-5963.
> HIVE-3861 upgrades the version of hbase used. This will get you past the
> problem in HBASE-5963 (which was fixed in 0.94.1) but fails with: HBASE-6396.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
