Yes I think its a good compromise to at least add compilation of -Phadoop-1 until we reach consensus on a timeline.
I have actually filed a JIRA for that half a year back, HIVE-7375, I will take a look. It was unfortunate this was deprioritzed in favor of running tests, which was not successful due to the time/resource taken. Thanks Szehon PS all precommit tests are actually broken right now due to the build machine not being able to checkout from ASF github, I have filed HIVE-10583 to fix it, will send a follow up email on that in another topic once that's resolved. On Fri, May 1, 2015 at 5:37 PM, Xuefu Zhang <xzh...@cloudera.com> wrote: > To close the loop, I have created HIVE-10597 to fix all the compilation > errors on Hadoop 1. Once this is in, @Szehon, could you please include > compilation on Hadoop-1 in precommit test? > > Whether or when to remove Hadoop 1 deserves a formal [DISCUSS] thread. > > Thanks, > Xuefu > > On Thu, Apr 30, 2015 at 12:53 PM, Sergey Shelukhin <ser...@hortonworks.com > > > wrote: > > > That runs the risk of not testing the code being committed against the > > Hadoop version for which it¹s actually going to be used. > > I¹d be +1 on deprecating Hadoop 1 support in a dot version (1.3?). It > will > > also provide impetus for people to upgrade. > > > > It will allow us to remove some shims tooŠ > > > > > > On 15/4/29, 12:13, "Xuefu Zhang" <xzh...@cloudera.com> wrote: > > > > >I think whether and when to deprecate Hadoop-1 is better to be moved to > an > > >official [DISCUSS] thread. What I disliked is the sentiment that I felt > > >when we decide to deprecate something just because it's unfortunately > > >broken for one reason or the other. Nevertheless, we should fix these > bugs > > >at this moment. We fixed such issues in previous release as well. > > > > > >I concur that we should have caught these breakages before they made > their > > >way into the code. Previously we discussed about running pre-commit > tests > > >alternatively or even randomly with either Hadoop-1 or Hadoop-2 profile. > > >Now it seems critical to do so. Thus, I support Szehon's proposal to > make > > >this happen. This is orthogonal to the deprecation discussion. > > > > > >Thanks, > > >Xuefu > > > > > >On Wed, Apr 29, 2015 at 11:30 AM, Szehon Ho <sze...@cloudera.com> > wrote: > > > > > >> Are you > > >> > personally willing to do the work to make sure that Hive works with > > >> Hadoop > > >> > 1.x? There are five new jiras (HIVE-10430, 10431, 10442, 10443, > 10444) > > >> > because > > >> > trunk currently does not work with Hadoop 1.x and no one noticed. > > >> > > >> We setup a Hadoop-1 build sometime back and made sure a lot of > breakage > > >> were fixed, but had not hooked it up to HiveQA (precommit) due to > > >>concerns > > >> of the time it would take to run these tests. It was a balance > between > > >> getting faster HiveQA results for everyone and testing all the paths, > > >>even > > >> today there are many paths we don't test. > > >> > > >> So if these JIRA's are fixed, would folks be amenable to alternate > > >>testing > > >> hadoop-1 and hadoop-2 on HiveQA precommit runs and for devs to not > > >>commit > > >> further breakages of Hadoop 1.x compat, for another release? Yes I do > > >> understand its a burden to maintain hadoop-1. > > >> > > >> if you use HDP, CDH or other hadoop distro then hadoop-1.x and > hive-1.2 > > >> > combination does not exist. > > >> > > > >> > > >> I think Apache is agnostic to specific vendors so should not discuss > > >>based > > >> on that, but vendors still have this combination, I know of users that > > >>use > > >> Hadoop-1 and latest Hive 1.1. > > >> > > >> Thanks, > > >> Szehon > > >> > > >> On Wed, Apr 29, 2015 at 11:07 AM, Alexander Pivovarov < > > >> apivova...@gmail.com> > > >> wrote: > > >> > > >> > HDP and CDH distro are based on hadoop-2.x since 2013-2014 > > >> > > > >> > if you use HDP, CDH or other hadoop distro then hadoop-1.x and > > >>hive-1.2 > > >> > combination does not exist. > > >> > hdp-1.3 has hive-0.11 > > >> > cdh-4.7 has hive-0.10 > > >> > > > >> > The only way to install hive-1.2 on hadoop-1.x is to do it manually. > > >> > But if you manually install hadoop components then most probably you > > >>will > > >> > install hadoop-2.x > > >> > > > >> > So, we can say that hadoop-1.x and hive-1.2 combination should not > > >>exist > > >> in > > >> > real life scenario. > > >> > > > >> > > > >> > > > >> > On Wed, Apr 29, 2015 at 8:03 AM, Owen O'Malley <omal...@apache.org> > > >> wrote: > > >> > > > >> > > On Tue, Apr 28, 2015 at 10:16 PM, Szehon Ho <sze...@cloudera.com> > > >> wrote: > > >> > > > > >> > > > Yes, question is whether users want to run latest Hive version > on > > >> > Hadoop > > >> > > > 1.x clusters. > > >> > > > > > >> > > > > >> > > But you ignored the more important question that I asked. Are you > > >> > > personally willing to do the work to make sure that Hive works > with > > >> > Hadoop > > >> > > 1.x? There are five new jiras (HIVE-10430, 10431, 10442, 10443, > > >>10444) > > >> > > because > > >> > > trunk currently does not work with Hadoop 1.x and no one noticed. > > >> > > > > >> > > Supporting Hadoop 1.x costs the Hive project a lot of work and > > >>provides > > >> > > very little in return. > > >> > > > > >> > > .. Owen > > >> > > > > >> > > > >> > > > > >
