Szehon, can we agree that hive-1.2 is the last hive version which supports hadoop-1. Starting from hive-1.3 it requires hadoop-2
So, most probably "organizations" will start to install hive-1.2 in the next 6 months. Then they will use it for 1-2 years until they want to upgrade to the next hive version. So, for the next 2 years "organizations" as good. They can use hive-1.2 with hadoop-1. But after 2 years (in 2017) they are kinda forced to uprate to hadoop-2 if they want to use hive-1.3 Alex On Wed, Apr 29, 2015 at 1:51 PM, Szehon Ho <sze...@cloudera.com> wrote: > Alex, I'd not confuse these two things (hadoop-1 and HiveCLI), after the > proposed change from hive script to wrap beeline, if we do the job right > then user should not notice much difference. There we can eliminate dev > work while not affecting user. Whereas migration of an organization from > using Hadoop-1 cluster to Hadoop-2 cluster is a much-bigger change. > > Thanks, > Szehon > > > > On Wed, Apr 29, 2015 at 1:27 PM, Thejas Nair <thejas.n...@gmail.com> > wrote: > > > I agree, the number of users who use Hive 1.1 and newer versions > > against Hadoop 1.x is likely to very small. (Most likely an orders of > > magnitude smaller than hive cli users in same versions). > > The big improvements in Hive 1.x line require Hadoop 2.x. Users who > > are motivated to upgrade to newer 1.x lines are likely to be using > > Hadoop 2.x . > > > > > > > > On Wed, Apr 29, 2015 at 12:41 PM, Alexander Pivovarov > > <apivova...@gmail.com> wrote: > > > Xuefu, > > > > > > Remember when it was beeline discussion you mentioned "Hive community > > don't > > > need to maintain two code paths" > > > Totally agree. Resources are limited. > > > > > > I believe the same approach is applicable here. > > > Lets assume hadoop-1 + hive-1.2 is going to be used by 1% of the users. > > > > > > We'd better spend resources on implementing new functionality (useful > for > > > 99% of the users) instead of spending resources to support hadoop-1 > > profile > > > (useful for 1% of the users). > > > > > > Alex > > > > > > > > > On Wed, Apr 29, 2015 at 12:13 PM, Xuefu Zhang <xzh...@cloudera.com> > > wrote: > > > > > >> I think whether and when to deprecate Hadoop-1 is better to be moved > to > > an > > >> official [DISCUSS] thread. What I disliked is the sentiment that I > felt > > >> when we decide to deprecate something just because it's unfortunately > > >> broken for one reason or the other. Nevertheless, we should fix these > > bugs > > >> at this moment. We fixed such issues in previous release as well. > > >> > > >> I concur that we should have caught these breakages before they made > > their > > >> way into the code. Previously we discussed about running pre-commit > > tests > > >> alternatively or even randomly with either Hadoop-1 or Hadoop-2 > profile. > > >> Now it seems critical to do so. Thus, I support Szehon's proposal to > > make > > >> this happen. This is orthogonal to the deprecation discussion. > > >> > > >> Thanks, > > >> Xuefu > > >> > > >> On Wed, Apr 29, 2015 at 11:30 AM, Szehon Ho <sze...@cloudera.com> > > wrote: > > >> > > >> > Are you > > >> > > personally willing to do the work to make sure that Hive works > with > > >> > Hadoop > > >> > > 1.x? There are five new jiras (HIVE-10430, 10431, 10442, 10443, > > 10444) > > >> > > because > > >> > > trunk currently does not work with Hadoop 1.x and no one noticed. > > >> > > > >> > We setup a Hadoop-1 build sometime back and made sure a lot of > > breakage > > >> > were fixed, but had not hooked it up to HiveQA (precommit) due to > > >> concerns > > >> > of the time it would take to run these tests. It was a balance > > between > > >> > getting faster HiveQA results for everyone and testing all the > paths, > > >> even > > >> > today there are many paths we don't test. > > >> > > > >> > So if these JIRA's are fixed, would folks be amenable to alternate > > >> testing > > >> > hadoop-1 and hadoop-2 on HiveQA precommit runs and for devs to not > > commit > > >> > further breakages of Hadoop 1.x compat, for another release? Yes I > do > > >> > understand its a burden to maintain hadoop-1. > > >> > > > >> > if you use HDP, CDH or other hadoop distro then hadoop-1.x and > > hive-1.2 > > >> > > combination does not exist. > > >> > > > > >> > > > >> > I think Apache is agnostic to specific vendors so should not discuss > > >> based > > >> > on that, but vendors still have this combination, I know of users > that > > >> use > > >> > Hadoop-1 and latest Hive 1.1. > > >> > > > >> > Thanks, > > >> > Szehon > > >> > > > >> > On Wed, Apr 29, 2015 at 11:07 AM, Alexander Pivovarov < > > >> > apivova...@gmail.com> > > >> > wrote: > > >> > > > >> > > HDP and CDH distro are based on hadoop-2.x since 2013-2014 > > >> > > > > >> > > if you use HDP, CDH or other hadoop distro then hadoop-1.x and > > hive-1.2 > > >> > > combination does not exist. > > >> > > hdp-1.3 has hive-0.11 > > >> > > cdh-4.7 has hive-0.10 > > >> > > > > >> > > The only way to install hive-1.2 on hadoop-1.x is to do it > manually. > > >> > > But if you manually install hadoop components then most probably > you > > >> will > > >> > > install hadoop-2.x > > >> > > > > >> > > So, we can say that hadoop-1.x and hive-1.2 combination should not > > >> exist > > >> > in > > >> > > real life scenario. > > >> > > > > >> > > > > >> > > > > >> > > On Wed, Apr 29, 2015 at 8:03 AM, Owen O'Malley < > omal...@apache.org> > > >> > wrote: > > >> > > > > >> > > > On Tue, Apr 28, 2015 at 10:16 PM, Szehon Ho < > sze...@cloudera.com> > > >> > wrote: > > >> > > > > > >> > > > > Yes, question is whether users want to run latest Hive version > > on > > >> > > Hadoop > > >> > > > > 1.x clusters. > > >> > > > > > > >> > > > > > >> > > > But you ignored the more important question that I asked. Are > you > > >> > > > personally willing to do the work to make sure that Hive works > > with > > >> > > Hadoop > > >> > > > 1.x? There are five new jiras (HIVE-10430, 10431, 10442, 10443, > > >> 10444) > > >> > > > because > > >> > > > trunk currently does not work with Hadoop 1.x and no one > noticed. > > >> > > > > > >> > > > Supporting Hadoop 1.x costs the Hive project a lot of work and > > >> provides > > >> > > > very little in return. > > >> > > > > > >> > > > .. Owen > > >> > > > > > >> > > > > >> > > > >> > > >
