Ok. After installing protobuf 2.5.0 I can compile trunk. But now I cannot compile Hadoop-2 branches. None of them. So if I switch between branches I need to reinstall protobuf?
Is there a consensus about going towards protobuf 2.5.0 upgrade in ALL versions? I did not get definite impression there is. If not it could be a pretty big disruption. Thanks, --Konst On Mon, Aug 12, 2013 at 3:19 PM, Alejandro Abdelnur <t...@cloudera.com>wrote: > I've just committed HADOOP-9845 to trunk (only trunk at the moment). > > To build trunk now you need protoc 2.5.0 (the build will fail with a > warning if you don't have it). > > We'd propagate this to the 2 branches once the precommit build is back to > normal and see things are OK. > > Thanks. > > > On Mon, Aug 12, 2013 at 2:57 PM, Alejandro Abdelnur <t...@cloudera.com > >wrote: > > > About to commit HADOOP-9845 to trunk, in 5 mins. This will make trunk use > > protoc 2.5.0. > > > > thx > > > > > > On Mon, Aug 12, 2013 at 11:47 AM, Giridharan Kesavan < > > gkesa...@hortonworks.com> wrote: > > > >> I can take care of re-installing 2.4 and installing 2.5 in a different > >> location. This would fix 2.0 branch builds as well. > >> Thoughts? > >> > >> -Giri > >> > >> > >> On Mon, Aug 12, 2013 at 11:37 AM, Alejandro Abdelnur <t...@cloudera.com > >> >wrote: > >> > >> > Giri, > >> > > >> > first of all, thanks for installing protoc 2.5.0. > >> > > >> > I didn't know we were installing them as the only version and not > >> driven by > >> > env/path settings. > >> > > >> > Now we have a bit of a problem, precommit builds are broken because of > >> > mismatch of protoc (2.5.0) and protobuf JAR( 2.4.1). > >> > > >> > We have to options: > >> > > >> > 1* commit HADOOP-9845 that will bring protobuf to 2.5.0 and iron out > any > >> > follow up issues. > >> > 2* reinstall protoc 2.4.1 in the jenkins machines and have 2.4.1 and > >> 2.5.0 > >> > coexisting > >> > > >> > My take would be to commit HADOOP-9845 in trunk, iron out any issues > an > >> > then merge it to the other branches. > >> > > >> > We need to sort this out quickly as precommits are not working. > >> > > >> > I'll wait till 3PM today for objections to option #1, if none I'll > >> commit > >> > it to trunk. > >> > > >> > Thanks. > >> > > >> > Alejandro > >> > > >> > > >> > > >> > On Mon, Aug 12, 2013 at 11:30 AM, Giridharan Kesavan < > >> > gkesa...@hortonworks.com> wrote: > >> > > >> > > Like I said protoc is upgraded from 2.4 to 2.5. 2.5 is in the > default > >> > path. > >> > > If we still need 2.4 I may have to install it. Let me know > >> > > > >> > > -Giri > >> > > > >> > > > >> > > On Sat, Aug 10, 2013 at 7:01 AM, Alejandro Abdelnur < > >> t...@cloudera.com > >> > > >wrote: > >> > > > >> > > > thanks giri, how do we set 2.4 or 2.5., what is the path to both > so > >> we > >> > > can > >> > > > use and env to set it in the jobs? > >> > > > > >> > > > thx > >> > > > > >> > > > Alejandro > >> > > > (phone typing) > >> > > > > >> > > > On Aug 9, 2013, at 23:10, Giridharan Kesavan < > >> gkesa...@hortonworks.com > >> > > > >> > > > wrote: > >> > > > > >> > > > > build slaves hadoop1-hadoop9 now has libprotoc 2.5.0 > >> > > > > > >> > > > > > >> > > > > > >> > > > > -Giri > >> > > > > > >> > > > > > >> > > > > On Fri, Aug 9, 2013 at 10:56 PM, Giridharan Kesavan < > >> > > > > gkesa...@hortonworks.com> wrote: > >> > > > > > >> > > > >> Alejandro, > >> > > > >> > >> > > > >> I'm upgrading protobuf on slaves hadoop1-hadoop9. > >> > > > >> > >> > > > >> -Giri > >> > > > >> > >> > > > >> > >> > > > >> On Fri, Aug 9, 2013 at 1:15 PM, Alejandro Abdelnur < > >> > t...@cloudera.com > >> > > > >wrote: > >> > > > >> > >> > > > >>> pinging again, I need help from somebody with sudo access to > the > >> > > hadoop > >> > > > >>> jenkins boxes to do this or to get sudo access for a couple of > >> > hours > >> > > to > >> > > > >>> set > >> > > > >>> up myself. > >> > > > >>> > >> > > > >>> Please!!! > >> > > > >>> > >> > > > >>> thx > >> > > > >>> > >> > > > >>> > >> > > > >>> On Thu, Aug 8, 2013 at 2:29 PM, Alejandro Abdelnur < > >> > > t...@cloudera.com > >> > > > >>>> wrote: > >> > > > >>> > >> > > > >>>> To move forward with this we need protoc 2.5.0 in the apache > >> > hadoop > >> > > > >>>> jenkins boxes. > >> > > > >>>> > >> > > > >>>> Who can help with this? I assume somebody at Y!, right? > >> > > > >>>> > >> > > > >>>> Thx > >> > > > >>>> > >> > > > >>>> > >> > > > >>>> On Thu, Aug 8, 2013 at 2:24 PM, Elliott Clark < > >> ecl...@apache.org> > >> > > > >>> wrote: > >> > > > >>>> > >> > > > >>>>> In HBase land we've pretty well discovered that we'll need > to > >> > have > >> > > > the > >> > > > >>>>> same version of protobuf that the HDFS/Yarn/MR servers are > >> > running. > >> > > > >>>>> That is to say there are issues with ever having 2.4.x and > >> 2.5.x > >> > on > >> > > > >>>>> the same class path. > >> > > > >>>>> > >> > > > >>>>> Upgrading to 2.5.x would be great, as it brings some new > >> classes > >> > we > >> > > > >>>>> could use. With that said HBase is getting pretty close to > a > >> > > rather > >> > > > >>>>> large release (0.96.0 aka The Singularity) so getting this > in > >> > > sooner > >> > > > >>>>> rather than later would be great. If we could get this into > >> > 2.1.0 > >> > > it > >> > > > >>>>> would be great as that would allow us to have a pretty easy > >> story > >> > > to > >> > > > >>>>> users with regards to protobuf version. > >> > > > >>>>> > >> > > > >>>>> On Thu, Aug 8, 2013 at 8:18 AM, Kihwal Lee < > >> kih...@yahoo-inc.com > >> > > > >> > > > >>> wrote: > >> > > > >>>>>> Sorry to hijack the thread but, I also wanted to mention > >> Avro. > >> > See > >> > > > >>>>> HADOOP-9672. > >> > > > >>>>>> The version we are using has memory leak and inefficiency > >> > issues. > >> > > > >>> We've > >> > > > >>>>> seen users running into it. > >> > > > >>>>>> > >> > > > >>>>>> Kihwal > >> > > > >>>>>> > >> > > > >>>>>> > >> > > > >>>>>> ________________________________ > >> > > > >>>>>> From: Tsuyoshi OZAWA <ozawa.tsuyo...@gmail.com> > >> > > > >>>>>> To: "common-dev@hadoop.apache.org" < > >> > common-dev@hadoop.apache.org> > >> > > > >>>>>> Cc: "hdfs-...@hadoop.apache.org" < > hdfs-...@hadoop.apache.org > >> >; > >> > " > >> > > > >>>>> yarn-...@hadoop.apache.org" <yarn-...@hadoop.apache.org>; " > >> > > > >>>>> mapreduce-...@hadoop.apache.org" < > >> > mapreduce-...@hadoop.apache.org> > >> > > > >>>>>> Sent: Thursday, August 8, 2013 1:59 AM > >> > > > >>>>>> Subject: Re: Upgrade to protobuf 2.5.0 for the 2.1.0 > release, > >> > > > >>>>> HADOOP-9845 > >> > > > >>>>>> > >> > > > >>>>>> > >> > > > >>>>>> Hi, > >> > > > >>>>>> > >> > > > >>>>>> About Hadoop, Harsh is dealing with this problem in > >> HADOOP-9346. > >> > > > >>>>>> For more detail, please see the JIRA ticket: > >> > > > >>>>>> https://issues.apache.org/jira/browse/HADOOP-9346 > >> > > > >>>>>> > >> > > > >>>>>> - Tsuyoshi > >> > > > >>>>>> > >> > > > >>>>>> On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur < > >> > > > >>> t...@cloudera.com> > >> > > > >>>>> wrote: > >> > > > >>>>>>> I' like to upgrade to protobuf 2.5.0 for the 2.1.0 > release. > >> > > > >>>>>>> > >> > > > >>>>>>> As mentioned in HADOOP-9845, Protobuf 2.5 has significant > >> > > benefits > >> > > > >>> to > >> > > > >>>>>>> justify the upgrade. > >> > > > >>>>>>> > >> > > > >>>>>>> Doing the upgrade now, with the first beta, will make > things > >> > > easier > >> > > > >>> for > >> > > > >>>>>>> downstream projects (like HBase) using protobuf and > adopting > >> > > Hadoop > >> > > > >>> 2. > >> > > > >>>>> If > >> > > > >>>>>>> we do the upgrade later, downstream projects will have to > >> > > support 2 > >> > > > >>>>>>> different versions and they my get in nasty waters due to > >> > > classpath > >> > > > >>>>> issues. > >> > > > >>>>>>> > >> > > > >>>>>>> I've locally tested the patch in a pseudo deployment of > >> > > 2.1.0-beta > >> > > > >>>>> branch > >> > > > >>>>>>> and it works fine (something is broken in trunk in the RPC > >> > layer > >> > > > >>>>> YARN-885). > >> > > > >>>>>>> > >> > > > >>>>>>> Now, to do this it will require a few things: > >> > > > >>>>>>> > >> > > > >>>>>>> * Make sure protobuf 2.5.0 is available in the jenkins box > >> > > > >>>>>>> * A follow up email to dev@ aliases indicating developers > >> > should > >> > > > >>>>> install > >> > > > >>>>>>> locally protobuf 2.5.0 > >> > > > >>>>>>> > >> > > > >>>>>>> Thanks. > >> > > > >>>>>>> > >> > > > >>>>>>> -- > >> > > > >>>>>>> Alejandro > >> > > > >>>> > >> > > > >>>> > >> > > > >>>> > >> > > > >>>> -- > >> > > > >>>> Alejandro > >> > > > >>> > >> > > > >>> > >> > > > >>> > >> > > > >>> -- > >> > > > >>> Alejandro > >> > > > >> > >> > > > >> > >> > > > > >> > > > >> > > >> > > >> > > >> > -- > >> > Alejandro > >> > > >> > > > > > > > > -- > > Alejandro > > > > > > -- > Alejandro >
