Re: [DISCUSS] About the details of JDK-8 support

2015-10-17 Thread Allen Wittenauer
On Oct 17, 2015, at 8:55 AM, Steve Loughran wrote: >> > > thx for doing this, I'll keep an eye on the builds Thanks for helping out too. You never know how software actually runs in the wild. I’m already hitting bugs that didn’t show up in testing. *sigh* > one question: which jdk8? Som

Re: [DISCUSS] About the details of JDK-8 support

2015-10-17 Thread Steve Loughran
> On 17 Oct 2015, at 15:42, Allen Wittenauer wrote: > > > > > FYI: > > I’ve switched HADOOP to Apache Yetus (from the YETUS-83 branch) in > multijdk mode with jdk7 and jdk8 turned on. (So java patches will take *twice > as long* since they aren’t done in parallel!). I’ll be ramping

Re: [DISCUSS] About the details of JDK-8 support

2015-10-17 Thread Allen Wittenauer
FYI: I’ve switched HADOOP to Apache Yetus (from the YETUS-83 branch) in multijdk mode with jdk7 and jdk8 turned on. (So java patches will take *twice as long* since they aren’t done in parallel!). I’ll be ramping up the other big features (docker and multiple maven repos) and watchi

Re: [DISCUSS] About the details of JDK-8 support

2015-10-15 Thread Tsuyoshi Ozawa
> We should target source-level support of JDK 8 too, around which you outlined > a bunch of issues around dependencies. I also found a bunch of issues around > generating documentation, site etc. I propose that we track them under the > umbrella JIRA and make progress there first. OK. I will s

Re: [DISCUSS] About the details of JDK-8 support

2015-10-15 Thread Sangjin Lee
+1 On Thu, Oct 15, 2015 at 7:57 AM, Steve Loughran wrote: > > > On 15 Oct 2015, at 14:42, Karthik Kambatla wrote: > > > > On Wed, Oct 14, 2015 at 4:28 PM, Allen Wittenauer > wrote: > > > >> > >> If people want, I could setup a cut off of yetus master to run the > jenkins > >> test-patch. (mul

Re: [DISCUSS] About the details of JDK-8 support

2015-10-15 Thread Steve Loughran
> On 15 Oct 2015, at 14:42, Karthik Kambatla wrote: > > On Wed, Oct 14, 2015 at 4:28 PM, Allen Wittenauer wrote: > >> >> If people want, I could setup a cut off of yetus master to run the jenkins >> test-patch. (multiple maven repos, docker support, multijdk support, … ) >> Yetus would get s

Re: [DISCUSS] About the details of JDK-8 support

2015-10-15 Thread Karthik Kambatla
On Wed, Oct 14, 2015 at 4:28 PM, Allen Wittenauer wrote: > > If people want, I could setup a cut off of yetus master to run the jenkins > test-patch. (multiple maven repos, docker support, multijdk support, … ) > Yetus would get some real world testing out of it and hadoop common-dev > could sto

Re: [DISCUSS] About the details of JDK-8 support

2015-10-15 Thread Steve Loughran
> On 15 Oct 2015, at 00:28, Allen Wittenauer wrote: > > > If people want, I could setup a cut off of yetus master to run the jenkins > test-patch. (multiple maven repos, docker support, multijdk support, … ) > Yetus would get some real world testing out of it and hadoop common-dev could > s

Re: [DISCUSS] About the details of JDK-8 support

2015-10-14 Thread Allen Wittenauer
If people want, I could setup a cut off of yetus master to run the jenkins test-patch. (multiple maven repos, docker support, multijdk support, … ) Yetus would get some real world testing out of it and hadoop common-dev could stop spinning in circles over some of the same issues month after mo

Re: [DISCUSS] About the details of JDK-8 support

2015-10-14 Thread Robert Kanter
The only problem with trying to get the JDK 8 trunk builds green (or blue I guess) is that it's like trying to hit a moving target because of how many new commits keep coming in. I was looking at fixing these a while ago, and managed to at least make them compile and fixed (or worked with others t

Re: [DISCUSS] About the details of JDK-8 support

2015-10-14 Thread Steve Loughran
> On 13 Oct 2015, at 17:32, Haohui Mai wrote: > > Just to echo Steve's idea -- if we're seriously considering supporting > JDK 8, maybe the first thing to do is to set up the jenkins to run > with JDK 8? I'm happy to help. Does anyone know who I can talk to if I > need to play around with all th

Re: [DISCUSS] About the details of JDK-8 support

2015-10-13 Thread Chen He
On Tue, Oct 13, 2015 at 3:00 AM, Jean-Baptiste Note wrote: > Hi, > > As far as security is concerned we (Criteo) are using CDH5 with JDK8 in > production with security enabled. > We reported some gripes with some specific java versions: > https://issues.cloudera.org/browse/DISTRO-732 > > I would

Re: [DISCUSS] About the details of JDK-8 support

2015-10-13 Thread Kihwal Lee
quot; ; dev ; "yarn-...@hadoop.apache.org" Sent: Tuesday, October 13, 2015 5:00 AM Subject: Re: [DISCUSS] About the details of JDK-8 support Hi, As far as security is concerned we (Criteo) are using CDH5 with JDK8 in production with security enabled. We reported some gripes with some

Re: [DISCUSS] About the details of JDK-8 support

2015-10-13 Thread Haohui Mai
Just to echo Steve's idea -- if we're seriously considering supporting JDK 8, maybe the first thing to do is to set up the jenkins to run with JDK 8? I'm happy to help. Does anyone know who I can talk to if I need to play around with all the Jenkins knob? ~Haohui On Tue, Oct 13, 2015 at 8:24 AM,

Re: [DISCUSS] About the details of JDK-8 support

2015-10-13 Thread Vinod Vavilapalli
If you see the community discussion thread on 2.8, my proposal was to support *both* JDK 7 and JDK 8 first. The last time we had discussion about dropping JDKs it wasn’t fun, so let’s not go there for now. In terms of runtime support for JDK 8, yes, there is vast evidence that things already wo

Re: [DISCUSS] About the details of JDK-8 support

2015-10-13 Thread Jean-Baptiste Note
Hi, As far as security is concerned we (Criteo) are using CDH5 with JDK8 in production with security enabled. We reported some gripes with some specific java versions: https://issues.cloudera.org/browse/DISTRO-732 I would bump the dependency to _u51 or later; _u40 _u45 do have a lot of problems w

Re: [DISCUSS] About the details of JDK-8 support

2015-10-13 Thread Tsuyoshi Ozawa
Thank you for sharing lots information and joining discussion. About the runtime support of JDK-8, let's describe it on wiki. It would be great for users to clarify which JDK version they should use. It's also useful to describe to use "1.8.0_40 or above" explicitly. Andrew, Elliott, could you up

Re: [DISCUSS] About the details of JDK-8 support

2015-10-09 Thread Sangjin Lee
Yes, at least for us, dropping the java 7 support (e.g. moving to java 8 source-wise) **at this point** would be an issue. I concur with the sentiment that we should preserve the java 7 support on branch-2 (not not move to java 8 source level) but can consider it for trunk. My 2 cents. Thanks, San

Re: [DISCUSS] About the details of JDK-8 support

2015-10-09 Thread Steve Loughran
> On 7 Oct 2015, at 22:39, Elliott Clark wrote: > > On Mon, Oct 5, 2015 at 5:35 PM, Tsuyoshi Ozawa wrote: > >> Do you have any concern about this? I’ve not >> tested with HBase yet. >> > > We've been running JDK 8u60 in production with Hadoop 2.6.X and HBase for > quite a while. Everything h

Re: [DISCUSS] About the details of JDK-8 support

2015-10-07 Thread Elliott Clark
On Mon, Oct 5, 2015 at 5:35 PM, Tsuyoshi Ozawa wrote: > Do you have any concern about this? I’ve not > tested with HBase yet. > We've been running JDK 8u60 in production with Hadoop 2.6.X and HBase for quite a while. Everything has been very stable for us. We're running and compiling with jdk8.

Re: [DISCUSS] About the details of JDK-8 support

2015-10-07 Thread Andrew Wang
> > > On 7 Oct 2015, at 17:23, Andrew Wang wrote: > > > > We've been supporting JDK8 as a runtime for CDH5 for a while now (meaning > > the full stack including HBase), so I agree that we're good there. > > > > > with Kerberos on? > > Yea, I haven't been that involved with our internal JDK validat

Re: [DISCUSS] About the details of JDK-8 support

2015-10-07 Thread Steve Loughran
> On 7 Oct 2015, at 17:23, Andrew Wang wrote: > > We've been supporting JDK8 as a runtime for CDH5 for a while now (meaning > the full stack including HBase), so I agree that we're good there. > with Kerberos on? > I'm against dropping JDK7 support though in branch-2. +1. We're only talking

Re: [DISCUSS] About the details of JDK-8 support

2015-10-07 Thread Andrew Wang
We've been supporting JDK8 as a runtime for CDH5 for a while now (meaning the full stack including HBase), so I agree that we're good there. I'm against dropping JDK7 support though in branch-2. Even bumping dependency versions scares me, since it often leads to downstream pain. Any comment about

Re: [DISCUSS] About the details of JDK-8 support

2015-10-07 Thread Steve Loughran
> On 7 Oct 2015, at 07:29, Masatake Iwasaki wrote: > > Thanks for clear summary, Tsuyoshi. > > I read some related past discussions. > > https://wiki.apache.org/hadoop/MovingToJdk7and8 > http://search-hadoop.com/m/uOzYtGSiCs1acRnh > http://search-hadoop.com/m/uOzYthdWJqpGdSZ1 > > Though th

Re: [DISCUSS] About the details of JDK-8 support

2015-10-06 Thread Masatake Iwasaki
Thanks for clear summary, Tsuyoshi. I read some related past discussions. https://wiki.apache.org/hadoop/MovingToJdk7and8 http://search-hadoop.com/m/uOzYtGSiCs1acRnh http://search-hadoop.com/m/uOzYthdWJqpGdSZ1 Though there seems to be no consensus about when to drop java 7 support yet, it

Re: [DISCUSS] About the details of JDK-8 support

2015-10-05 Thread Tsuyoshi Ozawa
I forgot to mention - we also need to upgrade Google Guice to 0.4.0 for source-level JDK8 support in addition to asm and cglib. https://github.com/google/guice/wiki/Guice40 > Some highlights: > Java8 runtime compatibility for Guice core & all extensions. Best regards, - Tsuyoshi On Tue, Oct 6,

[DISCUSS] About the details of JDK-8 support

2015-10-05 Thread Tsuyoshi Ozawa
Hi commiters and users of Hadoop stack, I’ll share the current status of JDK-8 support here. We can take a two-step approach to support JDK-8 - runtime-level support and source-level support. About runtime-level support, I’ve tested Hadoop stack with JDK-8 e.g. MapReduce, Spark, Tez, Flink on YA