Re: Thinking ahead to hadoop-2.6

Hi, I am worry about KMS and transparent encryption since there are quite many bugs discovered after it got merged to branch-2. It gives us an impression that the feature is not yet well tested. Indeed, transparent encryption is a complicated feature which changes the core part of HDFS. It i

Re: Thinking ahead to hadoop-2.6

I am also +1 for the end of next week. YARN-1492 (shared cache) is near the final stages of the review, and with a little more time it would make it easier to put it in. On Tue, Sep 23, 2014 at 4:48 PM, Subramaniam V K wrote: > +1 for end of next week. > > We have got all the patches for YARN-10

Re: Thinking ahead to hadoop-2.6

+1 for end of next week. We have got all the patches for YARN-1051 committed to the branch. We are currently fixing test-patch & plan to call a merge vote soon. Hopefully it will go through by end of next week. Thanks, Subru On Tue, Sep 23, 2014 at 4:31 PM, Karthik Kambatla wrote: > Looking at

[jira] [Created] (HADOOP-11122) Fix findbugs and order of key updates Abstract/ZK DelegationTokenSecretManagers

Karthik Kambatla created HADOOP-11122: - Summary: Fix findbugs and order of key updates Abstract/ZK DelegationTokenSecretManagers Key: HADOOP-11122 URL: https://issues.apache.org/jira/browse/HADOOP-11122

Re: Thinking ahead to hadoop-2.6

Looking at the patches, we might be able to get most of YARN-1492 in by the end of next week. There would be a couple of security items still remaining, but we can may be call the feature alpha-ready without them. On Tue, Sep 23, 2014 at 4:25 PM, Chris Trezzo wrote: > I would like to see the sha

Re: Thinking ahead to hadoop-2.6

I think we should delay cutting the branch beyond the end of the week then, seeing as the HDFS-6581 merge vote (assuming it goes through) doesn't close until 12am on Tuesday next week. Archival storage is also still finishing up user APIs and I had a few other q's posted on HDFS-7081, so again migh

Re: Thinking ahead to hadoop-2.6

I would like to see the shared cache (YARN-1492) make it in as well. We are going through the final review process (with Karthik and Vinod) and should be fairly close to complete. This is another feature that has been under development for a while. Thanks, Chris On Tue, Sep 23, 2014 at 4:00 PM, S

Re: Thinking ahead to hadoop-2.6

I actually would like to see both archival storage and single replica memory writes to be in 2.6 release. Archival storage is in the final stages of getting ready for branch-2 merge as Nicholas has already indicated on the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both of these featu

Re: Thinking ahead to hadoop-2.6

Hey Arun, Maybe we could do a quick run through of the Roadmap wiki and add/retarget things accordingly? I think the KMS and transparent encryption are ready to go. We've got a very few further bug fixes pending, but that's it. Two HDFS things that I think probably won't make the end of the week

Re: Thinking ahead to hadoop-2.6

Looks like most of the content is in and hadoop-2.6 is shaping up nicely. I'll create branch-2.6 by end of the week and we can go from there to stabilize it - hopefully in the next few weeks. Thoughts? thanks, Arun On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy wrote: > Folks, > > With hadoo

Re: In hindsight... Re: Thinking ahead to hadoop-2.6

Sorry, coming to discussion late. We all agreed that 2.6 would the *last* release supporting JDK6 and hadoop-2.7 would drop support for JDK6. We could easily do 2.7 right after 2.6 (maybe with few critical bug-fixes) with the defining feature of 2.7 being *JDK7 only*. I've checked with HBase, Pig

[jira] [Created] (HADOOP-11121) native libraries guide still mentions pre-built native libs

Allen Wittenauer created HADOOP-11121: - Summary: native libraries guide still mentions pre-built native libs Key: HADOOP-11121 URL: https://issues.apache.org/jira/browse/HADOOP-11121 Project: Hado

Re: Time to address the Guava version problem

I'm using curator elsewhere, it does log a lot (as does the ZK client), but it solves a lot of problem. It's being adopted more downstream too. I'm wondering if we can move the code to the extent we know it works with Guava 16, with the hadoop core being 16-compatible, but not actually migrated to

Re: Time to address the Guava version problem

At the same time, not being able to use Curator will require a lot of extra code, a lot of which we probably already have from the ZKRMStateStore, but it's not available to use in hadoop-auth. We'd need to create our own ZK libraries that Hadoop components can use, but (a) that's going to take a w

[jira] [Created] (HADOOP-11120) hadoop fs -rmr gives wrong advice

Allen Wittenauer created HADOOP-11120: - Summary: hadoop fs -rmr gives wrong advice Key: HADOOP-11120 URL: https://issues.apache.org/jira/browse/HADOOP-11120 Project: Hadoop Common Issue T

[jira] [Created] (HADOOP-11119) TrashPolicyDefault init pushes messages to command line

Allen Wittenauer created HADOOP-9: - Summary: TrashPolicyDefault init pushes messages to command line Key: HADOOP-9 URL: https://issues.apache.org/jira/browse/HADOOP-9 Project: Hadoop Co

Re: Time to address the Guava version problem

If we've broken compatibility in branch-2, that's a bug that we need to fix. HADOOP-10868 has not yet made it into a release; I don't see it as a justification for solidifying the breakage. -1 to upgrading Guava in branch-2. On Tue, Sep 23, 2014 at 3:06 AM, Steve Loughran wrote: > +1 to upgradi

Re: Time to address the Guava version problem

+1 to upgrading guava. Irrespective of downstream apps, the hadoop source tree is now internally inconsistent On 22 September 2014 17:56, Sangjin Lee wrote: > I agree that a more robust solution is to have better classloading > isolation. > > Still, IMHO guava (and possibly protobuf as well) sti

[jira] [Resolved] (HADOOP-11118) build hadoop by Java 7 in default

[ https://issues.apache.org/jira/browse/HADOOP-8?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-8. - Resolution: Duplicate > build hadoop by Java 7 in default >

Build failed in Jenkins: Hadoop-Common-0.23-Build #1081

See -- [...truncated 8263 lines...] Running org.apache.hadoop.io.TestBloomMapFile Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.881 sec Running org.apache.hadoop.io.TestObject