RE: Apache Hadoop 2.8.3 Release Plan

2017-11-20 Thread Zheng, Kai
e and >> timely fashion, let's try to address that... This is interesting. Do you aware any means to do that? Thanks! Regards, Kai -Original Message- From: Andrew Wang [mailto:andrew.w...@cloudera.com] Sent: Tuesday, November 21, 2017 2:22 PM To: Zheng, Kai Cc: Junping Du ;

RE: Apache Hadoop 2.8.3 Release Plan

2017-11-20 Thread Zheng, Kai
Hi Junping, Thank you for making 2.8.2 happen and now planning the 2.8.3 release. I have an ask, is it convenient to include the back port work for OSS connector module? We have some Hadoop users that wish to have it by default for convenience, though in the past they used it by back porting t

RE: [VOTE] Merge yarn-native-services branch into trunk

2017-11-09 Thread Zheng, Kai
Cool to have this feature! Thanks Jian and all. Regards, Kai -Original Message- From: Vinod Kumar Vavilapalli [mailto:vino...@apache.org] Sent: Tuesday, November 07, 2017 7:20 AM To: Jian He Cc: yarn-...@hadoop.apache.org; common-...@hadoop.apache.org; Hdfs-dev ; mapreduce-...@hadoop.a

RE: [DISCUSS] A final minor release off branch-2?

2017-11-06 Thread Zheng, Kai
Thanks Vinod. >> Of the top of my head, one of the biggest areas is application >> compatibility. When folks move from 2.x to 3.x, are their apps binary >> compatible? Source compatible? Or need changes? I thought these are good concerns from overall perspective. On the other hand, I've discuss

RE: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-09-02 Thread Zheng, Kai
Thanks Sammi. My non-binding +1 to make the release candidate. Regards, Kai -Original Message- From: Chen, Sammi Sent: Friday, September 02, 2016 4:59 PM To: Zheng, Kai ; Andrew Wang ; Arun Suresh Cc: common-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce

RE: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

2016-07-22 Thread Zheng, Kai
For the leveldb thing, wouldn't we have an alternative option in Java for the platforms where leveldb isn't supported yet due to whatever reasons. IMO, native library would be best to be used for optimization and production for performance. For development and pure Java platform, by default pure

RE: Looking to a Hadoop 3 release

2016-02-18 Thread Zheng, Kai
Thanks Andrew for driving this. Wonder if it's a good chance for HADOOP-12579 (Deprecate and remove WriteableRPCEngine) to be in. Note it's not an incompatible change, but feel better to be done in the major release. Regards, Kai -Original Message- From: Andrew Wang [mailto:andrew.w...@

RE: Hadoop encryption module as Apache Chimera incubator project

2016-02-03 Thread Zheng, Kai
rg Subject: Re: Hadoop encryption module as Apache Chimera incubator project Let's do one step at a time. There is a clear need for common encryption, and let's focus on making that happen. On Wed, Feb 3, 2016 at 4:48 PM, Zheng, Kai wrote: > I thought this discussion would switch

RE: Hadoop encryption module as Apache Chimera incubator project

2016-02-03 Thread Zheng, Kai
I thought this discussion would switch to common-dev@ now? >> Would it make sense to also package some of the compression libraries, and >> maybe some of the text processing from MapReduce? Evolving some of this code >> to a common library with few/no dependencies would be generally useful. As a

RE: Hadoop encryption module as Apache Chimera incubator project

2016-01-28 Thread Zheng, Kai
Give the forgotten reference: [1] https://github.com/apache/directory-kerby -Original Message- From: Zheng, Kai [mailto:kai.zh...@intel.com] Sent: Friday, January 29, 2016 9:10 AM To: hdfs-dev@hadoop.apache.org Subject: RE: Hadoop encryption module as Apache Chimera incubator project

RE: Hadoop encryption module as Apache Chimera incubator project

2016-01-28 Thread Zheng, Kai
Sounds good to have further discussions. Mind I have some questions, thanks. @Haifeng: Thanks Uma & Haifeng for your answers about how to scope and vision Chimera. It sounds good to me. So I guess we would prefer to use a generic project name like Chimera to make the project not tightly coupled

RE: Hadoop encryption module as Apache Chimera incubator project

2016-01-21 Thread Zheng, Kai
keep this mail thread focused >just on Chimera. Thank you! > >Uma and everyone, thank you for the proposal. +1 to proceed. > >--Chris Nauroth > > > > >On 1/20/16, 11:16 PM, "Zheng, Kai" wrote: > >>Thanks Uma. >> >>I have a quest

RE: Hadoop encryption module as Apache Chimera incubator project

2016-01-20 Thread Zheng, Kai
Thanks Uma. I have a question by the way, it's not about Chimera project, but about the mentioned advantage 1 and libhadoop.so installation problem. I copied the saying as below for convenience. >>1. As Chimera embedded the native in jar (similar to Snappy java), it solves >>the current issue

RE: Hadoop encryption module as Apache Chimera incubator project

2016-01-20 Thread Zheng, Kai
we are looking for is some level of autonomy then it would need to be a module with its own release train - or at least be able to. On Jan 20, 2016, at 9:18 PM, Zheng, Kai wrote: > Just a question. Becoming a separate jar/module in Apache Commons means > Chimera or the module can be

RE: Hadoop encryption module as Apache Chimera incubator project

2016-01-20 Thread Zheng, Kai
Just a question. Becoming a separate jar/module in Apache Commons means Chimera or the module can be released separately or in a timely manner, not coupling with other modules for release in the project? Thanks. Regards, Kai -Original Message- From: Aaron T. Myers [mailto:a...@cloudera.

RE: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

2015-11-03 Thread Zheng, Kai
Phase II out of 2.9, then we just need to change the target version of the Phase II umbrella rather than all the subtasks. On Mon, Nov 2, 2015 at 4:26 PM, Zheng, Kai wrote: > Yeah, so for the issues we recently resolved on trunk and are > addressing as follow-on tasks in Phase I, we would

RE: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

2015-11-02 Thread Zheng, Kai
should either move the jira under HDFS-8031 or update the jira component as "erasure-coding" when making further improvement or fixing bugs in EC. In this way it will be easier for later backporting EC to 2.9. On Mon, Nov 2, 2015 at 3:48 PM, Vinayakumar B wrote: > +1 for the idea

RE: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

2015-11-02 Thread Zheng, Kai
trunk. >>>>>>>>> >>>>>>>>> Thanks all for the contribution. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Yi Liu >>>>>>>>> >>&

RE: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk

2015-09-22 Thread Zheng, Kai
Non-binding +1 According to our extensive performance tests, striping + ISA-L coder based erasure coding not only can save storage, but also can increase the throughput of a client or a cluster. It will be a great addition to HDFS and its users. Based on the latest branch codes, we also observe

RE: IMPORTANT: testing patches for branches

2015-04-29 Thread Zheng, Kai
@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: IMPORTANT: testing patches for branches On Apr 22, 2015, at 11:34 PM, Zheng, Kai wrote: > Hi Allen, > > This sounds great. > >>> Naming a patch foo-HDFS-7285.00.patch should get te

RE: IMPORTANT: testing patches for branches

2015-04-22 Thread Zheng, Kai
Hi Allen, This sounds great. >> Naming a patch foo-HDFS-7285.00.patch should get tested on the HDFS-7285 >> branch. Does it happen locally in developer's machine when running test-patch.sh, or also mean something in Hadoop Jenkins building when a JIRA becoming patch available? Thanks. Regard

RE: Looking to a Hadoop 3 release

2015-03-04 Thread Zheng, Kai
Might I have some comments for this, just providing my thought. Thanks. >> If we start now, it might make it out by 2016. If we start now, >> downstreamers can start aligning themselves to land versions that suit at >> about the same time. Not only for down streamers to align with the long term

RE: 2.7 status

2015-03-04 Thread Zheng, Kai
Kumar Vavilapalli [mailto:vino...@hortonworks.com] Sent: Tuesday, March 03, 2015 11:31 AM To: Zheng, Kai Cc: mapreduce-...@hadoop.apache.org; hdfs-dev@hadoop.apache.org; Hadoop Common; yarn-...@hadoop.apache.org Subject: Re: 2.7 status Kai, please ping the reviewers that were already looking at

RE: 2.7 status

2015-03-02 Thread Zheng, Kai
Is it interested to get the following issues in the release ? Thanks ! HADOOP-10670 HADOOP-10671 Regards, Kai -Original Message- From: Yongjun Zhang [mailto:yzh...@cloudera.com] Sent: Monday, March 02, 2015 4:46 AM To: hdfs-dev@hadoop.apache.org Cc: Vinod Kumar Vavilapalli; Hadoop Commo

RE: Looking to a Hadoop 3 release

2015-03-02 Thread Zheng, Kai
Sorry for the bad. I thought it was sending to my colleagues. By the way, for the JDK8 support, we (Intel) would like to investigate further and help, thanks. Regards, Kai -Original Message- From: Zheng, Kai Sent: Tuesday, March 03, 2015 8:49 AM To: common-...@hadoop.apache.org

RE: Looking to a Hadoop 3 release

2015-03-02 Thread Zheng, Kai
JDK8 support is in the consideration, looks like many issues were reported and resolved already. https://issues.apache.org/jira/browse/HADOOP-11090 -Original Message- From: Andrew Wang [mailto:andrew.w...@cloudera.com] Sent: Tuesday, March 03, 2015 7:20 AM To: common-...@hadoop.apache.

RE: Anyone know how to mock a secured hdfs for unit test?

2014-06-30 Thread Zheng, Kai
erage with something that more closely approximates a secured deployment. Chris Nauroth Hortonworks http://hortonworks.com/ On Thu, Jun 26, 2014 at 7:27 AM, Zheng, Kai wrote: > Hi David, > > Quite some time ago I opened HADOOP-9952 and planned to create secured > MiniClusters by

RE: Anyone know how to mock a secured hdfs for unit test?

2014-06-26 Thread Zheng, Kai
Hi David, Quite some time ago I opened HADOOP-9952 and planned to create secured MiniClusters by making use of MiniKDC. Unfortunately since then I didn't get the chance to work on it yet. If you need something like that and would contribute, please let me know and see if anything I can help wit

RE: Replacing the JSP web UIs to HTML 5 applications

2013-10-29 Thread Zheng, Kai
> having /JMX for monitoring integration and a /JSON end point for the UI IMHO, this makes sense, especially for the long term. JMX interface serves as management console in admin perspective, WebUI serves as end user interface. Both might share same functionality codes, but that does not validat