[jira] [Created] (HDFS-7881) TestHftpFileSystem#testSeek fails in branch-2

2015-03-03 Thread Akira AJISAKA (JIRA)
Akira AJISAKA created HDFS-7881: --- Summary: TestHftpFileSystem#testSeek fails in branch-2 Key: HDFS-7881 URL: https://issues.apache.org/jira/browse/HDFS-7881 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-7880) Remove the tests for legacy Web UI in branch-2

2015-03-03 Thread Akira AJISAKA (JIRA)
Akira AJISAKA created HDFS-7880: --- Summary: Remove the tests for legacy Web UI in branch-2 Key: HDFS-7880 URL: https://issues.apache.org/jira/browse/HDFS-7880 Project: Hadoop HDFS Issue Type: Bu

[jira] [Resolved] (HDFS-7856) DFSOutputStream using outer independent class DataStreamer and Packet

2015-03-03 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng resolved HDFS-7856. - Resolution: Duplicate This isn't needed as the major work should be done while separating the inner classe

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Arun Murthy
Awesome, looks like we can just do this in a compatible manner - nothing else on the list seems like it warrants a (premature) major release. Thanks Vinod. Arun From: Vinod Kumar Vavilapalli Sent: Tuesday, March 03, 2015 2:30 PM To: common-...@hadoop.ap

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Vinod Kumar Vavilapalli
I started pitching in more on that JIRA. To add, I think we can and should strive for doing this in a compatible manner, whatever the approach. Marking and calling it incompatible before we see proposal/patch seems premature to me. Commented the same on JIRA: https://issues.apache.org/jira/bro

[jira] [Created] (HDFS-7879) hdfs.dll does not export functions of the public libhdfs API.

2015-03-03 Thread Chris Nauroth (JIRA)
Chris Nauroth created HDFS-7879: --- Summary: hdfs.dll does not export functions of the public libhdfs API. Key: HDFS-7879 URL: https://issues.apache.org/jira/browse/HDFS-7879 Project: Hadoop HDFS

[jira] [Created] (HDFS-7878) API - expose an unique file identifier

2015-03-03 Thread Sergey Shelukhin (JIRA)
Sergey Shelukhin created HDFS-7878: -- Summary: API - expose an unique file identifier Key: HDFS-7878 URL: https://issues.apache.org/jira/browse/HDFS-7878 Project: Hadoop HDFS Issue Type: Impr

Re: Dose hdfs support the configuration that different blocks can have different number of replcias?

2015-03-03 Thread Andrew Wang
Yup, definitely. Check out the -setrep command: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#setrep HTH, Andrew On Tue, Mar 3, 2015 at 11:49 AM, Lipeng Wan wrote: > Hi Andrew, > > Thanks for your reply! > Then is it possible for us to specify dif

Re: Dose hdfs support the configuration that different blocks can have different number of replcias?

2015-03-03 Thread Lipeng Wan
Hi Andrew, Thanks for your reply! Then is it possible for us to specify different replication factors for different files? Lipeng On Tue, Mar 3, 2015 at 2:38 PM, Andrew Wang wrote: > Hi Lipeng, > > Right now that is unsupported, replication is set on a per-file basis, not > per-block. > > Andre

Re: Dose hdfs support the configuration that different blocks can have different number of replcias?

2015-03-03 Thread Andrew Wang
Hi Lipeng, Right now that is unsupported, replication is set on a per-file basis, not per-block. Andrew On Tue, Mar 3, 2015 at 11:23 AM, Lipeng Wan wrote: > Hi devs, > > By default, hdfs creates same number of replicas for each block. Is it > possible for us to create more replicas for some of

Dose hdfs support the configuration that different blocks can have different number of replcias?

2015-03-03 Thread Lipeng Wan
Hi devs, By default, hdfs creates same number of replicas for each block. Is it possible for us to create more replicas for some of the blocks? Thanks! L. W.

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Allen Wittenauer
Between: * removing -finalize * breaking HDFS browsing * changing du’s output (in the 2.7 branch) * changing various names of metrics (either intentionally or otherwise) * changing the JDK release … and probably lots of other stuff in branch-2 I hav

Re: Questions about HDFS federation

2015-03-03 Thread Kihwal Lee
Federation is for sharing the data node storage spaces, not for sharing the data among multiple clusters. Check out viewfs instead. From: xeonmailinglist-yahoo To: hdfs-dev@hadoop.apache.org Sent: Tuesday, March 3, 2015 12:31 PM Subject: Re: Questions about HDFS federation I would

[jira] [Created] (HDFS-7877) Support maintenance state for datanodes

2015-03-03 Thread Ming Ma (JIRA)
Ming Ma created HDFS-7877: - Summary: Support maintenance state for datanodes Key: HDFS-7877 URL: https://issues.apache.org/jira/browse/HDFS-7877 Project: Hadoop HDFS Issue Type: New Feature

Re: Questions about HDFS federation

2015-03-03 Thread xeonmailinglist-yahoo
I would like to access the same file from hadoop-coc-1, and hadoop-coc-2 using HDFS federation. Does anyone knows if it is possible to share namespaces between 2 Namenodes in a HDFS federation, so that I can access the same file from different hosts? Thanks, On 03-03-2015 10:32, xeonmailing

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Andrew Wang
Hi Junping, thanks for your response, I view branch-3 as essentially the same size as our recent 2.x releases, with the exception of incompatible changes like classpath isolation and JDK8 target version. These, while perhaps not revolutionary, are still incompatible, and require a major version bu

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Andrew Wang
Hi Akira, thanks for responding, On Tue, Mar 3, 2015 at 4:04 AM, Akira AJISAKA wrote: > Thanks Andrew for bringing this up. > +1 mostly looks fine but I'm thinking it's not now to cut branch-3. > > > classpath isolation > > IMHO, classpath isolation is a good thing to do. > We should pay down th

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Andrew Wang
Hi Konst, thanks for taking a look. I think I essentially agree with your points. On Mon, Mar 2, 2015 at 11:04 PM, Konstantin Shvachko wrote: > Andrew, > > Hadoop 3 seems in general like a good idea to me. > 1. I did not understand if you propose to release 3.0 instead of 2.7 or in > addition? >

Re: 2.7 status

2015-03-03 Thread Sagar Thacker
Please unsubscribe me. On Tue, Mar 3, 2015 at 8:34 AM, Zheng, Kai wrote: > Is it interested to get the following issues in the release ? Thanks ! > > HADOOP-10670 > HADOOP-10671 > > Regards, > Kai > > -Original Message- > From: Yongjun Zhang [mailto:yzh...@cloudera.com] > Sent: Monday, M

Re: Questions about HDFS federation

2015-03-03 Thread Sagar Thacker
Please unsubscribe me.

Re: Questions about HDFS federation

2015-03-03 Thread Kihwal Lee
I suggest testing without federation first. That is, running two separate yarn/hdfs instances. Once that works properly, you can introduce federation in your config. Kihwal From: xeonmailinglist To: hdfs-dev@hadoop.apache.org Sent: Tuesday, March 3, 2015 4:32 AM Subject: Questions abo

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Junping Du
Thanks all for good discussions here. +1 on supporting Java 8 ASAP. In addition, I agree that we should separating this effort with cutting down Hadoop 3. IMO, Hadoop is still very cool today, and we should only consider Hadoop 3 until we have revolutionary feature (like YARN for 2.0) which dese

[jira] [Created] (HDFS-7876) DataNodes start to scan blocks earlier

2015-03-03 Thread Xinwei Qin (JIRA)
Xinwei Qin created HDFS-7876: - Summary: DataNodes start to scan blocks earlier Key: HDFS-7876 URL: https://issues.apache.org/jira/browse/HDFS-7876 Project: Hadoop HDFS Issue Type: Improvement

Re: The Activities of Apache Hadoop Community

2015-03-03 Thread Akira AJISAKA
Hi all, One year after the previous post, we collected and analyzed JIRA tickets again to investigate the activities of Apache Hadoop community in 2014. http://ajisakaa.blogspot.com/2015/02/the-activities-of-apache-hadoop.html As we expected in the previous post, the activities of Apache Hadoop

[jira] [Created] (HDFS-7875) Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated

2015-03-03 Thread nijel (JIRA)
nijel created HDFS-7875: --- Summary: Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated Key: HDFS-7875 URL: https://issues.apache.org/jira/browse/HDFS-7875 Project: Hadoop

Re: Looking to a Hadoop 3 release

2015-03-03 Thread Akira AJISAKA
Thanks Andrew for bringing this up. +1 mostly looks fine but I'm thinking it's not now to cut branch-3. > classpath isolation IMHO, classpath isolation is a good thing to do. We should pay down the technical dept ASAP. I'm willing to help. I'm thinking we can cut branch-3 and release 3.0 alpha

Questions about HDFS federation

2015-03-03 Thread xeonmailinglist
Hi, 1. The image below shows 2 YARN instances using the same federation of clouds. Is this possible? 2. I have configured in 2 hosts (hadooop-coc-1, and hadoop-coc-2) a federation of HDFS. In the configuration, I have set a namespace in each host, and a single data node (see image). The ser

[jira] [Created] (HDFS-7874) Even-though rollback image is created, after restarting namenode with "rollingUpgrade started" option , createdRollbackImages is set to false.

2015-03-03 Thread J.Andreina (JIRA)
J.Andreina created HDFS-7874: Summary: Even-though rollback image is created, after restarting namenode with "rollingUpgrade started" option , createdRollbackImages is set to false. Key: HDFS-7874 URL: https://issue

[jira] [Created] (HDFS-7873) OIV webhdfs premature close channel issue

2015-03-03 Thread Benoit Perroud (JIRA)
Benoit Perroud created HDFS-7873: Summary: OIV webhdfs premature close channel issue Key: HDFS-7873 URL: https://issues.apache.org/jira/browse/HDFS-7873 Project: Hadoop HDFS Issue Type: Bug