[jira] [Resolved] (HDFS-8275) Erasure Coding: Implement batched listing of enrasure coding zones

[ https://issues.apache.org/jira/browse/HDFS-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-8275. -- Resolution: Duplicate Hi Rakesh, this JIRA is duplicated with HDFS-8087. > Erasure Coding: Implement batched lis

[jira] [Created] (HDFS-8284) Add usage of tracing originated in DFSClient to doc

Masatake Iwasaki created HDFS-8284: -- Summary: Add usage of tracing originated in DFSClient to doc Key: HDFS-8284 URL: https://issues.apache.org/jira/browse/HDFS-8284 Project: Hadoop HDFS Iss

[jira] [Created] (HDFS-8283) DataStreamer cleanup and some minor improvement

Tsz Wo Nicholas Sze created HDFS-8283: - Summary: DataStreamer cleanup and some minor improvement Key: HDFS-8283 URL: https://issues.apache.org/jira/browse/HDFS-8283 Project: Hadoop HDFS I

[jira] [Created] (HDFS-8282) Erasure coding: move striped reading logic to StripedBlockUtil

Zhe Zhang created HDFS-8282: --- Summary: Erasure coding: move striped reading logic to StripedBlockUtil Key: HDFS-8282 URL: https://issues.apache.org/jira/browse/HDFS-8282 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-8281) Erasure Coding: implement parallel stateful reading for striped layout

Jing Zhao created HDFS-8281: --- Summary: Erasure Coding: implement parallel stateful reading for striped layout Key: HDFS-8281 URL: https://issues.apache.org/jira/browse/HDFS-8281 Project: Hadoop HDFS

[jira] [Created] (HDFS-8280) Code Cleanup in DFSInputStream

Jing Zhao created HDFS-8280: --- Summary: Code Cleanup in DFSInputStream Key: HDFS-8280 URL: https://issues.apache.org/jira/browse/HDFS-8280 Project: Hadoop HDFS Issue Type: Improvement Re

[jira] [Resolved] (HDFS-8279) Document how to specify storage types

[ https://issues.apache.org/jira/browse/HDFS-8279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-8279. - Resolution: Duplicate Thanks for the link to HDFS-7770 [~xyao]. I'll help you get that reviewed and

[jira] [Created] (HDFS-8279) Document how to specify storage types

Lars Francke created HDFS-8279: -- Summary: Document how to specify storage types Key: HDFS-8279 URL: https://issues.apache.org/jira/browse/HDFS-8279 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Resolved] (HDFS-8235) Erasure Coding: Create DFSStripedInputStream in DFSClient#open

[ https://issues.apache.org/jira/browse/HDFS-8235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8235. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed The .3 patch looks good to me

[jira] [Created] (HDFS-8278) HDFS Balancer should consider remaining storage % when checking for under-utilized machines

Gopal V created HDFS-8278: - Summary: HDFS Balancer should consider remaining storage % when checking for under-utilized machines Key: HDFS-8278 URL: https://issues.apache.org/jira/browse/HDFS-8278 Project: Ha

Re: fsck output compatibility question with regard to HDFS-7281

A lot of the summary information… but the key parts of “yo, these files are busted and here’s why” is not, IIRC. That’s one of the key items where people are parsing fsck output (and worse, usually under duress.) On Apr 28, 2015, at 12:23 PM, Mai Haohui wrote: > In terms of the monitoring, w

Re: fsck output compatibility question with regard to HDFS-7281

In terms of the monitoring, we have put a lot of information into the JMX output. It's relatively easy to use python / ruby / node.js to write your own tools to parse the information. In the longer term, it might also make sense to move some of our tools to based on the JMX output instead of makin

Re: fsck output compatibility question with regard to HDFS-7281

On Tue, Apr 28, 2015 at 11:25 AM, Allen Wittenauer wrote: > > On Apr 28, 2015, at 10:59 AM, Andrew Wang > wrote: > > > > This is also not something typically upheld by unix-y commands. BSD vs. > GNU > > already leads to incompatible flags and output. Most of these commands > > haven't been chang

Re: fsck output compatibility question with regard to HDFS-7281

Hi Rich, You're essentially proposing feature flags. We've discussed this before wrt namenode metadata, and the complexity from supporting the combinations of all the flags is substantial. It's even harder in the domain of shell output, since we don't even have a standard way of parsing. Also ima

Re: fsck output compatibility question with regard to HDFS-7281

On Apr 28, 2015, at 10:59 AM, Andrew Wang wrote: > > This is also not something typically upheld by unix-y commands. BSD vs. GNU > already leads to incompatible flags and output. Most of these commands > haven't been changed in 20 years, but that doesn't constitute a compat > guarantee.

Re: fsck output compatibility question with regard to HDFS-7281

I'm late to the discussion so I apologize if this has already been suggested. Can't we just add new options flags to include new cli output? Seems like that would work regardless of the cli being changed. If compatibility is broken it could be done as part of a major release. Eg. Add a

Re: fsck output compatibility question with regard to HDFS-7281

I'm surprised by this compatibility requirement. It's quite onerous, since it means we can't evolve the output at all. There's no standardized way to parse CLI output, so who knows what might break user scripts. e.g. if we wanted to display a "+" for ACLs in ls output, that'd be incompatible. Same

[jira] [Created] (HDFS-8277) Safemode enter fails when Standby NameNode is down

Hari Sekhon created HDFS-8277: - Summary: Safemode enter fails when Standby NameNode is down Key: HDFS-8277 URL: https://issues.apache.org/jira/browse/HDFS-8277 Project: Hadoop HDFS Issue Type: Bu

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #168

See Changes: [ozawa] MAPREDUCE-6341. Fix typo in mapreduce tutorial. Contributed by John Michael Luy. [xgong] YARN-3530. ATS throws exception on trying to filter results without [arp] HDFS-8205. CommandFormat#parse() should no

Hadoop-Hdfs-trunk-Java8 - Build # 168 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/168/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 5185 lines...] [INFO] [INFO] --- maven-enforcer-plu

Hadoop-Hdfs-trunk - Build # 2109 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2109/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 5186 lines...] [INFO] [INFO] --- maven-enforcer-plugin:1

Build failed in Jenkins: Hadoop-Hdfs-trunk #2109

See Changes: [ozawa] MAPREDUCE-6341. Fix typo in mapreduce tutorial. Contributed by John Michael Luy. [xgong] YARN-3530. ATS throws exception on trying to filter results without [arp] HDFS-8205. CommandFormat#parse() should not par

[jira] [Created] (HDFS-8276) LazyPersistFileScrubber should be disabled if scrubber interval configured zero

surendra singh lilhore created HDFS-8276: Summary: LazyPersistFileScrubber should be disabled if scrubber interval configured zero Key: HDFS-8276 URL: https://issues.apache.org/jira/browse/HDFS-8276

[jira] [Resolved] (HDFS-8189) ClientProtocol#createErasureCodingZone API was wrongly annotated as Idempotent

[ https://issues.apache.org/jira/browse/HDFS-8189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B resolved HDFS-8189. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Committed to HDFS-728

[jira] [Resolved] (HDFS-8230) Erasure Coding: Ignore DatanodeProtocol#DNA_ERASURE_CODING_RECOVERY commands from standbynode if any

[ https://issues.apache.org/jira/browse/HDFS-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B resolved HDFS-8230. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Committed to HDFS-728