Re: 0.23 & trunk tars, we'll we publishing 1 tar per component or a single tar? What about source tar?

Other approach would be asking which tar to build. mapred-tar (mapred and common) hdfs-tar (hdfs and common) hadoop-tar (all) In this case, hbase can just use hdfs-tar. -Bharath From: Ravi Teja To: mapreduce-...@hadoop.apache.org; common-...@hadoop.apache.org

[jira] [Created] (HDFS-2109) Store uMask as member variable to DFSClient.Conf

Affects Versions: 0.23.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.23.0 As a part of removing reference to conf in DFSClient, I am proposing replacing FsPermission.getUMask(conf) everywhere in DFSClient class with

[jira] [Created] (HDFS-2105) Remove the references to configuration object from the DFSClient library.

Issue Type: Bug Affects Versions: 0.23.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.23.0 This is an umbrella jira to track removing the references to conf object in DFSClient library. -- This message is automatically generated by

[jira] [Resolved] (HDFS-2103) Read lock must be released before acquiring a write lock

[ https://issues.apache.org/jira/browse/HDFS-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Mundlapudi resolved HDFS-2103. -- Resolution: Not A Problem Didn't notice the finally block, where read lock is rel

[jira] [Created] (HDFS-2103) Read lock must be released before acquiring a write lock

: 0.23.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.23.0 In FSNamesystem.getBlockLocationsUpdateTimes function, we have the following code: {code} for (int attempt = 0; attempt < 2; attempt++) { if (attempt == 0) { // fi

[jira] [Created] (HDFS-2094) Add metrics for write pipeline failures

: Bharath Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.23.0 Write pipeline can fail for various reasons like rpc connection issues, disk problem etc. I am proposing to add metrics to detect write pipeline issues. -- This message is automatically generated by JIRA. For

[jira] [Created] (HDFS-2092) Remove configuration object reference in DFSClient

Affects Versions: 0.23.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.23.0 At present, DFSClient stores reference to configuration object. Since, these configuration objects are pretty big at times can blot the processes which has

[jira] [Resolved] (HDFS-2072) Remove StringUtils.stringifyException(ie) in logger functions

[ https://issues.apache.org/jira/browse/HDFS-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Mundlapudi resolved HDFS-2072. -- Resolution: Duplicate > Remove StringUtils.stringifyException(ie) in logger functi

[jira] [Created] (HDFS-2072) Remove StringUtils.stringifyException(ie) in logger functions

Versions: 0.23.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.23.0 Apache logger api has an overloaded function which can take the message and exception. I am proposing to clean the logging code with this api. ie.: Change the code

[jira] [Created] (HDFS-2065) Fix NPE in DFSClient.getFileChecksum

Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.23.0 The following code can throw NPE if callGetBlockLocations returns null. If server returns null {code} List locatedblocks = callGetBlockLocations(namenode, src, 0, Long.MAX_VALUE).getLocatedBlocks

[jira] [Created] (HDFS-2057) Wait time to terminate the threads causing unit tests to take longer time

Issue Type: Bug Components: data-node Affects Versions: 0.20.204.0, 0.20.205.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.20.205.0 As a part of datanode process hang, this part of code was introduced in 0.20.204 to clean up

[jira] [Created] (HDFS-2030) Fix the usability of namenode upgrade command

Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Priority: Minor Fix For: 0.23.0 Fixing the Namenode upgrade option along the same line as Namenode format option. If clusterid is not given then clusterid will be automatically generated for the

[jira] [Created] (HDFS-2023) Backport of NPE for File.list and File.listFiles

Affects Versions: 0.20.205.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.20.205.0 Since we have multiple Jira's in trunk for common and hdfs, I am creating another jira for this issue. This patch addresses the following: 1. Pro

[jira] [Created] (HDFS-2019) Fix all the places where Java method File.list is used with FileUtil.list API

Issue Type: Bug Components: data-node Affects Versions: 0.23.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Priority: Minor Fix For: 0.23.0 This new method FileUtil.list will throw an exception when disk is bad rather

[jira] [Created] (HDFS-1942) If all Block Pool service threads exit then datanode should exit.

Components: data-node Affects Versions: 0.23.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Currently, if all block pool service threads exit, Datanode continue to run. This should be fixed. -- This message is automatically generated by JIRA. For

[jira] [Created] (HDFS-1941) Remove -genclusterid from NameNode startup options

Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Priority: Minor Currently, namenode -genclusterid is a helper utility to generate unique clusterid. This option is useless once namenode -format automatically generates the clusterid. -- This message is

[jira] [Created] (HDFS-1934) Fix NullPointerException when certain File APIs return null

Versions: 0.20.205.0, 0.23.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.20.205.0, 0.23.0 While testing Disk Fail Inplace, We encountered the NPE from this part of the code. File[] files = dir.listFiles(); for (File f : files

Re: Question about hadoop namenode -format -clusterid

Correct way to format a namenode : /bin/hdfs namenode -format -clusterid PS: Set your environment right like common home etc. Only first time it requires the cluster id, second time onwards it will remember cluster id and prompt you to format this particular cluster id. I have filed a Jira

[jira] [Created] (HDFS-1905) Improve the usability of namenode -format

Versions: 0.23.0 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Priority: Minor Fix For: 0.23.0 While setting up 0.23 version based cluster, i ran into this issue. When i issue a format namenode command, which got changed in 23, it should let

Re: run pig on hadoop

Hello Hiba, The file fileHDFS will be stored under user home directory. Lets say you run this command as 'hdfsclient' user. Then it should be located at /user/hdfsclient/fileHDFS. You could also specify the actual path where you want to store the file. Like, $PATH/bin/hadoop dfs -copyFromLocal

[jira] [Created] (HDFS-1843) Discover file not found early for file append

Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Priority: Minor Fix For: 0.23.0 For the append call, discover file not found exception early and avoid extra server call. -- This message is automatically generated by JIRA. For more

[jira] Created: (HDFS-1692) In secure mode, Datanode process doesn't exit when disks fail.

: Bug Components: data-node Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi In secure mode, when disks fail more than volumes tolerated, datanode process doesn't exit properly and it just hangs even though shutdown method is called. -- This

[jira] Created: (HDFS-1592) Datanode startup doesn't honor volumes.tolerated

.20.1 Reporter: Bharath Mundlapudi Assignee: Bharath Mundlapudi Fix For: 0.20.4 Datanode startup doesn't honor volumes.tolerated for hadoop 20 version. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Created: (HDFS-1061) Memory footprint optimization for INodeFile object.

: name-node Affects Versions: 0.22.0 Reporter: Bharath Mundlapudi Priority: Minor Fix For: 0.22.0 I am proposing a footprint optimization to merge blockReplication and preferredBlockSize fields into one 'long header' field in INodeFile class. Thi