[jira] [Created] (HDFS-2086) If the include hosts list contains host name, after restarting namenode, datanodes registrant is denied

/browse/HDFS-2086 Project: Hadoop HDFS Issue Type: Bug Reporter: Tanping Wang As the tile describes the problem: if the include host list contains host name, after restarting namenodes, the datanodes registrant is denied by namenodes. This is because after

[jira] [Resolved] (HDFS-1874) Programmatically start hdfs processes with JMX port open

[ https://issues.apache.org/jira/browse/HDFS-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanping Wang resolved HDFS-1874. Resolution: Won't Fix > Programmatically start hdfs processes with JMX p

[jira] [Created] (HDFS-2083) Adop JMXJsonServlet into HDFS in order to query statistics

Affects Versions: 0.23.0 Reporter: Tanping Wang Assignee: Tanping Wang Fix For: 0.23.0 HADOOP-7144 added JMXJsonServlet into Common. It gives the capability to query statistics and metrics exposed via JMX to be queried through HTTP. We adopt this into HDFS

[jira] [Created] (HDFS-2056) Update fetchdt usage

Update fetchdt usage Key: HDFS-2056 URL: https://issues.apache.org/jira/browse/HDFS-2056 Project: Hadoop HDFS Issue Type: Improvement Reporter: Tanping Wang Assignee: Tanping Wang

RE: snapshot

Federation touched snapshot. Using the same logic of snapshot, Federation extended taking snapshot to block pool level. https://issues.apache.org/jira/secure/attachment/12471757/Federation-upgrade.htm Regards, Tanping -Original Message- From: doducthan...@gmail.com [mailto:doducthan...@g

[jira] [Created] (HDFS-1998) make refresh-namodenodes.sh refreshing all namenodes

Affects Versions: 0.23.0 Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: 0.23.0 refresh-namenodes.sh is used to refresh name nodes in the cluster to check for updates of include/exclude list. It is used when

[jira] [Created] (HDFS-1995) Minor modification to both dfsclusterhealth and dfshealth pages for Web UI

Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: 0.23.0 Four small modifications/fixes: on dfshealthpage: 1) fix remaining% to be remaining / total ( it was mistaken as used

[jira] [Created] (HDFS-1986) Add an option for user to return http or https ports regardless of security is on/off in DFSUtil.getInfoServer()

://issues.apache.org/jira/browse/HDFS-1986 Project: Hadoop HDFS Issue Type: Bug Components: tools Affects Versions: 0.23.0 Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: 0.23.0 Currently

[jira] [Created] (HDFS-1953) Change name node mxbean name in cluster web console

Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: 0.23.0 name node mxbean name is changed after the new metrics framework is checked. Need to change this in ClusterJspHelper.java in order for cluster web console to work again

[jira] [Created] (HDFS-1881) After taking snapshot (upgrade) on Federation, the current directory of data node is emtpy.

Project: Hadoop HDFS Issue Type: Bug Reporter: Tanping Wang Assignee: Tanping Wang After taking a snapshot in Federation (by starting up namenode with option -upgrade), it appears that the current directory of data node does not contain the block files. We

[jira] [Resolved] (HDFS-1867) Remove a datanode from cluster if include list is not empty and this datanode is removed from both include and exclude lists

[ https://issues.apache.org/jira/browse/HDFS-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanping Wang resolved HDFS-1867. Resolution: Duplicate Solve this jira on HDFS-1773. > Remove a datanode from cluster if incl

[jira] [Reopened] (HDFS-1773) Remove a datanode from cluster if include list is not empty and this datanode is removed from both include and exclude lists

[ https://issues.apache.org/jira/browse/HDFS-1773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanping Wang reopened HDFS-1773: Port this jira to trunk. Mark HDFS-1867 as a duplication of this jira. > Remove a datanode f

[jira] [Created] (HDFS-1874) Programmatically start hdfs processes with JMX port open

Components: data-node, name-node Affects Versions: 0.23.0 Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: 0.23.0 Federation web console makes JMX calls to each name node in order to collect statistics from the name nodes

[jira] [Created] (HDFS-1873) Federation Cluster Management Web Console

Reporter: Tanping Wang Assignee: Tanping Wang Fix For: 0.23.0 The Federation cluster management console provides # Cluster summary information that shows overall cluster utilization. A list of the name nodes that reports the used space, files and directories, blocks

[jira] [Created] (HDFS-1867) Remove a datanode from cluster if include list is not empty and this datanode is removed from both include and exclude lists

URL: https://issues.apache.org/jira/browse/HDFS-1867 Project: Hadoop HDFS Issue Type: Improvement Reporter: Tanping Wang Priority: Minor Fix For: 0.23.0 This jira is to port HDFS-1773 into trunk. HDF-1773 was originally fixed on

[jira] [Resolved] (HDFS-1695) HDFS federation: Fix testOIV and TestDatanodeUtils

[ https://issues.apache.org/jira/browse/HDFS-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanping Wang resolved HDFS-1695. Resolution: Fixed > HDFS federation: Fix testOIV and TestDatanodeUt

[jira] [Resolved] (HDFS-1386) unregister namenode datanode info MXBean

[ https://issues.apache.org/jira/browse/HDFS-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanping Wang resolved HDFS-1386. Resolution: Duplicate Release Note: (was: Making it a blocker for 0.22 as this causes tests

[jira] [Created] (HDFS-1773) Remove a datanode from cluster if include list is not empty and this datanode is removed from both include and exclude lists

URL: https://issues.apache.org/jira/browse/HDFS-1773 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.20.203.1 Environment: branch-20-security Reporter: Tanping Wang Assignee: Tanping Wang Fix For: 0.20.4

[jira] Created: (HDFS-1758) Web UI JSP pages thread safety issue

-security Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor The set of JSP pages that web UI uses are not thread safe. We have observed some problems when requesting Live/Dead/Decommissioning pages from the web UI, incorrect page is displayed. To

[jira] Created: (HDFS-1738) change hdfs jmxget to return an empty string instead of null when an attribute value is not available.

/browse/HDFS-1738 Project: Hadoop HDFS Issue Type: Bug Components: tools Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Currently the tool, hdfs jmx returns null in case of an attribute value is not available. A null

[jira] Created: (HDFS-1705) HDFS federation: Balancer command throws NullPointerException

Components: balancer Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch Run balancer results in "java.lang.NullPointerException" error. $ hdfs balancer 1

[jira] Created: (HDFS-1703) HDFS federation: Start namenodes and secondary namenodes on hosts returned by hdfs getconf

Project: Hadoop HDFS Issue Type: Sub-task Components: scripts Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch Startup scripts should start

[jira] Created: (HDFS-1702) HDFS federation : Fix TestBackupNode? and TestRefreshNamenodes? test failure.

Issue Type: Sub-task Components: test Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch TestBackupNode fails when check point signature does not match

[jira] Created: (HDFS-1701) HDFS federation :TestHeartbeatHandling fails: wrong number of items in cmds array

HDFS Issue Type: Sub-task Reporter: Tanping Wang Testcase: testHeartbeat took 5.787 sec FAILED expected:<2> but was:<1> junit.framework.AssertionFailedError: expected:<2> but

[jira] Created: (HDFS-1695) HDFS federation: Fix testOIV and TestDatanodeUtils

Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch # TestOfflineImageViewer fails due to the newest fsimage versions not being listed as supported. We're missing

[jira] Created: (HDFS-1678) HDFS federation: Remove getBlockpool() for NameNodeMXBean in FSNameSystem.

Issue Type: Sub-task Components: name-node, test Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch There was getBlockpoolId() which is used by NameNodeMXBean

[jira] Created: (HDFS-1677) HDF federation: Fix TestFsck and TestListCorruptFileBlocks failures

: Sub-task Components: test Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch These tests are failing due to null pointer exception. -- This message is

[jira] Created: (HDFS-1674) HDFS Federation: Rename BlockPool class to BlockPoolSlice

Components: data-node Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch HDFS Federation (porting): The class BlockPool is poorly named as it does not represent an actual

[jira] Created: (HDFS-1664) HDFS federation :Add new
 column
s of Block
Pool
Used 
and
 Block
Pool
Used
%
 into JSP

Project: Hadoop HDFS Issue Type: New Feature Components: tools Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch 


Add new
 column
s into web UI

[jira] Created: (HDFS-1663) HDFS federation: Rename getPoolId() everywhere to getBlockPoolId()

Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch Rename getPoolId() everywhere to getBlockPoolId() to unify the method name. -- This message is automatically

[jira] Created: (HDFS-1662) HDFS federation: fix unit test case, TestCheckpoint and TestDataNodeMXBean

Issue Type: Bug Components: test Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch HDFS federation: TestCheckpoint fails due to getting block pool id is empty

[jira] Created: (HDFS-1659) HDFS Federation: BPOfferService exits after the first iteration of the loop

Issue Type: Bug Components: data-node Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch HDFS federation: BPOfferService#offerService() sets the

[jira] Created: (HDFS-1653) Block received is sent with invalid DatanodeRegistration

Components: data-node Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Fix For: Federation Branch When sending block received bpRegistration should be used. Also when datanode shutsdown, if there are no namenode threads created

[jira] Created: (HDFS-1651) Tests fail due to null pointer exception in Datanode#shutdown()

Components: test Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch While stopping the RPC proxy, in tests where there are no BPOfferService threads created results

[jira] Created: (HDFS-1650) Fix unit test case: TestReplication

Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch TestReplication fails due to null pointer exception. When corrupting/deleting blocks in TestReplication#testPendingReplicationRetry() the loop should run

[jira] Created: (HDFS-1648) Federation: Properly release in_use.lock created in BlockPoolStorage

: Bug Components: data-node Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Fix For: Federation Branch StorageDirectory locks in_use.lock file, when analyzeStorage() method is called during initialization. Since

[jira] Created: (HDFS-1635) Pump up LAYOUT_VERSION in FSConstants.java for Federation before committing to trunk

: Hadoop HDFS Issue Type: New Feature Affects Versions: Federation Branch Reporter: Tanping Wang Fix For: Federation Branch We need to pump up LAYOUT_VERSION for Federation release from current. We will wait until before committing to trunk to avoid conflicting

[jira] Created: (HDFS-1632) HDFS Federation: data node storage structure changes and introduce block pool storage for Federation.

/browse/HDFS-1632 Project: Hadoop HDFS Issue Type: New Feature Components: data-node Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Fix For: Federation Branch In Federation, we introduced block

[jira] Created: (HDFS-1428) HDFS federation : add cluster ID and block pool ID into Name node web UI

Type: Sub-task Reporter: Tanping Wang Assignee: Tanping Wang -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Created: (HDFS-1422) hadoop-hdfs-trunk is broken due to HADOOP-6951 change to hadoop-common-trunk

Issue Type: Bug Components: name-node Reporter: Tanping Wang The recent check in to hadoop-common breaks hadoop-hdfs trunk. The change was made to src/java/org/apache/hadoop/ipc/Server.java src/java/org/apache/hadoop/ipc/Server.java The change makes refresh() method

[jira] Created: (HDFS-1419) Federation: Three test cases need minor modification after the new block id change

HDFS Issue Type: Bug Components: test Affects Versions: Federation Branch Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Fix For: Federation Branch Three test cases M src/test/hdfs/org/apache/hadoop

[jira] Created: (HDFS-1414) HDFS federation : fix unit test cases

HDFS federation : fix unit test cases - Key: HDFS-1414 URL: https://issues.apache.org/jira/browse/HDFS-1414 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Tanping Wang -- This

[jira] Created: (HDFS-1401) TestFileConcurrentReader test case is still timing out / failing

Components: hdfs client Reporter: Tanping Wang Priority: Minor -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Created: (HDFS-1398) HDFS federation: Upgrade and rolling back of Federation

Reporter: Tanping Wang -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Created: (HDFS-1397) HDFS federation: Storage directory of VERSION(/ID) file

Reporter: Tanping Wang -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Created: (HDFS-1386) unregister namenode datanode info MXBean

Reporter: Tanping Wang Assignee: Tanping Wang Fix For: 0.22.0 Currently Namenode and Datanode web page needs to be scraped by scripts to get this information. Having an interface where this structured information is provided, will help building scripts around it. -- This

[jira] Resolved: (HDFS-1385) A tool to discover library dependency in HDFS for Eclipse project

[ https://issues.apache.org/jira/browse/HDFS-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanping Wang resolved HDFS-1385. Resolution: Duplicate > A tool to discover library dependency in HDFS for Eclipse proj

[jira] Created: (HDFS-1385) A tool to discover library dependency in HDFS for Eclipse project

: Improvement Components: tools Reporter: Tanping Wang Assignee: Tanping Wang Priority: Minor Proposal: Currently when opening a new HDFS eclipse project in Eclipse, Eclipse pulls in the library dependency based on .eclipse.templates/.classpath file. This

[jira] Created: (HDFS-1365) HDFS federation: propose ClusterID and BlockPoolID format

Reporter: Tanping Wang Assignee: Tanping Wang -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.