[jira] Created: (HDFS-983) NameNode transaction log corruption with bad filename
NameNode transaction log corruption with bad filename - Key: HDFS-983 URL: https://issues.apache.org/jira/browse/HDFS-983 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.20.1 Reporter: dhruba borthakur The SecondaryNamenode is unable to create checkpoints. The stack trace is attached. This is the second time we have seen this issue. Both these occurances happened with unprintable characters in the filename. I am wondering if there is an String-UTF8 encoding problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-913) TestRename won't run automatically from 'run-test-hdfs-faul-inject' target
[
https://issues.apache.org/jira/browse/HDFS-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas resolved HDFS-913.
--
Resolution: Fixed
Release Note: HDFS-913. Rename fault injection test TestRename.java to
TestFiRename.java to include it in tests run by ant target
run-test-hdfs-fault-inject.
Hadoop Flags: [Reviewed]
I committed the change.
> TestRename won't run automatically from 'run-test-hdfs-faul-inject' target
> --
>
> Key: HDFS-913
> URL: https://issues.apache.org/jira/browse/HDFS-913
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Reporter: Konstantin Boudnik
>Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
>
> Fault injection test classes suppose to have {{TestFi}} prefix. Otherwise,
> JUnit target won't pick them up as a part of the batch test run. Although,
> it's still possible to run test with different names with {{-Dtestcase=}}
> directive.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-984) Delegation Tokens should be persisted in Namenode
Delegation Tokens should be persisted in Namenode - Key: HDFS-984 URL: https://issues.apache.org/jira/browse/HDFS-984 Project: Hadoop HDFS Issue Type: New Feature Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey The Delegation tokens should be persisted in the FsImage and EditLogs so that they are valid to be used after namenode shutdown and restart. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-985) HDFS should issue multiple RPCs for listing a large directory
HDFS should issue multiple RPCs for listing a large directory - Key: HDFS-985 URL: https://issues.apache.org/jira/browse/HDFS-985 Project: Hadoop HDFS Issue Type: New Feature Reporter: Hairong Kuang Assignee: Hairong Kuang Fix For: 0.22.0 Currently HDFS issues one RPC from the client to the NameNode for listing a directory. However some directories are large that contain thousands or millions of items. Listing such large directories in one RPC has a few shortcomings: 1. The list operation holds the global fsnamesystem lock for a long time thus blocking other requests. If a large number (like thousands) of such list requests hit NameNode in a short period of time, NameNode will be significantly slowed down. Users end up noticing longer response time or lost connections to NameNode. 2. The response message is uncontrollable big. We observed a response as big as 50M bytes when listing a directory of 300 thousand items. Even with the optimization introduced at HDFS-946 that may be able to cut the response by 20-50%, the response size will still in the magnitude of 10 mega bytes. I propose to implement a directory listing using multiple RPCs. Here is the plan: 1. Each getListing RPC has an upper limit on the number of items returned. This limit could be configurable, but I am thinking to set it to be a fixed number like 500. 2. Each RPC additionally specifies a start position for this listing request. I am thinking to use the last item of the previous listing RPC as an indicator. Since NameNode stores all items in a directory as a sorted array, NameNode uses the last item to locate the start item of this listing even if the last item is deleted in between these two consecutive calls. This has the advantage of avoid duplicate entries at the client side. 3. The return value additionally specifies if the whole directory is done listing. If the client sees a false flag, it will continue to issue another RPC. This proposal will change the semantics of large directory listing in a sense that listing is no longer an atomic operation if a directory's content is changing while the listing operation is in progress. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
