Re: Feature request to provide DFSInputStream subclassing mechanism

Blocks are supposed to be an internal abstraction within HDFS, and aren't an inherent part of FileSystem (the user-visible class used to access all Hadoop filesystems). Is it possible to instead deal with files and offsets? On a read failure, you could open a stream to the same file on the backup

[jira] [Created] (HDFS-5079) Cleaning up NNHAStatusHeartbeat.State DatanodeProtocolProtos.

Konstantin Shvachko created HDFS-5079: - Summary: Cleaning up NNHAStatusHeartbeat.State DatanodeProtocolProtos. Key: HDFS-5079 URL: https://issues.apache.org/jira/browse/HDFS-5079 Project: Hadoop H

[jira] [Created] (HDFS-5078) Support file append in NFSv3 gateway to enable data streaming to HDFS

Brandon Li created HDFS-5078: Summary: Support file append in NFSv3 gateway to enable data streaming to HDFS Key: HDFS-5078 URL: https://issues.apache.org/jira/browse/HDFS-5078 Project: Hadoop HDFS

Re: Feature request to provide DFSInputStream subclassing mechanism

Thank you for the suggestion, but we don't see how simply wrapping a FileSystem object would be sufficient in our use case. The reason why is we need to catch and handle read exceptions at the block level. There aren't any public methods available in the high level FileSystem abstraction laye

[jira] [Created] (HDFS-5077) NPE in FSNamesystem.commitBlockSynchronization()

Konstantin Shvachko created HDFS-5077: - Summary: NPE in FSNamesystem.commitBlockSynchronization() Key: HDFS-5077 URL: https://issues.apache.org/jira/browse/HDFS-5077 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-4926) namenode webserver's page has a tooltip that is inconsistent with the datanode HTML link

[ https://issues.apache.org/jira/browse/HDFS-4926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-4926. - Resolution: Fixed Fix Version/s: 2.1.1-beta Hadoop Flags: Reviewed I've committed this to trun

[jira] [Created] (HDFS-5076) Create http servlets to enable querying NN's last applied transaction ID and most recent checkpoint's transaction ID

Jing Zhao created HDFS-5076: --- Summary: Create http servlets to enable querying NN's last applied transaction ID and most recent checkpoint's transaction ID Key: HDFS-5076 URL: https://issues.apache.org/jira/browse/HDFS-

[jira] [Created] (HDFS-5075) httpfs-config.sh calls out incorrect env script name

Timothy St. Clair created HDFS-5075: --- Summary: httpfs-config.sh calls out incorrect env script name Key: HDFS-5075 URL: https://issues.apache.org/jira/browse/HDFS-5075 Project: Hadoop HDFS

Re: Feature request to provide DFSInputStream subclassing mechanism

I don't think exposing DFSClient and DistributedFileSystem members is necessary to achieve what you're trying to do. We've got wrapper FileSystems like FilterFileSystem and ViewFileSystem which you might be able to use for inspiration, and the HCFS wiki lists some third-party FileSystems that might

Re: Feature request to provide DFSInputStream subclassing mechanism

Hi Jeff, Do you need to subclass or could you simply wrap? Generally composition as opposed to inheritance is a lot safer way of integrating software written by different parties, since inheritance exposes all the implementation details which are subject to change. -Todd On Wed, Aug 7, 2013 at 1

Re: Feature request to provide DFSInputStream subclassing mechanism

Hello Jeff Is it something that could go under HCFS project? http://wiki.apache.org/hadoop/HCFS (I might be wrong?) Joe On 8/7/13 10:59 AM, "Jeff Dost" wrote: >Hello, > >We work in a software development team at the UCSD CMS Tier2 Center. We >would like to propose a mechanism to allow one to

Feature request to provide DFSInputStream subclassing mechanism

Hello, We work in a software development team at the UCSD CMS Tier2 Center. We would like to propose a mechanism to allow one to subclass the DFSInputStream in a clean way from an external package. First I'd like to give some motivation on why and then will proceed with the details. We hav

Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release. As mentioned in HADOOP-9845, Protobuf 2.5 has significant benefits to justify the upgrade. Doing the upgrade now, with the first beta, will make things easier for downstream projects (like HBase) using protobuf and adopting Hadoop 2. If

Build failed in Jenkins: Hadoop-Hdfs-trunk #1484

See Changes: [cnauroth] HADOOP-9527. Add symlink support to LocalFileSystem on Windows. Contributed by Arpit Agarwal. [jeagles] YARN-985. Nodemanager should log where a resource was localized (Ravi Prakash via jeagles) [jing9] HAD

Hadoop-Hdfs-trunk - Build # 1484 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1484/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 11245 lines...] [INFO] --- maven-clean-plugin:2.4.1:clean