Re: Feature request to provide DFSInputStream subclassing mechanism

Hi Steve, Thank you very much for the reality check! Some more answers inline ... On 8/8/13 1:30 PM, Steve Loughran wrote: On 7 August 2013 10:59, Jeff Dost wrote: Hello, We work in a software development team at the UCSD CMS Tier2 Center. We would like to propose a mechanism to allow one

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

On Thu, Aug 8, 2013 at 3:34 PM, Roman Shaposhnik wrote: > I'd be huge +1 on a flag day migrating us to 2.5.x. > > We can run a set of validation tests in Bigtop on all components > that could be affected by it. Hadoop and HBase are the obvious > suspects. What else in the ecosystem could be affec

[jira] [Created] (HDFS-5081) DistributedFileSystem#listStatus() throws FileNotFoundException when target of symlink doesn't exist

Ted Yu created HDFS-5081: Summary: DistributedFileSystem#listStatus() throws FileNotFoundException when target of symlink doesn't exist Key: HDFS-5081 URL: https://issues.apache.org/jira/browse/HDFS-5081 Proj

[jira] [Created] (HDFS-5080) BootstrapStandby not working with QJM when the existing NN is active

Jing Zhao created HDFS-5080: --- Summary: BootstrapStandby not working with QJM when the existing NN is active Key: HDFS-5080 URL: https://issues.apache.org/jira/browse/HDFS-5080 Project: Hadoop HDFS

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

To move forward with this we need protoc 2.5.0 in the apache hadoop jenkins boxes. Who can help with this? I assume somebody at Y!, right? Thx On Thu, Aug 8, 2013 at 2:24 PM, Elliott Clark wrote: > In HBase land we've pretty well discovered that we'll need to have the > same version of protob

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

In HBase land we've pretty well discovered that we'll need to have the same version of protobuf that the HDFS/Yarn/MR servers are running. That is to say there are issues with ever having 2.4.x and 2.5.x on the same class path. Upgrading to 2.5.x would be great, as it brings some new classes we co

Re: Feature request to provide DFSInputStream subclassing mechanism

This is being targeted for release 2.3. 2.1.x release stream is for stabilizing. When it reaches stability, 2.2 GA will be released. The current features in development will make it to 2.3, including HDFS-2832. On Thu, Aug 8, 2013 at 2:04 PM, Matevz Tadel wrote: > Thanks Colin, I subscribed to

Re: Feature request to provide DFSInputStream subclassing mechanism

Thanks Colin, I subscribed to HDFS-2832 so that I can follow the development there. I assume this is targeting release 2.1. Best, Matevz On 08/08/13 12:10, Colin McCabe wrote: There is work underway to decouple the block layer and the namespace layer of HDFS from each other. Once this is don

Re: Feature request to provide DFSInputStream subclassing mechanism

On 7 August 2013 10:59, Jeff Dost wrote: > Hello, > > We work in a software development team at the UCSD CMS Tier2 Center. We > would like to propose a mechanism to allow one to subclass the > DFSInputStream in a clean way from an external package. First I'd like to > give some motivation on wh

Re: Feature request to provide DFSInputStream subclassing mechanism

There is work underway to decouple the block layer and the namespace layer of HDFS from each other. Once this is done, block behaviors like the one you describe will be easy to implement. It's a use case very similar to the hierarchical storage management (HSM) use case that we've discussed befor

Re: Feature request to provide DFSInputStream subclassing mechanism

Hi everybody, I'm jumping in as Jeff is away due to an unexpected annoyance involving Californian wildlife. On 8/7/13 7:47 PM, Andrew Wang wrote: Blocks are supposed to be an internal abstraction within HDFS, and aren't an inherent part of FileSystem (the user-visible class used to access all

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

Sorry to hijack the thread but, I also wanted to mention Avro. See HADOOP-9672. The version we are using has memory leak and inefficiency issues. We've seen users running into it. Kihwal From: Tsuyoshi OZAWA To: "common-...@hadoop.apache.org" Cc: "hdfs-dev@h

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

Another blocker, HADOOP-9850, has been committed. Kihwal From: Arun C Murthy To: Daryn Sharp Cc: "" ; "mapreduce-...@hadoop.apache.org" ; "yarn-...@hadoop.apache.org" ; "common-...@hadoop.apache.org" Sent: Thursday, August 1, 2013 1:30 PM Subject: Re: [

Jenkins build is back to normal : Hadoop-Hdfs-trunk #1485

See

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

Hi, About Hadoop, Harsh is dealing with this problem in HADOOP-9346. For more detail, please see the JIRA ticket: https://issues.apache.org/jira/browse/HADOOP-9346 - Tsuyoshi On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur wrote: > I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release.