[jira] [Created] (HADOOP-10643) Add NativeS3Fs that delgates calls from FileContext apis to native s3 fs implementation

2014-05-29 Thread Sumit Kumar (JIRA)
Sumit Kumar created HADOOP-10643: Summary: Add NativeS3Fs that delgates calls from FileContext apis to native s3 fs implementation Key: HADOOP-10643 URL: https://issues.apache.org/jira/browse/HADOOP-10643

[jira] [Created] (HADOOP-10642) Provide option to limit heap memory consumed by dynamic metrics2 metrics

2014-05-29 Thread Ted Yu (JIRA)
Ted Yu created HADOOP-10642: --- Summary: Provide option to limit heap memory consumed by dynamic metrics2 metrics Key: HADOOP-10642 URL: https://issues.apache.org/jira/browse/HADOOP-10642 Project: Hadoop Comm

[jira] [Resolved] (HADOOP-10628) Javadoc and few code style improvement for Crypto input and output streams

2014-05-29 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-10628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb resolved HADOOP-10628. --- Resolution: Fixed Thanks Yi, I committed this to fs-encryption. > Javadoc and few code st

Introducing ConsensusNode and a Coordination Engine

2014-05-29 Thread Konstantin Shvachko
Hello hadoop developers, I just opened two jiras proposing to introduce ConsensusNode into HDFS and a Coordination Engine into Hadoop Common. The latter should benefit HDFS and HBase as well as potentially other projects. See HDFS-6469 and HADOOP-10641 for details. The effort is based on the syst

[jira] [Created] (HADOOP-10641) Introduce Coordination Engine

2014-05-29 Thread Konstantin Shvachko (JIRA)
Konstantin Shvachko created HADOOP-10641: Summary: Introduce Coordination Engine Key: HADOOP-10641 URL: https://issues.apache.org/jira/browse/HADOOP-10641 Project: Hadoop Common Issue

[jira] [Created] (HADOOP-10640) Implement Namenode RPCs in HDFS native client

2014-05-29 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HADOOP-10640: - Summary: Implement Namenode RPCs in HDFS native client Key: HADOOP-10640 URL: https://issues.apache.org/jira/browse/HADOOP-10640 Project: Hadoop Comm

[jira] [Created] (HADOOP-10639) FileBasedKeyStoresFactory initialization is not using default for SSL_REQUIRE_CLIENT_CERT_KEY

2014-05-29 Thread Alejandro Abdelnur (JIRA)
Alejandro Abdelnur created HADOOP-10639: --- Summary: FileBasedKeyStoresFactory initialization is not using default for SSL_REQUIRE_CLIENT_CERT_KEY Key: HADOOP-10639 URL: https://issues.apache.org/jira/browse/H

[jira] [Reopened] (HADOOP-10589) NativeS3FileSystem throw NullPointerException when the file is empty

2014-05-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-10589: - Assignee: Steve Loughran not a duplicate. Same stack trace, but root cause is differe

[jira] [Resolved] (HADOOP-10589) NativeS3FileSystem throw NullPointerException when the file is empty

2014-05-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-10589. - Resolution: Duplicate Release Note: Duplicate of HADOOP-10533, though the stack tr

Re: Change proposal for FileInputFormat isSplitable

2014-05-29 Thread Niels Basjes
I forgot to ask a relevant question: What made the original proposed solution "incompatible"? To me it still seems to be a clean backward compatible solution that fixes this issue in a simple way. Perhaps Todd can explain why? Niels On May 29, 2014 2:17 PM, "Niels Basjes" wrote: > This is exact

Re: Change proposal for FileInputFormat isSplitable

2014-05-29 Thread Niels Basjes
This is exactly why I'm proposing a change that will either 'fix silently' (my original patch from 3 years ago) or 'break loudly' (my current proposal) old implementations. I'm convinced that ther are atleast 100 companies world wide that have a custom implementation with this bug and have no clue

Re: Change proposal for FileInputFormat isSplitable

2014-05-29 Thread Jay Vyas
I think breaking backwards compat is sensible since It's easily caught by the compiler and in this case where the alternative is a Runtime error that can result in terabytes of mucked up output. > On May 29, 2014, at 6:11 AM, Matt Fellows > wrote: > > As someone who doesn't really contribute

Re: Change proposal for FileInputFormat isSplitable

2014-05-29 Thread Matt Fellows
As someone who doesn't really contribute, just lurks, I could well be misinformed or under-informed, but I don't see why we can't deprecate a method which could cause dangerous side effects? People can still use the deprecated methods for backwards compatibility, but are discouraged by compiler war

Re: Change proposal for FileInputFormat isSplitable

2014-05-29 Thread Niels Basjes
My original proposal (from about 3 years ago) was to change the isSplitable method to return a safe default ( you can see that in the patch that is still attached to that Jira issue). For arguments I still do not fully understand this was rejected by Todd and Doug. So that is why my new proposal i

Re: Change proposal for FileInputFormat isSplitable

2014-05-29 Thread Matt Fellows
I could be missing something, but couldn't you just deprecate isSplitable (spelled incorrectly) and create a new isSplittable as described? On Thu, May 29, 2014 at 10:34 AM, Steve Loughran wrote: > On 28 May 2014 20:50, Niels Basjes wrote: > > > Hi, > > > > Last week I ran into this problem ag

Re: Change proposal for FileInputFormat isSplitable

2014-05-29 Thread Steve Loughran
On 28 May 2014 20:50, Niels Basjes wrote: > Hi, > > Last week I ran into this problem again > https://issues.apache.org/jira/browse/MAPREDUCE-2094 > > What happens here is that the default implementation of the isSplitable > method in FileInputFormat is so unsafe that just about everyone who > im

[jira] [Created] (HADOOP-10637) Add snapshot and several dfsadmin tests into TestCLI

2014-05-29 Thread Dasha Boudnik (JIRA)
Dasha Boudnik created HADOOP-10637: -- Summary: Add snapshot and several dfsadmin tests into TestCLI Key: HADOOP-10637 URL: https://issues.apache.org/jira/browse/HADOOP-10637 Project: Hadoop Common

[jira] [Resolved] (HADOOP-9987) HDFS Compatible ViewFileSystem

2014-05-29 Thread Gera Shegalov (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-9987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov resolved HADOOP-9987. --- Resolution: Duplicate > HDFS Compatible ViewFileSystem > -- > >