Re: [ANNOUNCE] Apache Hadoop 3.3.1 release

2021-06-15 Thread David
Thank you Wei-Chiu! Hopefully we can now move Hive forward! On Tue, Jun 15, 2021, 10:30 PM Wei-Chiu Chuang wrote: > Hi All, > > It gives me great pleasure to announce that the Apache Hadoop community has > voted to release Apache Hadoop 3.3.1. > > This is the first stable release of Apache Hado

[jira] [Created] (HDFS-16063) Add toString to EditLogFileInputStream

2021-06-10 Thread David Mollitor (Jira)
David Mollitor created HDFS-16063: - Summary: Add toString to EditLogFileInputStream Key: HDFS-16063 URL: https://issues.apache.org/jira/browse/HDFS-16063 Project: Hadoop HDFS Issue Type

Re: [DISCUSS] Hadoop 3.3.1 release

2021-03-10 Thread David
Hello, Still hoping for help here: https://issues.apache.org/jira/browse/HDFS-15790 Looks like it has been worked on, not sure how to best move it forward. On Wed, Mar 10, 2021, 12:21 PM Steve Loughran wrote: > I'm going to argue its too late to do IPv6 support close to a release, as > it's b

Re: [DISCUSS] Hadoop 3.3.1 release

2021-01-27 Thread David
Would be great for the Hive team to see this one as well: https://issues.apache.org/jira/browse/HDFS-15790 On Wed, Jan 27, 2021 at 8:28 PM Wei-Chiu Chuang wrote: > Hi all, > > Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated > more than 400 changes in the branch-3.3. A

[jira] [Resolved] (HDFS-15773) NameNode allocate five DataNodes for a small EC file (606 bytes) in log file

2021-01-20 Thread David (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David resolved HDFS-15773. -- Resolution: Information Provided > NameNode allocate five DataNodes for a small EC file (606 bytes) in

[jira] [Created] (HDFS-15773) NameNode allocate five DataNodes for a small file (606 bytes) in log file

2021-01-12 Thread David (Jira)
David created HDFS-15773: Summary: NameNode allocate five DataNodes for a small file (606 bytes) in log file Key: HDFS-15773 URL: https://issues.apache.org/jira/browse/HDFS-15773 Project: Hadoop HDFS

MDFS

2020-10-16 Thread David
Hello HDFS Gang, I've been playing around with a POC for a new distributed file system based on a lot of lessons learned from various Apache projects. You, also being a data nerd, I just wanted to share some thoughts with you since you may find it interesting. https://github.com/belugabehr/mdfs/

[jira] [Created] (HDFS-15393) Review of PendingReconstructionBlocks

2020-06-05 Thread David Mollitor (Jira)
David Mollitor created HDFS-15393: - Summary: Review of PendingReconstructionBlocks Key: HDFS-15393 URL: https://issues.apache.org/jira/browse/HDFS-15393 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-14902) NullPointer When Misconfigured

2019-10-08 Thread David Mollitor (Jira)
David Mollitor created HDFS-14902: - Summary: NullPointer When Misconfigured Key: HDFS-14902 URL: https://issues.apache.org/jira/browse/HDFS-14902 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-14899) Use Relative URLS in Hadoop HDFS RBF

2019-10-07 Thread David Mollitor (Jira)
David Mollitor created HDFS-14899: - Summary: Use Relative URLS in Hadoop HDFS RBF Key: HDFS-14899 URL: https://issues.apache.org/jira/browse/HDFS-14899 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-14898) Use Relative URLS in Hadoop HDFS HTTP FS

2019-10-07 Thread David Mollitor (Jira)
David Mollitor created HDFS-14898: - Summary: Use Relative URLS in Hadoop HDFS HTTP FS Key: HDFS-14898 URL: https://issues.apache.org/jira/browse/HDFS-14898 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-14872) Read HDFS Blocks in Random Order

2019-09-24 Thread David Mollitor (Jira)
David Mollitor created HDFS-14872: - Summary: Read HDFS Blocks in Random Order Key: HDFS-14872 URL: https://issues.apache.org/jira/browse/HDFS-14872 Project: Hadoop HDFS Issue Type: New

[jira] [Created] (HDFS-14866) NameNode stopRequested is Marked volatile

2019-09-20 Thread David Mollitor (Jira)
David Mollitor created HDFS-14866: - Summary: NameNode stopRequested is Marked volatile Key: HDFS-14866 URL: https://issues.apache.org/jira/browse/HDFS-14866 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-14865) Reduce Synchronization in DatanodeManager

2019-09-20 Thread David Mollitor (Jira)
David Mollitor created HDFS-14865: - Summary: Reduce Synchronization in DatanodeManager Key: HDFS-14865 URL: https://issues.apache.org/jira/browse/HDFS-14865 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-14864) DatanodeDescriptor Use Concurrent BlockingQueue

2019-09-20 Thread David Mollitor (Jira)
David Mollitor created HDFS-14864: - Summary: DatanodeDescriptor Use Concurrent BlockingQueue Key: HDFS-14864 URL: https://issues.apache.org/jira/browse/HDFS-14864 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-14863) Remove Synchronization From BlockPlacementPolicyDefault

2019-09-20 Thread David Mollitor (Jira)
David Mollitor created HDFS-14863: - Summary: Remove Synchronization From BlockPlacementPolicyDefault Key: HDFS-14863 URL: https://issues.apache.org/jira/browse/HDFS-14863 Project: Hadoop HDFS

[jira] [Created] (HDFS-14862) Review of MovedBlocks

2019-09-20 Thread David Mollitor (Jira)
David Mollitor created HDFS-14862: - Summary: Review of MovedBlocks Key: HDFS-14862 URL: https://issues.apache.org/jira/browse/HDFS-14862 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-14860) Clean Up StoragePolicySatisfyManager.java

2019-09-20 Thread David Mollitor (Jira)
David Mollitor created HDFS-14860: - Summary: Clean Up StoragePolicySatisfyManager.java Key: HDFS-14860 URL: https://issues.apache.org/jira/browse/HDFS-14860 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-14843) Double Synchronization in BlockReportLeaseManager

2019-09-10 Thread David Mollitor (Jira)
David Mollitor created HDFS-14843: - Summary: Double Synchronization in BlockReportLeaseManager Key: HDFS-14843 URL: https://issues.apache.org/jira/browse/HDFS-14843 Project: Hadoop HDFS

[jira] [Created] (HDFS-14842) ByteArrayManager Reduce Synchronization

2019-09-10 Thread David Mollitor (Jira)
David Mollitor created HDFS-14842: - Summary: ByteArrayManager Reduce Synchronization Key: HDFS-14842 URL: https://issues.apache.org/jira/browse/HDFS-14842 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-14841) Remove Class-Level Synchronization in LocalReplica

2019-09-10 Thread David Mollitor (Jira)
David Mollitor created HDFS-14841: - Summary: Remove Class-Level Synchronization in LocalReplica Key: HDFS-14841 URL: https://issues.apache.org/jira/browse/HDFS-14841 Project: Hadoop HDFS

[jira] [Created] (HDFS-14840) Use Java Conccurent Instead of Synchrnoization in BlockPoolTokenSecretManager

2019-09-10 Thread David Mollitor (Jira)
David Mollitor created HDFS-14840: - Summary: Use Java Conccurent Instead of Synchrnoization in BlockPoolTokenSecretManager Key: HDFS-14840 URL: https://issues.apache.org/jira/browse/HDFS-14840

[jira] [Created] (HDFS-14839) Use Java Concurrent BlockingQueue instead of Internal BlockQueue

2019-09-10 Thread David Mollitor (Jira)
David Mollitor created HDFS-14839: - Summary: Use Java Concurrent BlockingQueue instead of Internal BlockQueue Key: HDFS-14839 URL: https://issues.apache.org/jira/browse/HDFS-14839 Project: Hadoop

[jira] [Created] (HDFS-14837) Review of Block.java

2019-09-09 Thread David Mollitor (Jira)
David Mollitor created HDFS-14837: - Summary: Review of Block.java Key: HDFS-14837 URL: https://issues.apache.org/jira/browse/HDFS-14837 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-14799) Do Not Call Map containsKey In Conjunction with get

2019-08-29 Thread David Mollitor (Jira)
David Mollitor created HDFS-14799: - Summary: Do Not Call Map containsKey In Conjunction with get Key: HDFS-14799 URL: https://issues.apache.org/jira/browse/HDFS-14799 Project: Hadoop HDFS

[jira] [Created] (HDFS-14798) Synchronize invalidateBlocks in DatanodeDescriptor

2019-08-29 Thread David Mollitor (Jira)
David Mollitor created HDFS-14798: - Summary: Synchronize invalidateBlocks in DatanodeDescriptor Key: HDFS-14798 URL: https://issues.apache.org/jira/browse/HDFS-14798 Project: Hadoop HDFS

[jira] [Created] (HDFS-14790) Support Client Write Fan-Out

2019-08-28 Thread David Mollitor (Jira)
David Mollitor created HDFS-14790: - Summary: Support Client Write Fan-Out Key: HDFS-14790 URL: https://issues.apache.org/jira/browse/HDFS-14790 Project: Hadoop HDFS Issue Type: Improvement

New DataNode Implementation

2019-08-23 Thread David Mollitor
Hello Gang, Over the past few months, I've been dedicating my spare time to review the DataNode project. I've had several patches accepted over the past year but found working with the code base very challenging. It motivated me to create a new DataNode, backed by Spring Boot. I also wrote the

[jira] [Created] (HDFS-14727) Typo in analysesErrorMsg

2019-08-12 Thread David Mollitor (JIRA)
David Mollitor created HDFS-14727: - Summary: Typo in analysesErrorMsg Key: HDFS-14727 URL: https://issues.apache.org/jira/browse/HDFS-14727 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-14716) Pass CreateFile Flags To Subclasses of FileSystem

2019-08-09 Thread David Mollitor (JIRA)
David Mollitor created HDFS-14716: - Summary: Pass CreateFile Flags To Subclasses of FileSystem Key: HDFS-14716 URL: https://issues.apache.org/jira/browse/HDFS-14716 Project: Hadoop HDFS

[jira] [Created] (HDFS-14641) Possible Intermediate Block Report Race Condition

2019-07-09 Thread David Mollitor (JIRA)
David Mollitor created HDFS-14641: - Summary: Possible Intermediate Block Report Race Condition Key: HDFS-14641 URL: https://issues.apache.org/jira/browse/HDFS-14641 Project: Hadoop HDFS

[jira] [Created] (HDFS-14556) Spelling Mistake "gloablly"

2019-06-09 Thread David Mollitor (JIRA)
David Mollitor created HDFS-14556: - Summary: Spelling Mistake "gloablly" Key: HDFS-14556 URL: https://issues.apache.org/jira/browse/HDFS-14556 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-14487) Missing Space in Client Error Message

2019-05-12 Thread David Mollitor (JIRA)
David Mollitor created HDFS-14487: - Summary: Missing Space in Client Error Message Key: HDFS-14487 URL: https://issues.apache.org/jira/browse/HDFS-14487 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-14453) Improve Bad Sequence Number Error Message

2019-04-23 Thread David Mollitor (JIRA)
David Mollitor created HDFS-14453: - Summary: Improve Bad Sequence Number Error Message Key: HDFS-14453 URL: https://issues.apache.org/jira/browse/HDFS-14453 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-14452) Make Op valueOf Public

2019-04-23 Thread David Mollitor (JIRA)
David Mollitor created HDFS-14452: - Summary: Make Op valueOf Public Key: HDFS-14452 URL: https://issues.apache.org/jira/browse/HDFS-14452 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-14451) Incorrect header or version mismatch log message

2019-04-23 Thread David Mollitor (JIRA)
David Mollitor created HDFS-14451: - Summary: Incorrect header or version mismatch log message Key: HDFS-14451 URL: https://issues.apache.org/jira/browse/HDFS-14451 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-14395) Remove WARN Logging From Interrupts

2019-03-27 Thread David Mollitor (JIRA)
David Mollitor created HDFS-14395: - Summary: Remove WARN Logging From Interrupts Key: HDFS-14395 URL: https://issues.apache.org/jira/browse/HDFS-14395 Project: Hadoop HDFS Issue Type

Re: [Urgent] Question about Nexus repo and Hadoop release

2019-01-21 Thread David Nalley
I wonder if it's because there are no signatures on your key. --David On Mon, Jan 21, 2019 at 11:57 AM Wangda Tan wrote: > > Hi Brian, > > Here're links to my key: > > http://pool.sks-keyservers.net:11371/key/0xB3FA653D57300D45 > > http://pgp.mit.edu/pks/looku

Re: [Urgent] Question about Nexus repo and Hadoop release

2019-01-12 Thread David Nalley
On Sat, Jan 12, 2019 at 9:09 PM Wangda Tan wrote: > > Hi Devs, > > I'm currently rolling Hadoop 3.1.2 release candidate, however, I saw an issue > when I try to close repo in Nexus. > > Logs of https://repository.apache.org/#stagingRepositories > (orgapachehadoop-1183) shows hundreds of lines of

[jira] [Created] (HDFS-13406) Encryption zone creation fails with key names that are generally valid

2018-04-05 Thread David Tucker (JIRA)
David Tucker created HDFS-13406: --- Summary: Encryption zone creation fails with key names that are generally valid Key: HDFS-13406 URL: https://issues.apache.org/jira/browse/HDFS-13406 Project: Hadoop

[jira] [Created] (HDFS-13392) Incorrect length in Truncate CloseEvents

2018-04-03 Thread David Tucker (JIRA)
David Tucker created HDFS-13392: --- Summary: Incorrect length in Truncate CloseEvents Key: HDFS-13392 URL: https://issues.apache.org/jira/browse/HDFS-13392 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-12846) WebHDFS/Jetty misinterprets empty path

2017-11-21 Thread David Tucker (JIRA)
David Tucker created HDFS-12846: --- Summary: WebHDFS/Jetty misinterprets empty path Key: HDFS-12846 URL: https://issues.apache.org/jira/browse/HDFS-12846 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-11599) distcp interrupt does not kill hadoop job

2017-03-30 Thread David Fagnan (JIRA)
David Fagnan created HDFS-11599: --- Summary: distcp interrupt does not kill hadoop job Key: HDFS-11599 URL: https://issues.apache.org/jira/browse/HDFS-11599 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-11557) Empty directories may be recursively deleted without being listable

2017-03-21 Thread David Tucker (JIRA)
David Tucker created HDFS-11557: --- Summary: Empty directories may be recursively deleted without being listable Key: HDFS-11557 URL: https://issues.apache.org/jira/browse/HDFS-11557 Project: Hadoop HDFS

[jira] [Created] (HDFS-11169) GetBlockLocations returns a block when offset > filesize and file only has 1 block

2016-11-22 Thread David Tucker (JIRA)
David Tucker created HDFS-11169: --- Summary: GetBlockLocations returns a block when offset > filesize and file only has 1 block Key: HDFS-11169 URL: https://issues.apache.org/jira/browse/HDFS-11

[jira] [Resolved] (HDFS-10765) Log a messages if finalize metadata hasn't been done, but customer kicked off HDFS Balancer

2016-08-15 Thread David Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wang resolved HDFS-10765. --- Resolution: Fixed > Log a messages if finalize metadata hasn't been done, but customer ki

[jira] [Created] (HDFS-10765) Log a messages if finalize metadata hasn't been done, but customer kicked off HDFS Balancer

2016-08-15 Thread David Wang (JIRA)
David Wang created HDFS-10765: - Summary: Log a messages if finalize metadata hasn't been done, but customer kicked off HDFS Balancer Key: HDFS-10765 URL: https://issues.apache.org/jira/browse/HDFS-

[jira] [Created] (HDFS-9955) DataNode won't self-heal after some block dirs were manually misplaced

2016-03-14 Thread David Watzke (JIRA)
David Watzke created HDFS-9955: -- Summary: DataNode won't self-heal after some block dirs were manually misplaced Key: HDFS-9955 URL: https://issues.apache.org/jira/browse/HDFS-9955 Project: Hadoop

[jira] [Created] (HDFS-9851) Name node throws NPE when setPermission is called on a path that does not exist

2016-02-23 Thread David Yan (JIRA)
David Yan created HDFS-9851: --- Summary: Name node throws NPE when setPermission is called on a path that does not exist Key: HDFS-9851 URL: https://issues.apache.org/jira/browse/HDFS-9851 Project: Hadoop

[jira] [Created] (HDFS-9783) DataNode deadlocks after running hdfs mover

2016-02-09 Thread David Watzke (JIRA)
David Watzke created HDFS-9783: -- Summary: DataNode deadlocks after running hdfs mover Key: HDFS-9783 URL: https://issues.apache.org/jira/browse/HDFS-9783 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-8335) FSNamesystem/FSDirStatAndListingOp getFileInfo and getListingInt construct FSPermissionChecker regardless of isPermissionEnabled()

2015-05-06 Thread David Bryson (JIRA)
David Bryson created HDFS-8335: -- Summary: FSNamesystem/FSDirStatAndListingOp getFileInfo and getListingInt construct FSPermissionChecker regardless of isPermissionEnabled() Key: HDFS-8335 URL: https

[jira] [Created] (HDFS-5782) BlockListAsLongs should take lists of Replicas rather than concrete classes

2014-01-15 Thread David Powell (JIRA)
David Powell created HDFS-5782: -- Summary: BlockListAsLongs should take lists of Replicas rather than concrete classes Key: HDFS-5782 URL: https://issues.apache.org/jira/browse/HDFS-5782 Project: Hadoop

[jira] [Created] (HDFS-5689) FsDatasetImpl registers mbean using uninitialized DataNode UUID

2013-12-19 Thread David Powell (JIRA)
David Powell created HDFS-5689: -- Summary: FsDatasetImpl registers mbean using uninitialized DataNode UUID Key: HDFS-5689 URL: https://issues.apache.org/jira/browse/HDFS-5689 Project: Hadoop HDFS

[jira] [Created] (HDFS-5631) Expose interfaces required by FsDatasetSpi implementations

2013-12-04 Thread David Powell (JIRA)
David Powell created HDFS-5631: -- Summary: Expose interfaces required by FsDatasetSpi implementations Key: HDFS-5631 URL: https://issues.apache.org/jira/browse/HDFS-5631 Project: Hadoop HDFS

Re: HDFS single datanode cluster issues

2013-10-30 Thread David Mankellow
appreciated. Thanks, Dave On 30/10/2013 10:50, Allan Wilson wrote: Hi David How does your block replica count compare to the number of datanodes in your cluster? Anyway...I found this in the online doc. You may want to use the NEVER policy. dfs.client.block.write.replace-datanode-on

HDFS single datanode cluster issues

2013-10-30 Thread David Mankellow
tedFileSystem.append(DistributedFileSystem.java:81) at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:1106) The exception suggests we have already set " dfs.client.block.write.replace-datanode-on-failure.policy" to "NEVER" but hadoop ignores it. Any help would be appreciated. Thanks, David Mankellow

[jira] [Created] (HDFS-5194) Robust support for alternate FsDatasetSpi implementations

2013-09-12 Thread David Powell (JIRA)
David Powell created HDFS-5194: -- Summary: Robust support for alternate FsDatasetSpi implementations Key: HDFS-5194 URL: https://issues.apache.org/jira/browse/HDFS-5194 Project: Hadoop HDFS

[jira] [Created] (HDFS-3986) Add findHangingTests.sh from HBase to find hanging tests

2012-09-27 Thread David S. Wang (JIRA)
David S. Wang created HDFS-3986: --- Summary: Add findHangingTests.sh from HBase to find hanging tests Key: HDFS-3986 URL: https://issues.apache.org/jira/browse/HDFS-3986 Project: Hadoop HDFS

RE: Binding NameNode to another address

2011-11-30 Thread David Robson
Thanks Aaron, Have raised https://issues.apache.org/jira/browse/HDFS-2615 David -Original Message- From: Aaron T. Myers [mailto:a...@cloudera.com] Sent: Thursday, 1 December 2011 5:13 AM To: hdfs-dev@hadoop.apache.org Subject: Re: Binding NameNode to another address Hi David, You

[jira] [Created] (HDFS-2615) Parameter dfs.namenode.rpc-address does not get used when determining RPC address of NameNode

2011-11-30 Thread David Robson (Created) (JIRA)
Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.24.0 Reporter: David Robson When the NameNode is started it listens on two addresses. One defined by "dfs.namenode.servicerpc-address" and the other is de

Binding NameNode to another address

2011-11-29 Thread David Robson
Hi, Does anyone know anything about the dfs.namenode.rpc-address parameter? I posted the below to the hdfs-user list but thought I might get a better response from the dev list. David From: David Robson [mailto:david.rob...@quest.com] Sent: Friday, 25 November 2011 3:59 PM To: hdfs-u