[jira] [Resolved] (HDFS-9499) Fix typos in DFSAdmin.java

2019-07-15 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HDFS-9499. Resolution: Invalid Looks like it's already been resolved by another JIRA. > Fix

[jira] [Resolved] (HDFS-14487) Missing Space in Client Error Message

2019-06-18 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HDFS-14487. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.3.0 Thanks for the

[ANNOUNCE] Aaron Fabbri as Hadoop PMC

2019-06-17 Thread Daniel Templeton
I am very pleased to announce that Aaron Fabbri has now been added to the Hadoop PMC.  Welcome aboard, Aaron, and Congratulations! Daniel - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands,

Re: June Hadoop Community Meetup

2019-06-04 Thread Daniel Templeton
tions. Hope to see you there! Daniel On 5/23/19 10:57 AM, Daniel Templeton wrote: Hi, all!  I want to let you know that Cloudera is planning to host a contributors meetup on June 26 at our Palo Alto headquarters. We're still working out the details, but we're hoping to follow the format

June Hadoop Community Meetup

2019-05-23 Thread Daniel Templeton
Hi, all!  I want to let you know that Cloudera is planning to host a contributors meetup on June 26 at our Palo Alto headquarters.  We're still working out the details, but we're hoping to follow the format that Oath and LinkedIn followed during the last two.  Please feel free to reach out to m

[jira] [Created] (HDFS-14382) The hdfs fsck command docs do not explain the meaning of the reported fields

2019-03-19 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-14382: --- Summary: The hdfs fsck command docs do not explain the meaning of the reported fields Key: HDFS-14382 URL: https://issues.apache.org/jira/browse/HDFS-14382

[jira] [Created] (HDFS-14381) Add option to hdfs dfs -cat to ignore corrupt blocks

2019-03-19 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-14381: --- Summary: Add option to hdfs dfs -cat to ignore corrupt blocks Key: HDFS-14381 URL: https://issues.apache.org/jira/browse/HDFS-14381 Project: Hadoop HDFS

[jira] [Created] (HDFS-14121) Log message about the old hosts file format is misleading

2018-11-30 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-14121: --- Summary: Log message about the old hosts file format is misleading Key: HDFS-14121 URL: https://issues.apache.org/jira/browse/HDFS-14121 Project: Hadoop HDFS

[jira] [Created] (HDFS-14092) Remove two-step create/append in WebHdfsFileSystem

2018-11-20 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-14092: --- Summary: Remove two-step create/append in WebHdfsFileSystem Key: HDFS-14092 URL: https://issues.apache.org/jira/browse/HDFS-14092 Project: Hadoop HDFS

[jira] [Created] (HDFS-14015) Improve error handling in hdfsThreadDestructor in native thread local storage

2018-10-22 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-14015: --- Summary: Improve error handling in hdfsThreadDestructor in native thread local storage Key: HDFS-14015 URL: https://issues.apache.org/jira/browse/HDFS-14015

[jira] [Created] (HDFS-13913) LazyPersistFileScrubber.run() error handling is poor

2018-09-12 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-13913: --- Summary: LazyPersistFileScrubber.run() error handling is poor Key: HDFS-13913 URL: https://issues.apache.org/jira/browse/HDFS-13913 Project: Hadoop HDFS

[jira] [Created] (HDFS-13819) TestDirectoryScanner#testDirectoryScannerInFederatedCluster is flaky

2018-08-11 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-13819: --- Summary: TestDirectoryScanner#testDirectoryScannerInFederatedCluster is flaky Key: HDFS-13819 URL: https://issues.apache.org/jira/browse/HDFS-13819 Project

[jira] [Created] (HDFS-13800) Improve the error message when contacting an IPC port via a browser

2018-08-07 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-13800: --- Summary: Improve the error message when contacting an IPC port via a browser Key: HDFS-13800 URL: https://issues.apache.org/jira/browse/HDFS-13800 Project

Re: Read or Save specific blocks of a file

2018-04-23 Thread Daniel Templeton
I'm not aware of a way to work with blocks using the public APIs. The easiest way to do it is probably to retrieve the block IDs and then go grab those blocks from the data nodes' local file systems directly. Daniel On 4/23/18 9:05 AM, Thodoris Zois wrote: Hello list, I have a file on HDFS t

Re: Hadoop 3.1.0 release discussion

2018-01-31 Thread Daniel Templeton
ish that before code freeze date (Feb 08). + Sunil as well. Thanks, Wangda On Wed, Jan 17, 2018 at 4:31 PM, Daniel Templeton <mailto:dan...@cloudera.com>> wrote: What's the status on resource profiles?  I believe there are still a couple of open JIRAs to rethi

Re: Hadoop 3.1.0 release discussion

2018-01-17 Thread Daniel Templeton
What's the status on resource profiles?  I believe there are still a couple of open JIRAs to rethink some of the design choices. Daniel On 1/17/18 11:33 AM, Wangda Tan wrote: Hi All, Since we're fast approaching previously proposed feature freeze date (Jan 30, about 13 days from today). If yo

Re: Apache Hadoop 2.8.3 Release Plan

2017-11-21 Thread Daniel Templeton
Doh.  Mailer dropped some of the lists.  Replying again to avoid fragmenting the discussion... Still +1 to Andrew's comments. Daniel On 11/21/17 7:53 AM, Daniel Templeton wrote: +1 Daniel On 11/20/17 10:22 PM, Andrew Wang wrote: I'm against including new features in maintenanc

Re: Hadoop Compatability Guide, Part Deux: Developer Docs

2017-10-30 Thread Daniel Templeton
We've now gone a couple of rounds of reviews on HADOOP-14876, and a patch is posted for HADOOP-14875.  Feedback is very welcome.  Please take a look. Daniel On 10/14/17 8:46 AM, Daniel Templeton wrote: I just posted a first patch for HADOOP-14876 that adds downstream developer docs bas

Hadoop Compatability Guide, Part Deux: Developer Docs

2017-10-14 Thread Daniel Templeton
I just posted a first patch for HADOOP-14876 that adds downstream developer docs based on the overhauled compatibility guide from HADOOP-13714.  I would really appreciate some critical review of the doc, as it's much more likely to be read by downstream developers than the compatibility spec it

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC0)

2017-09-11 Thread Daniel Templeton
YARN-6622 is now committed to 2.9.  We could backport YARN-5258 and YARN-6622 for 2.8, but it'll take some editing.  We'll have to check to see what features are unsupported in 2.8 and remove those from the docs.  Not a huge effort overall, though.  Probably a hour's work.  I may have time to t

Re: DISCUSS: Hadoop Compatability Guidelines

2017-09-07 Thread Daniel Templeton
minor versions and how it should relate to the hadoop release version. Thoughts ? Cheers -Arun On Tue, Sep 5, 2017 at 10:38 AM, Daniel Templeton wrote: Good idea. I should have thought of that. :) Done. Daniel On 9/5/17 10:33 AM, Anu Engineer wrote: Could you please attach the PDFs t

Re: DISCUSS: Hadoop Compatability Guidelines

2017-09-05 Thread Daniel Templeton
Good idea.  I should have thought of that. :)  Done. Daniel On 9/5/17 10:33 AM, Anu Engineer wrote: Could you please attach the PDFs to the JIRA. I think the mailer is stripping them off from the mail. Thanks Anu On 9/5/17, 9:44 AM, "Daniel Templeton" wrote: Resending with

Re: DISCUSS: Hadoop Compatability Guidelines

2017-09-05 Thread Daniel Templeton
Resending with a broader audience, and reattaching the PDFs. Daniel On 9/4/17 9:01 AM, Daniel Templeton wrote: All, in prep for Hadoop 3 beta 1 I've been working on updating the compatibility guidelines on HADOOP-13714.  I think the initial doc is more or less complete, so I'd li

Re: [VOTE] Merge YARN-3926 (resource profile) to trunk

2017-08-26 Thread Daniel Templeton
ssues by far. Merge JIRA: YARN-7013 (Jenkins gave +1 already). Documentation: YARN-7056 Special thanks to a team of folks who worked hard and contributed towards this effort including design discussion/development/reviews, etc.: Varun Vasudev, Sunil Govind, Daniel Templeton, Vinod Vavilapalli,

[jira] [Created] (HDFS-12153) Tests use assertTrue(....equals(...)) instead of assertEquals()

2017-07-17 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-12153: --- Summary: Tests use assertTrue(equals(...)) instead of assertEquals() Key: HDFS-12153 URL: https://issues.apache.org/jira/browse/HDFS-12153 Project: Hadoop

[jira] [Created] (HDFS-11571) Typo in DataStorage exception message

2017-03-23 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-11571: --- Summary: Typo in DataStorage exception message Key: HDFS-11571 URL: https://issues.apache.org/jira/browse/HDFS-11571 Project: Hadoop HDFS Issue Type

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

2017-03-17 Thread Daniel Templeton
Thanks for the new RC, Junping. I built from source and tried it out on a 2-node cluster with HA enabled. I ran a pi job and some streaming jobs. I tested that localization and failover work correctly, and I played a little with the YARN and HDFS web UIs. I did encounter an old friend of mi

Re: [VOTE] Release cadence and EOL

2017-01-17 Thread Daniel Templeton
Thanks for driving this, Sangjin. Quick question, though: the subject line is "Release cadence and EOL," but I don't see anything about cadence in the proposal. Did I miss something? Daniel On 1/17/17 8:35 AM, Sangjin Lee wrote: Following up on the discussion thread on this topic ( https://s

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Daniel Templeton
I just downloaded the build tarball and deployed it on a 2-node cluster. It looks to me like it's compiled for the wrong platform: # file /usr/lib/hadoop/bin/container-executor /usr/lib/hadoop/bin/container-executor: setuid setgid Mach-O 64-bit executable I'm also seeing the no-native-librar

Re: Multiple namenodes

2016-07-22 Thread Daniel Templeton
On 7/22/16 12:23 PM, Kun Ren wrote: Thanks a lot for your reply, Daniel, very helpful. About (1) : I will consider this way, thanks. Also beside multiple clusters, are there any other options to do so? Thanks. HDFS does not support two active NNs in a single cluster. Each DN belongs to a s

Re: Multiple namenodes

2016-07-22 Thread Daniel Templeton
On 7/22/16 8:45 AM, Kun Ren wrote: (1). How to create/start multiple namenodes? Just pretend like you have two separate HDFS clusters and set them up that way. (That is actually what you have.) (2). Once I have multiple Namenodes running, do you think what is the best/simple way to change

[jira] [Created] (HDFS-10463) TestRollingFileSystemSinkWithHdfs needs some cleanup

2016-05-25 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-10463: --- Summary: TestRollingFileSystemSinkWithHdfs needs some cleanup Key: HDFS-10463 URL: https://issues.apache.org/jira/browse/HDFS-10463 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-9840) wheee

2016-02-21 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HDFS-9840. Resolution: Invalid Hadoop Flags: (was: Incompatible change,Reviewed) Fix

[jira] [Created] (HDFS-9782) RollingFileSystemSink should have configurable roll interval

2016-02-08 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-9782: -- Summary: RollingFileSystemSink should have configurable roll interval Key: HDFS-9782 URL: https://issues.apache.org/jira/browse/HDFS-9782 Project: Hadoop HDFS

[jira] [Created] (HDFS-9770) Namenode doesn't pass config to UGI in format

2016-02-05 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-9770: -- Summary: Namenode doesn't pass config to UGI in format Key: HDFS-9770 URL: https://issues.apache.org/jira/browse/HDFS-9770 Project: Hadoop HDFS

[jira] [Created] (HDFS-9769) Multiple tests use incorrect file permissions

2016-02-05 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-9769: -- Summary: Multiple tests use incorrect file permissions Key: HDFS-9769 URL: https://issues.apache.org/jira/browse/HDFS-9769 Project: Hadoop HDFS Issue

Re: Class Placement

2016-01-13 Thread Daniel Templeton
you do it right, you could have common test code in hadoop-common-test, which hadoop-hdfs-test could reuse On 7 Jan 2016, at 08:44, Allen Wittenauer wrote: On Jan 6, 2016, at 12:04 PM, Daniel Templeton wrote: I'm working on an HDFS sink for metrics2 to allow daemons to log metrics

[jira] [Created] (HDFS-9637) Add test for HADOOP-12702

2016-01-11 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-9637: -- Summary: Add test for HADOOP-12702 Key: HDFS-9637 URL: https://issues.apache.org/jira/browse/HDFS-9637 Project: Hadoop HDFS Issue Type: Improvement

Class Placement

2016-01-06 Thread Daniel Templeton
I'm working on an HDFS sink for metrics2 to allow daemons to log metrics into HDFS. In order to test the new sink, I need access to the MiniDFSCluster, which I don't have from the common package, where the code for all the other metrics2 sinks lives. Any objection to putting the unit test cod

TestDirectoryScanner.testThrottle() Failures

2015-12-16 Thread Daniel Templeton
Would someone please review and commit HDFS-9300 so that the testThrottle() test will stop failing. It's a 2-line patch. Thanks, Daniel

[jira] [Created] (HDFS-9339) Extend full test of KMS ACLs

2015-10-29 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-9339: -- Summary: Extend full test of KMS ACLs Key: HDFS-9339 URL: https://issues.apache.org/jira/browse/HDFS-9339 Project: Hadoop HDFS Issue Type: Test

[jira] [Created] (HDFS-9300) DirectoryScanner.testThrottle() is still a little flakey

2015-10-23 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-9300: -- Summary: DirectoryScanner.testThrottle() is still a little flakey Key: HDFS-9300 URL: https://issues.apache.org/jira/browse/HDFS-9300 Project: Hadoop HDFS

[jira] [Created] (HDFS-9295) Add a thorough test of the full KMS code path

2015-10-23 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-9295: -- Summary: Add a thorough test of the full KMS code path Key: HDFS-9295 URL: https://issues.apache.org/jira/browse/HDFS-9295 Project: Hadoop HDFS Issue

Re: [jira] [Created] (HDFS-9220) ChecksumException after writing less than 512 bytes

2015-10-09 Thread Daniel Templeton
Oops. I'm used to email replies automatically being appended to issues. :) Daniel On 10/9/15 3:13 PM, Yongjun Zhang wrote: Thanks Daniel, mind putting your comments in HDFS-9220? --Yongjun On Fri, Oct 9, 2015 at 10:11 AM, Daniel Templeton wrote: I saw a similar issue, and it went

Re: [jira] [Created] (HDFS-9220) ChecksumException after writing less than 512 bytes

2015-10-09 Thread Daniel Templeton
I saw a similar issue, and it went away when I backed out HDFS-4660 and HDFS-8722. Might be the same issue. On 10/9/15 6:02 AM, Bogdan Raducanu (JIRA) wrote: Bogdan Raducanu created HDFS-9220: - Summary: ChecksumException after writing less th

[jira] [Reopened] (HDFS-9176) TestDirectoryScanner#testThrottling often fails.

2015-10-05 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton reopened HDFS-9176: The latest build hit the same issue again, so it wasn't just a temporary environmental e

[jira] [Resolved] (HDFS-9176) TestDirectoryScanner#testThrottling often fails.

2015-10-05 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HDFS-9176. Resolution: Not A Problem The test failed twice is an odd way and hasn't failed since i

[jira] [Created] (HDFS-9179) fs.defaultFS should not be used on the server side

2015-09-30 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-9179: -- Summary: fs.defaultFS should not be used on the server side Key: HDFS-9179 URL: https://issues.apache.org/jira/browse/HDFS-9179 Project: Hadoop HDFS

[jira] [Created] (HDFS-9065) Include commas on # of files, blocks, total filesystem objects in NN Web UI

2015-09-11 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-9065: -- Summary: Include commas on # of files, blocks, total filesystem objects in NN Web UI Key: HDFS-9065 URL: https://issues.apache.org/jira/browse/HDFS-9065 Project

[jira] [Resolved] (HDFS-8989) Allow rate-limiting the DirectoryScanner via time-slicing

2015-09-02 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton resolved HDFS-8989. Resolution: Duplicate This is the same issue as HDFS-8873 as far as I can see. I'll ad

[jira] [Created] (HDFS-9001) DFSUtil.getNsServiceRpcUris() can return too many entries in a non-HA, non-federated cluster

2015-09-01 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-9001: -- Summary: DFSUtil.getNsServiceRpcUris() can return too many entries in a non-HA, non-federated cluster Key: HDFS-9001 URL: https://issues.apache.org/jira/browse/HDFS-9001

[jira] [Created] (HDFS-8950) NameNode refresh doesn't remove DataNodes

2015-08-24 Thread Daniel Templeton (JIRA)
Daniel Templeton created HDFS-8950: -- Summary: NameNode refresh doesn't remove DataNodes Key: HDFS-8950 URL: https://issues.apache.org/jira/browse/HDFS-8950 Project: Hadoop HDFS Issue