[jira] [Resolved] (HDFS-5000) DataNode configuration should allow specifying storage type

[ https://issues.apache.org/jira/browse/HDFS-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-5000. - Resolution: Fixed Fix Version/s: Heterogeneous Storage (HDFS-2832) Thanks Nicholas. I commi

Re: hsync is too slower than hflush

haha, thank you very much, I get it now. -- Best Regards, Haosong Huang Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Monday, August 26, 2013 at 11:18 AM, Andrew Wang wrote: > Ah, I forgot the checksum fsync, so two seeks. Even with 4k writes, 50ms > still feels in the right ballpa

Re: hsync is too slower than hflush

Ah, I forgot the checksum fsync, so two seeks. Even with 4k writes, 50ms still feels in the right ballpark. Best case it's ~20ms, still way slower than hflush. It's also worth asking if there's other dirty data waiting for writeback, since I believe it can also get written out on an fsync. hflush

[jira] [Created] (HDFS-5130) Add test for snapshot related FsShell and DFSAdmin commands

Binglin Chang created HDFS-5130: --- Summary: Add test for snapshot related FsShell and DFSAdmin commands Key: HDFS-5130 URL: https://issues.apache.org/jira/browse/HDFS-5130 Project: Hadoop HDFS

Re: hsync is too slower than hflush

In fact, I just write 4k in every hsync. Datenode would write checksum file and data file when I hsync data to datanode. Each of them would spent nearly 25ms, so a hsync call would spent nearly 50ms. But hflush is very fast, which spent both 1ms in write checksum and data. If a hsync would spent

Re: [jira] [Created] (HDFS-5129) JournalNode should also respect DFS_WEB_AUTHENTICATION_KERBERnOS_KEYTAB_KEY as NN and SNN

"Kai Zheng (JIRA)" 编写：

Re: [jira] [Resolved] (HDFS-5129) JournalNode should also respect DFS_WEB_AUTHENTICATION_KERBE.ROS_KEYTAB_KEY as NN and SNN

"Kai Zheng (JIRA)" 编写：

Re: hsync is too slower than hflush

50ms is believable. hsync makes each DN call fsync and wait for acks, so you'd expect at least a disk seek time (~10ms) with some extra time depending on how much unsync'd data is being written. So, just as some back of the envelope math, assuming a disk that can write at 100MB/s: 50ms - 10ms see

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

With 9 +1s including mine (6 binding) and no -1s, the vote passes. Thanks to all who voted. Arun On Aug 15, 2013, at 2:15 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs we saw since

Hadoop-Hdfs-0.23-Build - Build # 710 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/710/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 7866 lines...] [ERROR] location: class com.google.pro

Build failed in Jenkins: Hadoop-Hdfs-0.23-Build #710

See -- [...truncated 7673 lines...] [ERROR] symbol : class Parser [ERROR] location: package com.google.protobuf [ERROR]