Re: HADOOP-13636 and io.bytes.per.checksum
On 23 Sep 2016, at 18:55, Andrew Wang mailto:andrew.w...@cloudera.com>> wrote: Have you git blamed to dig up the original JIRA conversation? I think that deprecation predates many of us, so you might not get much historical perspective from the mailing list. its so old that it's lost in the history of the "show git history for selection" feature in IDEA which does exactly that. I'm happy to lend a +1 though, since like you said, it doesn't seem like that config key is going anywhere.
Re: HADOOP-13636 and io.bytes.per.checksum
> On Sep 24, 2016, at 4:24 AM, Steve Loughran wrote: > > > On 23 Sep 2016, at 18:55, Andrew Wang > mailto:andrew.w...@cloudera.com>> wrote: > > Have you git blamed to dig up the original JIRA conversation? I think that > deprecation predates many of us, so you might not get much historical > perspective from the mailing list. > > its so old that it's lost in the history of the "show git history for > selection" feature in IDEA which does exactly that. > > I'm happy to lend a +1 though, since like you said, it doesn't seem like that > config key is going anywhere. > dfs.bytes-per-checksum was added as part of HADOOP-6419 / 00de6db0e3976cf0108010ecbd6e73a5ceab7b1f. It looks like the internal-to-Yahoo! commit that was part of that one accidentally included a lot more than what was documented. So if there is any ASF-level documentation of the change, it's going to be in a JIRA patch somewhere, likely in the 0.20 timeline. - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: HADOOP-13636 and io.bytes.per.checksum
> On Sep 24, 2016, at 5:11 AM, Allen Wittenauer > wrote: > > >> On Sep 24, 2016, at 4:24 AM, Steve Loughran wrote: >> >> >> On 23 Sep 2016, at 18:55, Andrew Wang >> mailto:andrew.w...@cloudera.com>> wrote: >> >> Have you git blamed to dig up the original JIRA conversation? I think that >> deprecation predates many of us, so you might not get much historical >> perspective from the mailing list. >> >> its so old that it's lost in the history of the "show git history for >> selection" feature in IDEA which does exactly that. >> >> I'm happy to lend a +1 though, since like you said, it doesn't seem like >> that config key is going anywhere. >> > > dfs.bytes-per-checksum was added as part of HADOOP-6419 / > 00de6db0e3976cf0108010ecbd6e73a5ceab7b1f. It looks like the > internal-to-Yahoo! commit that was part of that one accidentally included a > lot more than what was documented. So if there is any ASF-level documentation > of the change, it's going to be in a JIRA patch somewhere, likely in the 0.20 > timeline. > OK neat. It was added then, but not actually used until HDFS-631 which you'll find in the old hadoop-hdfs tree as part of 0.21. So it was trapped in the project split... which is why it's mainly stuff outside of HDFS that is still using it. I have a hunch that HDFS-2000 was basically misguided(?): it was deprecated for HDFS's internal usage but needed in common for non-HDFS file systems to set. So it's not REALLY deprecated in the classic sense. Definitely need better documentation here. :/ - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/ [Sep 23, 2016 9:01:30 AM] (stevel) HADOOP-13643. Math error in AbstractContractDistCpTest. Contributed by [Sep 23, 2016 1:41:02 PM] (brahma) HDFS-10894. Remove the redundant charactors for command -saveNamespace [Sep 23, 2016 1:51:34 PM] (brahma) HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs [Sep 23, 2016 5:37:46 PM] (shv) HDFS-10843. Update space quota when a UC block is completed rather than [Sep 23, 2016 7:09:25 PM] (kihwal) YARN-5622. TestYarnCLI.testGetContainers fails due to mismatched date [Sep 23, 2016 8:26:57 PM] (weichiu) HDFS-10876. Dispatcher#dispatch should log IOException stacktrace. [Sep 23, 2016 9:10:11 PM] (shv) HDFS-10866. Fix Eclipse Java compile errors related to generic -1 overall The following subsystems voted -1: asflicense unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.server.TestContainerManagerSecurity cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/diff-compile-javac-root.txt [168K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/diff-checkstyle-root.txt [16M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/diff-patch-pylint.txt [16K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/whitespace-eol.txt [11M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/diff-javadoc-javadoc-root.txt [2.2M] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt [268K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-nativetask.txt [124K] asflicense: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/174/artifact/out/patch-asflicense-problems.txt [4.0K] Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/104/ [Sep 23, 2016 1:41:02 PM] (brahma) HDFS-10894. Remove the redundant charactors for command -saveNamespace [Sep 23, 2016 1:51:34 PM] (brahma) HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs [Sep 23, 2016 5:37:46 PM] (shv) HDFS-10843. Update space quota when a UC block is completed rather than [Sep 23, 2016 7:09:25 PM] (kihwal) YARN-5622. TestYarnCLI.testGetContainers fails due to mismatched date [Sep 23, 2016 8:26:57 PM] (weichiu) HDFS-10876. Dispatcher#dispatch should log IOException stacktrace. [Sep 23, 2016 9:10:11 PM] (shv) HDFS-10866. Fix Eclipse Java compile errors related to generic -1 overall The following subsystems voted -1: compile unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.yarn.server.nodemanager.recovery.TestNMLeveldbStateStoreService hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService hadoop.yarn.server.nodemanager.TestNodeManagerShutdown hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager hadoop.yarn.server.timeline.TestRollingLevelDB hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices hadoop.yarn.server.timeline.TestTimelineDataManager hadoop.yarn.server.timeline.TestLeveldbTimelineStore hadoop.yarn.server.timeline.recovery.TestLeveldbTimelineStateStore hadoop.yarn.server.timeline.TestRollingLevelDBTimelineStore hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer hadoop.yarn.server.timelineservice.storage.common.TestRowKeys hadoop.yarn.server.timelineservice.storage.common.TestKeyConverters hadoop.yarn.server.timelineservice.storage.common.TestSeparator hadoop.yarn.server.resourcemanager.TestRM hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore hadoop.yarn.server.resourcemanager.TestRMRestart hadoop.yarn.server.resourcemanager.TestResourceTrackerService hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.client.api.impl.TestNMClient hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore hadoop.yarn.server.timeline.TestOverrideTimelineStoreYarnClient hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorage hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun hadoop.yarn.server.timelineservice.storage.TestPhoenixOfflineAggregationWriterImpl hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.mapred.TestShuffleHandler hadoop.mapreduce.v2.hs.TestHistoryServerLeveldbStateStoreService Timed out junit tests : org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache org.apache.hadoop.mapred.TestMRIntermediateDataEncryption org.apache.hadoop.mapred.TestMROpportunisticMaps compile: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/104/artifact/out/patch-compile-root.txt [308K] cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/104/artifact/out/patch-compile-root.txt [308K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/104/artifact/out/patch-compile-root.txt [308K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/104/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [192K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/104/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/104/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-applicationhistoryservice.txt [52K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/104/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice.txt [20K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/104/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_had
