[jira] [Created] (HDFS-10844) libhdfs_threaded_hdfs_static and libhdfs_zerocopy_hdfs_static are failing
Akira Ajisaka created HDFS-10844:
Summary: libhdfs_threaded_hdfs_static and
libhdfs_zerocopy_hdfs_static are failing
Key: HDFS-10844
URL: https://issues.apache.org/jira/browse/HDFS-10844
Project: Hadoop HDFS
Issue Type: Bug
Components: libhdfs
Reporter: Akira Ajisaka
>From qbt report:
{noformat}
Failed CTEST tests :
test_test_libhdfs_threaded_hdfs_static
test_test_libhdfs_zerocopy_hdfs_static
{noformat}
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/156/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Reopened] (HDFS-9038) DFS reserved space is erroneously counted towards non-DFS used.
[
https://issues.apache.org/jira/browse/HDFS-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula reopened HDFS-9038:
Reopening issue to run jenkins on branch-2.7 patch...
> DFS reserved space is erroneously counted towards non-DFS used.
> ---
>
> Key: HDFS-9038
> URL: https://issues.apache.org/jira/browse/HDFS-9038
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Affects Versions: 2.7.1
>Reporter: Chris Nauroth
>Assignee: Brahma Reddy Battula
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: GetFree.java, HDFS-9038-002.patch, HDFS-9038-003.patch,
> HDFS-9038-004.patch, HDFS-9038-005.patch, HDFS-9038-006.patch,
> HDFS-9038-007.patch, HDFS-9038-008.patch, HDFS-9038-009.patch,
> HDFS-9038-010.patch, HDFS-9038-011.patch, HDFS-9038-branch-2.7.patch,
> HDFS-9038.patch
>
>
> HDFS-5215 changed the DataNode volume available space calculation to consider
> the reserved space held by the {{dfs.datanode.du.reserved}} configuration
> property. As a side effect, reserved space is now counted towards non-DFS
> used. I don't believe it was intentional to change the definition of non-DFS
> used. This issue proposes restoring the prior behavior: do not count
> reserved space towards non-DFS used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0
> On 6 Sep 2016, at 17:30, Andrew Wang wrote: > > Thanks everyone for voting and helping to validate alpha1. The VOTE closes > with 16 +1s, 6 of them binding PMC votes, and no -1s. > > I'll go ahead and wrap up the release, will send an announcement out likely > tomorrow once the mirrors have caught up. > > Best, > Andrew > Sorry I'm late with this; I was running all the tests last night and had to redo some today. Bandwidth issues. If I hadn't missed the vote, i'd have gone +0.5 I couldn't do enough due diligence to be confident all was well. But those bits I did do (s3, azure, openstack) are all good. -checked out the source, rebuilt locally, ran the Hadoop-aws s3a test suite against s3 ireland ,openstack against rackspace and azure against Azure. All well there. -built slider. Compilation failed there because Container has added things that Slider's mock containers don't implement. This is well within the compatibility scopes, albeit inconvenient. - I haven't done a full spark test run as don't have the time right now, and haven't been running locally be confident that all works. Sorry -I did do the SPARK-7481 cloud tests. These were *really* slow, but I was doing the hadoop trunk s3, azure and swift tests in different windows; I suspect I was just using up too much CPU, RAM and bandwidth. The windows build failed irrespective of whether I had -Pnative set or not. I think theres a script there that needs windows support. Were it a real release I'd veto it for that, as it meant I wouldn't be able to build the windows native libraries for the good of everyone. Filed: HADOOP-13586 Anyway, good to see the first 3.0 alpha out the door, let's see what surfaces in the field -Steve - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[VOTE] Merge HADOOP-13341
I’d like to call for a vote to run for 5 days (ending Mon 12, 2016 at 7AM PT) to merge the HADOOP-13341 feature branch into trunk. This branch was developed exclusively by me. As usual with large shell script changes, it's been broken up into several smaller commits to make it easier to read. The core of the functionality is almost entirely in hadoop-functions.sh with the majority of the rest of the new additions either being documentation or test code. In addition, large swaths of code is removed from the hadoop, hdfs, mapred, and yarn executables. Here's a quick summary: * makes the rules around _OPTS consistent across all the projects * makes it possible to provide custom _OPTS for every hadoop, hdfs, mapred, and yarn subcommand * with the exception of deprecations, removes all of the custom daemon _OPTS handling sprinkled around the hadoop, hdfs, mapred, and yarn subcommands * removes the custom handling handling of HADOOP_CLIENT_OPTS and makes it consistent for non-daemon subcommands * makes the _USER blocker consistent with _OPTS as well as providing better documentation around this feature's existence. Note that this is an incompatible change against -alpha1. * by consolidating all of this code, makes it possible to finally fix a good chunk of the "directory name containing spaces blows up the bash code" problems that's been around since the beginning of the project Thanks! - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/157/ [Sep 6, 2016 8:39:45 AM] (rohithsharmaks) YARN-5608. TestAMRMClient.setup() fails with ArrayOutOfBoundsException. [Sep 7, 2016 9:05:33 AM] (kai.zheng) HADOOP-13218. Migrate other Hadoop side tests to prepare for removing [Sep 6, 2016 2:31:45 PM] (vvasudev) YARN-5576. Allow resource localization while container is running. [Sep 6, 2016 3:02:42 PM] (kihwal) HADOOP-13549. Eliminate intermediate buffer for server-side PB encoding. [Sep 6, 2016 4:36:21 PM] (cnauroth) HADOOP-13447. Refactor S3AFileSystem to support introduction of separate [Sep 6, 2016 4:51:55 PM] (wang) Add CHANGES and release notes for 3.0.0-alpha1 to site [Sep 6, 2016 5:38:04 PM] (cdouglas) HDFS-9847. HDFS configuration should accept time units. Contributed by [Sep 6, 2016 6:02:39 PM] (cnauroth) HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by [Sep 6, 2016 6:44:26 PM] (xiao) HDFS-10835. Fix typos in httpfs.sh. Contributed by John Zhuge. [Sep 6, 2016 6:48:35 PM] (xiao) HDFS-10841. Remove duplicate or unused variable in appendFile(). [Sep 6, 2016 8:37:21 PM] (arp) HDFS-9038. DFS reserved space is erroneously counted towards non-DFS [Sep 7, 2016 3:54:17 AM] (xiao) HADOOP-13558. UserGroupInformation created from a Subject incorrectly [Sep 7, 2016 5:40:20 AM] (kasha) YARN-5616. Clean up WeightAdjuster. (Yufei Gu via kasha) -1 overall The following subsystems voted -1: asflicense unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed CTEST tests : test_test_libhdfs_threaded_hdfs_static test_test_libhdfs_zerocopy_hdfs_static Failed junit tests : hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens hadoop.yarn.server.TestContainerManagerSecurity hadoop.yarn.server.TestMiniYarnClusterNodeUtilization hadoop.mapred.TestMROpportunisticMaps hadoop.mapred.TestReduceFetch hadoop.mapred.TestMerge hadoop.mapreduce.TestMapReduceLazyOutput hadoop.mapred.TestMRIntermediateDataEncryption hadoop.mapred.TestLazyOutput hadoop.mapreduce.TestLargeSort hadoop.mapred.TestReduceFetchFromPartialMem hadoop.mapreduce.v2.TestMRJobsWithProfiler hadoop.mapreduce.lib.output.TestJobOutputCommitter hadoop.mapreduce.security.ssl.TestEncryptedShuffle hadoop.mapreduce.v2.TestMROldApiJobs hadoop.mapred.TestJobCleanup hadoop.mapreduce.v2.TestSpeculativeExecution hadoop.mapred.TestClusterMRNotification hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken hadoop.mapreduce.v2.TestMRAMWithNonNormalizedCapabilities hadoop.mapreduce.v2.TestMRJobs hadoop.mapred.TestJobName hadoop.mapreduce.TestMRJobClient hadoop.mapred.TestClusterMapReduceTestCase hadoop.mapred.TestAuditLogger hadoop.mapreduce.security.TestMRCredentials hadoop.mapred.TestMRTimelineEventHandling hadoop.mapreduce.v2.TestMiniMRProxyUser hadoop.mapreduce.v2.TestMRJobsWithHistoryService hadoop.mapred.TestMiniMRClientCluster hadoop.mapred.TestMiniMRChildTask hadoop.mapreduce.TestChild hadoop.mapreduce.security.TestBinaryTokenFile hadoop.mapred.TestJobCounters hadoop.streaming.TestMultipleCachefiles hadoop.streaming.TestFileArgs hadoop.streaming.TestSymLink hadoop.streaming.TestMultipleArchiveFiles hadoop.mapred.gridmix.TestGridmixSubmission hadoop.mapred.gridmix.TestLoadJob hadoop.mapred.gridmix.TestSleepJob hadoop.mapred.gridmix.TestDistCacheEmulation hadoop.tools.TestDistCh Timed out junit tests : org.apache.hadoop.http.TestHttpServerLifecycle org.apache.hadoop.mapred.TestMiniMRClasspath org.apache.hadoop.mapred.TestJobSysDirWithDFS org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/157/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/157/artifact/out/diff-compile-javac-root.txt [168K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/157/artifact/out/diff-checkstyle-root.txt [16M] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/157/artifact/out/diff-patch-pylint.txt [16K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/157/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs:
[jira] [Created] (HDFS-10845) Update hdfs-default after HDFS configuration accept time units
Yiqun Lin created HDFS-10845: Summary: Update hdfs-default after HDFS configuration accept time units Key: HDFS-10845 URL: https://issues.apache.org/jira/browse/HDFS-10845 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Yiqun Lin Assignee: Yiqun Lin Priority: Minor Update the hdfs-default.xml to match the change in type after HDFS-9847. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
