Re: Fixing flaky tests in Apache Hadoop

Thank you Akira and We-Chiu. IMHO, the citation is more than just flaky tests. It has more depth: - Every developer stays committed to keep the code healthy. - Those flaky tests are actually "*bugs*" that need to be fixed. It is evident that there is a major problem in handling the resources as I

[jira] [Resolved] (HDFS-12122) TestEditLogTailer is flaky

[ https://issues.apache.org/jira/browse/HDFS-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein resolved HDFS-12122. -- Release Note: I could not reproduce this. I did not see it in the Qbt reports either. Without l

[jira] [Resolved] (HDFS-4723) Occasional failure in TestDFSClientRetries#testGetFileChecksum because the number of available xcievers is set too low

[ https://issues.apache.org/jira/browse/HDFS-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein resolved HDFS-4723. - Release Note: I cannot reproduce the same stack trace. HDFS-15461 investigates addresses the same

[jira] [Resolved] (HDFS-11052) TestPersistBlocks#TestRestartDfsWithFlush flaky failure

[ https://issues.apache.org/jira/browse/HDFS-11052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein resolved HDFS-11052. -- Release Note: The issue is pretty old without providing enough information. Resolution: In

Re: Fixing flaky tests in Apache Hadoop

I also wondered if the hardware was too stressed since all Hadoop related projects all use the same set of Jenkins servers. However, HBase just recently moved to their own dedicated machines, so I'm actually surprised to see a lot of resource related failures even now. On Thu, Oct 22, 2020 at 2:03

Apache Hadoop qbt Report: trunk+JDK11 on Linux/x86_64

For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/35/ [Oct 21, 2020 5:59:35 AM] (Kihwal Lee) HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein. [Oct 21, 2020 12:01:13 PM] (noreply) HADOOP-17305. Fix ITestCustomSigner to work wi

Re: Fixing flaky tests in Apache Hadoop

Thanks for raising the issue, Akira and Ahmed, Fixing flaky tests is a thankless job so I want to take this opportunity to recognize the time and effort. We will always have flaky tests due to bad tests or simply infra issues. Fixing flaky tests will take time but if they are not addressed it was

Fixing flaky tests in Apache Hadoop

Hi Hadoop developers, Now there are a lot of failing unit tests and there is an issue to tackle this bad situation. https://issues.apache.org/jira/browse/HDFS-15646 Although this issue is in HDFS project, this issue is related to all the Hadoop developers. Please check the above URL, read the des

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64

For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/302/ [Oct 21, 2020 5:59:35 AM] (Kihwal Lee) HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein. [Oct 21, 2020 12:01:13 PM] (noreply) HADOOP-17305. Fix ITestCustomSigner to work wi

[jira] [Created] (HDFS-15646) Track failing tests in HDFS

Ahmed Hussein created HDFS-15646: Summary: Track failing tests in HDFS Key: HDFS-15646 URL: https://issues.apache.org/jira/browse/HDFS-15646 Project: Hadoop HDFS Issue Type: Bug Com

Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64

For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/93/ No changes -1 overall The following subsystems voted -1: asflicense hadolint jshint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: c