[
https://issues.apache.org/jira/browse/HDFS-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin resolved HDFS-12691.
--
Resolution: Later
> Ozone: Decrease interval time of SCMBlockDeletingService for improving the
> effici
Weiwei Yang created HDFS-12701:
--
Summary: More fine-grained locks in ShortCircuitCache
Key: HDFS-12701
URL: https://issues.apache.org/jira/browse/HDFS-12701
Project: Hadoop HDFS
Issue Type: Impr
For more details, see
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/10/
[Oct 23, 2017 4:58:04 PM] (epayne) YARN-4163: Audit getQueueInfo and
getApplications calls
[Oct 23, 2017 5:47:35 PM] (arp) HDFS-12683. DFSZKFailOverController re-order
logic for logging
[Oct 23, 2017 6:17
> On Oct 23, 2017, at 12:50 PM, Allen Wittenauer
> wrote:
>
>
>
> With no other information or access to go on, my current hunch is that one of
> the HDFS unit tests is ballooning in memory size. The easiest way to kill a
> Linux machine is to eat all of the RAM, thanks to overcommit and t
Elek, Marton created HDFS-12702:
---
Summary: Ozone: Add hugo to the dev docker image
Key: HDFS-12702
URL: https://issues.apache.org/jira/browse/HDFS-12702
Project: Hadoop HDFS
Issue Type: Sub-tas
+1 (non-binding)
- Verified all hashes and checksums
- Built from source on macOS 10.12.6, Java 1.8.0u65
- Deployed a pseudo cluster
- Ran some example jobs
Thanks,
Eric
On Tue, Oct 24, 2017 at 12:59 AM, Mukul Kumar Singh
wrote:
> Thanks Junping,
>
> +1 (non-binding)
>
> I built from source o
Daryn Sharp created HDFS-12703:
--
Summary: Exceptions are fatal to decommissioning monitor
Key: HDFS-12703
URL: https://issues.apache.org/jira/browse/HDFS-12703
Project: Hadoop HDFS
Issue Type: B
Daryn Sharp created HDFS-12704:
--
Summary: FBR may corrupt block state
Key: HDFS-12704
URL: https://issues.apache.org/jira/browse/HDFS-12704
Project: Hadoop HDFS
Issue Type: Bug
Compone
+1 (non-binding)
- Build from source 1.8.0_111
- Deployed on 3 node secure setup
- Ran few mapreduce jobs with multiple users.
- Verified basic resource localization
- Failover of Resource Manager.
- Log aggregation verification
- Sanity check of JHS
Thanks
Bibin
-
Daryn Sharp created HDFS-12705:
--
Summary: WebHdfsFileSystem exceptions should retain the caused by
exception
Key: HDFS-12705
URL: https://issues.apache.org/jira/browse/HDFS-12705
Project: Hadoop HDFS
For more details, see
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/567/
[Oct 23, 2017 4:43:41 PM] (epayne) YARN-4163: Audit getQueueInfo and
getApplications calls
[Oct 23, 2017 5:47:16 PM] (arp) HDFS-12683. DFSZKFailOverController re-order
logic for logging
[Oct 23, 2017 6:12:
Arpit Agarwal created HDFS-12706:
Summary: Allow overriding HADOOP_SHELL_EXECNAME
Key: HDFS-12706
URL: https://issues.apache.org/jira/browse/HDFS-12706
Project: Hadoop HDFS
Issue Type: Improv
Thanks Junping for getting this out.
+1 (non-binding)
* Built from source on CentOS 7.3.1611, jdk1.8.0_111
* Deployed 3 node cluster
* Ran some sample jobs
* Ran balancer
* Operate HDFS from command line: ls, put, dfsadmin etc
* HDFS Namenode UI looks good
Thanks,
Rakesh
On Fri, Oct 20, 2017 a
Thanks for all your hard work Junping!
* Checked signature.
* Ran a sleep job.
* Checked NN File browser UI works.
+1 (binding)
Cheers
Ravi
On Tue, Oct 24, 2017 at 12:26 PM, Rakesh Radhakrishnan
wrote:
> Thanks Junping for getting this out.
>
> +1 (non-binding)
>
> * Built from source on Cent
[
https://issues.apache.org/jira/browse/HDFS-12686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen resolved HDFS-12686.
--
Resolution: Duplicate
Since HDFS-12682 should be able to handle this, and it's pretty hard to split
tha
Allen,
Do we have any solid evidence to show the HDFS unit tests going through
the roof are due to serious memory leak by HDFS? Normally, I don't expect
memory leak are identified in our UTs - mostly, it (test jvm gone) is just
because of test or deployment issues.
Unless there is con
Just curious, Junping what would "solid evidence" look like? Is the
supposition here that the memory leak is within HDFS test code rather than
library runtime code? How would such a distinction be shown?
On Tue, Oct 24, 2017 at 4:06 PM, Junping Du wrote:
> Allen,
> Do we have any solid evid
In general, the "solid evidence" of memory leak comes from analysis of
heapdump, jastack, gc log, etc. In many cases, we can locate/conclude which
piece of code are leaking memory from the analysis.
Unfortunately, I cannot find any conclusion from previous comments and it even
cannot tell which
Sean/Junping-
Ignoring the epistemology, it's a problem. Let's figure out what's
causing memory to balloon and then we can work out the appropriate
remedy.
Is this reproducible outside the CI environment? To Junping's point,
would YETUS-561 provide more detailed information to aid debugging? -C
[
https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Zhang reopened HDFS-12502:
--
> nntop should support a category based on FilesInGetListingOps
> --
+1 (binding)
Thanks a lot, Junping!
I built and installed the source on a 6-node pseudo cluster. I simple sleep and
streaming jobs that exercised intra-queue and inter-queue preemption, and used
user weights.
-Eric
From: Junping Du
To: "common-...@hadoop.apache.org" ;
"hdfs-dev@hadoop.a
Bharat Viswanadham created HDFS-12707:
-
Summary: start-all script is missing ozone start
Key: HDFS-12707
URL: https://issues.apache.org/jira/browse/HDFS-12707
Project: Hadoop HDFS
Issue T
My plan is currently to:
* switch some of Hadoop’s Yetus jobs over to my branch with the YETUS-561
patch to test it out.
* if the tests work, work on getting YETUS-561 committed to yetus master
* switch jobs back to ASF yetus master either post-YETUS-561 or without it if
it doesn’t work
* go
FWIW we've been running branch-3.0 unit tests successfully internally,
though we have separate jobs for Common, HDFS, YARN, and MR. The failures
here are probably a property of running everything in the same JVM, which
I've found problematic in the past due to OOMs.
On Tue, Oct 24, 2017 at 4:04 PM
> On Oct 24, 2017, at 4:10 PM, Andrew Wang wrote:
>
> FWIW we've been running branch-3.0 unit tests successfully internally, though
> we have separate jobs for Common, HDFS, YARN, and MR. The failures here are
> probably a property of running everything in the same JVM, which I've found
> pro
Allen, can we bump up the maven surefire heap size to max (if it already is
not) for the branch-2 nightly build and see if it helps?
Thanks,
Subru
On Tue, Oct 24, 2017 at 4:22 PM, Allen Wittenauer
wrote:
>
> > On Oct 24, 2017, at 4:10 PM, Andrew Wang
> wrote:
> >
> > FWIW we've been running br
fang zhenyi created HDFS-12708:
--
Summary: Fix hdfs haadmin usage
Key: HDFS-12708
URL: https://issues.apache.org/jira/browse/HDFS-12708
Project: Hadoop HDFS
Issue Type: Improvement
R
[
https://issues.apache.org/jira/browse/HDFS-12532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula reopened HDFS-12532:
-
Assignee: Brahma Reddy Battula
> DN Reg can Fail when principal doesn't contain
If you want to do failover of NameNode (or even kill NameNode process)
doing shell operation (like ./hdfs haadmin -getServiceState nn) is
mandatory from program.
For DataNode there is one function in DFSClient#datanodeReport which
provides list of LIVE Datanode.
To avoid shell operations of hdfs
29 matches
Mail list logo
