[
https://issues.apache.org/jira/browse/HDDS-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Elek resolved HDDS-2325.
-------------------------------
Fix Version/s: 0.5.0
Resolution: Fixed
> BenchMarkDatanodeDispatcher genesis test is failing with NPE
> ------------------------------------------------------------
>
> Key: HDDS-2325
> URL: https://issues.apache.org/jira/browse/HDDS-2325
> Project: Hadoop Distributed Data Store
> Issue Type: New Feature
> Reporter: Marton Elek
> Assignee: Marton Elek
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.5.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> ## What changes were proposed in this pull request?
> Genesis is a microbenchmark tool for Ozone based on JMH
> ([https://openjdk.java.net/projects/code-tools/jmh/).]
>
> Due to the recent Datanode changes the BenchMarkDatanodeDispatcher is failing
> with NPE:
>
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.hadoop.ozone.container.common.interfaces.Handler.<init>(Handler.java:69)
> at
> org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.<init>(KeyValueHandler.java:114)
> at
> org.apache.hadoop.ozone.container.common.interfaces.Handler.getHandlerForContainerType(Handler.java:78)
> at
> org.apache.hadoop.ozone.genesis.BenchMarkDatanodeDispatcher.initialize(BenchMarkDatanodeDispatcher.java:115)
> at
> org.apache.hadoop.ozone.genesis.generated.BenchMarkDatanodeDispatcher_createContainer_jmhTest._jmh_tryInit_f_benchmarkdatanodedispatcher0_G(BenchMarkDatanodeDispatcher_createContainer_jmhTest.java:438)
> at
> org.apache.hadoop.ozone.genesis.generated.BenchMarkDatanodeDispatcher_createContainer_jmhTest.createContainer_Throughput(BenchMarkDatanodeDispatcher_createContainer_jmhTest.java:71)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:453)
> at
> org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:437)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> And this is the just the biggest problem there are a few other problems. I
> propose the following fixes:
> *fix 1*: NPE is thrown because the 'context' object is required by
> KeyValueHandler/Handler classes.
> In fact the context is not required, we need two functionalities/info from
> the context: the ability to send icr (IncrementalContainerReport) and the ID
> of the datanode.
> Law of Demeter principle suggests to have only the minimum required
> information from other classes.
> For example instead of having context but using only
> context.getParent().getDatanodeDetails().getUuidString() we can have only the
> UUID string which makes more easy to test (unit and benchmark) the
> Handler/KeyValueHandler.
> This is the biggest (but still small change) in this patch: I started to use
> the datanodeId and an icrSender instead of having the full context.
> *fix 2,3:* There were a few other problems. The scmId was missing if the
> writeChunk was called from Benchmark and and the Checksum was also missing.
> *fix 4:* I also had a few other problems: very huge containers are used
> (default 5G) and as the benchmark starts with creating 100 containers it
> requires 500G space by default. I adjusted the container size to make it
> possible to run on local machine.
>
> ## How this patch can be tested?
> {code:java}
> ./ozone genesis -benchmark=BenchMarkDatanodeDispatcher.writeChunk{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
