Anybody can help me with this? On Fri, Dec 17, 2010 at 12:45 PM, Min Zhou <coderp...@gmail.com> wrote: > Hi all, > > After apply the patch from HADOOP-6713 into our hadoop source tree, I > did a benchmark on multi-readers RPC, and didn't found any throughput > improvement . > The attachment of this mail is a patch of my rpc benmark. Test > scenario listed below. > > Network: Gigabit LAN > CPU: 8 core Intel(R) Xeon(R) CPU E5420 @ 2.50GHz > Memory: 8GB > > ================== scenario 1 ======================= > server side: > # export HADOOP_OPS="-Xmx4096m -Xms256m" > # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -server > -handlers 1 -readers 10 -durationPerRPC 100 > > client side: > # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -client > -host hdp38 -p 38668 -req 100 -threads 10 -size 8 -arraySize 200000 > 10/12/17 12:25:07 INFO ipc.RPCBenchmark: Starting 100 rpc call(s). > 10/12/17 12:26:02 INFO ipc.RPCBenchmark: # operations: 100 > 10/12/17 12:26:02 INFO ipc.RPCBenchmark: Elapsed Time: 55073 ms > 10/12/17 12:26:02 INFO ipc.RPCBenchmark: Ops per sec: 1.815771793800955 > 10/12/17 12:26:02 INFO ipc.RPCBenchmark: Average Time: 5249 ms > > ================== scenario 2 ======================= > server side: > # export HADOOP_OPS="-Xmx4096m -Xms256m" > # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -server > -handlers 10 -readers 10 -durationPerRPC 100 > > client side: > # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -client > -host hdp38 -p 38668 -req 100 -threads 10 -size 8 -arraySize 200000 > 10/12/17 12:28:23 INFO ipc.RPCBenchmark: Starting 100 rpc call(s). > 10/12/17 12:29:11 INFO ipc.RPCBenchmark: # operations: 100 > 10/12/17 12:29:11 INFO ipc.RPCBenchmark: Elapsed Time: 47784 ms > 10/12/17 12:29:11 INFO ipc.RPCBenchmark: Ops per sec: 2.092750711535242 > 10/12/17 12:29:11 INFO ipc.RPCBenchmark: Average Time: 4581 ms > > ================== scenario 3 ======================= > server side: > # export HADOOP_OPS="-Xmx4096m -Xms256m" > # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -server > -handlers 10 -readers 1 -durationPerRPC 100 > > client side: > # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -client > -host hdp38 -p 38668 -req 100 -threads 10 -size 8 -arraySize 200000 > 10/12/17 12:31:19 INFO ipc.RPCBenchmark: Starting 100 rpc call(s). > 10/12/17 12:32:08 INFO ipc.RPCBenchmark: # operations: 100 > 10/12/17 12:32:08 INFO ipc.RPCBenchmark: Elapsed Time: 49155 ms > 10/12/17 12:32:08 INFO ipc.RPCBenchmark: Ops per sec: 2.034381039568711 > 10/12/17 12:32:08 INFO ipc.RPCBenchmark: Average Time: 4722 ms > > ================== scenario 4 ======================= > server side: > # export HADOOP_OPS="-Xmx4096m -Xms256m > # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -server > -handlers 1 -readers 1 -durationPerRPC 100 > > client side: > # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -client > -host hdp38 -p 38668 -req 100 -threads 10 -size 8 -arraySize 200000 > 10/12/17 12:36:04 INFO ipc.RPCBenchmark: Starting 100 rpc call(s). > 10/12/17 12:36:59 INFO ipc.RPCBenchmark: # operations: 100 > 10/12/17 12:36:59 INFO ipc.RPCBenchmark: Elapsed Time: 55054 ms > 10/12/17 12:36:59 INFO ipc.RPCBenchmark: Ops per sec: 1.816398445162931 > 10/12/17 12:36:59 INFO ipc.RPCBenchmark: Average Time: 5278 ms > > Statistics above indicates that a single reader can consume 200,000 > 8-bytes Strings very fast. Ten concurrent readers should consume the > same data more quickly, but reader isn't the bottleneck of RPC at > least in these scenarios. Can you give an example which scenario > HADOOP-6713 will take affect? > > Thanks, > Min > -- > My research interests are distributed systems, parallel computing and > bytecode based virtual machine. > > My profile: > http://www.linkedin.com/in/coderplay > My blog: > http://coderplay.javaeye.com >
-- My research interests are distributed systems, parallel computing and bytecode based virtual machine. My profile: http://www.linkedin.com/in/coderplay My blog: http://coderplay.javaeye.com
