[jira] [Resolved] (HADOOP-7339) Introduce a buffered checksum for avoiding frequently calls on Checksum.update()

[ https://issues.apache.org/jira/browse/HADOOP-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhou resolved HADOOP-7339. -- Resolution: Won't Fix > Introduce a buffered checksum for avoiding frequently

[jira] [Created] (HADOOP-7339) Introduce a buffered checksum for avoiding frequently calls on Checksum.update()

Common Issue Type: Improvement Components: util Reporter: Min Zhou We found that PureJavaCRC32/CRC32.update() is the TOP 1 of the methods consuming CPU in a map side, and in reduce side, it cost a lots of CPU too. IFileOutputStream would frequently call

Re: Questions on HADOOP-6713

Anybody can help me with this? On Fri, Dec 17, 2010 at 12:45 PM, Min Zhou wrote: > Hi all, > > After apply the patch from HADOOP-6713 into our hadoop source tree,  I > did a benchmark on multi-readers RPC, and didn't found any  throughput > improvement . > The attachment

Questions on HADOOP-6713

Hi all, After apply the patch from HADOOP-6713 into our hadoop source tree, I did a benchmark on multi-readers RPC, and didn't found any throughput improvement . The attachment of this mail is a patch of my rpc benmark. Test scenario listed below. Network: Gigabit LAN CPU: 8 core Intel(R) Xeon(

Why hadoop jobs need setup and cleanup phases which would consume a lot of time ?

Hi all, Why hadoop jobs need setup and cleanup phases which would consume a lot of time ? Why could not us archieve it like a distributed RDBMS does a master process coordinates all salve nodes through socket. I think that will save plenty of time if there won't be any setups and cleanups. Wha