Build failed in Jenkins: Hadoop-Common-trunk #499

See Changes: [eli] HDFS-3758. TestFuseDFS test failing. Contributed by Colin Patrick McCabe [tucu] HADOOP-8681. add support for HTTPS to the web UIs. (tucu) [todd] HDFS-3695. Genericize format() to non-file JournalManagers. Contrib

Checksum Error during Reduce Phase hadoop-1.0.2

Hi, I am running a Terasort with a cluster of 8 nodes.The map phase completes but when the reduce phase is around 68-70% I get this following error. * 12/08/10 11:02:36 INFO mapred.JobClient: Task Id : attempt_201208101018_0001_r_27_0, Status : FAILED java.lang.RuntimeException: problem adva

Setting number of parallel Reducers and Mappers for optimal performance

Hi, I was trying to optimize Hadoop-1.0.2 performance by setting *mapred.tasktracker.map.tasks.maximum ,**mapred.tasktracker.reduce.tasks.maximum* such that the entire memory is utilized. The tuning of this parameter is given as (CPUS > 2) ? (CPUS * 0.50): 1 for reduce and (CPUS > 2) ? (CPUS * 0.

Re: Setting number of parallel Reducers and Mappers for optimal performance

Pavan, A very important factor is how much CPU and how many spindles you have... Your proposal for memory (44G in all) seems reasonable. However, if you have 12 spindles and sufficient CPU I'd do something like 10 or 12 maps of 2G each and 6 reduces with 3G/4G each depending on how you want to

Re: Setting number of parallel Reducers and Mappers for optimal performance

Arun, Thanks a lot for your response. I am running on a 16 core Xeon processor and 12 spindles.So running 12 Mappers with 2G and 6 Reducers with 3G might give me the best performance.Also is there a general formula to arrive at those numbers? On Fri, Aug 10, 2012 at 7:34 PM, Arun C Murthy wro