RE: last map task taking too long

Hi The input is a plain text file. I use the parameters specified in the input file to launch a process on each machine and then collect the results back. I am not using cached files. Everything needed is contained in the job jar file. Each map task is supposed to finish within one minute. Here

RE: last map task taking too long

Hi, Can you provide info on the input like compression etc? Also, are you using cached files in your map tasks? It might be helpful if you paste the logs here after blanking your system specific info., as then one can find out where till the reduce it went or if the copy phase started at all. T

Re: last map task taking too long

Basically, it finishes what it is supposed to do (I view the logs to find out), but does not move onto the reduce stage. Ted Dunning wrote: > > Is that last map task actually running, or is it pending? > > On Tue, Sep 29, 2009 at 5:57 PM, achilles852 wrote: > >> >> Hey.. I am trying to write

Re: last map task taking too long

Is that last map task actually running, or is it pending? On Tue, Sep 29, 2009 at 5:57 PM, achilles852 wrote: > > Hey.. I am trying to write a small mapreduce program. I launch a few map > tasks, each of which should complete within a certain time (say 5 > minutes)... all the tasks complete with

last map task taking too long

Hey.. I am trying to write a small mapreduce program. I launch a few map tasks, each of which should complete within a certain time (say 5 minutes)... all the tasks complete within 5 minutes except the last one - which takes around 10 times more the time taken by all other map tasks.any idea w

Re: Developing Hadoop and HDFS

Thanks for your interest, Geoff. Yes, finding open JIRAS and contributing patches is very helpful. We also maintain a wishlist of projects that one could work on: http://wiki.apache.org/hadoop/ProjectSuggestions. In addition, please do consider documentation and example work as well, as this

Developing Hadoop and HDFS

Hello, Yes, another person looking to contribute to and develop Hadoop. I'm looking to start off small, fixing a few bugs before moving into larger stuff. First, a bit of background: Years ago I had the idea of creating a semi-decentralized distributed file system. The idea came when I was workin

[jira] Created: (HADOOP-6291) Confusing warn message from Configuration

Confusing warn message from Configuration - Key: HADOOP-6291 URL: https://issues.apache.org/jira/browse/HADOOP-6291 Project: Hadoop Common Issue Type: Bug Components: conf Affects Version

[jira] Resolved: (HADOOP-6284) Any hadoop commands crashing jvm (SIGBUS) when /tmp (tmpfs) is full

[ https://issues.apache.org/jira/browse/HADOOP-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE resolved HADOOP-6284. Resolution: Fixed I have committed this. Thanks, Koji! > Any hadoop comm

[jira] Created: (HADOOP-6290) AutoInputFormat + (larger) bzip2 files cause multiple runs over same file

AutoInputFormat + (larger) bzip2 files cause multiple runs over same file - Key: HADOOP-6290 URL: https://issues.apache.org/jira/browse/HADOOP-6290 Project: Hadoop Common