Re: Hadoop + MPI

On Nov 21, 2011, at 5:04 PM, wrote: > Ralph, > > Yes, I have completed the first step, although I would really like that > code to be part of the MPI Application Master (Chris Douglas suggested a > way to do this at ApacheCon). > > Regarding the remaining steps, I have been following discussio

Re: Hadoop + MPI

Ralph, Yes, I have completed the first step, although I would really like that code to be part of the MPI Application Master (Chris Douglas suggested a way to do this at ApacheCon). Regarding the remaining steps, I have been following discussions on the open mpi mailing lists, and reading code fo

Re: Hadoop + MPI

Hi Milind Glad to hear of the progress - I recall our earlier conversation. I gather you have completed step 1 (wireup) - have you given any thought to the other two steps? Anything I can do to help? Ralph On Nov 21, 2011, at 4:47 PM, wrote: > Hi Ralph, > > I spoke with Jeff Squyres at SC

Re: Hadoop + MPI

Milind, Great news. Any chance you can upload a patch as it is? I am sure, others can help cleaning it up. I am willing to help smoothen it out and am sure Ralph can provide feedback as well. thanks mahadev On Mon, Nov 21, 2011 at 3:47 PM, wrote: > Hi Ralph, > > I spoke with Jeff Squyres  at

Re: Hadoop + MPI

Hi Ralph, I spoke with Jeff Squyres at SC11, and updated him on the status of my OpenMPI port on Hadoop Yarn. To update everyone, I have OpenMPI examples running on #Yarn, although it requires some code cleanup and refactoring, however that can be done as a later step. Currently, the MPI proces

Re: Hadoop + MPI

On Nov 21, 2011, at 4:43 PM, Arun C Murthy wrote: > Hi Ralph, > > Welcome! > > We'd absolutely love to have OpenMPI integrated with Hadoop! > > In fact, there already has been a bunch of discussions running OpenMPI on > what we call MR2 (aka YARN), documented here: > https://issues.apache.or

Re: Hadoop + MPI

Hi Ralph, Welcome! We'd absolutely love to have OpenMPI integrated with Hadoop! In fact, there already has been a bunch of discussions running OpenMPI on what we call MR2 (aka YARN), documented here: https://issues.apache.org/jira/browse/MAPREDUCE-2911. YARN is our effort to re-imagine Ha

Hadoop + MPI

Hi folks I am a lead developer in the Open MPI community, mostly focused on integrating that package with various environments. Over the last few months, I've had a couple of people ask me about MPI support within Hadoop - i.e., they want to run MPI applications under the Hadoop umbrella. I've

[jira] [Created] (HADOOP-7850) hadoop rpm does not create the appropriate link for the native files

hadoop rpm does not create the appropriate link for the native files Key: HADOOP-7850 URL: https://issues.apache.org/jira/browse/HADOOP-7850 Project: Hadoop Common Issue Ty

[jira] [Created] (HADOOP-7849) hadoop rpm overwrites the existing /etc/hadoop/hadoop-env.sh

hadoop rpm overwrites the existing /etc/hadoop/hadoop-env.sh Key: HADOOP-7849 URL: https://issues.apache.org/jira/browse/HADOOP-7849 Project: Hadoop Common Issue Type: Bug Affe

[jira] [Created] (HADOOP-7848) hadoop rpm does not set the correct ownership and permission for the task-controller

hadoop rpm does not set the correct ownership and permission for the task-controller Key: HADOOP-7848 URL: https://issues.apache.org/jira/browse/HADOOP-7848 Project:

[jira] [Created] (HADOOP-7847) There are two findbugs warnings on branch-20-security

There are two findbugs warnings on branch-20-security - Key: HADOOP-7847 URL: https://issues.apache.org/jira/browse/HADOOP-7847 Project: Hadoop Common Issue Type: Bug Affects Versions:

[jira] [Created] (HADOOP-7846) download pages for mapred/hdfs don't match hadoop-common

download pages for mapred/hdfs don't match hadoop-common Key: HADOOP-7846 URL: https://issues.apache.org/jira/browse/HADOOP-7846 Project: Hadoop Common Issue Type: Bug Compon

[jira] [Created] (HADOOP-7845) Link to downloads page from hadoop.apache.org

Link to downloads page from hadoop.apache.org - Key: HADOOP-7845 URL: https://issues.apache.org/jira/browse/HADOOP-7845 Project: Hadoop Common Issue Type: Improvement Components: document

[jira] [Resolved] (HADOOP-7063) ChecksumFileSystem should implement flush()

[ https://issues.apache.org/jira/browse/HADOOP-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G resolved HADOOP-7063. - Resolution: Duplicate > ChecksumFileSystem should implement flush() > --

Re: Dropping CHANGES.txt?

On Nov 21, 2011, at 12:13 PM, Doug Cutting wrote: > On 11/21/2011 11:49 AM, Matt Foley wrote: >> The advantage of CHANGES.txt is that it merges when the branch merges. So >> as long as the developers stick to reasonably normal merge and commit >> processes, it truly reflects what is in the curre

Re: Dropping CHANGES.txt?

On 11/21/2011 11:49 AM, Matt Foley wrote: > The advantage of CHANGES.txt is that it merges when the branch merges. So > as long as the developers stick to reasonably normal merge and commit > processes, it truly reflects what is in the current state of any given > branch, at least at the level of

RE: Dropping CHANGES.txt?

It sounds like CHANGES.txt has been helping to resolve issues for Matt, and taking it away will open the floodgates. I'm not familiar enough with the Jira procedures yet, but my usual expectation is that there will be some review between a developer marking the resolution "resolved" and (don't

Re: Dropping CHANGES.txt?

Hi Todd, As you know, one of the tasks of a release manager is to gather a correct and complete statement of all changes in a release, and include them in the Release Notes. As I've done the release management for 0.20-security, I've had the fun of seeing all the deficiencies of both CHANGES.txt a

Re: Dropping CHANGES.txt?

On Mon, Nov 21, 2011 at 11:01 AM, Todd Lipcon wrote: > I was thinking... our use of CHANGES.txt is a bit silly - we basically > maintain all of our changes in three places (a) JIRA fixVersion field, > (b) CHANGES.txt, and (c) the SVN changelog. It seems to me that we get > very little value out of

[jira] [Resolved] (HADOOP-7796) HADOOP-7773 introduced 7 new findbugs warnings

[ https://issues.apache.org/jira/browse/HADOOP-7796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HADOOP-7796. - Resolution: Duplicate Fix Version/s: 0.24.0 Target Version/s: 0.23.0, 0.

Dropping CHANGES.txt?

I was thinking... our use of CHANGES.txt is a bit silly - we basically maintain all of our changes in three places (a) JIRA fixVersion field, (b) CHANGES.txt, and (c) the SVN changelog. It seems to me that we get very little value out of CHANGES.txt -- the contents are way too large (and terse) for

Re: Not able to compile 0.23 - may be HADOOP-6614

Sorry about that guys, I messed up the merge of HADOOP-6614 from trunk. On Mon, Nov 21, 2011 at 9:47 AM, Harsh J wrote: > Ah my bad. Yes, branch-0.23 is separated. I was compiling trunk and noticed > this to be a non issue :) > > On 21-Nov-2011, at 10:18 PM, Praveen Sripati wrote: > >> Harsh, >>

Re: Not able to compile 0.23 - may be HADOOP-6614

Ah my bad. Yes, branch-0.23 is separated. I was compiling trunk and noticed this to be a non issue :) On 21-Nov-2011, at 10:18 PM, Praveen Sripati wrote: > Harsh, > > Branch-0.23 has "File workDir;" while the trunk has "final File workDir;" > in RunJar.java. > > For the 0.23 build, should I ge

Re: Not able to compile 0.23 - may be HADOOP-6614

Praveen, There is a fix being worked on in HADOOP-7843. Hope that helps. thanks mahadev On Mon, Nov 21, 2011 at 8:48 AM, Praveen Sripati wrote: > Harsh, > > Branch-0.23 has "File workDir;" while the trunk has "final File workDir;" > in RunJar.java. > > For the 0.23 build, should I get the code

Re: Not able to compile 0.23 - may be HADOOP-6614

HADOOP-7843 is filed to address the issue branch-0.23 On Mon, Nov 21, 2011 at 8:52 AM, Praveen Sripati wrote: > Hi, > > I got the latest code from branch-0.23 and got the below error. Think this > is related to HADOOP-6614. > > [ERROR] > > /home

[jira] [Created] (HADOOP-7843) compilation failing because workDir not initialized in RunJar.java

compilation failing because workDir not initialized in RunJar.java -- Key: HADOOP-7843 URL: https://issues.apache.org/jira/browse/HADOOP-7843 Project: Hadoop Common Issue Type:

Re: Not able to compile 0.23 - may be HADOOP-6614

Harsh, Branch-0.23 has "File workDir;" while the trunk has "final File workDir;" in RunJar.java. For the 0.23 build, should I get the code from branch-0.23 or trunk? I thought that the 0.23 release was build from branch-0.23. Thanks, Praveen On Mon, Nov 21, 2011 at 8:58 PM, Harsh J wrote: > T

Re: Not able to compile 0.23 - may be HADOOP-6614

This appears to have already been fixed since a7654ba4324a1cc831fcef3988d9dca767bcdef3. I can compile the trunk right now -- try a fetch+rebase/pull/up again perhaps? But yes, that value has to be final as the compiler noted. On 21-Nov-2011, at 8:22 PM, Praveen Sripati wrote: > Hi, > > I got

Not able to compile 0.23 - may be HADOOP-6614

Hi, I got the latest code from branch-0.23 and got the below error. Think this is related to HADOOP-6614. [ERROR] /home/praveensripati/Hadoop/0.23/source/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/RunJar.java:[171,31] local variable workDir is accessed from within in

Build failed in Jenkins: Hadoop-Common-0.23-Build #72

See Changes: [mattf] CHANGES.txt formatting cleanup [eli] HADOOP-6614. svn merge -c 1204388 from trunk [eli] HADOOP-7783. svn merge -c 1204376 from trunk [eli] HDFS-2514. svn merge -c 1204370 from trunk [todd] MAPREDUCE-3437.

[jira] [Created] (HADOOP-7842) a compress tool for test compress/decompress locally use hadoop compress codecs

a compress tool for test compress/decompress locally use hadoop compress codecs --- Key: HADOOP-7842 URL: https://issues.apache.org/jira/browse/HADOOP-7842 Project: Hadoop Com