Build failed in Jenkins: Hadoop-Common-trunk #136

2011-08-23 Thread Apache Jenkins Server
See Changes: [acmurthy] MAPREDUCE-2649. Handling of finished applications in RM. Contributed by Thomas Graves. [szetszwo] HDFS-2266. Add Namesystem and SafeMode interfaces to avoid directly referring to FSNamesystem in BlockManag

Re: how to pass a hdfs file to a c++ process

2011-08-23 Thread Robert Evans
Hadoop streaming is the simplest way to do this, if you program is set up to take stdin as its input, write to stdout for the output, and each record "file" in your case is a single line of text. You need to be able to have it work with the following shell script Hadoop fs -cat | head -1 | ./m

Re: how to pass a hdfs file to a c++ process

2011-08-23 Thread Arun C Murthy
On Aug 22, 2011, at 12:57 PM, Zhixuan Zhu wrote: > Hi All, > > I'm using hadoop-0.20.2 to try out some simple tasks. I asked a question > about FileInputFormat a few days ago and get some prompt replys from > this forum and it helped a lot. Thanks again! Now I have another > question. I'm trying

RE: how to pass a hdfs file to a c++ process

2011-08-23 Thread Zhixuan Zhu
I'll actually invoke one executable from each of my map. Because this C++ program has been implemented and used in the past, I just want to integrate it to our Hadoop map/reduce without having to re-implement the process in java. So my map is going to be very simple with just calling the process an

Re: how to pass a hdfs file to a c++ process

2011-08-23 Thread Arun Murthy
That is a normal use case. I'd encourage you to use Java MR (even pig/hive). If you really want to use your legacy app use streaming with a map cmd such as 'hadoop fs -cat | mylegacyexe' Arun Sent from my iPhone On Aug 23, 2011, at 8:00 AM, Zhixuan Zhu wrote: > I'll actually invoke one exec

RE: how to pass a hdfs file to a c++ process

2011-08-23 Thread Zhixuan Zhu
Thank you very much! 'hadoop fs -cat | mylegacyexe' is exactly the kind of method I came up with and was going to try it out. I'm glad to hear that it's actually an "official" alternative. Thanks again. This is a great forum! Grace -Original Message- From: Arun Murthy [mailto:a...@hor

[jira] [Created] (HADOOP-7573) hadoop should log configuration reads

2011-08-23 Thread Ari Rabkin (JIRA)
hadoop should log configuration reads - Key: HADOOP-7573 URL: https://issues.apache.org/jira/browse/HADOOP-7573 Project: Hadoop Common Issue Type: Improvement Components: conf Affects Version

[jira] [Created] (HADOOP-7574) Improvement for FSshell -stat

2011-08-23 Thread XieXianshan (JIRA)
Improvement for FSshell -stat - Key: HADOOP-7574 URL: https://issues.apache.org/jira/browse/HADOOP-7574 Project: Hadoop Common Issue Type: Improvement Components: fs Affects Versions: 0.23.0