RE: Question about invoking an executable from Hadoop mapper

2011-08-25 Thread Zhixuan Zhu
A shell script actually worked great! Thanks so much! Grace -Original Message- From: Robert Evans [mailto:ev...@yahoo-inc.com] Sent: Thursday, August 25, 2011 11:32 AM To: common-dev@hadoop.apache.org Subject: Re: Question about invoking an executable from Hadoop mapper I think this is

Re: Question about invoking an executable from Hadoop mapper

2011-08-25 Thread Robert Evans
I think this is a java issue. I don't think that it is launching a shell to run your command. I think it is just splitting on white space and then passing all the args to hadoop. What you want to do is to run sh -e 'hadoop dfs -cat file| myExec' Or with streaming white a small shell script t

Question about invoking an executable from Hadoop mapper

2011-08-25 Thread Zhixuan Zhu
I think this should be a common use case. I'm trying to invoke an executable from my mapper. Because my executable takes steaming input, I used something like the following: String lCmdStr = "hadoop dfs -cat file | myExec"; Process lChldProc = Runtime.getRuntime().exec(lCmdStr); All executable a