On Feb 23, 2009, at 2:01 AM, Bing TANG wrote:
Hi, everyone, Could somdone tell me the principle of "-file" when using Hadoop Streaming. I want to ship a big file to Slaves, so how it works? Hadoop uses "SCP" to copy? How does Hadoop deal with -file option?
No, -file just copies the file from the local filesystem to HDFS, and the DistributedCache copies it to the local filesystem of the node on which the map/reduce task runs.
Arun
