Alex Holmes created HADOOP-10820:
------------------------------------
Summary: Empty entry in libjars results in working directory being
recursively localized
Key: HADOOP-10820
URL: https://issues.apache.org/jira/browse/HADOOP-10820
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Alex Holmes
Priority: Minor
An empty token (e.g. "a.jar,,b.jar") in the -libjars option causes the current
working directory to be recursively localized.
Here's an example of this in action (using Hadoop 2.2.0):
{code}
# create a temp directory and touch three JAR files
mkdir -p tmp/path && cd tmp && touch a.jar b.jar c.jar path/d.jar
# Run an example job only specifying two of the JARs.
# Include an empty entry in libjars.
hadoop jar
/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar pi
-libjars a.jar,,c.jar 2 1000000000
# As the job is running examine the localized directory in HDFS.
# Notice that not only are the two JAR's specified in libjars copied,
# but in addition the contents of the working directory are also recursively
copied.
$ hadoop fs -lsr
/tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars
/tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/a.jar
/tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/c.jar
/tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp
/tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp/a.jar
/tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp/b.jar
/tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp/c.jar
/tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp/path
/tmp/hadoop-yarn/staging/aholmes/.staging/job_1404752711144_0018/libjars/tmp/path/d.jar
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
