Doroszlai, Attila created HDDS-1877:
---------------------------------------
Summary: hadoop31-mapreduce fails due to wrong HADOOP_VERSION
Key: HDDS-1877
URL: https://issues.apache.org/jira/browse/HDDS-1877
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: test
Affects Versions: 0.5.0, 0.4.1
Reporter: Doroszlai, Attila
Assignee: Doroszlai, Attila
hadoop31-mapreduce fails with:
{noformat:title=https://elek.github.io/ozone-ci/byscane/byscane-nightly-gl52x/acceptance/smokeresult/log.html#s1-s2-t2-k2-k2}
JAR does not exist or is not a normal file:
/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar
{noformat}
because 3.1 test is being run with {{HADOOP_VERSION=3}}:
{noformat:title=https://raw.githubusercontent.com/elek/ozone-ci/master/byscane/byscane-nightly-gl52x/acceptance/output.log}
Creating network "hadoop31_default" with the default driver
Pulling nm (flokkr/hadoop:3)...
3: Pulling from flokkr/hadoop
Digest: sha256:62e3488e64ff8c0406752fc4f263ae2549e04fedf02534469913c496c6a89d78
Status: Downloaded newer image for flokkr/hadoop:3
{noformat}
which has Hadoop 3.2.0 instead of 3.1.2:
{noformat:title=docker run -it --entrypoint /bin/bash flokkr/hadoop:3 -c 'ls
-la /opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples*'}
-rw-r--r-- 1 hadoop flokkr 316570 Jan 8 2019
/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.0.jar
{noformat}
{noformat:title=docker run -it --entrypoint /bin/bash flokkr/hadoop:3.1.2 -c
'ls -la /opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples*'}
-rw-r--r-- 1 hadoop flokkr 316380 Jan 29 2019
/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar
{noformat}
This only happens with {{acceptance.sh}}, not when directly using
{{test-all.sh}}, because the former explicitly defines {{HADOOP_VERSION}}:
{noformat:title=https://github.com/apache/hadoop/blob/d4ab9aea6f9cbcdcaf48b821e5be04b4e952b133/hadoop-ozone/dev-support/checks/acceptance.sh#L19}
export HADOOP_VERSION=3
{noformat}
so the correct value from {{.env}} file is ignored:
{noformat:title=https://github.com/apache/hadoop/blob/d4ab9aea6f9cbcdcaf48b821e5be04b4e952b133/hadoop-ozone/dist/src/main/compose/ozone-mr/hadoop31/.env#L21}
HADOOP_VERSION=3.1.2
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
