Hi Hadoop developers
The other day I created a tool for launching multiple node hadoop cluster on
docker container.
You can easily launch multiple node hadoop cluster from your Hadoop source
code.
It is useful for testing and debugging. Actually I often use it before
submitting a patch to Hadoop project.
https://github.com/Lewuathe/docker-hadoop-cluster
<https://github.com/Lewuathe/docker-hadoop-cluster>
And I also updated to build the latest trunk image automatically and upload
onto Docker Hub.
So you can easily check and test the latest trunk branch in the environment
which is more close to actual usage.
If you already installed docker and docker-compose, what needed is
docker-compose.yml like this.
version: '2'
services:
master:
image: lewuathe/hadoop-master
ports:
- "9870:9870"
- "8088:8088"
- "19888:19888"
- "8188:8188"
container_name: "master"
slave1:
image: lewuathe/hadoop-slave
container_name: "slave1"
depends_on:
- master
ports:
- "9901:9864"
- "8041:8042"
slave2:
image: lewuathe/hadoop-slave
container_name: "slave2"
depends_on:
- master
ports:
- "9902:9864"
- "8042:8042"
The usage in detail is described in the repository.
https://github.com/Lewuathe/docker-hadoop-cluster/blob/master/README.md
<https://github.com/Lewuathe/docker-hadoop-cluster/blob/master/README.md>
I would be glad if you use this tool for developing and debugging and make our
development more efficient.
Please give me any feedbacks to me. Thanks you!
————————
Kai Sasaki
mail: [email protected] <mailto:[email protected]>
github: https://github.com/Lewuathe <https://github.com/Lewuathe>
