Thanks all the feedbacks.
I created an issue:
https://issues.apache.org/jira/browse/HADOOP-14898
Let's continue the discussion there.
Thanks,
Marton
On 09/08/2017 02:45 PM, Marton, Elek wrote:
TL;DR: I propose to create official hadoop images and upload them to the
dockerhub.
GOAL/SCOPE: I would like improve the existing documentation with
easy-to-use docker based recipes to start hadoop clusters with various
configuration.
The images also could be used to test experimental features. For example
ozone could be tested easily with these compose file and configuration:
https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
Or even the configuration could be included in the compose file:
https://github.com/elek/hadoop/blob/docker-2.8.0/example/docker-compose.yaml
I would like to create separated example compose files for federation,
ha, metrics usage, etc. to make it easier to try out and understand the
features.
CONTEXT: There is an existing Jira
https://issues.apache.org/jira/browse/HADOOP-13397
But it’s about a tool to generate production quality docker images
(multiple types, in a flexible way). If no objections, I will create a
separated issue to create simplified docker images for rapid prototyping
and investigating new features. And register the branch to the dockerhub
to create the images automatically.
MY BACKGROUND: I am working with docker based hadoop/spark clusters
quite a while and run them succesfully in different environments
(kubernetes, docker-swarm, nomad-based scheduling, etc.) My work is
available from here: https://github.com/flokkr but they could handle
more complex use cases (eg. instrumenting java processes with btrace, or
read/reload configuration from consul).
And IMHO in the official hadoop documentation it’s better to suggest
to use official apache docker images and not external ones (which could
be changed).
Please let me know if you have any comments.
Marton
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
