Hadoop conf/ servlet improvements
---------------------------------
Key: HADOOP-6610
URL: https://issues.apache.org/jira/browse/HADOOP-6610
Project: Hadoop Common
Issue Type: Improvement
Components: conf
Affects Versions: 0.22.0
Reporter: Steve Loughran
Priority: Minor
I'm playing with the conf/ servlet, trying to do a workflow that
# pulls down the conf servlet from a well known URL (this is trickier when your
VMs are dynamic, but possible)
# saves it locally, using {{<get>}} task
# {{<get>}} some info on the machines in the allocated cluster, like their
external hostnames
# SCP in the configuration files, JAR files needed to submit work,
# submit work via SSH
I have to SSH as the VMs have different internal/external addresses; HDFS gets
upset.
Some issues I've found so far
# It's good to set expires headers on everything; HADOOP-6607 covers that
# Having sorted conf values makes it easier to locate properties, otherwise you
have to save it to a text editor and search around
# the <!-- Loaded from Unknown --> option makes things noisy
# Saving as a java.util.Properties would let me pull these things into a build
file or other tool very easily. This is easy to test too.
# Have a comment at the top listing when the conf was generated, and the
hostname. Maybe even make them conf values
More tricky is the conf options that are dynamic, things like
{code}
<property><!--Loaded from
Unknown--><name>dfs.datanode.address</name><value>0.0.0.0:0</value></property>
{code}
These show what the node was started with, not what it actually got. I am doing
a workaround there with my code (setting the actual values in the conf file
with {{live.dfs.datanode.address}}, etc, and extracting them that way. I don't
want to lose the original values, but do want the real ones
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
