Hey gang!
I’d like to call a vote to run for 7 days (ending May 16 at 13:30 PT)
to merge the HADOOP-12930 feature branch into trunk. This branch was developed
exclusively by me as per the discussion two months ago as a way to make what
would be a rather large patch hopefully easier to review. The vast majority of
the branch is code movement in the same file, additional license headers, maven
assembly hooks for distribution, and variable renames. Not a whole lot of new
code, but a big diff file none-the-less.
This branch modifies the ‘hadoop’, ‘hdfs’, ‘mapred’, and ‘yarn’
commands to allow for subcommands to be added or modified at runtime. This
allows for individual users or entire sites to tweak the execution environment
to suit their local needs. For example, it has been a practice for some
locations to change the distcp jar out for a custom one. Using this
functionality, it is possible that the ‘hadoop distcp’ command could run the
local version without overwriting the bundled jar and for existing
documentation (read: results from Internet searches) to work as written without
modification. This has the potential to be a huge win, especially for:
* advanced end users looking to supplement the Apache Hadoop
experience
* operations teams that may be able to leverage existing
documentation without having to remain local “exception” docs
* development groups wanting an easy way to trial experimental
features
Additionally, this branch includes the following, related changes:
* Adds the first unit tests for the ‘hadoop’ command
* Adds the infrastructure for hdfs script testing and the first
unit test for the ‘hdfs’ command
* Modifies the hadoop-tools components to be dynamic rather
than hard coded
* Renames the shell profiles for hdfs, mapred, and yarn to be
consistent with other bundled profiles, including the ones introduced in this
branch
Documentation, including a ‘hello world’-style example, is in the
UnixShellGuide markdown file. (Of course!)
I am at ApacheCon this week if anyone wants to discuss in-depth.
Thanks!
P.S.,
There are still two open sub-tasks. These are blocked by other issues
so that we may add unit testing to the shell code in those respective areas.
I’ll covert to full issues after HADOOP-12930 is closed.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
