>> Python as runtime requirement. Are you planing to migrate all BASH scripts provided by Hadoop (or dynamically created -ie launcher scripts) to Python?
I don't intend to mandate use of Python. Rather, I want there to be a cross-platform option available. Things that are best done in platform-specific manner, should be done in shell for linux, and powershell for windows. But things that are best done in a platform-independent way, can be, with a lower long-term maintenance cost than using different scripts per platform. This means that some, but not all, existing scripts may naturally migrate to Python as the overall system is ported to Windows. Hopefully when someone is porting a script that can be well done in a platform-independent way, they will be able to choose Python and write a single script that can replace the shell script and make it unnecessary to maintain two scripts (doing the same job but in different languages!) going forward. >> What else in the current build, besides saveVersion.sh, you see as candidate to be migrated to Python? I have a greatly improved version of src/docs/relnotes.py that I would like to submit, for auto-gen of release notes. That's all that I have on my hotlist right now, although I anticipate that some of the shell scripts invoked by ant may be natural candidates. >> How are you planning to define what Python modules can be used? Will developers have to install them manually? That's something the community will work out, the same way they decide what library jars to include, and when to upgrade those versions. But first, let's get an agreement in principle that this is the direction we want to go. Cheers, --Matt On Thu, Nov 29, 2012 at 3:26 PM, Alejandro Abdelnur <t...@cloudera.com>wrote: > Matt, thanks for the clarification. > > I may have missed the main point of the PROPOSAL thread then. I personally > want to continue the discussion before voting. > > * Phyton as runtime requirement. Are you planing to migrate all BASH > scripts provided by Hadoop (or dynamically created -ie launcher scripts) > to Phyton? > * What else in the current build, besides saveVersion.sh, you see as > candidate to be migrated to Phyton? > * How are you planning to define what Phyton modules can be used? Will > developers have to install them manually? > > Cheers > > > On Thu, Nov 29, 2012 at 2:39 PM, Matt Foley <mfo...@hortonworks.com> > wrote: > > > Hi Alejandro, > > Please see in-line below. > > > > On Mon, Nov 26, 2012 at 1:52 PM, Alejandro Abdelnur <t...@cloudera.com> > > wrote: > > > > > Matt, > > > > > > The scope of this vote seems different from what was discussed in the > > > PROPOSAL thread. > > > In the PROPOSAL thread you indicated this was for Hadoop1 because it is > > ANT > > > based. And the main reason was to remove saveVersion.sh. > > > Your #3 was not discussed in the proposal, was it? > > > > > > > The item #3 was in my original statement of the problem, with which I > > started the proposal thread. In fact, the thread title was "[PROPOSAL] > > introduce Python as build-time and run-time dependency for Hadoop and > > throughout Hadoop stack". It is true that only one or two people chose > to > > discuss #3 further in that thread. > > > > The point is not just to replace a single script, but to provide a means > to > > do cross-platform scripts, which will over time replace many > > non-platform-specific scripts written in platform-specific languages. > > > > > > > > > > It seems this vote is dragging much more stuff it was originally > > discussed. > > > I think you should suspend the vote, recap the motivation and then > > restart > > > the vote. > > > > > > > I respectfully disagree. I believe a careful reading of the cited > > discussion thread, plus my own statement of the vote, provides sufficient > > background for a thoughtful decision on the subject. Presumably so do > the > > ten other people who had already voted before you made that comment. > > > > If several other people want more discussion first, please speak up. > > Thanks, > > --Matt > > > > As things are laid out at the moment my vote is: > > > > > > -1 (It still seems an overkill to introduce a new runtime requirement > for > > > building to replace a script.) > > > +1 (I think this is the right way to simplify the build) > > > -1 (AFAIK there is not such requirement at the moment, and if it comes > it > > > would be in the form of an AM, which I'd argue it should leave outside > of > > > Hadoop) > > > > > > Thx > > > > > > > > > On Mon, Nov 26, 2012 at 1:16 PM, Giridharan Kesavan < > > > gkesa...@hortonworks.com> wrote: > > > > > > > +1, +1, +1 > > > > > > > > -Giri > > > > > > > > > > > > On Sat, Nov 24, 2012 at 12:13 PM, Matt Foley <ma...@apache.org> > wrote: > > > > > > > > > For discussion, please see previous thread "[PROPOSAL] introduce > > Python > > > > as > > > > > build-time and run-time dependency for Hadoop and throughout Hadoop > > > > stack". > > > > > > > > > > This vote consists of three separate items: > > > > > > > > > > 1. Contributors shall be allowed to use Python as a > > > platform-independent > > > > > scripting language for build-time tasks, and add Python as a > > build-time > > > > > dependency. > > > > > Please vote +1, 0, -1. > > > > > > > > > > 2. Contributors shall be encouraged to use Maven tasks in > combination > > > > with > > > > > either plug-ins or Groovy scripts to do cross-platform build-time > > > tasks, > > > > > even under ant in Hadoop-1. > > > > > Please vote +1, 0, -1. > > > > > > > > > > 3. Contributors shall be allowed to use Python as a > > > platform-independent > > > > > scripting language for run-time tasks, and add Python as a run-time > > > > > dependency. > > > > > Please vote +1, 0, -1. > > > > > > > > > > Note that voting -1 on #1 and +1 on #2 essentially REQUIRES > > > contributors > > > > to > > > > > use Maven plug-ins or Groovy as the only means of cross-platform > > > > build-time > > > > > tasks, or to simply continue using platform-dependent scripts as is > > > being > > > > > done today. > > > > > > > > > > Vote closes at 12:30pm PST on Saturday 1 December. > > > > > --------- > > > > > Personally, my vote is +1, +1, +1. > > > > > I think #2 is preferable to #1, but still has many unknowns in it, > > and > > > > > until those are worked out I don't want to delay moving to > > > cross-platform > > > > > scripts for build-time tasks. > > > > > > > > > > Best regards, > > > > > --Matt > > > > > > > > > > > > > > > > > > > > > -- > > > Alejandro > > > > > > > > > -- > Alejandro >
