[jira] [Created] (HADOOP-9084) TotalOrderPartitioner fails on hadoop running on top of gpfs (or any parallel or distributed filesystem)

giovanni delussu created HADOOP-9084: Summary: TotalOrderPartitioner fails on hadoop running on top of gpfs (or any parallel or distributed filesystem) Key: HADOOP-9084 URL: https://issues.apache.org/jira/brow

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

This predates me, so I don't know the rationale for repackaging Tomcat inside HTTPFS. I suspect that there was a desire to create a fully stand-alone distribution package, including a full web server. The Maven Jetty plugin isn't directly applicable to this use case. I don't know why it was deci

[jira] [Created] (HADOOP-9083) Port HADOOP-9020 Add a SASL PLAIN server to branch 1

Yu Gao created HADOOP-9083: -- Summary: Port HADOOP-9020 Add a SASL PLAIN server to branch 1 Key: HADOOP-9083 URL: https://issues.apache.org/jira/browse/HADOOP-9083 Project: Hadoop Common Issue Type:

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

pretty much all of the j* in JSR223 land is abomination of one sort or another, actually :) jruby is good because you can run rails application on standard Java infrastructure which is way easier to maintain, then obscure Ruby application servers.

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Dne 22.11.2012 1:14, Chris Nauroth napsal(a): The trickiest maintenance issue is hadoop-hdfs-httpfs, where we unpack and repack a Tomcat. why its not possible to just ship WAR file? Its seems to be special purpose app and they needs hand security setup anyway and intergration with existing fir

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

On Thu, Nov 22, 2012 at 12:58AM, Radim Kolar wrote: > I know that Python compatibility can be worked around. I used Python > for few years and wrote about 70k LOC in it until it started to > irritate me that every new version has incompatibilities such as 2.4 > vs 2.3 vs 2.5 and it makes maintainin

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Unfortunately, there are a couple of spots where it gets really messy and directory-wide rules fail to cover it. The trickiest maintenance issue is hadoop-hdfs-httpfs, where we unpack and repack a Tomcat. Initially, I tried to do this using only the ant plugin, but I wound up with a ton of differ

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

/Ignore Python 3 for the time being, it's a completely different language with incompatible syntax and semantics that doesn't support several currently-important platforms. Maybe in a few years sane people can consider moving to it, but for now it's best to just stick with the compatible subset

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Dne 21.11.2012 22:44, Chris Nauroth napsal(a): Sorry, to clarify my point a little more, Ant does allow you to make declarations to explicitly set the desired file permissions via the fileMode attribute of a tarfileset. there are just 2 directories /bin and /sbin with executable files. Its proba

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

On Wed, Nov 21, 2012 at 1:50 PM, Konstantin Boudnik wrote: > The main point of my argument expressed in a lesser than 100 words: adding > Python that is inconsistent across different Linux distros and has a history > of backward incompatibilities (2.6 vs 2.5, 3.0 vs earlier, etc.) doesn't seem > t

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Ditto... On Wed, Nov 21, 2012 at 01:14PM, Matt Foley wrote: > Cos, > Please see in-line. > > On Wed, Nov 21, 2012 at 12:00 PM, Konstantin Boudnik wrote: > > > I like Alejandro's idea about Maven for a few of reasons: > > - bringing in a scripting environment which is known for its > > inter

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Sorry, to clarify my point a little more, Ant does allow you to make declarations to explicitly set the desired file permissions via the fileMode attribute of a tarfileset. However, it does not have the capability to preserve whatever permissions were naturally created on files earlier in the buil

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

On Wed, Nov 21, 2012 at 09:46PM, Radim Kolar wrote: > > >Why not do a maven-plugin to do that? > maven plugins are difficult to maintain. its better to use inline > scripts, with something like this: > > http://docs.codehaus.org/display/GMAVEN/Home;jsessionid=E29093B96230BBB4461F02A1718A6B71 Exa

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Dne 21.11.2012 22:03, Chris Nauroth napsal(a): For creation of the distribution tarballs, the Maven Ant Plugin (and actually the underlying Ant tool) cannot preserve file permissions or symlinks. maven assembly plugin can deal with file permissions. not sure about symlinks. I do not remember dis

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Cos, Please see in-line. On Wed, Nov 21, 2012 at 12:00 PM, Konstantin Boudnik wrote: > I like Alejandro's idea about Maven for a few of reasons: > - bringing in a scripting environment which is known for its > inter-version > idiosyncrasies just because Windows can't handle trivial shell >

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

I worked on some of the Python build scripting that currently resides in branch-trunk-win. Initially, my goal was to keep a "pure" Maven implementation to the greatest degree possible without external scripting, but I encountered a few problems: 1. One approach is to try to express all of the bui

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Why not do a maven-plugin to do that? maven plugins are difficult to maintain. its better to use inline scripts, with something like this: http://docs.codehaus.org/display/GMAVEN/Home;jsessionid=E29093B96230BBB4461F02A1718A6B71

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

I like Alejandro's idea about Maven for a few of reasons: - bringing in a scripting environment which is known for its inter-version idiosyncrasies just because Windows can't handle trivial shell scripting looks like an overkill to me - relative to above, there's a chance that Python's

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Got it, thx. BTW, for branch-1, how about doing an ant task as part of the build that does that. Thx On Wed, Nov 21, 2012 at 11:44 AM, Matt Foley wrote: > Hi Alejandro, > For build-time issues in branch-2 and beyond, this may make sense (although > I'm concerned about obscuring functionality

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Hi Alejandro, For build-time issues in branch-2 and beyond, this may make sense (although I'm concerned about obscuring functionality in a way that only maven experts will be able to understand). In the particular case of saveVersion.sh, I'd be happy to see it done automatically by the build tools

Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Hey Matt, We already require java/mvn/protoc/cmake/forrest (forrest is hopefully on its way out with the move of docs to APT) Why not do a maven-plugin to do that? Colin already has something to simplify all the cmake calls from the builds using a maven-plugin (https://issues.apache.org/jira/bro

[jira] [Created] (HADOOP-9082) Select and document a platform-independent scripting language for use in Hadoop environment

Matt Foley created HADOOP-9082: -- Summary: Select and document a platform-independent scripting language for use in Hadoop environment Key: HADOOP-9082 URL: https://issues.apache.org/jira/browse/HADOOP-9082

[PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

This discussion started in HADOOP-8924 , where it was proposed to replace the build-time utility "saveVersion.sh" with a python script. This would require Python as a build-time dependency. Here's the background: Those of us involved in the bran

[jira] [Created] (HADOOP-9081) Port TestWinUtils from branch-1-win to branch-trunk-win

Chris Nauroth created HADOOP-9081: - Summary: Port TestWinUtils from branch-1-win to branch-trunk-win Key: HADOOP-9081 URL: https://issues.apache.org/jira/browse/HADOOP-9081 Project: Hadoop Common

[jira] [Created] (HADOOP-9080) Update bylaws to specify PMC chair voting

Eli Collins created HADOOP-9080: --- Summary: Update bylaws to specify PMC chair voting Key: HADOOP-9080 URL: https://issues.apache.org/jira/browse/HADOOP-9080 Project: Hadoop Common Issue Type: T

[jira] [Created] (HADOOP-9079) LocalDirAllocator throws ArithmeticException

Jimmy Xiang created HADOOP-9079: --- Summary: LocalDirAllocator throws ArithmeticException Key: HADOOP-9079 URL: https://issues.apache.org/jira/browse/HADOOP-9079 Project: Hadoop Common Issue Type

[jira] [Created] (HADOOP-9078) enhance unit-test coverage of class org.apache.hadoop.fs.FileContext

Ivan A. Veselovsky created HADOOP-9078: -- Summary: enhance unit-test coverage of class org.apache.hadoop.fs.FileContext Key: HADOOP-9078 URL: https://issues.apache.org/jira/browse/HADOOP-9078 Proj

[jira] [Created] (HADOOP-9077) Make streaming to output rawbytes and typed bytes only

Miguel Hernandez created HADOOP-9077: Summary: Make streaming to output rawbytes and typed bytes only Key: HADOOP-9077 URL: https://issues.apache.org/jira/browse/HADOOP-9077 Project: Hadoop Common

[jira] [Created] (HADOOP-9076) Fix QA bot in YARN project

Radim Kolar created HADOOP-9076: --- Summary: Fix QA bot in YARN project Key: HADOOP-9076 URL: https://issues.apache.org/jira/browse/HADOOP-9076 Project: Hadoop Common Issue Type: Task C

commit access to hadoop

what it takes to gain commit access to hadoop?