Re: [DISCUSS] hadoop-thirdparty 1.0.0 release

2020-02-22 Thread Tsuyoshi Ozawa
Thanks Viany for starting discussion. +1 hadoop-thirdparty's release is required for 3.3.0 release because the package includes shaded protobuf which 3.3.0 branch will depend on. The plan sounds reasonable to me. - Tsuyoshi On Sat, Feb 22, 2020 at 6:31 AM Ayush Saxena wrote: > > Thanx Vinay fo

Re: [DISCUSS] making Ozone a separate Apache project

2020-05-14 Thread Tsuyoshi Ozawa
+1 Thanks - Tsuyoshi On Thu, May 14, 2020 at 19:42 Rakesh Radhakrishnan wrote: > +1 > > Thanks, > Rakesh > > On Wed, May 13, 2020 at 1:23 PM Elek, Marton wrote: > > > > > > > I would like to start a discussion to make a separate Apache project for > > Ozone > > > > > > > > ### HISTORY [1] > >

[DISCUSS] About the details of JDK-8 support

2015-10-05 Thread Tsuyoshi Ozawa
Hi commiters and users of Hadoop stack, I’ll share the current status of JDK-8 support here. We can take a two-step approach to support JDK-8 - runtime-level support and source-level support. About runtime-level support, I’ve tested Hadoop stack with JDK-8 e.g. MapReduce, Spark, Tez, Flink on YA

Re: [DISCUSS] About the details of JDK-8 support

2015-10-05 Thread Tsuyoshi Ozawa
Tue, Oct 6, 2015 at 9:35 AM, Tsuyoshi Ozawa wrote: > Hi commiters and users of Hadoop stack, > > I’ll share the current status of JDK-8 support here. We can take a > two-step approach to support JDK-8 - runtime-level support and > source-level support. > > About runtime-level su

Re: [DISCUSS] About the details of JDK-8 support

2015-10-13 Thread Tsuyoshi Ozawa
the > sentiment that we should preserve the java 7 support on branch-2 (not not > move to java 8 source level) but can consider it for trunk. My 2 cents. > > Thanks, > Sangjin > > On Fri, Oct 9, 2015 at 10:43 AM, Steve Loughran > wrote: > >> >> > On 7 Oc

Re: [DISCUSS] About the details of JDK-8 support

2015-10-15 Thread Tsuyoshi Ozawa
> We should target source-level support of JDK 8 too, around which you outlined > a bunch of issues around dependencies. I also found a bunch of issues around > generating documentation, site etc. I propose that we track them under the > umbrella JIRA and make progress there first. OK. I will s

Re: Planning for Apache Hadoop 2.6.2

2015-10-21 Thread Tsuyoshi Ozawa
Thanks for your notification, Sangjin. I'll backport YARN-3798 soon. - Tsuyoshi On Wed, Oct 21, 2015 at 10:09 AM, Sangjin Lee wrote: > If you have backported bugfixes to 2.7.2, please take a moment to consider > if it is relevant (and important) for 2.6.x too, and if so backport it to > branch-2

Re: Planning for Apache Hadoop 2.6.2

2015-10-21 Thread Tsuyoshi Ozawa
Done. - Tsuyoshi On Wed, Oct 21, 2015 at 8:53 PM, Tsuyoshi Ozawa wrote: > Thanks for your notification, Sangjin. I'll backport YARN-3798 soon. > > - Tsuyoshi > > On Wed, Oct 21, 2015 at 10:09 AM, Sangjin Lee wrote: >> If you have backported bugfixes to 2.7.2, please

Re: Java 8 + Jersey updates

2015-10-26 Thread Tsuyoshi Ozawa
Hi Steve, Thanks for your help. > 2. it's "significant" This change includes upgrading not only Jersey, but also its dependencies like grizzly, asm, and so on. > I'll try to rebuild a YARN app (slider) with the patch to see how it fares It helps us a lot. I'd like to suggest that the incompat

Re: Java 8 + Jersey updates

2015-10-26 Thread Tsuyoshi Ozawa
targetting this only at trunk / > 3.0 based on the "target version" and the incompatibility discussion? > > best, > Colin > > On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa wrote: > >> Hi Steve, >> >> Thanks for your help. >> >> >

Re: 2.7.2 release plan

2015-10-27 Thread Tsuyoshi Ozawa
same risk of regressions or > > incompatibilities as code changes. > > > > --Chris Nauroth > > > > > > > > > > On 7/16/15, 12:38 AM, "Tsuyoshi Ozawa" > > wrote: > > > >> Hi, > >> > >> thank you for startin

Re: 2.7.2 release plan

2015-10-28 Thread Tsuyoshi Ozawa
nk we need HDFS-8950 and HDFS-7725 in 2.7.2.It should be easy to > backport/cherry-pick HDFS-7725. For HDFS-8950, it will be nice if Ming can > chime in. > Kihwal > > From: Tsuyoshi Ozawa > To: "common-...@hadoop.apache.org" > Cc: Chris Nauroth ; "yarn-.

Re: 2.7.2 release plan

2015-10-28 Thread Tsuyoshi Ozawa
the support in Mapreduce for labels in >> MAPREDUCE-6304, >> >> Regards, >> + Naga >> >> From: Tsuyoshi Ozawa [oz...@apache.org] >> Sent: Wednesday, October 28, 2015 14:28 >> To: yarn-...@hadoop.apache.org >> Cc: hdfs-dev@hadoop.apache.org; common-...@

Re: Update on potential Gerrit integration

2015-11-10 Thread Tsuyoshi Ozawa
Thanks, Zhe. It helps us to decide which tools we should use. - Tsuyoshi On Tuesday, 10 November 2015, Zhe Zhang wrote: > Hi, > > Per our earlier discussion on the "Github integration for Hadoop" thread, I > surveyed HBase, Flume, and HTrace about their interest on Gerrit. The > feedbacks have

Re: Jenkins stability and patching

2015-11-22 Thread Tsuyoshi Ozawa
Thank you for starting discussion, Steve. It sounds good to me. I'll check the test failures. - Tsuyoshi On Sun, Nov 22, 2015 at 9:21 PM, Steve Loughran wrote: > > Jenkins is pretty much dead in the water these days; a test run that works is > a rare miracle rather than the default state. Whic

Re: [VOTE] Release Apache Hadoop 2.6.3 RC0

2015-12-12 Thread Tsuyoshi Ozawa
Hi Junping, Thank you for starting the voting. I cannot access the tar.gz file because of permission error. Could you check the permission to access the files? > Forbidden > You don't have permission to access > /~junping_du/hadoop-2.6.3-RC0/hadoop-2.6.3-RC0-src.tar.gz on this server. Thanks, -

Re: [VOTE] Release Apache Hadoop 2.6.3 RC0

2015-12-16 Thread Tsuyoshi Ozawa
+1 (binding) - verified checksums - built Spark and Tez with -Dhadoop.version=2.6.3. About Tez, ran tests and all of them passed. Thanks, - Tsuyoshi On Thu, Dec 17, 2015 at 1:30 AM, Sangjin Lee wrote: > +1 (non-binding) > > - downloaded source and binary and verified the signatures (although I

Re: [Important] Lots of 2.8.0 commits are in branch-2 only

2015-12-16 Thread Tsuyoshi Ozawa
Thanks Wangda for reporting issue. I'm also +1 for recreating a new branch-2.8 based on current branch-2.8 too. After that, we can revert 2.9.0-only patches on the branch. I will help you if necessary, so please let me know. Thanks, - Tsuyoshi On Thu, Dec 17, 2015 at 10:51 AM, Jing Zhao wrote: >

Re: [VOTE] Release Apache Hadoop 2.7.2 RC1

2015-12-21 Thread Tsuyoshi Ozawa
+1 - downloaded src and bin tar balls and verified signatures. - built Tez and Spark with 2.7.2 artifacts and JDK7. - ran tests of Tez with 2.7.2 artifacts, it passed. FYI: YARN-4348, reported by Jian, is one of critical issues of 2.7.2 release.It's better to release 2.7.3 as soon as possible aft

Re: [VOTE] Release Apache Hadoop 2.7.2 RC1

2015-12-23 Thread Tsuyoshi Ozawa
Hi Vinod, thank you for the clarification. > - Pull these 16 tickets into 2.7.2 and roll a new RC > > What do people think? Do folks expect “any fix in 2.6.3 to be there in all > > releases that get out after 2.6.3 release date (December 16th)”? I personally prefer to pull these tickets into 2

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-07-02 Thread Tsuyoshi OZAWA
Hi Arun, I think MAPREDUCE-5221 should be merged because it's a missing feature from MRv1. Thanks, Tsuyoshi On Wed, Jul 3, 2013 at 9:10 AM, Ramya Sunil wrote: > -1. > Some of the cli and distcp system tests which use hftp:// and webhdfs:// > are failing on secure cluster (HDFS-4841 and HDFS-495

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-07-03 Thread Tsuyoshi OZAWA
Hi Arun, Some bug fixes about MapReduce should be included in next release. MAPREDUCE-5221 should be merged because it's a lost feature of MRv2 compared to MRv1. MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be merged immediately. # I sent the previous email only to hdfs-dev

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-08-05 Thread Tsuyoshi OZAWA
+1 (non-binding) * verified md5 of the source code. * build the package from source code. * run basic hdfs commands. * run MapReduce example jobs(random text writer, wordcount, and pi) over YARN in single node. - Tsuyoshi On Tue, Jul 30, 2013 at 10:29 PM, Arun C Murthy wrote: > Folks, > > I've

Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845

2013-08-08 Thread Tsuyoshi OZAWA
Hi, About Hadoop, Harsh is dealing with this problem in HADOOP-9346. For more detail, please see the JIRA ticket: https://issues.apache.org/jira/browse/HADOOP-9346 - Tsuyoshi On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur wrote: > I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release.

Re: [ACTION NEEDED]: protoc 2.5.0 in trunk/branch-2/branch-2.1-beta/branch-2.1.0-beta

2013-08-15 Thread Tsuyoshi OZAWA
Thanks for sharing! We also need to update Wiki or some documents, don't we? http://wiki.apache.org/hadoop/HowToContribute On Thu, Aug 15, 2013 at 8:03 AM, Alejandro Abdelnur wrote: > Following up on this. > > HADOOP-9845 & HADOOP-9872 have been committed > to trunk/branch-2/branch-2.1-beta/branc

Re: [VOTE] Release Apache Hadoop 2.2.0

2013-10-13 Thread Tsuyoshi OZAWA
+1 (non-binding) - Verified md5 checksums and signature. - Build from source, and run some example MR jobs in single node setup. On Sun, Oct 13, 2013 at 7:42 PM, Siddharth Seth wrote: > +1 (binding) > > Verified checksums and signature. Built from source and ran some simple MR > and Tez jobs. >

Re: [VOTE] Release Apache Hadoop 2.2.0

2013-10-15 Thread Tsuyoshi OZAWA
Congrats! On Tue, Oct 15, 2013 at 9:45 PM, Mattmann, Chris A (398J) wrote: > Congrats bros! > > ++ > Chris Mattmann, Ph.D. > Senior Computer Scientist > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 171-266B, Mailst

Re: [VOTE] Release Apache Hadoop 2.2.0

2013-10-16 Thread Tsuyoshi OZAWA
I don't have Windows environment, but one user who tried 2.2.0 release on Windows reported that released tar ball doesn't contain "winutil.exe" and cannot run any commands. Thanks, Tsuyoshi On Tue, Oct 15, 2013 at 10:38 PM, Tsuyoshi OZAWA wrote: > Congrats! > > On

Re: [VOTE] Release Apache Hadoop 2.2.0

2013-10-16 Thread Tsuyoshi OZAWA
Filed the problems as https://issues.apache.org/jira/browse/HADOOP-10051. On Wed, Oct 16, 2013 at 12:00 AM, Tsuyoshi OZAWA wrote: > I don't have Windows environment, but one user who tried 2.2.0 release > on Windows reported that released tar ball doesn't contain > "win

Re: Managing docs with hadoop-1 & hadoop-2

2013-10-18 Thread Tsuyoshi OZAWA
>> # Have stable1/stable2 links: >> http://hadoop.apache.org/docs/stable1 -> hadoop-1.x >> http://hadoop.apache.org/docs/stable2 -> hadoop-2.x > +1, would also make: > current -> stable2(since v2 is the latest) > stable -> stable1 (for compatibility) +1, Eli's proposal looks good to

Re: release-2.2.0 tag?

2013-11-07 Thread Tsuyoshi OZAWA
Hi Mohammad, IIUC, 2.2.0 is the latest released version. branch-2.2 is the latest branch under development. On Fri, Nov 8, 2013 at 11:52 AM, Mohammad Islam wrote: > Hi, > I could not find out any release tag for 2.2.0. > Which branch should I use instead for latest released version? branch-2.2.0

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-26 Thread Tsuyoshi Ozawa
+1(binding), - Tsuyoshi On Tue, Mar 20, 2018 at 14:21 Owen O'Malley wrote: > All, > > Following our discussions on the previous thread (Merging branch HDFS-7240 > to trunk), I'd like to propose the following: > > * HDSL become a subproject of Hadoop. > * HDSL will release separately from Hadoop

Re: [VOTE] Release Apache Hadoop 2.7.2 RC2

2016-01-20 Thread Tsuyoshi Ozawa
+1 (binding) - Verified checksum. - Built from source code. - Ran some tests and mr examples. Thanks, - Tsuyoshi On Fri, Jan 15, 2016 at 1:57 PM, Vinod Kumar Vavilapalli wrote: > Hi all, > > I've created an updated release candidate RC2 for Apache Hadoop 2.7.2. > > As discussed before, this is

Re: [VOTE] Release Apache Hadoop 2.6.4 RC0

2016-02-09 Thread Tsuyoshi Ozawa
+1(binding) - Verified signatures. - Ran tests for Apache Tez with the artifacts. All tests passed. - Ran local cluster and ran some examples on it. - Tsuyoshi On Wed, Feb 10, 2016 at 2:18 AM, Sunil Govind wrote: > +1 (non-binding) with one note, > > - Installed tar ball from source and brought

Re: Review HDFS-7588 please?

2016-02-15 Thread Tsuyoshi Ozawa
Thanks for the ping, I'll also check them. Regards, - Tsuyoshi On Mon, Feb 15, 2016 at 3:23 PM, Haohui Mai wrote: > Thanks for the ping. I'll take a look at it. > > Regards, > Haohui > > > > On Fri, Feb 12, 2016 at 3:56 PM, Ravi Prakash wrote: >> Hi! >> >> There are a few subtasks of HDFS-7588

Slack channel for Hadoop developers

2016-04-22 Thread Tsuyoshi Ozawa
Hi, I created slack channel for Hadoop community unofficially and experimentally: https://hadoopdev.slack.com/ I know that there is IRC channel and it's good to log. However, Slack is very also good tool to join easily and have a communication interactively. It will be also useful to join followi

[DISCUSS] Merge Jersey upgrade(HADOOP-9613) to trunk

2016-05-09 Thread Tsuyoshi Ozawa
Hi developers, We’ve worked on upgrading jersey(HADOOP-9613) for a years. It's essential change to support complication with JDK8. It’s almost there. One concern to merge this to trunk is incompatibility. After the release of Jersey 1.13, the root element whose content is empty collection is chan

Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)

2016-05-10 Thread Tsuyoshi Ozawa
> Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. Sounds good. To do so, we need to check the blockers of 3.0.0-alpha RC, especially upgrading all dependencies which use refractions at first. Thanks, - Tsuyoshi On Tue, May 10, 2016 at 8:32 AM, Akira AJISAKA wrote: > Hi de

Re: Save the Date: Apache Hadoop Bug Bash / May 8th!

2015-03-19 Thread Tsuyoshi Ozawa
Hi Allen, Thank you for the great suggestion. Let's do this! Thanks, - Tsuyoshi On Fri, Mar 20, 2015 at 2:34 AM, Allen Wittenauer wrote: > > Hi folks, > > There are ~6,000 Hadoop JIRA issues that have gone unaddressed, > including ~900 with patches waiting to be reviewed. Among other

Re: [VOTE] Release Apache Hadoop 2.7.0 RC0

2015-04-13 Thread Tsuyoshi Ozawa
+1 (non-binding) - Verified checksums on source and binary tar ball. - Built from source and launched single cluster. - Checked to launch some jobs successfully. - Killing application via RM UI. Thanks, - Tsuyoshi On Mon, Apr 13, 2015 at 7:43 AM, Junping Du wrote: > +1. (Non-binding) > - Downlo

Re: [VOTE] Release Apache Hadoop 2.7.0 RC0

2015-04-15 Thread Tsuyoshi Ozawa
Hey Akira, could you target the issues to 2.7.1? Thanks, - Tsuyoshi On Wed, Apr 15, 2015 at 2:51 PM, Akira AJISAKA wrote: > +0 (non-binding) > > - Downloaded source > - Built from source > - Ran a pseudo-distributed cluster > - Read documents downloaded from binary tarball > - Looked Web UIs > >

[DISCUSS] Using maven-jarjar-plugin for avoiding classpath conflicts

2015-06-09 Thread Tsuyoshi Ozawa
Hi, Recently, I've tackled with dependency problems about Guava, Jetty, and Jersey. Essentially, it's similar to "DLL hell". I've seen that Google Guice uses jarjar-maven-plugin to avoid classpath conflicts between user-side dependency and library-side dependency. http://sonatype.github.io/jarja

Re: [DISCUSS] Using maven-jarjar-plugin for avoiding classpath conflicts

2015-06-09 Thread Tsuyoshi Ozawa
gt; shading the hadoop client. Have you reviewed his proposal? > > Best, > Andrew > > On Tue, Jun 9, 2015 at 12:17 PM, Tsuyoshi Ozawa wrote: > >> Hi, >> >> Recently, I've tackled with dependency problems about Guava, Jetty, >> and Jersey. Essentially, it&#

Re: 2.7.1 status

2015-06-15 Thread Tsuyoshi Ozawa
Hi Vinod, Thank you for working to release 2.7! YARN-3798 looks a critical issue for branch-2.7. I'll mark it as a blocker of 2.7.1 if possible. Thanks, - Tsuyoshi On Tue, Jun 16, 2015 at 6:58 AM, Vinod Kumar Vavilapalli wrote: > We are down to one blocker and a few critical tickets. I’ll try t

Re: [VOTE] Release Apache Hadoop 2.7.1 RC0

2015-07-03 Thread Tsuyoshi Ozawa
Thank you for starting voting, Vinod. I tried to untar the tarball, but the command exited with an error. Is binary tarball broken? $ tar xzvf hadoop-2.7.1-RC0.tar.gz ... hadoop-2.7.1/share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib/hadoop-common-2.7.1.jar gzip: stdin: unexpected end of fil

Re: [VOTE] Release Apache Hadoop 2.7.1 RC0

2015-07-03 Thread Tsuyoshi Ozawa
tar (GNU tar) 1.23 > > uname -a > Linux a.com 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC 2014 > x86_64 x86_64 x86_64 GNU/Linux > > I was able to expand the tarball. > > Can you use another machine ? > > Cheers > > > On Fri, Jul 3, 2015 at 9:53 AM,

Re: IMPORTANT: automatic changelog creation

2015-07-08 Thread Tsuyoshi Ozawa
+1, thanks Allen and Andrew for taking lots effort! > Is there any possibility that, we can restrict someone from editing the > issue in jira once its marked as "closed" after release? Vinay's comment looks considerable for us to me. What do you think? - Tsuyoshi On Wed, Jul 8, 2015 at 3:57 PM,

Re: 2.7.2 release plan

2015-07-16 Thread Tsuyoshi Ozawa
Hi, thank you for starting the discussion about 2.7.2 release. > The focus obviously is to have blocker issues [2], bug-fixes and *no* features / improvements. I've committed YARN-3170, which is an improvement of documentation. I thought documentation pages which can be fit into branch-2.7 can b

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

2015-09-21 Thread Tsuyoshi Ozawa
+1 (non-binding) 1. Built from source. 2. Verified signature of bin tar ball and source tar ball. 3. Running some examples. 4. Running Spark on YARN with spark-shell. It works well. I think 4 verifies compatibility is preserved. Thanks everyone for working to release 2.6.1. Thanks, - Tsuyoshi O

Re: data center aware hadoop?

2012-09-22 Thread Tsuyoshi OZAWA
> Or do you plan to have different data across sites and then run MR jobs > across them? This would be an interesting problem, but its way above the FS. MAPREDUCE-4502 relates to this problem. Please check it out if you have interests. https://issues.apache.org/jira/browse/MAPREDUCE-4502 - Tsuyos

Re: Hadoop CI with alternate architectures.

2016-05-18 Thread Tsuyoshi Ozawa
Hi Asanjar, Thanks for your contribution! I'm ashamed to say, but I don't know how to change the build machine. Adding "to" Allen, who is a PMC of Yetus project. Hey Allen, do you know how to add Power based Jenkins slave(s) to Apache Hadoop CI? Thanks, - Tsuyoshi On Wed, May 18, 2016 at 11:18

[DICUSS] Upgrading Guice to 4.0(HADOOP-12064)

2016-06-27 Thread Tsuyoshi Ozawa
Hi developers, I will plan to upgrade Google Guice dependency on trunk. The change also includes asm and cglib upgrade. I checked following points: * Both HDFS and YARN UIs work well. * All webIU-related tests pass as described on HADOOP-12064. * Ran mapreduce job, and it works well. https://iss

Re: [DICUSS] Upgrading Guice to 4.0(HADOOP-12064)

2016-06-29 Thread Tsuyoshi Ozawa
No objections here? - Tsuyoshi On Mon, Jun 27, 2016 at 6:01 AM, Tsuyoshi Ozawa wrote: > Hi developers, > > I will plan to upgrade Google Guice dependency on trunk. The change > also includes asm and cglib upgrade. > I checked following points: > > * Both HDFS and YARN U

Re: [DICUSS] Upgrading Guice to 4.0(HADOOP-12064)

2016-06-29 Thread Tsuyoshi Ozawa
te: >> > My strong expectation is that we’ll have a version of classpath >> > isolation in our first release of 3.x. I’m planning to spending some cycles >> > right away on this. >> > >> > Assuming classpath isolation gets in, it is reasonable to bump

Re: [DICUSS] Upgrading Guice to 4.0(HADOOP-12064)

2016-06-29 Thread Tsuyoshi Ozawa
I forgot to mention about importance point: it's a blocker issue to compile Hadoop with JDK8. Hence, we need to merge it on both client side and server slide anyway. Thanks, - Tsuyoshi On Wed, Jun 29, 2016 at 12:24 PM, Tsuyoshi Ozawa wrote: > Thanks Vinod, Sangjin, Sean for your

Re: [DICUSS] Upgrading Guice to 4.0(HADOOP-12064)

2016-06-30 Thread Tsuyoshi Ozawa
n 29, 2016 at 12:27 PM, Tsuyoshi Ozawa wrote: >> >> I forgot to mention about importance point: it's a blocker issue to >> compile Hadoop with JDK8. Hence, we need to merge it on both client >> side and server slide anyway. >> >> Thanks, >> - Tsuy

Re: [DICUSS] Upgrading Guice to 4.0(HADOOP-12064)

2016-07-07 Thread Tsuyoshi Ozawa
u, Jun 30, 2016 at 12:03 PM, Tsuyoshi Ozawa wrote: >> >> Thanks, Andrew. >> >> Based on discussion here, I would like to merge it into *trunk* if >> there are no objection tomorrow. >> >> Thanks, >> - Tsuyoshi >> >> On Wed, Jun 29, 201

[DISCUSS] Official Docker Image at release time

2016-07-19 Thread Tsuyoshi Ozawa
Hi developers, Klaus mentioned the availability of an official docker image of Apache Hadoop. Is it time that we start to distribute an official docker image at release time? http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629

Re: [DISCUSS] Official Docker Image at release time

2016-07-20 Thread Tsuyoshi Ozawa
Forwarding this discussion to Klaus. - Tsuyoshi On Tue, Jul 19, 2016 at 4:46 PM, Tsuyoshi Ozawa wrote: > Hi developers, > > Klaus mentioned the availability of an official docker image of Apache > Hadoop. Is it time that we start to distribute an official docker > image

[DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

2016-07-21 Thread Tsuyoshi Ozawa
Hi developers, I'd like to discuss how to make an advance towards dependency management in Apache Hadoop trunk code since there has been lots work about updating dependencies in parallel. Summarizing recent works and activities as follows: 0) Currently, we have merged minimum update dependencies

Re: [DISCUSS] Official Docker Image at release time

2016-07-21 Thread Tsuyoshi Ozawa
sues.apache.org/jira/browse/HADOOP-13397 > > > Da (Klaus) Ma (马达), PMP® | Software Architect > IBM Spectrum, STG, IBM GCG > +86-10-8245 4084 | klaus1982...@gmail.com | http://k82.me > > On Jul 21, 2016, at 13:33, Tsuyoshi Ozawa wrote: > > Forwarding this discussion to Klaus.

Re: [DISCUSS] Official Docker Image at release time

2016-07-21 Thread Tsuyoshi Ozawa
ncies. > This can be achieved by flexible Dockerfile which uses configuration > passed from environment variables. > > Thanks > > Kai Sasaki > > > On Jul 19, 2016, at 4:46 PM, Tsuyoshi Ozawa > wrote: > > Hi developers, > > Klaus mentioned the availability of an offic

[DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-26 Thread Tsuyoshi Ozawa
Hi Vinod, Thanks all guys for starting discussion! My suggestion is adding the date when branch cut is done: like 3.0.0-alpha1-20160724, 2.8.0-20160730 or something. Pros:-) It's totally ordered. If we have a policy such as backporting to maintainance branches after the date, users can find that

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-26 Thread Tsuyoshi Ozawa
> Andrew: I bet many would assume it's the release date, like how Ubuntu releases are numbered. Good point. Maybe I confuse you because of lack of explanation. I assume that "branch-cut off timing" mean the timing of freezing branch like when starting the release vote. It's because that the relea

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-27 Thread Tsuyoshi Ozawa
this problem is also pretty rare in practice, since users normally > upgrade to the highest maintenance release within a major/minor. Thus > they'll only hit this if their upgrade cycle is faster than it takes for a > change released in e.g. 2.6.x to then also be released in a 2.7.x

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-09-02 Thread Tsuyoshi Ozawa
Thanks for making this forward hardly, Andrew! In fact, I hit an error similar to Eric's one regardless of configuring yarn.app.mapreduce.am.env and mapreduce.admin.user.env... does anyone hit same error? 2016-09-02 16:49:35,223 INFO mapreduce.Job: The url to track the job: http://ip-172-31-7-12

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-09-02 Thread Tsuyoshi Ozawa
I'm sorry, very big typo I did... s/hardly/very hard/ Best - Tsuyoshi On Sat, Sep 3, 2016 at 1:53 AM, Tsuyoshi Ozawa wrote: > Thanks for making this forward hardly, Andrew! > > In fact, I hit an error similar to Eric's one regardless of > configuring yarn.ap

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-09-03 Thread Tsuyoshi Ozawa
ne regardless of configuring yarn.app.mapreduce.am.env and mapreduce.admin.user.env... does anyone hit same error? This was because of completely my wrong configuration. Please ignore this. Best, - Tsuyoshi On Sat, Sep 3, 2016 at 1:53 AM, Tsuyoshi Ozawa wrote: > Thanks for making this forwar

About 2.6.5 and 2.7.4 release

2016-09-08 Thread Tsuyoshi Ozawa
Hi developers, Chiwan Park reported Hadoop incompatibility by the change of HADOOP-11252. Now, HADOOP-13579 is committed to branch-2.6 and branch-2.7. Should we release 2.6.5 and 2.7.4 soon? Thanks, - Tsuyoshi - To unsubscribe,

Re: About 2.6.5 and 2.7.4 release

2016-09-08 Thread Tsuyoshi Ozawa
> Chiwan Park reported Hadoop incompatibility by the change of HADOOP-11252. > Now, HADOOP-13579 is committed to branch-2.6 and branch-2.7. complement: HADOOP-13579 fixes the problem of incompatibility by HADOOP-11252. Thanks, - Tsuyoshi On Fri, Sep 9, 2016 at 2:42 PM, Tsuyoshi Ozawa

Can we update protobuf's version on trunk?

2017-03-27 Thread Tsuyoshi Ozawa
Dear Hadoop developers, After shaded client, introduced by HADOOP-11804, is merged, we can more easily update some dependency with minimizing the impact of backward compatibility on trunk. (Thanks Sean and Sanjin for taking the issue!) Then, is it time to update protobuf's version to the latest o

Re: Can we update protobuf's version on trunk?

2017-03-27 Thread Tsuyoshi Ozawa
Forwarding to common-dev, hdfs-dev, mapreduce-dev too. Thanks - Tsuyoshi 2017年3月27日(月) 21:16 Tsuyoshi Ozawa : > Dear Hadoop developers, > > After shaded client, introduced by HADOOP-11804, is merged, > we can more easily update some dependency with minimizing the impact &

Re: Can we update protobuf's version on trunk?

2017-03-28 Thread Tsuyoshi Ozawa
> Stack mentioned a compatibility mode that uses the proto2 semantics. We can define protobuf's version by using syntax = "proto2" or syntax = "proto3" in proto files with proto3 compiler. I think this is what Stack mentioned as compatibility mode. https://github.com/golang/protobuf/blob/master/p

Re: Can we update protobuf's version on trunk?

2017-03-28 Thread Tsuyoshi Ozawa
'version at compilation time by using syntax = "proto2""... - Tsuyoshi On Wed, Mar 29, 2017 at 9:13 AM, Tsuyoshi Ozawa wrote: >> Stack mentioned a compatibility mode that uses the proto2 semantics. > > We can define protobuf's version by using syntax = "proto2&quo

Re: [VOTE] Release Apache Hadoop 2.3.0

2014-02-13 Thread Tsuyoshi OZAWA
+1 (non-binding) * Built from source. * Ran a pseudo-cluster with manual RM HA. * Ran a few MR examples as tests. Thanks, Tsuyoshi On Tue, Feb 11, 2014 at 6:49 AM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc0) for hadoop-2.3.0 that I would like to > get released. > >

Re: [VOTE] Release Apache Hadoop 2.4.0

2014-04-03 Thread Tsuyoshi OZAWA
Azurry, can you file the RM-failover related bug as a JIRA? I'm going to reproduce it in my local. Thanks, - Tsuyoshi On Fri, Apr 4, 2014 at 7:47 AM, Azuryy wrote: > Did you tested RM failover on Hive? There is bug. > > > Sent from my iPhone5s > >> On 2014年4月4日, at 2:12, Xuan Gong wrote: >> >>

Re: [VOTE] Release Apache Hadoop 2.4.0

2014-04-03 Thread Tsuyoshi OZAWA
Hi, Ran tests and confirmed that some tests(TestSymlinkLocalFSFileSystem) fail. The log of the test failure is as follows: https://gist.github.com/oza/9965197 Should we fix or disable the feature? Thanks, - Tsuyoshi On Mon, Mar 31, 2014 at 6:22 PM, Arun C Murthy wrote: > Folks, > > I've creat

Re: [VOTE] Release Apache Hadoop 2.4.0

2014-04-03 Thread Tsuyoshi OZAWA
JIRA about this failure) Now I'm checking the problem reported by Azuryy. Thanks, - Tsuyoshi On Fri, Apr 4, 2014 at 8:55 AM, Tsuyoshi OZAWA wrote: > Hi, > > Ran tests and confirmed that some tests(TestSymlinkLocalFSFileSystem) fail. > The log of the test failure is as f

Re: [VOTE] Release Apache Hadoop 2.4.0

2014-04-08 Thread Tsuyoshi OZAWA
Hi Arun, I apologize for the late response. If the problems are recognized correctly, +1 for the release(non-binding). * Ran examples on pseudo distributed cluster. * Ran tests. * Built from source. Let's fix the problems at the target version(2.4.1). Thanks, - Tsuyoshi On Wed, Apr 9, 2014 at

Re: Thinking ahead

2014-04-14 Thread Tsuyoshi OZAWA
Hi, +1 for 2.4.1 release(non-binding). There are some critical issues including YARN-1861 and YARN-1929 about RM-HA. IMO, we should include the fixes. One question is whether we can include RM-HA phase 2(YARN-556) in 2.5 by mid-May. It's still work-in-progress. Thanks, - Tsuyoshi On Mon, Apr 1

Re: Policy on adding timeouts to tests

2014-04-16 Thread Tsuyoshi OZAWA
Hi Karthik, Some tests with servers like MiniCluster or ZK can never end because of unexpected busy loop or something if the tests don't have timeouts. It can blocks the other jobs of Jenkins server. Therefore, IMHO, we should add timeouts when we write tests with them. Thanks, - Tsuyoshi On Wed

Re: hadoop-2.5 - June end?

2014-06-10 Thread Tsuyoshi OZAWA
IMHO, it's useful to release 2.5 as intermediate release. The working items which are not ready for 2.5 will be target for 2.6, right? Thanks, - Tsuyoshi On Tue, Jun 10, 2014 at 5:03 PM, wrote: > Arun, > > Any possibility of getting HDFS-6376 committed and backported into 2.5? > > Dave Marion

Re: hadoop-2.5 - June end?

2014-06-23 Thread Tsuyoshi OZAWA
IMHO, I'd like to +1 to include YARN-1492 and work-preserving restart in 2.6. I believe they are good improvement, but they are work-in-progress and it takes a bit time to be done. Thanks, - Tsuyoshi On Tue, Jun 24, 2014 at 7:13 AM, Jian He wrote: > Just found the roadmap includes a wrong jira l

Re: [VOTE] Change by-laws on release votes: 5 days instead of 7

2014-06-24 Thread Tsuyoshi OZAWA
+1(non-binding) On Wed, Jun 25, 2014 at 1:55 AM, Hitesh Shah wrote: > +1 (binding) > > — Hitesh > > On Jun 24, 2014, at 1:53 AM, Arun C Murthy wrote: > >> Folks, >> >> As discussed, I'd like to call a vote on changing our by-laws to change >> release votes from 7 days to 5. >> >> I've attached

Re: [VOTE] Release Apache Hadoop 2.4.1

2014-06-26 Thread Tsuyoshi OZAWA
+1(non-binding) - Built from source. - Ran tests. -- Reproduced HADOOP-10510 on AWS instance(Ubuntu Server 14.04 LTS (PV) - ami-bddaa2bc (64-bit)). I agree with that this problem will be fixed after releases when voting at 2.4.0. I believe that this problem is environment-depend one. - Deployed si

Re: Branching 2.5

2014-07-16 Thread Tsuyoshi OZAWA
Hi Wangda, The following link is same link as Karthik mentioned: https://issues.apache.org/jira/browse/YARN-2247?jql=project%20in%20(Hadoop%2C%20HDFS%2C%20YARN%2C%20%22Hadoop%20Map%2FReduce%22)%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Target%20Version%2Fs%22%20%3D%202.5.0%20AND%20priority

Re: [DISCUSS] Assume Private-Unstable for classes that are not annotated

2014-07-23 Thread Tsuyoshi OZAWA
+1 for Karthik's suggestion. - Tsuyoshi On Wed, Jul 23, 2014 at 6:54 AM, Karthik Kambatla wrote: > Hi devs > > As you might have noticed, we have several classes and methods in them that > are not annotated at all. This is seldom intentional. Avoiding incompatible > changes to all these classes

Re: [DISCUSS] Migrate from svn to git for source control?

2014-08-04 Thread Tsuyoshi OZAWA
Thank you for great suggestion, Karthik. +1(non-binding) to use git. I'm also using private git repository. Additionally, I have one question. Will we accept github-based development like Apache Spark? IHMO, it allow us to leverage Hadoop development, because the cost of sending pull request is ver

Re: [DISCUSS] Migrate from svn to git for source control?

2014-08-04 Thread Tsuyoshi OZAWA
pens up the possibility for better review workflows, but we can > discuss that on a different thread. > > A few different people have also mentioned Gerrit, so that'd be in the > running along with Github (and I guess ReviewBoard). > > Thanks, > Andrew > > > On Mon, A

Re: [VOTE] Release Apache Hadoop 2.5.0 RC2

2014-08-11 Thread Tsuyoshi OZAWA
-1 (non-binding). * Downloaded source and verified signature. * Built from source. * Ran tests and some MR jobs. * Ran RM-HA with manual failover mode. - Documents are missing. Good catch, Akira. As he mentioned, the documents including javadocs are missing from the binary tar ball. We should inc

Re: [VOTE] Migration from subversion to git for version control

2014-08-11 Thread Tsuyoshi OZAWA
+1(non-binding) Thanks, - Tsuyoshi On Sat, Aug 9, 2014 at 11:57 AM, Karthik Kambatla wrote: > I have put together this proposal based on recent discussion on this topic. > > Please vote on the proposal. The vote runs for 7 days. > >1. Migrate from subversion to git for version control. >

Re: [VOTE] Release Apache Hadoop 2.5.0 RC2

2014-08-11 Thread Tsuyoshi OZAWA
x27;t change the source. > > Thanks > Karthik > > > On Mon, Aug 11, 2014 at 5:51 AM, Tsuyoshi OZAWA > wrote: > >> -1 (non-binding). >> >> * Downloaded source and verified signature. >> * Built from source. >> * Ran tests and some MR jobs. >

Re: [VOTE] Release Apache Hadoop 2.5.0 RC2

2014-08-12 Thread Tsuyoshi OZAWA
hadoop-2.5.0-RC2/ >>> >>> Please note that the binary tar ball is signed by a new gpg key, so please >>> re-import keys. (I lost my original private key). The source tar ball, >>> signature and checksum are untouched. >>> >>> >>> On Mon

Re: [DISCUSS] Switch to log4j 2

2014-08-14 Thread Tsuyoshi OZAWA
Hi, Steve has started discussion titled "use SLF4J APIs in new modules?" as a related topic. http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201404.mbox/%3cca+4kjvv_9cmmtdqzcgzy-chslyb1wkgdunxs7wrheslwbuh...@mail.gmail.com%3E It sounds good to me to use asynchronous logging when we log

Re: Apache Hadoop 2.5.0 published tarballs are missing some txt files

2014-08-18 Thread Tsuyoshi OZAWA
Hi, Thanks Karthik for preparing new tar ball. Verified MD5, SHA1, SHA224, SHA256, SHA384, SHA512 checksums. It looks good to me. Thanks, - Tsuyoshi On Tue, Aug 19, 2014 at 4:42 AM, Karthik Kambatla wrote: > Hi devs > > Tsuyoshi just brought it to my notice that the published tarballs don't >

Re: [VOTE] Release Apache Hadoop 2.5.1 RC0

2014-09-09 Thread Tsuyoshi OZAWA
* Verified signatures. * Verified LICENSE.txt NOTICE.txt README.txt are included in both source tar ball and bin tar ball. +1 once Akira's comment is addressed. Thanks, - Tsuyoshi On Tue, Sep 9, 2014 at 3:37 PM, Akira AJISAKA wrote: > -0 (non-binding) > > In the document, "Apache Hadoop 2.5.1

Re: [VOTE] Release Apache Hadoop 2.6.0

2014-11-10 Thread Tsuyoshi OZAWA
Hi Arun, Could you confirm the link and permission to the files is correct? I got a following error: Forbidden You don't have permission to access /~acmurthy/hadoop-2.6.0-rc0/hadoop-2.6.0-src.tar.gz on this server. On Tue, Nov 11, 2014 at 11:52 AM, Arun C Murthy wrote: > Folks, > > I've create

Re: [VOTE] Release Apache Hadoop 2.6.0

2014-11-17 Thread Tsuyoshi OZAWA
+0(non-binding) HDFS-6833 is critical issue for us - could you help us to merge it into 2.6? Thanks, - Tsuyoshi On Sat, Nov 15, 2014 at 1:03 PM, Hitesh Shah wrote: > +1 (binding) > > Built Hadoop from source, compiled Tez against the hadoop jars pushed to > staging repo and ran a few example T

Re: [VOTE] Release Apache Hadoop 2.6.0

2014-11-17 Thread Tsuyoshi OZAWA
h replication > factor 3), we should perhaps target this into 2.6.1 and not block this > release. Thoughts? > > On Mon, Nov 17, 2014 at 4:17 AM, Tsuyoshi OZAWA > wrote: > >> +0(non-binding) >> >> HDFS-6833 is critical issue for us - could you help us to merge it

Re: [VOTE] Release Apache Hadoop 2.6.0

2014-11-18 Thread Tsuyoshi OZAWA
Hi, I also found that a following command fails with hadoop-2.6.0-rc1-src.tar.gz: $ mvn package -Pdocs -DskipTests -Dtar I filed it as HADOOP-11316. We should fix it if possible. Or, should we fix it for 2.6.1? Thanks, - Tsuyoshi On Tue, Nov 18, 2014 at 3:04 PM, Tsuyoshi OZAWA wrote

  1   2   >