Re: [DISCUSS] Mysql-connector-java is GPL licensed; Hadoop can't use it.

2023-06-09 Thread Owen O'Malley
his case, RBF is used by a small set of Hadoop users, moreover an even > smaller subset of them will choose to use the mysql backend. > > (And there need to be instructions for how to acquire the mysql connector > jar since it's not shipped in the convenience binary) > > On Thu,

Re: [DISCUSS] Mysql-connector-java is GPL licensed; Hadoop can't use it.

2023-06-08 Thread Owen O'Malley
The relevant url is https://www.apache.org/legal/resolved.html#optional On Fri, Jun 9, 2023 at 6:54 AM Owen O'Malley wrote: > We are allowed to use category X software in optional components. > Furthermore, the dependency is marked as provided, so it won't be pulled > into a

Re: [DISCUSS] Mysql-connector-java is GPL licensed; Hadoop can't use it.

2023-06-08 Thread Owen O'Malley
We are allowed to use category X software in optional components. Furthermore, the dependency is marked as provided, so it won't be pulled into a transitive closure. You are right that I should have included a comment about that on the original jira. .. Owen On Fri, Jun 9, 2023 at 1:58 AM Wei-C

[jira] [Resolved] (HDFS-16917) Add transfer rate quantile metrics for DataNode reads

2023-02-27 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16917. -- Fix Version/s: 3.4.0 Resolution: Fixed > Add transfer rate quantile met

[jira] [Resolved] (HDFS-16890) RBF: Add period state refresh to keep router state near active namenode's

2023-02-27 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16890. -- Fix Version/s: 3.4.0 3.3.6 Resolution: Fixed > RBF: Add peri

[jira] [Resolved] (HDFS-16901) RBF: Routers should propagate the real user in the UGI via the caller context

2023-02-22 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16901. -- Fix Version/s: 3.4.0 3.3.6 Resolution: Fixed Thanks, Simba!

[jira] [Resolved] (HDFS-16895) NamenodeHeartbeatService should use credentials of logged in user

2023-02-07 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16895. -- Fix Version/s: 3.4.0 3.3.5 Assignee: Hector Sandoval Cha

[jira] [Resolved] (HDFS-16886) Fix documentation for StateStoreRecordOperations#get(Class ..., Query ...)

2023-01-11 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16886. -- Fix Version/s: 3.4.0 3.3.5 Resolution: Fixed > Fix documenta

[jira] [Resolved] (HDFS-16877) Namenode doesn't use alignment context in TestObserverWithRouter

2023-01-06 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16877. -- Fix Version/s: 3.4.0 Assignee: Simbarashe Dzinamarira Resolution: Fixed

[jira] [Resolved] (HDFS-16851) RBF: Add a utility to dump the StateStore

2022-11-29 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16851. -- Fix Version/s: 3.3.6 3.4.0 Resolution: Fixed > RBF: Add a ut

[jira] [Resolved] (HDFS-16847) RBF: StateStore writer should not commit tmp fail if there was an error in writing the file.

2022-11-28 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16847. -- Fix Version/s: 3.4.0 3.3.5 Resolution: Fixed I committed

[jira] [Resolved] (HDFS-16845) Add configuration flag to enable observer reads on routers without using ObserverReadProxyProvider

2022-11-28 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16845. -- Fix Version/s: 3.4.0 3.3.5 2.10.3 Resol

[jira] [Created] (HDFS-16856) [RBF] Refactor router admin command to use HDFS AdminHelper class

2022-11-28 Thread Owen O'Malley (Jira)
Owen O'Malley created HDFS-16856: Summary: [RBF] Refactor router admin command to use HDFS AdminHelper class Key: HDFS-16856 URL: https://issues.apache.org/jira/browse/HDFS-16856 Project: Hadoop

[jira] [Created] (HDFS-16851) [RBF] Utility to textually dump the StateStore

2022-11-21 Thread Owen O'Malley (Jira)
Owen O'Malley created HDFS-16851: Summary: [RBF] Utility to textually dump the StateStore Key: HDFS-16851 URL: https://issues.apache.org/jira/browse/HDFS-16851 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-16844) [RBF] The routers should be resiliant against exceptions from StateStore

2022-11-18 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16844. -- Fix Version/s: 3.4.0 3.3.5 Resolution: Fixed > [RBF] The

[jira] [Resolved] (HDFS-16843) [RBF] The routers should be resiliant against exceptions from StateStore

2022-11-15 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16843. -- Resolution: Duplicate > [RBF] The routers should be resiliant against excepti

[jira] [Created] (HDFS-16844) [RBF] The routers should be resiliant against exceptions from StateStore

2022-11-15 Thread Owen O'Malley (Jira)
Owen O'Malley created HDFS-16844: Summary: [RBF] The routers should be resiliant against exceptions from StateStore Key: HDFS-16844 URL: https://issues.apache.org/jira/browse/HDFS-16844 Pr

[jira] [Created] (HDFS-16843) [RBF] The routers should be resiliant against exceptions from StateStore

2022-11-15 Thread Owen O'Malley (Jira)
Owen O'Malley created HDFS-16843: Summary: [RBF] The routers should be resiliant against exceptions from StateStore Key: HDFS-16843 URL: https://issues.apache.org/jira/browse/HDFS-16843 Pr

[jira] [Resolved] (HDFS-16836) StandbyCheckpointer can still trigger rollback fs image after RU is finalized

2022-11-15 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16836. -- Fix Version/s: 3.4.0 3.3.5 Resolution: Fixed I just committed

[jira] [Created] (HDFS-16778) Separate out the logger for which DN is picked by a DFSInputStream

2022-09-19 Thread Owen O'Malley (Jira)
Owen O'Malley created HDFS-16778: Summary: Separate out the logger for which DN is picked by a DFSInputStream Key: HDFS-16778 URL: https://issues.apache.org/jira/browse/HDFS-16778 Project: H

[jira] [Resolved] (HDFS-16767) RBF: Support observer node from Router-Based Federation

2022-09-14 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16767. -- Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed I just committed

review please?

2022-07-14 Thread Owen O'Malley
We have a critical issue that has been occasionally crashing our NameNode's with thread exhaustion. I'd love a review for the patch - https://github.com/apache/hadoop/pull/4527 Thanks, Owen

[jira] [Resolved] (HDFS-16518) KeyProviderCache close cached KeyProvider with Hadoop ShutdownHookManager

2022-03-30 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16518. -- Fix Version/s: 3.4.0 2.10.2 3.3.3 Resol

[jira] [Resolved] (HDFS-16517) In 2.10 the distance metric is wrong for non-DN machines

2022-03-23 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16517. -- Fix Version/s: 2.10.2 Resolution: Fixed > In 2.10 the distance metric is wrong

[jira] [Created] (HDFS-16517) In 2.10 the distance metric is wrong for non-DN machines

2022-03-21 Thread Owen O'Malley (Jira)
Owen O'Malley created HDFS-16517: Summary: In 2.10 the distance metric is wrong for non-DN machines Key: HDFS-16517 URL: https://issues.apache.org/jira/browse/HDFS-16517 Project: Hadoop

[jira] [Resolved] (HDFS-16495) RBF should prepend the client ip rather than append it.

2022-03-14 Thread Owen O'Malley (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16495. -- Fix Version/s: 3.4.0 3.2.4 Resolution: Fixed > RBF should

[jira] [Created] (HDFS-16495) RBF should prepend the client ip rather than append it.

2022-03-04 Thread Owen O'Malley (Jira)
Owen O'Malley created HDFS-16495: Summary: RBF should prepend the client ip rather than append it. Key: HDFS-16495 URL: https://issues.apache.org/jira/browse/HDFS-16495 Project: Hadoop

Re: Backport of Router-Based Federation to 2.10.

2021-12-11 Thread Owen O'Malley
anc-2.10 ..? As rolling upgrade > support is available from 2 to 3, any chance to upgrade to branch-3.X,so > that so many features and improvements will be available..? > > >> On Thu, 9 Dec 2021 at 5:55 AM, Owen O'Malley wrote: >> >> All, >> I've sta

Backport of Router-Based Federation to 2.10.

2021-12-08 Thread Owen O'Malley
All, I've started backporting Router-Based Federation work from trunk back into Hadoop 2.10. The goal is to backport everything that is relevant in Hadoop 2. Would anyone outside of LinkedIn be interested in it? Thanks, Owen

[jira] [Created] (HDFS-16253) Add a toString implementation to DFSInputStream

2021-10-04 Thread Owen O'Malley (Jira)
Owen O'Malley created HDFS-16253: Summary: Add a toString implementation to DFSInputStream Key: HDFS-16253 URL: https://issues.apache.org/jira/browse/HDFS-16253 Project: Hadoop HDFS

Re: Collect feedback for HDFS-15638

2020-10-16 Thread Owen O'Malley
I'm very -1 on adding these semantics. When you create the table's directory, set the default ACL. That will have exactly the effect that you are looking for without creating additional semantics. .. Owen On Fri, Oct 16, 2020 at 7:02 PM Xinli shang wrote: > Hi all, > > I opened https://issues.

Re: [DISCUSS] About creation of Hadoop Thirdparty repository for shaded artifacts

2019-09-27 Thread Owen O'Malley
I'm very unhappy with this direction. In particular, I don't think git is a good place for distribution of binary artifacts. Furthermore, the PMC shouldn't be releasing anything without a release vote. I'd propose that we make a third party module that contains the *source* of the pom files to bui

Re: [VOTE] Moving Submarine to a separate Apache project proposal

2019-09-06 Thread Owen O'Malley
Since you don't have any Apache Members, I'll join to provide Apache oversight. .. Owen On Fri, Sep 6, 2019 at 1:38 PM Owen O'Malley wrote: > +1 for moving to a new project. > > On Sat, Aug 31, 2019 at 10:19 PM Wangda Tan wrote: > >> Hi all, >> >&

Re: [VOTE] Moving Submarine to a separate Apache project proposal

2019-09-06 Thread Owen O'Malley
+1 for moving to a new project. On Sat, Aug 31, 2019 at 10:19 PM Wangda Tan wrote: > Hi all, > > As we discussed in the previous thread [1], > > I just moved the spin-off proposal to CWIKI and completed all TODO parts. > > > https://cwiki.apache.org/confluence/display/HADOOP/Submarine+Project+Sp

[jira] [Created] (HDFS-14244) hdfs++ doesn't add necessary libraries to dynamic library link

2019-01-30 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-14244: Summary: hdfs++ doesn't add necessary libraries to dynamic library link Key: HDFS-14244 URL: https://issues.apache.org/jira/browse/HDFS-14244 Project: H

[NOTIFICATION] Hadoop trunk rebased

2018-04-26 Thread Owen O'Malley
As we discussed in hdfs-dev@hadoop, I did a force push to Hadoop's trunk to replace the Ozone merge with a rebase. That means that you'll need to rebase your branches. .. Owen

Re: [RESULT][VOTE] Merge HDDS (HDFS-7240) *code* into trunk

2018-04-26 Thread Owen O'Malley
> > Thanks > Xiaoyu > > > > > > *From: *Owen O'Malley > *Date: *Wednesday, April 25, 2018 at 11:15 AM > *To: *Chris Douglas > *Cc: *Xiaoyu Yao , Jitendra Pandey < > jiten...@hortonworks.com>, Hdfs-dev > *Subject: *Re: [RESULT][VOTE] Merge HDDS

Re: [RESULT][VOTE] Merge HDDS (HDFS-7240) *code* into trunk

2018-04-25 Thread Owen O'Malley
*sigh* I asked you guys to rebase this before merge. It is tempting to squash it into a single commit. .. Owen On Wed, Apr 25, 2018 at 11:06 AM, Chris Douglas wrote: > This really made a mess of trunk. Periodically merging trunk into the > HDFS-7240 branch, then merging the whole thing back, c

[RESULT][VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-27 Thread Owen O'Malley
Ok, with a lot of +1's, one +0, and no -1's the vote passes. We have a new subproject! We should resolve the final name and then create a jira instance for it. Thanks everyone, Owen

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Owen O'Malley
On Thu, Mar 22, 2018 at 11:09 AM, Lei Xu wrote: > > I have one concrete question about how this HDSL subproject being > separated: Ozone / HDSL was designed in the current way to re-use the > existing HDFS code base as much as possible, thus today for this > container service is in DataNode itsel

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-21 Thread Owen O'Malley
On Tue, Mar 20, 2018 at 8:34 PM, 郑锴(铁杰) wrote: > >>* HDSL become a subproject of Hadoop. > I'm not compfortable with the HDSL name, as Konstantin mentioned. H-DSL > looks like a DSL language at the first glance. > Let's start a separate thread about the name. My general guidance on naming things

[VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-20 Thread Owen O'Malley
All, Following our discussions on the previous thread (Merging branch HDFS-7240 to trunk), I'd like to propose the following: * HDSL become a subproject of Hadoop. * HDSL will release separately from Hadoop. Hadoop releases will not contain HDSL and vice versa. * HDSL will get its own jira instan

Re: [VOTE] Merging branch HDFS-7240 to trunk

2018-03-19 Thread Owen O'Malley
Andrew and Daryn, Do you have any feedback on the proposal? Otherwise, we can start a vote for "adoption of new codebase" tomorrow. .. Owen On Wed, Mar 14, 2018 at 1:50 PM, Owen O'Malley wrote: > This discussion seems to have died down coming closer consensus without a >

Re: [VOTE] Merging branch HDFS-7240 to trunk

2018-03-14 Thread Owen O'Malley
This discussion seems to have died down coming closer consensus without a resolution. I'd like to propose the following compromise: * HDSL become a subproject of Hadoop. * HDSL will release separately from Hadoop. Hadoop releases will not contain HDSL and vice versa. * HDSL will get its own jira

Re: [VOTE] Merging branch HDFS-8707 (native HDFS client) to trunk

2018-03-13 Thread Owen O'Malley
+1 (binding) .. Owen On Sun, Mar 11, 2018 at 6:20 PM, Chris Douglas wrote: > +1 (binding) -C > > On Thu, Mar 8, 2018 at 9:31 AM, Jim Clampffer > wrote: > > Hi Everyone, > > > > The feedback was generally positive on the discussion thread [1] so I'd > > like to start a formal vote for merging H

Re: [VOTE] Merging branch HDFS-7240 to trunk

2018-03-09 Thread Owen O'Malley
Hi Joep, On Tue, Mar 6, 2018 at 6:50 PM, J. Rottinghuis wrote: Obviously when people do want to use Ozone, then having it in the same repo > is easier. The flipside is that, separate top-level project in the same > repo or not, it adds to the Hadoop releases. > Apache projects are about the gro

Re: [VOTE] Merging branch HDFS-7240 to trunk

2018-03-02 Thread Owen O'Malley
On Thu, Mar 1, 2018 at 11:03 PM, Andrew Wang wrote: Owen mentioned making a Hadoop subproject; we'd have to > hash out what exactly this means (I assume a separate repo still managed by > the Hadoop project), but I think we could make this work if it's more > attractive than incubation or a new T

Re: [VOTE] Merging branch HDFS-7240 to trunk

2018-03-01 Thread Owen O'Malley
I think it would be good to get this in sooner rather than later, but I have some thoughts. 1. It is hard to tell what has changed. git rebase -i tells me the branch has 722 commits. The rebase failed with a conflict. It would really help if you rebased to current trunk. 2. I think Ozo

Re: [DISCUSS] Merging HDFS-8707 (C++ HDFS client) to trunk

2018-03-01 Thread Owen O'Malley
+1 on the merge. We've been using it on the trunk of ORC for a while. It will be great to have it released by Hadoop. .. Owen On Thu, Mar 1, 2018 at 10:31 AM, Vinayakumar B wrote: > Definitely this would be great addition. Kudos to everyone's contributions. > > I am not a C++ expert. So cannot

[CFP] Dataworks Summit San Jose Call for Presentations closes Friday 9 Feb

2018-02-08 Thread Owen O'Malley
All, Dataworks Summit San Jose 2018 is June 17-21. The call for abstracts is open through February 9th. One of the tracks is Big Compute And Storage, which is great for talks about Hadoop. You can submit an abstract at https://dataworkssummit.com/san-jose-2018/ Thanks, Owen

Hadoop Summit EU 2017

2016-11-04 Thread Owen O'Malley
The DataWorks Summit EU 2017 (including Hadoop Summit) is going to be in Munich April 5-6 2017 . I’ve pasted the text from the CFP below. Would you like to share your knowledge with the best and brightest in the data community? If so, we encourage you to submit an abstract for DataWorks Summit wi

Re: Hadoop encryption module as Apache Chimera incubator project

2016-01-27 Thread Owen O'Malley
On Wed, Jan 27, 2016 at 9:59 AM, Gangumalla, Uma wrote: > I think Chimera goal is to enhance even for other use cases. Naturally. > For Hadoop, CTR mode should be enough today, This isn't true. Hadoop should use better encryption for RPC and shuffle, both of which should not use CTR. > I

Re: Hadoop encryption module as Apache Chimera incubator project

2016-01-26 Thread Owen O'Malley
Sorry to be coming in to this discussion late. Rather than pull the code out of Hadoop, may I suggest instead making it a separate subproject within Hadoop itself? I'd suggest letting it release independently of Hadoop, since it will need a much faster cadence that Hadoop proper does. It should als

[jira] [Created] (HDFS-9025) fix compilation issues on arch linux

2015-09-04 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-9025: --- Summary: fix compilation issues on arch linux Key: HDFS-9025 URL: https://issues.apache.org/jira/browse/HDFS-9025 Project: Hadoop HDFS Issue Type: Sub

[DISCUSS] Create branch for HDFS-8707

2015-07-01 Thread Owen O'Malley
All, Haohui and I would like to pull code for an async io c++ hdfs client from https://github.com/haohui/libhdfspp as a new development branch. The motivation is to get a high performance asynchronous c++ hdfs client and the driving need is to support reading ORC files from HDFS without a jvm in

[jira] [Created] (HDFS-8707) Implement an async pure c++ HDFS client

2015-07-01 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-8707: --- Summary: Implement an async pure c++ HDFS client Key: HDFS-8707 URL: https://issues.apache.org/jira/browse/HDFS-8707 Project: Hadoop HDFS Issue Type

Re: Moving to JDK7, JDK8 and new major releases

2014-06-25 Thread Owen O'Malley
On Tue, Jun 24, 2014 at 4:44 PM, Alejandro Abdelnur wrote: > After reading this thread and thinking a bit about it, I think it should be > OK such move up to JDK7 in Hadoop I agree with Alejandro. Changing minimum JDKs is not an incompatible change and is fine in the 2 branch. (Although I think

[jira] [Resolved] (HDFS-3699) HftpFileSystem should try both KSSL and SPNEGO when authentication is required

2013-10-01 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-3699. - Resolution: Won't Fix Using KSSL is strongly deprecated and should be avoided in secur

[jira] [Resolved] (HDFS-3983) Hftp should support both SPNEGO and KSSL

2013-10-01 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-3983. - Resolution: Won't Fix Target Version/s: (was: ) KSSL is deprecated and sh

Re: Release numbering for branch-2 releases

2013-02-04 Thread Owen O'Malley
I think that using "-(alpha,beta)" tags on the release versions is a really bad idea. All releases should follow the strictly numeric (Major.Minor.Patch) pattern that we've used for all of the releases except the 2.0.x ones. -- Owen On Mon, Feb 4, 2013 at 11:53 AM, Stack wrote: > On Mon, Feb 4

[jira] [Resolved] (HDFS-4012) HftpFileSystem shouldn't implicity fetch delegation tokens

2012-10-09 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-4012. - Resolution: Not A Problem This is not a problem. > HftpFileSystem s

[jira] [Resolved] (HDFS-4011) WebHdfsFileSystem shouldn't implicitly fetch delegation tokens

2012-10-09 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-4011. - Resolution: Invalid This is a feature. > WebHdfsFileSystem shouldn'

[jira] [Resolved] (HDFS-4010) Remove unused TokenRenewer implementation from WebHdfsFileSystem and HftpFileSystem

2012-10-09 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-4010. - Resolution: Won't Fix They are used via the java ServiceLoader

[jira] [Resolved] (HDFS-4009) WebHdfsFileSystem and HftpFileSystem don't need delegation tokens

2012-10-09 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-4009. - Resolution: Won't Fix This is a feature, not a bug. In particular, if your kerberos ti

[jira] [Created] (HDFS-3993) The KSSL class should not limit the ssl ciphers

2012-10-01 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-3993: --- Summary: The KSSL class should not limit the ssl ciphers Key: HDFS-3993 URL: https://issues.apache.org/jira/browse/HDFS-3993 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-3749) Disable check for jsvc on windows

2012-08-01 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-3749: --- Summary: Disable check for jsvc on windows Key: HDFS-3749 URL: https://issues.apache.org/jira/browse/HDFS-3749 Project: Hadoop HDFS Issue Type

Re: OSGi and classloaders

2012-07-09 Thread Owen O'Malley
Changing the configurations is a big and very touchy job. It is touchy in that it is very exposed to the users and many many applications assume the configuration is dealt with in particular ways. It is a requirement to maintain compatibility and thus that needs to be factored in to the work. Furth

[jira] [Resolved] (HDFS-2386) with security enabled fsck calls lead to handshake_failure and hftp fails throwing the same exception in the logs

2012-06-25 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-2386. - Resolution: Invalid Fixed via HDFS-2617. > with security enabled fsck ca

[jira] [Created] (HDFS-3466) The SPNEGO filter for the NameNode should come out of the web keytab file

2012-05-25 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-3466: --- Summary: The SPNEGO filter for the NameNode should come out of the web keytab file Key: HDFS-3466 URL: https://issues.apache.org/jira/browse/HDFS-3466 Project: H

[jira] [Created] (HDFS-3461) HFTP should use the same port & protocol for getting the delegation token

2012-05-23 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-3461: --- Summary: HFTP should use the same port & protocol for getting the delegation token Key: HDFS-3461 URL: https://issues.apache.org/jira/browse/HDFS-3461 Projec

[jira] [Created] (HDFS-3374) hdfs' TestDelegationToken fails intermittently with a race condition

2012-05-04 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-3374: --- Summary: hdfs' TestDelegationToken fails intermittently with a race condition Key: HDFS-3374 URL: https://issues.apache.org/jira/browse/HDFS-3374 Project: H

[jira] [Created] (HDFS-3348) After HDFS-2617 can't use 0.0.0.0 in dfs.http.address

2012-05-02 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-3348: --- Summary: After HDFS-2617 can't use 0.0.0.0 in dfs.http.address Key: HDFS-3348 URL: https://issues.apache.org/jira/browse/HDFS-3348 Project: Hadoop HDFS

[jira] [Created] (HDFS-3345) Primary and secondary Principals must be the same

2012-05-01 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-3345: --- Summary: Primary and secondary Principals must be the same Key: HDFS-3345 URL: https://issues.apache.org/jira/browse/HDFS-3345 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-3316) The tar ball doesn't include jsvc any more

2012-04-23 Thread Owen O'Malley (JIRA)
Owen O'Malley created HDFS-3316: --- Summary: The tar ball doesn't include jsvc any more Key: HDFS-3316 URL: https://issues.apache.org/jira/browse/HDFS-3316 Project: Hadoop HDFS Issue

Re: [RESULT] - [VOTE] Rename hadoop branches post hadoop-1.x

2012-03-29 Thread Owen O'Malley
On Wed, Mar 28, 2012 at 5:11 PM, Doug Cutting wrote: > On 03/28/2012 12:39 PM, Owen O'Malley wrote: > > [ ... ] So the RM of the 2 branch needs to make the call of what > > should be 2.1 vs 3.0. > > I thought these were community decisions, not RM decisions, no? >

Re: [RESULT] - [VOTE] Rename hadoop branches post hadoop-1.x

2012-03-28 Thread Owen O'Malley
On Wed, Mar 28, 2012 at 12:32 PM, Todd Lipcon wrote: But new features also go to trunk. And if none of our new features are > incompatible, why do we anticipate that trunk is 3.0? > Let's imagine that we already had a 2.0.0 release. Now we want to add features like HA. The only place to put that

Re: [RESULT] - [VOTE] Rename hadoop branches post hadoop-1.x

2012-03-28 Thread Owen O'Malley
I disagree. Trunk should become branch-3 once someone wants to start stabilizing it. Arun is going to need the minor versions for when he adds features. X.Y.Z Z = bug fixes Y = minor release (compatible, adds features) X = major release (incompatible) So from branch-2 will come branch-2.0 with t

Re: svn commit: r1304067 - in /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project: ./ bin/ conf/ hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/ hadoop-mapreduce-exam

2012-03-22 Thread Owen O'Malley
To me, I'd much much rather have the human readable description of what is being fixed and I mostly could care less about which subversion commit it corresponds to. I'd be all for using the CHANGE.txt description as the commit message for both trunk and the branches. -- Owen

[jira] [Created] (HDFS-2856) Fix block protocol so that Datanodes don't require root or jvsc

2012-01-29 Thread Owen O'Malley (Created) (JIRA)
ement Components: contrib/raid, scripts Reporter: Owen O'Malley Since we send the block tokens unencrypted to the datanode, we currently start the datanode as root and get a secure (< 1024) port. If we have the datanode generate a nonce and send it on the conne

[jira] [Resolved] (HDFS-224) I propose a tool for creating and manipulating a new abstraction, Hadoop Archives.

2012-01-27 Thread Owen O'Malley (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-224. Resolution: Duplicate We have a different version of harchives. > I propos

[jira] [Created] (HDFS-2619) Remove my personal email address from the libhdfs build file.

2011-12-01 Thread Owen O'Malley (Created) (JIRA)
Components: build Reporter: Owen O'Malley Assignee: Owen O'Malley When I first wrote the libhdfs autoconf/automake stuff, I incorrectly put my email address in the AC_INIT line, which means if something goes wrong, you get: {quote} > configure: WARNING:     #

[jira] [Created] (HDFS-2467) HftpFileSystem uses incorrect compare for finding delegation tokens

2011-10-18 Thread Owen O'Malley (Created) (JIRA)
Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.23.0 When looking for hdfs delegation tokens, Hftp converts the service to a string and compares it to a text. -- This message is automatically generated by JIRA. If you think it was sent i

Re: Retire 0.20-append branch?

2011-10-03 Thread Owen O'Malley
On Mon, Oct 3, 2011 at 2:50 PM, Todd Lipcon wrote: > In essence, that means that we would: > 1) Ask contributors to provide patches against 0.20-security branch > instead of 0.20-append +1 > 2) Move the branch itself into some kind of attic/ directory You can just delete it. Subversion always k

[jira] [Resolved] (HDFS-2257) HftpFileSystem should implement getDelegationTokens

2011-09-29 Thread Owen O'Malley (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-2257. - Resolution: Not A Problem HFTP already has a getDelegationToken, which it overrides

[jira] [Resolved] (HDFS-2326) hftp token renewal uses wrong port

2011-09-29 Thread Owen O'Malley (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-2326. - Resolution: Invalid Fix Version/s: (was: 0.20.205.0) > hftp token rene

[jira] [Resolved] (HDFS-2328) hftp throws NPE if security is not enabled on remote cluster

2011-09-14 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-2328. - Resolution: Fixed I committed this to 20-s and 205. I'll commit it to trunk as part of

[jira] [Created] (HDFS-2156) rpm should only require the same major version as common

2011-07-15 Thread Owen O'Malley (JIRA)
: Owen O'Malley The rpm for hdfs should only require the same major version (eg. 0.23) of common. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HDFS-1666) TestAuthorizationFilter is failing

2011-06-10 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-1666. - Resolution: Fixed Fix Version/s: 0.22.0 Assignee: Todd Lipcon This was comm

[jira] [Resolved] (HDFS-1952) FSEditLog.open() appears to succeed even if all EDITS directories fail

2011-06-10 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-1952. - Resolution: Fixed Resolving this, since it was committed to trunk. > FSEditLog.open() ap

Re: [Discuss] Merge federation branch HDFS-1052 into trunk

2011-04-28 Thread Owen O'Malley
On Apr 27, 2011, at 10:12 PM, Konstantin Shvachko wrote: > The question is whether this is a > * Code Change, > which requires Lazy consensus of active committers or a > * Adoption of New Codebase, > which needs Lazy 2/3 majority of PMC members This is a code change, just like all of our jiras. T

Re: [Discuss] Merge federation branch HDFS-1052 into trunk

2011-04-27 Thread Owen O'Malley
On Apr 26, 2011, at 11:34 PM, suresh srinivas wrote: >> 2. I assume that merging requires a vote. I am sure people who know bylaws >> better than I do will correct me if it is not true. >> Did I miss the vote? >> > > > As regards to voting, since I was not sure about the procedure, I had > cons

[jira] [Created] (HDFS-1811) Create scripts to decommission datanodes

2011-04-05 Thread Owen O'Malley (JIRA)
Create scripts to decommission datanodes Key: HDFS-1811 URL: https://issues.apache.org/jira/browse/HDFS-1811 Project: Hadoop HDFS Issue Type: Improvement Reporter: Owen O'M

[jira] [Created] (HDFS-1810) Remove duplicate jar entries from common

2011-04-05 Thread Owen O'Malley (JIRA)
Remove duplicate jar entries from common Key: HDFS-1810 URL: https://issues.apache.org/jira/browse/HDFS-1810 Project: Hadoop HDFS Issue Type: Improvement Reporter: Owen O'M

[jira] [Created] (HDFS-1809) Lack of proper DefaultMetricsSystem initialization breaks some tests

2011-04-05 Thread Owen O'Malley (JIRA)
: Bug Reporter: Owen O'Malley Assignee: Suresh Srinivas Fix For: 0.20.203.0 Following tests are failing: TestHDFSServerPorts TestNNLeaseRecovery TestSaveNamespace -- This message is automatically generated by JIRA. For more information on JIRA, see:

[jira] Created: (HDFS-1729) Improve metrics for measuring NN startup costs.

2011-03-07 Thread Owen O'Malley (JIRA)
Reporter: Owen O'Malley Assignee: Matt Foley Fix For: 0.20.100 Current logging and metrics are insufficient to diagnose latency problems in cluster startup. Add: 1. better logs in both Datanode and Namenode for Initial Block Report processing, to

[jira] Resolved: (HDFS-1192) refreshSuperUserGroupsConfiguration should use server side configuration for the refresh (for HADOOP-6815)

2010-10-10 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-1192. - Resolution: Fixed Fix Version/s: 0.22.0 Hadoop Flags: [Reviewed] This jira w

Re: hadoop.job.ugi backwards compatibility

2010-09-13 Thread Owen O'Malley
On Mon, Sep 13, 2010 at 11:10 AM, Todd Lipcon wrote: > Yep, but there are plenty of 10 node clusters out there that do important > work at small startups or single-use-case installations, too. We need to > provide scalability and security features that work for the 100+ node > clusters but also no

Re: hadoop.job.ugi backwards compatibility

2010-09-13 Thread Owen O'Malley
On Mon, Sep 13, 2010 at 10:05 AM, Todd Lipcon wrote: > This is not MR-specific, since the strangely named hadoop.job.ugi determines > HDFS permissions as well. Yeah, after I hit send, I realized that I should have used common-dev. This is really a dev issue. > "or the user must write a custom g

[jira] Resolved: (HDFS-98) creating a file in hdfs should not automatically create the parent directories

2010-09-12 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-98?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-98. --- Resolution: Won't Fix Leaving the old FileSystem behavior and fixing it in FileContex

Re: Integrating Lustre and HDFS

2010-06-10 Thread Owen O'Malley
> Your local storage should get used for MR. Use Lustre via file:// > (LocalFileSystem, iirc) > instead of HDFS via hdfs:// (DistributedFileSystem, irrc) as the default file > system type. If Lustre has integrated checksums, you'll want to use the RawLocalFileSystem instead of LocalFileSystem.

  1   2   >