[jira] [Created] (HDDS-977) Exclude dependency-reduced-pom.xml from ozone rat check

2019-01-11 Thread Elek, Marton (JIRA)
Elek, Marton created HDDS-977:
-

 Summary: Exclude dependency-reduced-pom.xml from ozone rat check
 Key: HDDS-977
 URL: https://issues.apache.org/jira/browse/HDDS-977
 Project: Hadoop Distributed Data Store
  Issue Type: Bug
Reporter: Elek, Marton
Assignee: Elek, Marton


As of now we have one (false positive) rat violation:

 
{code:java}
hadoop-ozone/ozonefs/target/rat.txt: !? 
/home/elek/projects/hadoop/hadoop-ozone/ozonefs/dependency-reduced-pom.xml
{code}
As it's generated during the build, it could be safely ignored.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org



Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-11 Thread Zoltan Haindrich

Hello,

I would like to note that it seems like 3.2.0-RC1 release misses some source attachments (as all releases lately). David Phillips just commented on that jira yesterday; and 
I've just noticed that a release vote is already going onso I think now is the best time to talk about this - because https://issues.apache.org/jira/browse/HADOOP-15205 
is open now for almost a year.


This might be just a documentation related issue; but then the HowToRelease doc 
should be updated.
Steve Loughran was able to publish the artifacts in question for 2.7.7 - but 
releases before and after that are missing these source attachements.

People working on downstream projects (or at least me) may find it harder to 
work with hadoop packages; beacuse of the missing source attachments.

example artifact which misses the sources:
https://repository.apache.org/content/repositories/orgapachehadoop-1178/org/apache/hadoop/hadoop-mapreduce-client-core/3.2.0/

cheers,
Zoltan

On 1/8/19 12:42 PM, Sunil G wrote:

Hi folks,


Thanks to all of you who helped in this release [1] and for helping to vote
for RC0. I have created second release candidate (RC1) for Apache Hadoop
3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/


RC tag in git is release-3.2.0-RC1.



The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1178/


This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.



3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org



Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2019-01-11 Thread Apache Jenkins Server
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/

[Jan 10, 2019 4:02:36 AM] (bharat) HDDS-941. Rename ChunkGroupInputStream to 
keyInputStream and
[Jan 10, 2019 11:58:39 AM] (stevel) HADOOP-16040. ABFS: Bug fix for 
tolerateOobAppends configuration.
[Jan 10, 2019 12:02:58 PM] (stevel) HADOOP-15662. Better exception handling of 
DNS errors.
[Jan 10, 2019 12:07:09 PM] (stevel) HADOOP-16036. WASB: Disable jetty logging 
configuration announcement.
[Jan 10, 2019 7:21:37 PM] (kihwal) HADOOP-15909. KeyProvider class should 
implement Closeable. Contributed




-1 overall


The following subsystems voted -1:
asflicense findbugs hadolint pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.security.ssl.TestSSLFactory 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/diff-compile-javac-root.txt
  [336K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/diff-checkstyle-root.txt
  [17M]

   hadolint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/diff-patch-hadolint.txt
  [4.0K]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/diff-patch-pylint.txt
  [60K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/whitespace-eol.txt
  [9.3M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/whitespace-tabs.txt
  [1.1M]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-hdds_framework.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-ozone_client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-ozone_common.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-ozone_s3gateway.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/branch-findbugs-hadoop-ozone_tools.txt
  [8.0K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/diff-javadoc-javadoc-root.txt
  [752K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
  [168K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [328K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [84K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1013/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server

[VOTE] - HDDS-4 Branch merge

2019-01-11 Thread Anu Engineer
Since I have not heard any concerns, I will start a VOTE thread now.
This vote will run for 7 days and will end on Jan/18/2019 @ 8:00 AM PST.

I will start with my vote, +1 (Binding)

Thanks
Anu


-- Forwarded message -
From: Anu Engineer 
Date: Mon, Jan 7, 2019 at 5:10 PM
Subject: [Discuss] - HDDS-4 Branch merge
To: , 


Hi All,

I would like to propose a merge of HDDS-4 branch to the Hadoop trunk.
HDDS-4 branch implements the security work for HDDS and Ozone.

HDDS-4 branch contains the following features:
- Hadoop Kerberos and Tokens support
- A Certificate infrastructure used by Ozone and HDDS.
- Audit Logging and parsing support (Spread across trunk and HDDS-4)
- S3 Security Support - AWS Signature Support.
- Apache Ranger Support for Ozone

I will follow up with a formal vote later this week if I hear no
objections. AFAIK, the changes are isolated to HDDS/Ozone and should not
impact any other Hadoop project.

Thanks
Anu


Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-11 Thread Gabor Bota
  Thanks for the work Sunil!

  +1 (non-binding)

  checked out git tag release-3.2.0-RC1.
  hadoop-aws integration (mvn verify) test run was successful on eu-west-1
(a known issue is there, it's fixed in trunk)
  built from source on Mac OS X 10.14.2, java version 8.0.181-oracle
  deployed on a 3 node cluster
  verified pi job, teragen, terasort and teravalidate

  Regards,
  Gabor Bota

On Fri, Jan 11, 2019 at 1:11 PM Zoltan Haindrich  wrote:

> Hello,
>
> I would like to note that it seems like 3.2.0-RC1 release misses some
> source attachments (as all releases lately). David Phillips just commented
> on that jira yesterday; and
> I've just noticed that a release vote is already going onso I think
> now is the best time to talk about this - because
> https://issues.apache.org/jira/browse/HADOOP-15205
> is open now for almost a year.
>
> This might be just a documentation related issue; but then the
> HowToRelease doc should be updated.
> Steve Loughran was able to publish the artifacts in question for 2.7.7 -
> but releases before and after that are missing these source attachements.
>
> People working on downstream projects (or at least me) may find it harder
> to work with hadoop packages; beacuse of the missing source attachments.
>
> example artifact which misses the sources:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1178/org/apache/hadoop/hadoop-mapreduce-client-core/3.2.0/
>
> cheers,
> Zoltan
>
> On 1/8/19 12:42 PM, Sunil G wrote:
> > Hi folks,
> >
> >
> > Thanks to all of you who helped in this release [1] and for helping to
> vote
> > for RC0. I have created second release candidate (RC1) for Apache Hadoop
> > 3.2.0.
> >
> >
> > Artifacts for this RC are available here:
> >
> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/
> >
> >
> > RC tag in git is release-3.2.0-RC1.
> >
> >
> >
> > The maven artifacts are available via repository.apache.org at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1178/
> >
> >
> > This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm
> PST.
> >
> >
> >
> > 3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
> > additions
> >
> > are the highlights of this release.
> >
> > 1. Node Attributes Support in YARN
> >
> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >
> > 3. Support service upgrade via YARN Service API and CLI
> >
> > 4. HDFS Storage Policy Satisfier
> >
> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >
> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >
> > 7. Improvements in Router-based HDFS federation
> >
> >
> >
> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >
> > I have done few testing with my pseudo cluster. My +1 to start.
> >
> >
> >
> > Regards,
> >
> > Sunil
> >
> >
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > ORDER BY fixVersion ASC
> >
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>


[jira] [Created] (HDFS-14200) Add emptyTrash option to purge trash immediately

2019-01-11 Thread Stephen O'Donnell (JIRA)
Stephen O'Donnell created HDFS-14200:


 Summary: Add emptyTrash option to purge trash immediately
 Key: HDFS-14200
 URL: https://issues.apache.org/jira/browse/HDFS-14200
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: namenode
Reporter: Stephen O'Donnell
Assignee: Stephen O'Donnell


I have always felt the HDFS trash is missing a simple way to empty the current 
users trash immediately. We have "expunge" but in my experience supporting 
clusters, end users find this confusing. When most end users run expunge, they 
really want to empty their trash immediately and get confused when expunge does 
not do this.

This can result in users performing somewhat dangerous "skipTrash" operations 
on the trash to free up space. The alternative, which most users will not 
figure out on their own is:

# Run the expunge command once - this will move the current folder to a 
checkpoint and remove any old checkpoints older than the retention interval
# Wait over 1 minute and then run expunge again, overriding fs.trash.interval 
to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 
-expunge.

With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" 
that purges everything in the logged in users Trash directories immediately.

How would the community feel about adding this new option? I will upload a 
patch for comments.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org



Re: [VOTE] - HDDS-4 Branch merge

2019-01-11 Thread Ajay Kumar
+1 (non-binding)

On 1/11/19, 7:40 AM, "Anu Engineer"  wrote:

Since I have not heard any concerns, I will start a VOTE thread now.
This vote will run for 7 days and will end on Jan/18/2019 @ 8:00 AM PST.

I will start with my vote, +1 (Binding)

Thanks
Anu


-- Forwarded message -
From: Anu Engineer 
Date: Mon, Jan 7, 2019 at 5:10 PM
Subject: [Discuss] - HDDS-4 Branch merge
To: , 


Hi All,

I would like to propose a merge of HDDS-4 branch to the Hadoop trunk.
HDDS-4 branch implements the security work for HDDS and Ozone.

HDDS-4 branch contains the following features:
- Hadoop Kerberos and Tokens support
- A Certificate infrastructure used by Ozone and HDDS.
- Audit Logging and parsing support (Spread across trunk and HDDS-4)
- S3 Security Support - AWS Signature Support.
- Apache Ranger Support for Ozone

I will follow up with a formal vote later this week if I hear no
objections. AFAIK, the changes are isolated to HDDS/Ozone and should not
impact any other Hadoop project.

Thanks
Anu



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org



Re: [VOTE] - HDDS-4 Branch merge

2019-01-11 Thread Jitendra Pandey
+1 (binding)

On 1/11/19, 10:26 AM, "Ajay Kumar"  wrote:

+1 (non-binding)

On 1/11/19, 7:40 AM, "Anu Engineer"  wrote:

Since I have not heard any concerns, I will start a VOTE thread now.
This vote will run for 7 days and will end on Jan/18/2019 @ 8:00 AM PST.

I will start with my vote, +1 (Binding)

Thanks
Anu


-- Forwarded message -
From: Anu Engineer 
Date: Mon, Jan 7, 2019 at 5:10 PM
Subject: [Discuss] - HDDS-4 Branch merge
To: , 


Hi All,

I would like to propose a merge of HDDS-4 branch to the Hadoop trunk.
HDDS-4 branch implements the security work for HDDS and Ozone.

HDDS-4 branch contains the following features:
- Hadoop Kerberos and Tokens support
- A Certificate infrastructure used by Ozone and HDDS.
- Audit Logging and parsing support (Spread across trunk and HDDS-4)
- S3 Security Support - AWS Signature Support.
- Apache Ranger Support for Ozone

I will follow up with a formal vote later this week if I hear no
objections. AFAIK, the changes are isolated to HDDS/Ozone and should not
impact any other Hadoop project.

Thanks
Anu



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org




-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org



Re: [VOTE] - HDDS-4 Branch merge

2019-01-11 Thread Dinesh Chitlangia
+1 (non binding)

Thanks,
Dinesh


On 1/11/19, 1:53 PM, "Jitendra Pandey"  wrote:

+1 (binding)

On 1/11/19, 10:26 AM, "Ajay Kumar"  wrote:

+1 (non-binding)

On 1/11/19, 7:40 AM, "Anu Engineer"  wrote:

Since I have not heard any concerns, I will start a VOTE thread now.
This vote will run for 7 days and will end on Jan/18/2019 @ 8:00 AM 
PST.

I will start with my vote, +1 (Binding)

Thanks
Anu


-- Forwarded message -
From: Anu Engineer 
Date: Mon, Jan 7, 2019 at 5:10 PM
Subject: [Discuss] - HDDS-4 Branch merge
To: , 


Hi All,

I would like to propose a merge of HDDS-4 branch to the Hadoop 
trunk.
HDDS-4 branch implements the security work for HDDS and Ozone.

HDDS-4 branch contains the following features:
- Hadoop Kerberos and Tokens support
- A Certificate infrastructure used by Ozone and HDDS.
- Audit Logging and parsing support (Spread across trunk and 
HDDS-4)
- S3 Security Support - AWS Signature Support.
- Apache Ranger Support for Ozone

I will follow up with a formal vote later this week if I hear no
objections. AFAIK, the changes are isolated to HDDS/Ozone and 
should not
impact any other Hadoop project.

Thanks
Anu



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org




-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org





Re: [VOTE] - HDDS-4 Branch merge

2019-01-11 Thread Ajay Kumar
+1 (binding)


On 1/11/19, 7:40 AM, "Anu Engineer"  wrote:

Since I have not heard any concerns, I will start a VOTE thread now.
This vote will run for 7 days and will end on Jan/18/2019 @ 8:00 AM PST.

I will start with my vote, +1 (Binding)

Thanks
Anu


-- Forwarded message -
From: Anu Engineer 
Date: Mon, Jan 7, 2019 at 5:10 PM
Subject: [Discuss] - HDDS-4 Branch merge
To: , 


Hi All,

I would like to propose a merge of HDDS-4 branch to the Hadoop trunk.
HDDS-4 branch implements the security work for HDDS and Ozone.

HDDS-4 branch contains the following features:
- Hadoop Kerberos and Tokens support
- A Certificate infrastructure used by Ozone and HDDS.
- Audit Logging and parsing support (Spread across trunk and HDDS-4)
- S3 Security Support - AWS Signature Support.
- Apache Ranger Support for Ozone

I will follow up with a formal vote later this week if I hear no
objections. AFAIK, the changes are isolated to HDDS/Ozone and should not
impact any other Hadoop project.

Thanks
Anu





-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org



Re: [VOTE] - HDDS-4 Branch merge

2019-01-11 Thread Hanisha Koneru
+1 (binding)

Thanks,
Hanisha









On 1/11/19, 7:40 AM, "Anu Engineer"  wrote:

>Since I have not heard any concerns, I will start a VOTE thread now.
>This vote will run for 7 days and will end on Jan/18/2019 @ 8:00 AM PST.
>
>I will start with my vote, +1 (Binding)
>
>Thanks
>Anu
>
>
>-- Forwarded message -
>From: Anu Engineer 
>Date: Mon, Jan 7, 2019 at 5:10 PM
>Subject: [Discuss] - HDDS-4 Branch merge
>To: , 
>
>
>Hi All,
>
>I would like to propose a merge of HDDS-4 branch to the Hadoop trunk.
>HDDS-4 branch implements the security work for HDDS and Ozone.
>
>HDDS-4 branch contains the following features:
>- Hadoop Kerberos and Tokens support
>- A Certificate infrastructure used by Ozone and HDDS.
>- Audit Logging and parsing support (Spread across trunk and HDDS-4)
>- S3 Security Support - AWS Signature Support.
>- Apache Ranger Support for Ozone
>
>I will follow up with a formal vote later this week if I hear no
>objections. AFAIK, the changes are isolated to HDDS/Ozone and should not
>impact any other Hadoop project.
>
>Thanks
>Anu

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org



[jira] [Created] (HDFS-14201) Ability to disallow safemode NN to become active

2019-01-11 Thread Xiao Liang (JIRA)
Xiao Liang created HDFS-14201:
-

 Summary: Ability to disallow safemode NN to become active
 Key: HDFS-14201
 URL: https://issues.apache.org/jira/browse/HDFS-14201
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: auto-failover
Affects Versions: 2.9.2, 3.1.1
Reporter: Xiao Liang
Assignee: Xiao Liang


Currently with HA, Namenode in safemode can be possibly selected as active, for 
availability of both read and write, Namenodes not in safemode are better 
choices to become active though.

It can take tens of minutes for a cold started Namenode to get out of safemode, 
especially when there are large number of files and blocks in HDFS, that means 
if a Namenode in safemode become active, the cluster will be not fully 
functioning for quite a while, even if it can while there is some Namenode not 
in safemode.

The proposal here is to add an option, to allow Namenode to report itself as 
UNHEALTHY to ZKFC, if it's in safemode, so as to only allow fully functioning 
Namenode to become active, improving the general availability of the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org



Re: [VOTE] - HDDS-4 Branch merge

2019-01-11 Thread Bharat Viswanadham
+1 (binding)


Thanks,
Bharat


On 1/11/19, 11:04 AM, "Hanisha Koneru"  wrote:

+1 (binding)

Thanks,
Hanisha









On 1/11/19, 7:40 AM, "Anu Engineer"  wrote:

>Since I have not heard any concerns, I will start a VOTE thread now.
>This vote will run for 7 days and will end on Jan/18/2019 @ 8:00 AM PST.
>
>I will start with my vote, +1 (Binding)
>
>Thanks
>Anu
>
>
>-- Forwarded message -
>From: Anu Engineer 
>Date: Mon, Jan 7, 2019 at 5:10 PM
>Subject: [Discuss] - HDDS-4 Branch merge
>To: , 
>
>
>Hi All,
>
>I would like to propose a merge of HDDS-4 branch to the Hadoop trunk.
>HDDS-4 branch implements the security work for HDDS and Ozone.
>
>HDDS-4 branch contains the following features:
>- Hadoop Kerberos and Tokens support
>- A Certificate infrastructure used by Ozone and HDDS.
>- Audit Logging and parsing support (Spread across trunk and HDDS-4)
>- S3 Security Support - AWS Signature Support.
>- Apache Ranger Support for Ozone
>
>I will follow up with a formal vote later this week if I hear no
>objections. AFAIK, the changes are isolated to HDDS/Ozone and should not
>impact any other Hadoop project.
>
>Thanks
>Anu

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org