Re: [VOTE] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-12 Thread Yongjun Zhang
insert calls back into the call queue. That's > > completely unacceptable. A buggy or malicious client can easily cause > > livelock in the IPC layer with handlers only looping on calls that never > > satisfy the condition. Why is this not implemented via > RetriableExcepti

Re: [VOTE] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-05 Thread Yongjun Zhang
Great work guys. Wonder if we can elaborate what's impact of not having #2 fixed, and why #2 is not needed for the feature to complete? 2. Need to fix automatic failover with ZKFC. Currently it does not doesn't know about ObserverNodes trying to convert them to SBNs. Thanks. --Yongjun On Wed, D

[jira] [Created] (HDFS-14012) Add diag info in RetryInvocationHandler

2018-10-20 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-14012: Summary: Add diag info in RetryInvocationHandler Key: HDFS-14012 URL: https://issues.apache.org/jira/browse/HDFS-14012 Project: Hadoop HDFS Issue Type

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-09-09 Thread Yongjun Zhang
hadoop.apache.org/releases.html page. > > I just did it, FYI: https://svn.apache.org/viewvc?view=revision&revision= > 1837967 > > Thanks > +Vinod > > > On May 31, 2018, at 10:48 PM, Yongjun Zhang wrote: > > Greetings all, > > I've created the first re

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-07-10 Thread Yongjun Zhang
--Yongjun On Mon, Jul 9, 2018 at 2:08 PM, Jonathan Eagles wrote: > Thank you, Yongjun Zhang for resolving this issue for me. I have verified > the 3.0.3 build is now working for me for tez to specify as a hadoop > dependency. > > As for release procedure, can someone comment o

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-07-09 Thread Yongjun Zhang
today and now 2.9.1 is pushed to the mvn repo. >> >> You can double check it. I suspect sometimes Nexus may fail to notify >> user when this is unexpected failures. >> >> >> >> >> >> Bests, >> >> Sammi >> >> *From:* Yongj

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-16 Thread Yongjun Zhang
can resume the build with the > command > [ERROR] mvn -rf :hadoop-shim > > Not seeing the artifact published here as well. > https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-project > > On Tue, Jun 12, 2018 at 6:44 PM, Yongjun Zhang > wrote: > >> Thank

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-12 Thread Yongjun Zhang
Verified that various queue config properties for CS are refreshable > - Verified streaming jobs work as expected > - Verified that user weights work as expected > - Verified that FairOrderingPolicy in a CS queue will evenly assign > resources > - Verified running yarn shell applicatio

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-08 Thread Yongjun Zhang
istributed HDFS with KMS >>- Ran basic hdfs commands with encryption >>- Sanity checked webui and logs >> >> >> >> -Xiao >> >> On Fri, Jun 8, 2018 at 10:34 AM, Brahma Reddy Battula < >> brahmareddy.batt...@hotmail.com> wrote: >>

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-08 Thread Yongjun Zhang
Hello, PMC votes are needed for release, can I get some help from PMCs? Really appreciate it! --Yongjun On Thu, Jun 7, 2018 at 6:15 PM, Yongjun Zhang wrote: > Thank you all very much for the testing, feedback and discussion! > > I was able to build outside docker, by following

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-07 Thread Yongjun Zhang
BTW, thanks Allen and Steve for discussing and suggestion about the site build problem I hit earlier, I did the following step mvn install -DskipTests before doing the steps Nanda listed helped to solve the problems. --Yongjun On Thu, Jun 7, 2018 at 6:15 PM, Yongjun Zhang wrote: > Th

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-07 Thread Yongjun Zhang
Thank you all very much for the testing, feedback and discussion! I was able to build outside docker, by following the steps Nanda described, I saw the same problem; then I tried 3.0.2 released a while back, it has the same issue. As Allen pointed out, it seems the steps to build site are not cor

[jira] [Created] (HDFS-13663) Should throw exception when incorrect block size is set

2018-06-07 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-13663: Summary: Should throw exception when incorrect block size is set Key: HDFS-13663 URL: https://issues.apache.org/jira/browse/HDFS-13663 Project: Hadoop HDFS

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-06 Thread Yongjun Zhang
bu Asanuma > wrote: > >> Sorry, I forgot to write my vote in the last message. >> >> +1 (non-binding) >> >> Regards, >> Takanobu >> >> > -Original Message- >> > From: Takanobu Asanuma [mailto:tasan...@yahoo-corp.jp] >> >

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-02 Thread Yongjun Zhang
Hi Gabor, I got the git tag in, it's release-3.0.3-RC0. Would you please give it a try? It should correspond to commit 37fd7d752db73d984dc31e0cdfd590d252f5e075 Author: Yongjun Zhang Date: Wed May 30 00:07:33 2018 -0700 Update version to 3.0.3 to prepare for 3.0.3 release T

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-02 Thread Yongjun Zhang
t; - basic sanity check of NN, RM UI > > > > Thanks, > > Ajay > > > > On 6/2/18, 12:45 AM, "Shashikant Banerjee" < > sbaner...@hortonworks.com> > > wrote: > > > > Hi Yongjun, > >

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-01 Thread Yongjun Zhang
jobs work as expected > - Verified that user weights work as expected > - Verified that FairOrderingPolicy in a CS queue will evenly assign > resources > - Verified running yarn shell application runs as expected > > > > > > > > On Friday, June 1, 2018,

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-01 Thread Yongjun Zhang
>> https://issues.apache.org/jira/issues/?filter=12343997 > > Even after logging in, I am not able to see any jiras. > It says "The requested filter doesn't exist or is private." > > On Fri, Jun 1, 2018 at 10:41 AM, Yongjun Zhang > wrote: > >> Thank

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-06-01 Thread Yongjun Zhang
ems to be private. I can't see the >> blocker list. >> https://issues.apache.org/jira/issues/?filter=12343997 >> >> Meanwhile , I will be working on testing the release. >> >> Thanks >> Shashi >> On 6/1/18, 11:18 AM, "Yongjun Zhang" wr

[VOTE] Release Apache Hadoop 3.0.3 (RC0)

2018-05-31 Thread Yongjun Zhang
Greetings all, I've created the first release candidate (RC0) for Apache Hadoop 3.0.3. This is our next maintenance release to follow up 3.0.2. It includes about 249 important fixes and improvements, among which there are 8 blockers. See https://issues.apache.org/jira/issues/?filter=12343997 The

Re: Apache Hadoop 3.0.3 Release plan

2018-05-31 Thread Yongjun Zhang
rther. Sorry for the inconvenience. --Yongjun On Wed, May 30, 2018 at 11:49 AM, Yongjun Zhang wrote: > Hi, > > The build issues are all solved, and I have cut the 3.0.3 branch and close > to get a build out. Since it's taking me a bit more time (I expect to send > vote invitation

Re: Apache Hadoop 3.0.3 Release plan

2018-05-30 Thread Yongjun Zhang
Sorry, forgot to mention Junping Du, who also helped me a lot. Many thanks Junping! Thanks. --Yongjun On Wed, May 30, 2018 at 11:49 AM, Yongjun Zhang wrote: > Hi, > > The build issues are all solved, and I have cut the 3.0.3 branch and close > to get a build out. Since it'

Re: Apache Hadoop 3.0.3 Release plan

2018-05-30 Thread Yongjun Zhang
ndrew Wang, Eddy Xu who helped when I tried to solve the build issues. At this point, please be aware of the existence of branch-3.0, branch-3.0.3. Best, --Yongjun On Sat, May 26, 2018 at 11:52 PM, Yongjun Zhang wrote: > HI, > > I did build before cut branch and hit some issues, h

Re: Apache Hadoop 3.0.3 Release plan

2018-05-26 Thread Yongjun Zhang
HI, I did build before cut branch and hit some issues, have not got to the bottom, will cut branch after the build issues are resolved. Thanks. --Yongjun On Sat, May 26, 2018 at 1:46 PM, Yongjun Zhang wrote: > Hi All, > > I will be working on cutting the 3.0.3 branch and trying a bu

Re: Apache Hadoop 3.0.3 Release plan

2018-05-26 Thread Yongjun Zhang
Hi All, I will be working on cutting the 3.0.3 branch and trying a build today. Thanks. --Yongjun On Wed, May 23, 2018 at 3:31 PM, Yongjun Zhang wrote: > Thanks Eric. Sounds good. I may try to see if I can do the branching/RC > sooner. > > --Yongjun > > > On Wed, Ma

Re: Apache Hadoop 3.0.3 Release plan

2018-05-23 Thread Yongjun Zhang
> too early. > > Echoing Eric Payne, I think we should wait to cut the branch until we are > actually creating the RC to vote on (i.e. on May 29 or 30 if the vote is to > be on May 30). > > Eric > > > > On Wed, May 23, 2018 at 4:11 PM, Yongjun Zhang > wrote: >

Re: Apache Hadoop 3.0.3 Release plan

2018-05-23 Thread Yongjun Zhang
in 3.0.3 release. This is a fix that > will enable HBase to use Hadoop 3.0.x in the production line. > > thanks > Vrushali > > > On Tue, May 8, 2018 at 10:24 AM, Yongjun Zhang > wrote: > >> Thanks Wei-Chiu and Haibo for the feedback! >> >> Good thing is th

Re: [ANNOUNCE] Apache Hadoop 2.9.1 Release

2018-05-19 Thread Yongjun Zhang
please > refer to CHANGES [2] and RELEASENOTES [3]. > - For download, please got to download page[4] > > Thank you all for contributing to the Apache Hadoop 2.9.1. > > > Last, thanks Yongjun Zhang, Junping Du, Andrew Wang and Chris Douglas for > your help and support. > &

Re: Apache Hadoop 3.0.3 Release plan

2018-05-08 Thread Yongjun Zhang
>> downstream projects can't pick it up (after the fact that HBase is one of >> the most important projects around Hadoop). >> >> On Mon, May 7, 2018 at 1:19 PM, Yongjun Zhang >> wrote: >> >> > Hi Eric, >> > >> > Thanks for the

Re: Apache Hadoop 3.0.3 Release plan

2018-05-07 Thread Yongjun Zhang
-Eric Payne > > > > > > On Monday, May 7, 2018, 12:09:00 AM CDT, Yongjun Zhang < > yjzhan...@apache.org> wrote: > > > > > > Hi All, > > > > We have released Apache Hadoop 3.0.2 in April of this year [1]. Since then, > there are quite some

Apache Hadoop 3.0.3 Release plan

2018-05-06 Thread Yongjun Zhang
Hi All, > We have released Apache Hadoop 3.0.2 in April of this year [1]. Since then, there are quite some commits done to branch-3.0. To further improve the quality of release, we plan to do 3.0.3 release now. The focus of 3.0.3 will be fixing blockers (3), critical bugs (17) and bug fixes (~130)

[jira] [Created] (HDFS-13315) Add a test for the issue reported in HDFS-11481 which is fixed by HDFS-10997.

2018-03-19 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-13315: Summary: Add a test for the issue reported in HDFS-11481 which is fixed by HDFS-10997. Key: HDFS-13315 URL: https://issues.apache.org/jira/browse/HDFS-13315 Project

[jira] [Created] (HDFS-13115) Handle inode of a given inodeId already deleted

2018-02-06 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-13115: Summary: Handle inode of a given inodeId already deleted Key: HDFS-13115 URL: https://issues.apache.org/jira/browse/HDFS-13115 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-13101) Yet another fsimage corruption related to snapshot

2018-02-01 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-13101: Summary: Yet another fsimage corruption related to snapshot Key: HDFS-13101 URL: https://issues.apache.org/jira/browse/HDFS-13101 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-13100) Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs.

2018-02-01 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-13100: Summary: Handle IllegalArgumentException in when GETSERVERDEFAULTS is not implemented in webhdfs. Key: HDFS-13100 URL: https://issues.apache.org/jira/browse/HDFS-13100

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

2017-12-20 Thread Yongjun Zhang
Congratulations Andrew and all, a great milestone! Thanks Andrew for driving it! --Yongjun On Tue, Dec 19, 2017 at 9:10 AM, Jonathan Kelly wrote: > Thanks, Andrew! > > On Mon, Dec 18, 2017 at 4:54 PM Andrew Wang > wrote: > > > Thanks for the spot, I just pushed a correct tag. I can't delete th

[jira] [Created] (HDFS-12866) Recursive delete of a large directory or snapshot makes namenode unresponsive

2017-11-28 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12866: Summary: Recursive delete of a large directory or snapshot makes namenode unresponsive Key: HDFS-12866 URL: https://issues.apache.org/jira/browse/HDFS-12866 Project

Re: If primary replica is unresponsive, hsync() hangs

2017-09-11 Thread Yongjun Zhang
Thanks for finding the issue Wei-Chiu. I agree hsync should be handling DN failure similarly as write-pipeline recovery, as you stated. If it's not doing that, it should be fixed. --Yongjun On Mon, Sep 11, 2017 at 10:53 AM, Wei-Chiu Chuang wrote: > Hello my dear HDFS dev colleagues, > > It app

[jira] [Resolved] (HDFS-12296) Add a field to FsServerDefaults to tell if external attribute provider is enabled

2017-09-08 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang resolved HDFS-12296. -- Resolution: Won't Fix > Add a field to FsServerDefaults to tell if external attribute

[jira] [Resolved] (HDFS-12294) Let distcp to bypass external attribute provider when calling getFileStatus etc at source cluster

2017-09-08 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang resolved HDFS-12294. -- Resolution: Won't Fix > Let distcp to bypass external attribute provider when

[jira] [Created] (HDFS-12404) Name of config introduced by HDFS-12357 need to be changed from authorization.provider to attribute.provider

2017-09-07 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12404: Summary: Name of config introduced by HDFS-12357 need to be changed from authorization.provider to attribute.provider Key: HDFS-12404 URL: https://issues.apache.org/jira/browse

[jira] [Created] (HDFS-12360) TestLeaseRecoveryStriped.testLeaseRecovery failure

2017-08-26 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12360: Summary: TestLeaseRecoveryStriped.testLeaseRecovery failure Key: HDFS-12360 URL: https://issues.apache.org/jira/browse/HDFS-12360 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-12357) Let NameNode to bypass external attribute provider for special user

2017-08-25 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12357: Summary: Let NameNode to bypass external attribute provider for special user Key: HDFS-12357 URL: https://issues.apache.org/jira/browse/HDFS-12357 Project: Hadoop

[jira] [Resolved] (HDFS-12202) Provide new set of FileSystem API to bypass external attribute provider

2017-08-16 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang resolved HDFS-12202. -- Resolution: Won't Fix > Provide new set of FileSystem API to bypass external attribute

[jira] [Created] (HDFS-12296) Add a field to FsServerDefaults to tell if external attirbute provider is enabled

2017-08-13 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12296: Summary: Add a field to FsServerDefaults to tell if external attirbute provider is enabled Key: HDFS-12296 URL: https://issues.apache.org/jira/browse/HDFS-12296

[jira] [Created] (HDFS-12295) NameNode to support file path prefix /.reserved/bypassExtAttr

2017-08-12 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12295: Summary: NameNode to support file path prefix /.reserved/bypassExtAttr Key: HDFS-12295 URL: https://issues.apache.org/jira/browse/HDFS-12295 Project: Hadoop HDFS

[jira] [Created] (HDFS-12294) Support /.reserved/bypassExtAttr prefix to file path so getFileStatus etc can bypass external attribute provider

2017-08-12 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12294: Summary: Support /.reserved/bypassExtAttr prefix to file path so getFileStatus etc can bypass external attribute provider Key: HDFS-12294 URL: https://issues.apache.org/jira

[jira] [Created] (HDFS-12202) Provide new set of FileSystem API to bypass external attribute provider

2017-07-26 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12202: Summary: Provide new set of FileSystem API to bypass external attribute provider Key: HDFS-12202 URL: https://issues.apache.org/jira/browse/HDFS-12202 Project

[jira] [Created] (HDFS-12191) Provide option to not capture the accessTime change of a file to snapshot if no other modification has been done

2017-07-23 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12191: Summary: Provide option to not capture the accessTime change of a file to snapshot if no other modification has been done Key: HDFS-12191 URL: https://issues.apache.org/jira

[jira] [Created] (HDFS-12190) Enable 'hdfs dfs -stat' to display access time

2017-07-21 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12190: Summary: Enable 'hdfs dfs -stat' to display access time Key: HDFS-12190 URL: https://issues.apache.org/jira/browse/HDFS-12190 Project: Hadoop HDFS

[jira] [Created] (HDFS-12162) Update listStatus document to describe the behavior when the argument is a file

2017-07-19 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12162: Summary: Update listStatus document to describe the behavior when the argument is a file Key: HDFS-12162 URL: https://issues.apache.org/jira/browse/HDFS-12162

[jira] [Created] (HDFS-12139) liststatus returns incorrect pathSuffix for path of file

2017-07-13 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-12139: Summary: liststatus returns incorrect pathSuffix for path of file Key: HDFS-12139 URL: https://issues.apache.org/jira/browse/HDFS-12139 Project: Hadoop HDFS

[jira] [Created] (HDFS-11976) Examine code base for cases that exception is thrown from finally block and fix it

2017-06-14 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-11976: Summary: Examine code base for cases that exception is thrown from finally block and fix it Key: HDFS-11976 URL: https://issues.apache.org/jira/browse/HDFS-11976

[jira] [Created] (HDFS-11974) Fsimage transfer failed due to socket timeout, but logs doesn't show that

2017-06-14 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-11974: Summary: Fsimage transfer failed due to socket timeout, but logs doesn't show that Key: HDFS-11974 URL: https://issues.apache.org/jira/browse/HDFS-11974 Pr

[jira] [Created] (HDFS-11938) Logs for KMS delegation token lifecycle

2017-06-06 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-11938: Summary: Logs for KMS delegation token lifecycle Key: HDFS-11938 URL: https://issues.apache.org/jira/browse/HDFS-11938 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-11914) Add more diagnosis info for fsimage transfer failure.

2017-06-02 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-11914: Summary: Add more diagnosis info for fsimage transfer failure. Key: HDFS-11914 URL: https://issues.apache.org/jira/browse/HDFS-11914 Project: Hadoop HDFS

Re: Jenkins test failure

2017-05-17 Thread Yongjun Zhang
Regards, > Akira > > On 2017/05/17 18:38, Yongjun Zhang wrote: > >> Hi, >> >> I saw quite a few jenkins test failure for patches uploaded to jira, >> >> For example, >> https://builds.apache.org/job/PreCommit-HADOOP-Build/12347/console >> >&g

Jenkins test failure

2017-05-17 Thread Yongjun Zhang
Hi, I saw quite a few jenkins test failure for patches uploaded to jira, For example, https://builds.apache.org/job/PreCommit-HADOOP-Build/12347/console apache-yetus-2971eff/yetus-project/pom.xml Modes: Sentinel MultiJDK Jenkins Robot Docker ResetRepo UnitTests Processing: HADOOP-14407 HA

[jira] [Created] (HDFS-11799) Introduce a config to allow setting up write pipeline with single node

2017-05-10 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-11799: Summary: Introduce a config to allow setting up write pipeline with single node Key: HDFS-11799 URL: https://issues.apache.org/jira/browse/HDFS-11799 Project: Hadoop

[jira] [Created] (HDFS-11706) Enable fallback to regular distcp when distcp failed with snapshot diff

2017-04-26 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-11706: Summary: Enable fallback to regular distcp when distcp failed with snapshot diff Key: HDFS-11706 URL: https://issues.apache.org/jira/browse/HDFS-11706 Project

[jira] [Created] (HDFS-11656) RetryInvocationHandler may report ANN as SNN in messages.

2017-04-14 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-11656: Summary: RetryInvocationHandler may report ANN as SNN in messages. Key: HDFS-11656 URL: https://issues.apache.org/jira/browse/HDFS-11656 Project: Hadoop HDFS

[jira] [Created] (HDFS-11385) DistCp CopyCommitter should issue warning instead of throw exception when failed to delete file.

2017-02-01 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-11385: Summary: DistCp CopyCommitter should issue warning instead of throw exception when failed to delete file. Key: HDFS-11385 URL: https://issues.apache.org/jira/browse/HDFS-11385

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha2 RC0

2017-01-24 Thread Yongjun Zhang
Thanks Andrew much for the work here! +1 (binding). - Downloaded both binary and src tarballs - Verified md5 checksum and signature for both - Built from source tarball - Deployed 2 pseudo clusters, one with the released tarball and the other with what I built from source, and did the following

[jira] [Created] (HDFS-11292) log lastWrittenTxId in logSyncAll

2017-01-04 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-11292: Summary: log lastWrittenTxId in logSyncAll Key: HDFS-11292 URL: https://issues.apache.org/jira/browse/HDFS-11292 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-11040) Add documentation for HDFS-9820 distcp improvement

2016-10-20 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-11040: Summary: Add documentation for HDFS-9820 distcp improvement Key: HDFS-11040 URL: https://issues.apache.org/jira/browse/HDFS-11040 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-10993) rename may fail without a clear message indicating the failure reason.

2016-10-10 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10993: Summary: rename may fail without a clear message indicating the failure reason. Key: HDFS-10993 URL: https://issues.apache.org/jira/browse/HDFS-10993 Project: Hadoop

Re: [VOTE] Release Apache Hadoop 2.6.5 (RC1)

2016-10-07 Thread Yongjun Zhang
Hi Sangjin, Thanks a lot for your work here. My +1 (binding). - Downloaded both binary and src tarballs - Verified md5 checksum and signature for both - Build from source tarball - Deployed 2 pseudo clusters, one with the released tarball and the other with what I built from source, and did the

[jira] [Created] (HDFS-10961) Flaky test TestSnapshotFileLength.testSnapshotfileLength

2016-10-04 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10961: Summary: Flaky test TestSnapshotFileLength.testSnapshotfileLength Key: HDFS-10961 URL: https://issues.apache.org/jira/browse/HDFS-10961 Project: Hadoop HDFS

[jira] [Created] (HDFS-10943) rollEditLog expects empty EditsDoubleBuffer.bufCurrent which is not guaranteed

2016-09-30 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10943: Summary: rollEditLog expects empty EditsDoubleBuffer.bufCurrent which is not guaranteed Key: HDFS-10943 URL: https://issues.apache.org/jira/browse/HDFS-10943 Project

[jira] [Created] (HDFS-10942) Incorrect handling of flushing edit logs to JN

2016-09-30 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10942: Summary: Incorrect handling of flushing edit logs to JN Key: HDFS-10942 URL: https://issues.apache.org/jira/browse/HDFS-10942 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-10919) Provide admin/debug tool to dump out info of a given block

2016-09-27 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10919: Summary: Provide admin/debug tool to dump out info of a given block Key: HDFS-10919 URL: https://issues.apache.org/jira/browse/HDFS-10919 Project: Hadoop HDFS

[jira] [Created] (HDFS-10887) Provide admin/debug tool to dump block map

2016-09-21 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10887: Summary: Provide admin/debug tool to dump block map Key: HDFS-10887 URL: https://issues.apache.org/jira/browse/HDFS-10887 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-10812) ntermittent failure of TestRBWBlockInvalidation.testRWRInvalidation

2016-08-27 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10812: Summary: ntermittent failure of TestRBWBlockInvalidation.testRWRInvalidation Key: HDFS-10812 URL: https://issues.apache.org/jira/browse/HDFS-10812 Project: Hadoop

[jira] [Created] (HDFS-10811) Intermittent failure of TestRBWBlockInvalidation.testBlockInvalidationWhenRBWReplicaMissedInDN

2016-08-27 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10811: Summary: Intermittent failure of TestRBWBlockInvalidation.testBlockInvalidationWhenRBWReplicaMissedInDN Key: HDFS-10811 URL: https://issues.apache.org/jira/browse/HDFS-10811

[jira] [Resolved] (HDFS-10575) webhdfs fails with filenames including semicolons

2016-08-24 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang resolved HDFS-10575. -- Resolution: Duplicate > webhdfs fails with filenames including semicol

[jira] [Resolved] (HDFS-10788) fsck NullPointerException when it encounters corrupt replicas

2016-08-24 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang resolved HDFS-10788. -- Resolution: Duplicate Thanks guys, I'm marking it as duplicate of HDFS-9958. &

Re: Issue in handling checksum errors in write pipeline

2016-07-30 Thread Yongjun Zhang
Hi Brahma, Thanks for reporting the issue. If your problem is really a network issue, then your proposed solution sounds reasonable to me, and it's different than what HDFS-6937 intends to solve. I think we can create a new jira for your issue. Here is why: HDFS-6937's scenario is that we keep r

[jira] [Created] (HDFS-10698) Test org.apache.hadoop.cli.TestHDFSCLI fails in trunk

2016-07-27 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10698: Summary: Test org.apache.hadoop.cli.TestHDFSCLI fails in trunk Key: HDFS-10698 URL: https://issues.apache.org/jira/browse/HDFS-10698 Project: Hadoop HDFS

[jira] [Created] (HDFS-10667) Report more accurate info about data corruption location

2016-07-20 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10667: Summary: Report more accurate info about data corruption location Key: HDFS-10667 URL: https://issues.apache.org/jira/browse/HDFS-10667 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-10587) Incorrect offset/length calculation in pipeline recovery causes block corruption

2016-07-20 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang resolved HDFS-10587. -- Resolution: Duplicate Fix Version/s: 2.7.1, 2.6.4 Closing this jira as duplicate of HDFS

[jira] [Created] (HDFS-10652) Add a unit test for HDFS-4660

2016-07-18 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10652: Summary: Add a unit test for HDFS-4660 Key: HDFS-10652 URL: https://issues.apache.org/jira/browse/HDFS-10652 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-10632) DataXceiver to report the length of the block it's receiving

2016-07-14 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10632: Summary: DataXceiver to report the length of the block it's receiving Key: HDFS-10632 URL: https://issues.apache.org/jira/browse/HDFS-10632 Project: Hadoop

[jira] [Created] (HDFS-10625) VolumeScanner to report why a block is found bad

2016-07-13 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10625: Summary: VolumeScanner to report why a block is found bad Key: HDFS-10625 URL: https://issues.apache.org/jira/browse/HDFS-10625 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-10624) VolumeScanner to report why a block is found bad

2016-07-13 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10624: Summary: VolumeScanner to report why a block is found bad Key: HDFS-10624 URL: https://issues.apache.org/jira/browse/HDFS-10624 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-10603) Flaky test org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot.testWithCheckpoint

2016-07-08 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10603: Summary: Flaky test org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot.testWithCheckpoint Key: HDFS-10603 URL: https://issues.apache.org/jira/browse/HDFS

[jira] [Created] (HDFS-10396) Using -diff option with DistCp may get "Comparison method violates its general contract" exception

2016-05-12 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10396: Summary: Using -diff option with DistCp may get "Comparison method violates its general contract" exception Key: HDFS-10396 URL: https://issues.apache.org/jira/browse/

[jira] [Created] (HDFS-10387) DataTransferProtocol#writeBlock missing some javadocs

2016-05-10 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10387: Summary: DataTransferProtocol#writeBlock missing some javadocs Key: HDFS-10387 URL: https://issues.apache.org/jira/browse/HDFS-10387 Project: Hadoop HDFS

[jira] [Created] (HDFS-10386) DataTransferProtocol#writeBlock missing some javadocs

2016-05-10 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10386: Summary: DataTransferProtocol#writeBlock missing some javadocs Key: HDFS-10386 URL: https://issues.apache.org/jira/browse/HDFS-10386 Project: Hadoop HDFS

[jira] [Created] (HDFS-10376) setOwner call is not run as the specified user in TestPermission

2016-05-06 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10376: Summary: setOwner call is not run as the specified user in TestPermission Key: HDFS-10376 URL: https://issues.apache.org/jira/browse/HDFS-10376 Project: Hadoop HDFS

[jira] [Created] (HDFS-10333) Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk

2016-04-26 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10333: Summary: Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk Key: HDFS-10333 URL: https://issues.apache.org/jira/browse/HDFS-10333 Project: Hadoop

[jira] [Created] (HDFS-10314) Propose a new tool that wraps around distcp to "restore" changes on target cluster

2016-04-19 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10314: Summary: Propose a new tool that wraps around distcp to "restore" changes on target cluster Key: HDFS-10314 URL: https://issues.apache.org/jira/browse/

[jira] [Created] (HDFS-10313) Distcp does not check the order of snapshot names passed to -diff

2016-04-19 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10313: Summary: Distcp does not check the order of snapshot names passed to -diff Key: HDFS-10313 URL: https://issues.apache.org/jira/browse/HDFS-10313 Project: Hadoop HDFS

[jira] [Created] (HDFS-10263) Reversed snapshot diff report contains incorrect entries

2016-04-05 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10263: Summary: Reversed snapshot diff report contains incorrect entries Key: HDFS-10263 URL: https://issues.apache.org/jira/browse/HDFS-10263 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-10211) Add more info to DelegationTokenIdentifier#toString for better supportability

2016-03-24 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang resolved HDFS-10211. -- Resolution: Duplicate > Add more info to DelegationTokenIdentifier#toString for bet

[jira] [Created] (HDFS-10211) Add more info to DelegationTokenIdentifier#toString for better supportability

2016-03-24 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-10211: Summary: Add more info to DelegationTokenIdentifier#toString for better supportability Key: HDFS-10211 URL: https://issues.apache.org/jira/browse/HDFS-10211 Project

[jira] [Created] (HDFS-9939) Possible performance improvement by increasing buf size in DecompressorStream in HDFS

2016-03-10 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-9939: --- Summary: Possible performance improvement by increasing buf size in DecompressorStream in HDFS Key: HDFS-9939 URL: https://issues.apache.org/jira/browse/HDFS-9939

Re: CHANGES.txt is gone from trunk, branch-2, branch-2.8

2016-03-03 Thread Yongjun Zhang
That's nice, thanks Andrew and Allen. --Yongjun On Thu, Mar 3, 2016 at 9:11 PM, Andrew Wang wrote: > Hi all, > > With the inclusion of HADOOP-12651 going back to branch-2.8, CHANGES.txt > and release notes are now generated by Yetus. I've gone ahead and deleted > the manually updated CHANGES.tx

[jira] [Created] (HDFS-9889) Update balancer/mover document about HDFS-6133 feature

2016-03-02 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-9889: --- Summary: Update balancer/mover document about HDFS-6133 feature Key: HDFS-9889 URL: https://issues.apache.org/jira/browse/HDFS-9889 Project: Hadoop HDFS Issue

Re: Looking to a Hadoop 3 release

2016-02-19 Thread Yongjun Zhang
Thanks Andrew for initiating the effort! +1 on pushing 3.x with extended alpha cycle, and continuing the more stable 2.x releases. --Yongjun On Thu, Feb 18, 2016 at 5:58 PM, Andrew Wang wrote: > Hi Kai, > > Sure, I'm open to it. It's a new major release, so we're allowed to make > these kinds

[jira] [Created] (HDFS-9820) Improve distcp to support efficient restore

2016-02-17 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-9820: --- Summary: Improve distcp to support efficient restore Key: HDFS-9820 URL: https://issues.apache.org/jira/browse/HDFS-9820 Project: Hadoop HDFS Issue Type: New

  1   2   3   >