Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-28 Thread Zhe Zhang
+1 on RC1 - Verified signatures - Built from source (RHEL7) - Ran single node cluster and tested basic HDFS commands Thanks for putting this together Jonathan! On Mon, Oct 28, 2019 at 7:54 PM zhankun tang wrote: > Thanks for the efforts! Jonathan! > > +1 (non-binding) on RC1. > - Set up a sing

Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk

2015-09-29 Thread Zhe Zhang
Thanks everyone who has participated in this discussion. With 7 +1's (5 binding and 2 non-binding), and no -1, this vote has passed. I will do a final 'git merge' with trunk and work with Andrew to merge the branch to trunk. I'll update on this thread when the merge is done

Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk

2015-10-05 Thread Zhe Zhang
Thanks Vinay for capturing the issue and Uma for offering the help. --- Zhe Zhang On Mon, Oct 5, 2015 at 12:19 PM, Gangumalla, Uma wrote: > Vinay, > > > I would merge them as part of HDFS-9182. > > Thanks, > Uma > > > > On 10/5/15, 12:48 AM, "Vinayakuma

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

2015-11-02 Thread Zhe Zhang
Thanks Vinod for the proposal and Andrew/Jing for the comments. 2.9 sounds a good plan. As Andrew pointed out, 2.8 is already quite big. And even when disabled, EC logic has been baked in NN pretty deeply. Do we have a tentative date or estimate for 2.9? --- Zhe Zhang On Mon, Nov 2, 2015 at 1

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

2015-11-04 Thread Zhe Zhang
To add to the above discussion on umbrella JIRA: the COMMON side changes of EC have been tracked under HADOOP-11264 (before merge) and HADOOP-11842 (after merge). --- Zhe Zhang On Tue, Nov 3, 2015 at 4:40 PM, Vinod Vavilapalli wrote: > That makes sense. > > Thanks for the discussion

Update on potential Gerrit integration

2015-11-09 Thread Zhe Zhang
Hi, Per our earlier discussion on the "Github integration for Hadoop" thread, I surveyed HBase, Flume, and HTrace about their interest on Gerrit. The feedbacks have been positive: HBase: http://mail-archives.apache.org/mod_mbox/hbase-dev/201511.mbox/%3CCAAjoTXkeAR5xjNfWtEP87Y%3DPToaL7pyPeE2o1C4N5

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

2015-12-08 Thread Zhe Zhang
still having some questions about the stability of EC code. Maybe we should give Ming/Daryn some more time to reply to this thread? I think we should also address Elliot's comments above regarding major vs. minor releases, how that impacts the 2.9 upgrade timing etc. Thanks, --- Zhe Zhang On

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

2016-01-07 Thread Zhe Zhang
de is as Elliot stated, this will delay the upgrade to 2.9 for some customers. On Tue, Dec 8, 2015 at 2:54 PM, Zhe Zhang wrote: > Hi Vinod, > > Thanks for the update. After seeing the comment from Elliot I checked with > Ming Ma and Daryn Sharp offline, to get feedback based on thei

Re: [VOTE] Merge feature branch YARN-5734 (API based scheduler configuration) to trunk, branch-3.0, branch-2

2017-10-02 Thread Zhe Zhang
> > There were 15 subtasks completed for this feature. > > Huge thanks to everyone who helped with reviews, commits, guidance, and > technical discussion/design, including Carlo Curino, Xuan Gong, Subru > Krishnan, Min Shen, Konstantin Shvachko, Carl Steinbach, Wangda Tan, Vinod &g

Re: [VOTE] Release Apache Hadoop 2.7.5 (RC1)

2017-12-14 Thread Zhe Zhang
> http://home.apache.org/~shv/hadoop-2.7.5-RC1/releasenotes.html > > > > The RC0 is available at: http://home.apache.org/~shv/hadoop-2.7.5-RC1/ > > > > Please give it a try and vote on this thread. The vote will run for 5 > days > > ending 12/13/2017. > > > > My up to date public key is available from: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > Thanks, > > --Konstantin > > > > > > > > > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: [VOTE] Release Apache Hadoop 2.7.6 (RC0)

2018-04-13 Thread Zhe Zhang
t; > My up to date public key is available from: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > Thanks, > > --Konstantin > > > --------- > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: Looking to a Hadoop 3 release

2016-02-22 Thread Zhe Zhang
Thanks Andrew for driving the effort! +1 (non-binding) on starting the 3.0 release process now with 3.0 as an alpha. I wanted to echo Andrew's point that backporting EC to branch-2 is a lot of work. Considering that no concrete backporting plan has been proposed, it seems quite uncertain whether

Re: CHANGES.txt is gone from trunk, branch-2, branch-2.8

2016-03-03 Thread Zhe Zhang
CHANGES.txt updates will still > be > > necessary for backports to older release lines like 2.6.x and 2.7.x. > > > > Happy committing! > > > > Best, > > Andrew > > > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: Rise of the Dragon

2016-03-18 Thread Zhe Zhang
Seems like someone tried to "clone" all erasure coding JIRAs. On Wed, Mar 16, 2016 at 12:30 PM Colin P. McCabe wrote: > Who is creating all these dragon JIRAs? > > Colin >

[VOTE] Merge HDFS-7285 (erasure coding) branch to trunk

2015-09-22 Thread Zhe Zhang
nding votes are very welcome as well. And here's my non-binding +1. Thanks, --- Zhe Zhang

Re: [DISCUSS] Increased use of feature branches

2016-06-10 Thread Zhe Zhang
Thanks for the notes Andrew, Junping, Karthik. Here are some of my understandings: - Trunk is the "latest and greatest" of Hadoop. If a user starts using Hadoop today, without legacy workloads, trunk is what he/she should use. - Therefore, each commit to trunk should be transactional -- atomic, c

Re: HDFS Balancer - Recommended Threshold Value

2016-08-10 Thread Zhe Zhang
ay to pass param > only to move blocks from Over Utilized ( 98-100%) to under utilized ? > > > Pls advise! > > > > > Regards, > Senthil > -- Zhe Zhang Apache Hadoop Commiter http://zhe-thoughts.github.io/about/ | @oldcap

Re: HDFS Balancer - Balancing Effectively ?

2016-08-11 Thread Zhe Zhang
Thanks Nicholas for the pointers; the first 2 links are not working. On Thu, Aug 11, 2016 at 10:09 AM Tsz Wo Sze wrote: > Hi Senthi > The Balancer performance was improved dramatically recently [1]. I am not > sure if you know about the new conf and parameters; see [2]. If you are > interested

Re: NFS Gateway Performance

2016-08-11 Thread Zhe Zhang
is going through Single Machine So this causing > performance issue .. I tried with small data set in case of huge data set > how it will perform ?? > > > Direct HDFS put gives better performance .. Any Suggestion ? > > > Regards, > Senthil > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-08-30 Thread Zhe Zhang
Thanks Andrew for the great work! It's really exciting to finally see a Hadoop 3 RC. I noticed CHANGES and RELEASENOTES markdown files which were not in previous RCs like 2.7.3. What are good tools to verify them? I tried reading them on IntelliJ but format looks odd. I'm still testing the RC: -

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

2016-08-30 Thread Zhe Zhang
+1 (non-binding) Did the following on 7 RHEL 6.6 servers - Downloaded and built from source - Downloaded and verified checksum of the binary tar.gz file - Setup a cluster with 1 NN and 6 DNs - Tried regular HDFS commands - Tried EC commands (listPolicies, getPolicy, setPolicy), they work fine - Ve

Listing large directories via WebHDFS

2016-10-19 Thread Zhe Zhang
Hi, The regular HDFS client (DistributedFileSystem) throttles the workload of listing large directories by dividing the work into batches, something like below: {code} // fetch the first batch of entries in the directory DirectoryListing thisListing = dfs.listPaths( src, HdfsFileSt

Re: Listing large directories via WebHDFS

2016-10-19 Thread Zhe Zhang
ion features for webhdfs/httpfs via > https://issues.apache.org/jira/browse/HDFS-10784 and > https://issues.apache.org/jira/browse/HDFS-10823, to provide better > control. > > Best, > > -Xiao > > On Wed, Oct 19, 2016 at 2:08 PM, Zhe Zhang wrote: > > Hi, > >

Re: Updated 2.8.0-SNAPSHOT artifact

2016-11-01 Thread Zhe Zhang
repository.apache.org/content/repositories/snapshots/org/ap > >>>> ach > >>>> e/hadoop/hadoop-yarn-api/ > >>>> > >>>> Are there any plans to start producing updated SNAPSHOT artifacts > >>>> for current hadoop development lines? > >>>>

Re: Multiple node Hadoop cluster on Docker for test and debugging

2016-11-09 Thread Zhe Zhang
blob/master/README.md> > > I would be glad if you use this tool for developing and debugging and make > our development more efficient. > Please give me any feedbacks to me. Thanks you! > > > Kai Sasaki > mail: lewua...@me.com <mailto:lewua...@me.com> > github: https://github.com/Lewuathe <https://github.com/Lewuathe> > > > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Jenkins patch-compile fails on trunk

2016-11-10 Thread Zhe Zhang
https://builds.apache.org/job/PreCommit-HDFS-Build/17517/artifact/patchprocess/patch-compile-root.txt Looks related to yarn-ui? The HDFS-10872 patch doesn't touch YARN, but the patch-compile failed at YARN. Thanks, Zhe

Re: About 2.7.4 Release

2017-05-03 Thread Zhe Zhang
browse/YARN-4367>. SLS > YARN-4612 <https://issues.apache.org/jira/browse/YARN-4612>. SLS > > - Node labeling > MAPREDUCE-6304 <https://issues.apache.org/jira/browse/MAPREDUCE-6304> > YARN-2943 <https://issues.apache.org/jira/browse/YARN-2943> > YARN-4109

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-08-02 Thread Zhe Zhang
Apache release policy > > > >>> http://www.apache.org/legal/release-policy.html > > > >>> I edited the HowToReleasePreDSBCR page to make sure people don't > use > > > >>> Jenkins option for building. > > > >>> > >

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-08-02 Thread Zhe Zhang
>> > > On Jul 31, 2017, at 11:20 AM, Konstantin Shvachko < > >>> shv.had...@gmail.com> > >>> > wrote: > >>> > > > >>> > > https://wiki.apache.org/hadoop/HowToReleasePreDSBCR > >>> > > >>> > FYI: > >>> > > >>> > If you are using ASF Jenkins to create an ASF release > >>> > artifact, it's pretty much an automatic vote failure as any such > >>> release is > >>> > in violation of ASF policy. > >>> > > >>> > > >>> > >> > >> > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: [RESULT] [VOTE] Release Apache Hadoop 2.7.4 (RC0)

2017-08-07 Thread Zhe Zhang
r contributing to the release, testing it, and > voting. > > Binding +1s (7) > Zhe Zhang > Jason Lowe > Eric Payne > Sunil G > Akira Ajisaka > Chris Douglas > Konstantin Shvachko > > Non-binding +1s (13) > John Zhuge > Surendra Lilhore > Masatake Iwa

Re: Controlling the block placement and the file placement in HDFS writes

2014-12-18 Thread Zhe Zhang
BlockPlacementPolicy from inode or file path to DataNode address. My 2 cents.. -- Zhe Zhang Software Engineer, Cloudera https://sites.google.com/site/zhezhangresearch/

Re: [VOTE] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-05 Thread Zhe Zhang
4. As testing progresses, we continue fixing non-critical bugs like > HDFS-14116. > > I attached a unified patch to the umbrella jira for the review and Jenkins > build. > Please vote on this thread. The vote will run for 7 days until Wed Dec 12. > > Thanks, > --Konstantin > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: [VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-15 Thread Zhe Zhang
will run for 7 days until Wed Dec 21. > > Thanks, > --Konstantin > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-02-01 Thread Zhe Zhang
+1 on the proposal and looking forward to the progress of the project! On Thu, Jan 31, 2019 at 10:51 PM Weiwei Yang wrote: > Thanks for proposing this Wangda, my +1 as well. > It is amazing to see the progress made in Submarine last year, the > community grows fast and quiet collaborative. I can

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

2019-02-04 Thread Zhe Zhang
gt; Wangda Tan > > [1] > > https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E > > > -- Zhe Zhang Apache Hadoop Committer http://zhe-thoughts.github.io/about/ | @oldcap

Re: [VOTE] Moving branch-2 precommit/nightly test builds to java 8

2019-02-04 Thread Zhe Zhang
/ > https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-YARN-Build/ > > https://builds.apache.org/view/H-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/ > > [3] > > https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/ > > Jonathan

Re: [VOTE] Merge YARN-8200 to branch-2 and branch-3.0

2019-08-26 Thread Zhe Zhang
+1 (binding) As Jonathan said this feature in branch-2 has been running stably for over a year. Therefore I’m supportive to the merge On Mon, Aug 26, 2019 at 2:00 PM Jim Brennan wrote: > +1 (non-binding). > I have built branch-2 with the latest YARN-8200 patch > (YARN-8200-branch-2.003.patch).

Re: [VOTE] Moving Submarine to a separate Apache project proposal

2019-09-06 Thread Zhe Zhang
My late +1. Me and my colleagues have been involved in Submarine development (in collaboration with our TonY project). I think Submarine will be valuable as a new Apache project. On Fri, Sep 6, 2019 at 10:57 PM Dinesh Chitlangia wrote: > +1 > > -Dinesh > > > > > On Fri, Sep 6, 2019 at 11:23 PM 俊

[jira] [Resolved] (HDFS-7285) Erasure Coding Support inside HDFS

2015-09-30 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7285. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 I just did a final {{git merge

[jira] [Created] (HDFS-9229) Expose size of NameNode directory as a metric

2015-10-12 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9229: --- Summary: Expose size of NameNode directory as a metric Key: HDFS-9229 URL: https://issues.apache.org/jira/browse/HDFS-9229 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-9280) Document NFS gateway export point parameter

2015-10-21 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9280: --- Summary: Document NFS gateway export point parameter Key: HDFS-9280 URL: https://issues.apache.org/jira/browse/HDFS-9280 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-9329) TestBootstrapStandby#testRateThrottling is flaky because fsimage size is smaller than IO buffer size

2015-10-28 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9329: --- Summary: TestBootstrapStandby#testRateThrottling is flaky because fsimage size is smaller than IO buffer size Key: HDFS-9329 URL: https://issues.apache.org/jira/browse/HDFS-9329

[jira] [Created] (HDFS-9341) Simulated slow disk in SimulatedFSDataset

2015-10-29 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9341: --- Summary: Simulated slow disk in SimulatedFSDataset Key: HDFS-9341 URL: https://issues.apache.org/jira/browse/HDFS-9341 Project: Hadoop HDFS Issue Type: Test

[jira] [Created] (HDFS-9342) Erasure coding: client should update and commit block based on acknowledged size

2015-10-29 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9342: --- Summary: Erasure coding: client should update and commit block based on acknowledged size Key: HDFS-9342 URL: https://issues.apache.org/jira/browse/HDFS-9342 Project: Hadoop

[jira] [Created] (HDFS-9344) QUEUE_WITH_CORRUPT_BLOCKS is no longer needed

2015-10-29 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9344: --- Summary: QUEUE_WITH_CORRUPT_BLOCKS is no longer needed Key: HDFS-9344 URL: https://issues.apache.org/jira/browse/HDFS-9344 Project: Hadoop HDFS Issue Type: Bug

[jira] [Resolved] (HDFS-9344) QUEUE_WITH_CORRUPT_BLOCKS is no longer needed

2015-10-30 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9344. - Resolution: Invalid Thanks Xiao! Good analysis! Closing this as invalid. > QUEUE_WITH_CORRUPT_BLOCKS

[jira] [Created] (HDFS-9386) Erasure coding: updateBlockForPipeline sometimes returns non-striped block for striped file

2015-11-05 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9386: --- Summary: Erasure coding: updateBlockForPipeline sometimes returns non-striped block for striped file Key: HDFS-9386 URL: https://issues.apache.org/jira/browse/HDFS-9386

[jira] [Created] (HDFS-9403) Erasure coding: some EC tests are missing timeout

2015-11-09 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9403: --- Summary: Erasure coding: some EC tests are missing timeout Key: HDFS-9403 URL: https://issues.apache.org/jira/browse/HDFS-9403 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2015-11-09 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9405: --- Summary: When starting a file, NameNode should generate EDEK in a separate thread Key: HDFS-9405 URL: https://issues.apache.org/jira/browse/HDFS-9405 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-9496) Erasure coding: an erasure codec throughput benchmark tool

2015-12-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9496. - Resolution: Duplicate Thanks Rui for clarifying this. > Erasure coding: an erasure codec through

[jira] [Resolved] (HDFS-9529) Extend Erasure Code to support POWER Chip acceleration

2015-12-09 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9529. - Resolution: Duplicate Thanks Qijun for confirming this. Resolving this one as dup. > Extend Erasure C

[jira] [Resolved] (HDFS-7691) Handle hflush and hsync in the best optimal way possible during online Erasure encoding

2015-12-15 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7691. - Resolution: Duplicate Thanks Vinay for confirming. Let's track the hflush related efforts under HDFS

[jira] [Created] (HDFS-9576) HTrace: collect path/offset/length information on read and write operations

2015-12-18 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9576: --- Summary: HTrace: collect path/offset/length information on read and write operations Key: HDFS-9576 URL: https://issues.apache.org/jira/browse/HDFS-9576 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-9578) Incorrect default value (typo) in hdfs-default.xml

2015-12-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9578. - Resolution: Duplicate Thanks for reporting this [~tianyin]. It has been fixed in trunk already

[jira] [Created] (HDFS-9644) Update encryption documentation to reflect nested EZs

2016-01-12 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9644: --- Summary: Update encryption documentation to reflect nested EZs Key: HDFS-9644 URL: https://issues.apache.org/jira/browse/HDFS-9644 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-12502) nntop should support category based on FilesInGetListingOps

2017-09-19 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-12502: Summary: nntop should support category based on FilesInGetListingOps Key: HDFS-12502 URL: https://issues.apache.org/jira/browse/HDFS-12502 Project: Hadoop HDFS

[jira] [Reopened] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-12502: -- > nntop should support a category based on FilesInGetListing

[jira] [Created] (HDFS-12888) NameNode web UI shows stale config values after cli refresh

2017-12-04 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-12888: Summary: NameNode web UI shows stale config values after cli refresh Key: HDFS-12888 URL: https://issues.apache.org/jira/browse/HDFS-12888 Project: Hadoop HDFS

[jira] [Created] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-01-22 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9688: --- Summary: Test the effect of nested encryption zones in HDFS downgrade Key: HDFS-9688 URL: https://issues.apache.org/jira/browse/HDFS-9688 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-9698) Long running Balancer should renew TGT

2016-01-25 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9698: --- Summary: Long running Balancer should renew TGT Key: HDFS-9698 URL: https://issues.apache.org/jira/browse/HDFS-9698 Project: Hadoop HDFS Issue Type: Bug

[jira] [Resolved] (HDFS-9172) Erasure Coding: Move DFSStripedIO stream related classes to hadoop-hdfs-client

2016-02-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-9172. - Resolution: Invalid > Erasure Coding: Move DFSStripedIO stream related classes to hadoop-hdfs-cli

[jira] [Created] (HDFS-9785) Remove unused TrashPolicy#getInstance code

2016-02-09 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9785: --- Summary: Remove unused TrashPolicy#getInstance code Key: HDFS-9785 URL: https://issues.apache.org/jira/browse/HDFS-9785 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-9799) Reimplement getCurrentTrashDir to remove incompatibility

2016-02-12 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9799: --- Summary: Reimplement getCurrentTrashDir to remove incompatibility Key: HDFS-9799 URL: https://issues.apache.org/jira/browse/HDFS-9799 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-9844) Correct path creation in getTrashRoot to handle root dir

2016-02-22 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-9844: --- Summary: Correct path creation in getTrashRoot to handle root dir Key: HDFS-9844 URL: https://issues.apache.org/jira/browse/HDFS-9844 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-7929) inotify unable fetch pre-upgrade edit log segments once upgrade starts

2015-03-13 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-7929: --- Summary: inotify unable fetch pre-upgrade edit log segments once upgrade starts Key: HDFS-7929 URL: https://issues.apache.org/jira/browse/HDFS-7929 Project: Hadoop HDFS

[jira] [Created] (HDFS-7936) Erasure coding: resolving conflicts when merging with HDFS-7903 and HDFS-7435

2015-03-16 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-7936: --- Summary: Erasure coding: resolving conflicts when merging with HDFS-7903 and HDFS-7435 Key: HDFS-7936 URL: https://issues.apache.org/jira/browse/HDFS-7936 Project: Hadoop HDFS

[jira] [Created] (HDFS-7955) Improve naming of classes, methods, and variables related to block replication and recovery

2015-03-18 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-7955: --- Summary: Improve naming of classes, methods, and variables related to block replication and recovery Key: HDFS-7955 URL: https://issues.apache.org/jira/browse/HDFS-7955

[jira] [Resolved] (HDFS-7369) Erasure coding: distribute recovery work for striped blocks to DataNode

2015-03-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7369. - Resolution: Fixed > Erasure coding: distribute recovery work for striped blocks to DataN

[jira] [Created] (HDFS-7969) Erasure coding: lease recovery for striped block groups

2015-03-20 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-7969: --- Summary: Erasure coding: lease recovery for striped block groups Key: HDFS-7969 URL: https://issues.apache.org/jira/browse/HDFS-7969 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-7793) Refactor DFSOutputStream separating DataStreamer out

2015-03-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7793. - Resolution: Fixed > Refactor DFSOutputStream separating DataStreamer

[jira] [Created] (HDFS-8010) Erasure coding: extend UnderReplicatedBlocks to accurately handle striped blocks

2015-03-27 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8010: --- Summary: Erasure coding: extend UnderReplicatedBlocks to accurately handle striped blocks Key: HDFS-8010 URL: https://issues.apache.org/jira/browse/HDFS-8010 Project: Hadoop

[jira] [Created] (HDFS-8030) HDFS Erasure Coding Phase II -- EC with contiguous layout

2015-03-31 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8030: --- Summary: HDFS Erasure Coding Phase II -- EC with contiguous layout Key: HDFS-8030 URL: https://issues.apache.org/jira/browse/HDFS-8030 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-8031) Follow-on work for erasure coding phase I (striping layout)

2015-03-31 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8031: --- Summary: Follow-on work for erasure coding phase I (striping layout) Key: HDFS-8031 URL: https://issues.apache.org/jira/browse/HDFS-8031 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-7617) Add unit tests for editlog transactions for EC

2015-03-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7617. - Resolution: Fixed Fix Version/s: HDFS-7285 > Add unit tests for editlog transactions for

[jira] [Resolved] (HDFS-7689) Add periodic checker to find the corrupted EC blocks/files

2015-03-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7689. - Resolution: Duplicate > Add periodic checker to find the corrupted EC blocks/fi

[jira] [Resolved] (HDFS-7783) Support appending to a striping layout file

2015-03-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7783. - Resolution: Duplicate > Support appending to a striping layout f

[jira] [Resolved] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

2015-03-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7729. - Resolution: Duplicate > Add logic to DFSOutputStream to support writing a file in striping lay

[jira] [Resolved] (HDFS-7981) getStoragePolicy() regards HOT policy as EC policy

2015-03-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7981. - Resolution: Won't Fix HDFS-7839 is close to finish. > getStoragePolicy() regards HOT policy as E

[jira] [Created] (HDFS-8032) Erasure coding: use sentinels in BlockInfoStriped and shrink size of indices

2015-03-31 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8032: --- Summary: Erasure coding: use sentinels in BlockInfoStriped and shrink size of indices Key: HDFS-8032 URL: https://issues.apache.org/jira/browse/HDFS-8032 Project: Hadoop HDFS

[jira] [Created] (HDFS-8033) Erasure coding: stateful (non-positional) read from files in striped layout

2015-03-31 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8033: --- Summary: Erasure coding: stateful (non-positional) read from files in striped layout Key: HDFS-8033 URL: https://issues.apache.org/jira/browse/HDFS-8033 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-7839) Erasure coding: implement facilities in NameNode to create and manage EC zones

2015-04-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7839. - Resolution: Fixed Fix Version/s: HDFS-7285 > Erasure coding: implement facilities in NameNode

[jira] [Resolved] (HDFS-7969) Erasure coding: NameNode support for lease recovery of striped block groups

2015-04-06 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7969. - Resolution: Fixed Fix Version/s: HDFS-7285 > Erasure coding: NameNode support for lease recovery

[jira] [Created] (HDFS-8076) Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible

2015-04-07 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8076: --- Summary: Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible Key: HDFS-8076 URL: https://issues.apache.org/jira/browse/HDFS-8076 Project: Hadoop

[jira] [Resolved] (HDFS-7782) Erasure coding: pread from files in striped layout

2015-04-07 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7782. - Resolution: Fixed Fix Version/s: HDFS-7285 > Erasure coding: pread from files in striped lay

[jira] [Created] (HDFS-8077) Erasure coding: fix bug in EC zone and symlinks

2015-04-07 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8077: --- Summary: Erasure coding: fix bug in EC zone and symlinks Key: HDFS-8077 URL: https://issues.apache.org/jira/browse/HDFS-8077 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-8117) More accurate verification in SimulatedFSDataset: replace DEFAULT_DATABYTE with patterned data

2015-04-09 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8117: --- Summary: More accurate verification in SimulatedFSDataset: replace DEFAULT_DATABYTE with patterned data Key: HDFS-8117 URL: https://issues.apache.org/jira/browse/HDFS-8117

[jira] [Resolved] (HDFS-8124) DFSStripedInputStream fails to read data after one stripe

2015-04-10 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8124. - Resolution: Duplicate The HDFS-8120 patch also fixes this issue. > DFSStripedInputStream fails to r

[jira] [Resolved] (HDFS-8122) Erasure Coding: Support specifying ECSchema during creation of ECZone

2015-04-13 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8122. - Tags: reviewed Resolution: Fixed Fix Version/s: HDFS-7285 > Erasure Coding: Supp

[jira] [Resolved] (HDFS-8114) Erasure coding: Add auditlog FSNamesystem#createErasureCodingZone if this operation fails

2015-04-13 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8114. - Resolution: Fixed Fix Version/s: HDFS-7285 Thanks Rakesh for the work and Uma for the review! I

[jira] [Created] (HDFS-8178) QJM doesn't purge empty and corrupt inprogress edits files

2015-04-17 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8178: --- Summary: QJM doesn't purge empty and corrupt inprogress edits files Key: HDFS-8178 URL: https://issues.apache.org/jira/browse/HDFS-8178 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-8167) BlockManager.addBlockCollectionWithCheck should check if the block is a striped block

2015-04-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8167. - Resolution: Fixed > BlockManager.addBlockCollectionWithCheck should check if the block is a > s

[jira] [Created] (HDFS-8188) Erasure coding: refactor client-related code to sync with HDFS-8082 and HDFS-8169

2015-04-20 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8188: --- Summary: Erasure coding: refactor client-related code to sync with HDFS-8082 and HDFS-8169 Key: HDFS-8188 URL: https://issues.apache.org/jira/browse/HDFS-8188 Project: Hadoop

[jira] [Resolved] (HDFS-8188) Erasure coding: refactor client-related code to sync with HDFS-8082 and HDFS-8169

2015-04-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8188. - Resolution: Fixed Fix Version/s: HDFS-7285 Thanks Nicholas for the prompt feedback! I've comm

[jira] [Created] (HDFS-8191) Fix byte to integer casting in SimulatedFSDataset#simulatedByte

2015-04-20 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8191: --- Summary: Fix byte to integer casting in SimulatedFSDataset#simulatedByte Key: HDFS-8191 URL: https://issues.apache.org/jira/browse/HDFS-8191 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-8216) TestDFSStripedOutputStream should use BlockReaderTestUtil to create BlockReader

2015-04-21 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8216. - Resolution: Fixed Hadoop Flags: Reviewed > TestDFSStripedOutputStream should use BlockReaderTestU

[jira] [Resolved] (HDFS-8212) DistributedFileSystem.createErasureCodingZone should pass schema in FileSystemLinkResolver

2015-04-21 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8212. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed

[jira] [Resolved] (HDFS-8156) Add/implement necessary APIs even we just have the system default schema

2015-04-22 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8156. - Resolution: Fixed Hadoop Flags: Reviewed Thanks Kai for the contribution and Nicholas for the

[jira] [Created] (HDFS-8239) Erasure coding: [bug] should always allocate unique striped block group IDs

2015-04-23 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8239: --- Summary: Erasure coding: [bug] should always allocate unique striped block group IDs Key: HDFS-8239 URL: https://issues.apache.org/jira/browse/HDFS-8239 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-8239) Erasure coding: [bug] should always allocate unique striped block group IDs

2015-04-23 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8239. - Resolution: Duplicate > Erasure coding: [bug] should always allocate unique striped block group

[jira] [Resolved] (HDFS-8228) Erasure Coding: SequentialBlockGroupIdGenerator#nextValue may cause block id conflicts

2015-04-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8228. - Resolution: Fixed Assignee: Jing Zhao (was: Zhe Zhang) Hadoop Flags: Reviewed > Eras

  1   2   3   >