[jira] [Created] (HDFS-16341) Add block placement policy desc

guo created HDFS-16341: -- Summary: Add block placement policy desc Key: HDFS-16341 URL: https://issues.apache.org/jira/browse/HDFS-16341 Project: Hadoop HDFS Issue Type: Improvement Components

[jira] [Created] (HDFS-14786) A new block placement policy tolerating availability zone failure

Mingliang Liu created HDFS-14786: Summary: A new block placement policy tolerating availability zone failure Key: HDFS-14786 URL: https://issues.apache.org/jira/browse/HDFS-14786 Project: Hadoop HDFS

[jira] [Created] (HDFS-13156) HDFS Block Placement Policy - Client Local Rack

BELUGA BEHR created HDFS-13156: -- Summary: HDFS Block Placement Policy - Client Local Rack Key: HDFS-13156 URL: https://issues.apache.org/jira/browse/HDFS-13156 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-12008) Improve the available-space block placement policy

Kihwal Lee created HDFS-12008: - Summary: Improve the available-space block placement policy Key: HDFS-12008 URL: https://issues.apache.org/jira/browse/HDFS-12008 Project: Hadoop HDFS Issue Type

Re: HDFS Erasuring Coding Block placement policy related reconstruction work not scheduled appropriately

Thanks Rakesh for your comments. I’ve filed a jira for discussion https://issues.apache.org/jira/browse/HDFS-10530 Looking forward to discussing, and reach to an agreement. Thanks, Rui On 6/9/16, 18:05, "Rakesh Radhakrishnan" wrote: Thanks Rui for reporting this. With "RS-DEFAULT-6-3-64k EC

[jira] [Created] (HDFS-10530) BlockManager reconstruction work scheduling should correctly adhere to EC block placement policy

GAO Rui created HDFS-10530: -- Summary: BlockManager reconstruction work scheduling should correctly adhere to EC block placement policy Key: HDFS-10530 URL: https://issues.apache.org/jira/browse/HDFS-10530

Re: HDFS Erasuring Coding Block placement policy related reconstruction work not scheduled appropriately

Thanks Rui for reporting this. With "RS-DEFAULT-6-3-64k EC policy" EC file will have 6 data blocks and 3 parity blocks. Like you described initially the cluster has 5 racks, so the first 5 data blocks will use those racks. Now while adding rack-6, reconstruction task will be scheduled for placing

HDFS Erasuring Coding Block placement policy related reconstruction work not scheduled appropriately

Hi all, We found out that under RS-DEFAULT-6-3-64k EC policy, if an EC file was witten to 5 racks, reconstruction work would be scheduled if the 6th rack is added. While adding the 7th rack or more racks will not trigger reconstruction work. Based on “BlockPlacementPolicyRackFaultTolerant.java”,

[jira] [Created] (HDFS-10036) CLONE - Erasure coding: Make block placement policy for EC file configurable

dragon created HDFS-10036: - Summary: CLONE - Erasure coding: Make block placement policy for EC file configurable Key: HDFS-10036 URL: https://issues.apache.org/jira/browse/HDFS-10036 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-9361) Default block placement policy causes TestReplaceDataNodeOnFailure to fail intermittently

also agree this is not a problem. The test can be configured to ignore load factor. > Default block placement policy causes TestReplaceDataNodeOnFailure to fail > intermittently > - > >

[jira] [Created] (HDFS-9361) Default block placement policy causes TestReplaceDataNodeOnFailure to fail intermittently

Wei-Chiu Chuang created HDFS-9361: - Summary: Default block placement policy causes TestReplaceDataNodeOnFailure to fail intermittently Key: HDFS-9361 URL: https://issues.apache.org/jira/browse/HDFS-9361

[jira] [Created] (HDFS-9083) Replication violates block placement policy.

Rushabh S Shah created HDFS-9083: Summary: Replication violates block placement policy. Key: HDFS-9083 URL: https://issues.apache.org/jira/browse/HDFS-9083 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-9014) Block placement policy with respect to DN free space

ock placement policy with respect to DN free space > > > Key: HDFS-9014 > URL: https://issues.apache.org/jira/browse/HDFS-9014 > Project: Hadoop HDFS > Issue Type: Improvement

[jira] [Created] (HDFS-9014) Block placement policy with respect to DN free space

Max Lapan created HDFS-9014: --- Summary: Block placement policy with respect to DN free space Key: HDFS-9014 URL: https://issues.apache.org/jira/browse/HDFS-9014 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-8789) Block Placement policy migrator

Chris Trezzo created HDFS-8789: -- Summary: Block Placement policy migrator Key: HDFS-8789 URL: https://issues.apache.org/jira/browse/HDFS-8789 Project: Hadoop HDFS Issue Type: New Feature

HDFS-8131:a space balanced block placement policy

Hi all: We implemented an balanced block placement policy in HDFS-8131. https://issues.apache.org/jira/browse/HDFS-8131 Could someone help to review and push this patch? This policy will choose the low used percent datanodes for new blocks with a little high possibility. In a not long term

[jira] [Created] (HDFS-8186) Erasure coding: Make block placement policy for EC file configurable

Walter Su created HDFS-8186: --- Summary: Erasure coding: Make block placement policy for EC file configurable Key: HDFS-8186 URL: https://issues.apache.org/jira/browse/HDFS-8186 Project: Hadoop HDFS

[jira] [Created] (HDFS-8131) Implement a space blance block placement policy

Liu Shaohui created HDFS-8131: - Summary: Implement a space blance block placement policy Key: HDFS-8131 URL: https://issues.apache.org/jira/browse/HDFS-8131 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-7892) a block placement policy with best fault tolerance

[ https://issues.apache.org/jira/browse/HDFS-7892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su resolved HDFS-7892. - Resolution: Duplicate > a block placement policy with best fault tolera

[jira] [Created] (HDFS-7892) a block placement policy with best fault tolerance

Walter Su created HDFS-7892: --- Summary: a block placement policy with best fault tolerance Key: HDFS-7892 URL: https://issues.apache.org/jira/browse/HDFS-7892 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-7891) a block placement policy with best fault tolerance

Walter Su created HDFS-7891: --- Summary: a block placement policy with best fault tolerance Key: HDFS-7891 URL: https://issues.apache.org/jira/browse/HDFS-7891 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-7613) Block placement policy for erasure coding groups

Zhe Zhang created HDFS-7613: --- Summary: Block placement policy for erasure coding groups Key: HDFS-7613 URL: https://issues.apache.org/jira/browse/HDFS-7613 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-6454) Configurable Block Placement Policy

Guo Ruijing created HDFS-6454: - Summary: Configurable Block Placement Policy Key: HDFS-6454 URL: https://issues.apache.org/jira/browse/HDFS-6454 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-4931) Extend the block placement policy interface to utilize the location information of previously stored files

Jihoon Son created HDFS-4931: Summary: Extend the block placement policy interface to utilize the location information of previously stored files Key: HDFS-4931 URL: https://issues.apache.org/jira/browse/HDFS-4931

[jira] [Created] (HDFS-4884) [Umbrella] Block Placement Policy Optimizer

Junping Du created HDFS-4884: Summary: [Umbrella] Block Placement Policy Optimizer Key: HDFS-4884 URL: https://issues.apache.org/jira/browse/HDFS-4884 Project: Hadoop HDFS Issue Type: Bug

Re: Block placement Policy

Have a look at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault. This is default placement policy, you should be able to extend this to implement your own policy and set config parameter "dfs.block.replicator.classname" to point to your class On Sat, May 4, 2013 at 8:57 P

Block placement Policy

I have read somewhere that a user can specified his own ReplicaPlacementPolicy. How can I specify my own ReplicaPlacementPolicy? I you have any sample ReplicaPlacementPolicy, then please share it.. -- *With regards ---* *Mohammad Mustaqeem*, M.Tech (CSE) MNNIT Allahabad 9026604270

[jira] [Reopened] (HDFS-2547) Design doc is wrong about default block placement policy.

[ https://issues.apache.org/jira/browse/HDFS-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J reopened HDFS-2547: --- > Design doc is wrong about default block placement pol

[jira] [Created] (HDFS-2547) Design doc is wrong about default block placement policy.

Design doc is wrong about default block placement policy. - Key: HDFS-2547 URL: https://issues.apache.org/jira/browse/HDFS-2547 Project: Hadoop HDFS Issue Type: Bug

Re: Local block placement policy, request

I'm moving this to hdfs-dev, where it belongs. There are so many failure scenarios that trying to shoe horn this functionality just isn't going to work in any realistic way. What happens when the local DataNode process dies? What happens when the local DataNode process h

[jira] Created: (HDFS-1451) Extend the pluggable block placement policy interface to handle/allow file-level intelligence

Extend the pluggable block placement policy interface to handle/allow file-level intelligence - Key: HDFS-1451 URL: https://issues.apache.org/jira/browse/HDFS-1451

[jira] Created: (HDFS-1094) Intelligent block placement policy to decrease probability of block loss

Intelligent block placement policy to decrease probability of block loss Key: HDFS-1094 URL: https://issues.apache.org/jira/browse/HDFS-1094 Project: Hadoop HDFS Issue