Re: btrfs convert running out of space

2015-01-19 Thread Chris Murphy
> On Mon, Jan 19, 2015 at 5:13 PM, Gareth Pye wrote: > I just tried from a slightly different tack, after doing another > -dusage=2 pass I did the following: > # btrfs balance start -v -dconvert=raid1 -dsoft -dusage=96 /data > Dumping filters: flags 0x1, state 0x0, force is off > DATA (flags 0x3

[PATCH 05/15] Btrfs: Fix a jump typo of nodatasum_case to avoid wrong WARN_ON()

2015-01-19 Thread Zhaolei
From: Zhao Lei if (sctx->is_dev_replace && !is_metadata && !have_csum) { ... goto nodatasum_case; } ... nodatasum_case: WARN_ON(sctx->is_dev_replace); In above code, nodatasum_case marker should be moved after WARN_ON(). Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrf

[PATCH 04/15] Btrfs: add ref_count and free function for btrfs_bio

2015-01-19 Thread Zhaolei
From: Zhao Lei 1: ref_count is simple than current RBIO_HOLD_BBIO_MAP_BIT flag to keep btrfs_bio's memory in raid56 recovery implement. 2: free function for bbio will make code clean and flexible, plus forced data type checking in compile. Changelog v1->v2: Rename following by David Sterb

[PATCH 02/15] Btrfs: sort raid_map before adding tgtdev stripes

2015-01-19 Thread Zhaolei
From: Zhao Lei It can avoid complex calculation of real stripes in sort, moreover, we can clean up code of sorting tgtdev_map because it will be in order initially. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 22 -- 1 file changed, 8 insertions

[PATCH 07/15] Btrfs: Cleanup btrfs_bio_counter_inc_blocked()

2015-01-19 Thread Zhaolei
From: Zhao Lei 1: Remove no-need DEFINE_WAIT(wait) 2: Add likely() for BTRFS_FS_STATE_DEV_REPLACING condition 3: Use while loop instead of goto Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/dev-replace.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff -

[PATCH 01/15] Btrfs: fix a out-of-bound access of raid_map

2015-01-19 Thread Zhaolei
From: Zhao Lei We add the number of stripes on target devices into bbio->num_stripes if we are under device replacement, and we just sort the raid_map of those stripes that not on the target devices, so if when we need real raid_map, we need skip the stripes on the target devices. Signed-off-by:

[PATCH 03/15] Btrfs: Make raid_map array be inlined in btrfs_bio structure

2015-01-19 Thread Zhaolei
From: Zhao Lei It can make code more simple and clear, we need not care about free bbio and raid_map together. Signed-off-by: Miao Xie Signed-off-by: Zhao Lei --- fs/btrfs/raid56.c | 77 ++--- fs/btrfs/raid56.h | 11 +++--- fs/btrfs/scrub.c | 31

[PATCH 06/15] Btrfs: Remove noneed force_write in scrub_write_block_to_dev_replace

2015-01-19 Thread Zhaolei
From: Zhao Lei It is always 1 in this place, because !1 case was already jumped out in previous code. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 19 +++ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub

[PATCH 10/15] Btrfs: Separate finding-right-mirror and writing-to-target's process in scrub_handle_errored_block()

2015-01-19 Thread Zhaolei
From: Zhao Lei In corrent code, code of finding-right-mirror and writing-to-target are mixed in logic, if we find a right mirror but failed in writing to target, it will treat as "hadn't found right block", and fill the target with sblock_bad. Actually, "failed in writing to target" does not mea

[PATCH 09/15] Btrfs: Break loop when reach BTRFS_MAX_MIRRORS in scrub_setup_recheck_block()

2015-01-19 Thread Zhaolei
From: Zhao Lei Use break instead of useless loop should be more suitable in this case. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index c93c43f..9afc6dd 100644 ---

[PATCH 11/15] Btrfs: Combine per-page recover in dev-replace and scrub

2015-01-19 Thread Zhaolei
From: Zhao Lei The code are similar, combine them to make code clean and easy to maintenance. Some lost condition are also completed with benefit of this combination. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 120 ++-

[PATCH 08/15] Btrfs: btrfs_rm_dev_replace_blocked(): Use wait_event()

2015-01-19 Thread Zhaolei
From: Zhao Lei Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/dev-replace.c | 13 ++--- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 92109b7..5ec03d9 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs

[PATCH 15/15] Rename all ref_count to refs in struct

2015-01-19 Thread Zhaolei
From: Zhao Lei refs is better than ref_count to record a struct's ref count. Signed-off-by: Zhao Lei Suggested-by: David Sterba --- fs/btrfs/scrub.c | 26 +- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index 94f2b

[PATCH 14/15] Btrfs: Introduce BTRFS_BLOCK_GROUP_RAID56_MASK to check raid56 simply

2015-01-19 Thread Zhaolei
From: Zhao Lei So we can check raid56 with: (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) instead of long: (map->type & (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID6)) Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/ctree.h | 3 +++ fs/btrfs/inode.c | 3 +-- fs/btrfs/s

[PATCH 13/15] Btrfs: Include map_type in raid_bio

2015-01-19 Thread Zhaolei
From: Zhao Lei Corrent code use many kinds of "clever" way to determine operation target's raid type, as: raid_map != NULL or raid_map[MAX_NR] == RAID[56]_Q_STRIPE To make code easy to maintenance, this patch put raid type into bbio, and we can always get raid type from bbio with a "stupid

[PATCH 12/15] Btrfs: Simplify scrub_setup_recheck_block()'s argument

2015-01-19 Thread Zhaolei
From: Zhao Lei scrub_setup_recheck_block() have many arguments but most of them can be get from one of them, we can remove them to make code clean. Some other cleanup for that function also included in this patch. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 25 +

[PATCH v3 00/15] Btrfs: Cleanup for raid56 scrub

2015-01-19 Thread Zhaolei
From: Zhao Lei Hi, everyone, These are v3 of cleanup patchs for raid56's scrub functions, it is based on review for new-developed raid56's scrub code. Some small typo-fix and cleanup for other functions are also included in this patchset. Changelog v2->v3: 1: Drop [PATCH 10/16] Btrfs: Avoid tr

RE: btrfs convert running out of space

2015-01-19 Thread Lakshmi_Narayanan_Du
Hi , I am newbie to file system and just started to understand the concepts of btrfs. Correct me if I am wrong .I tried to reproduce the issue but could not get the issue. Please see the below snippet of the logs and let me know whether I am missing any context/steps to get the bug btrfs

Re: [PATCH 1/6] btrfs: add support for processing pending changes

2015-01-19 Thread Qu Wenruo
Original Message Subject: [PATCH 1/6] btrfs: add support for processing pending changes From: David Sterba To: Date: 2014年11月14日 18:33 There are some actions that modify global filesystem state but cannot be performed at the time of request, but later at the transaction commi

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-19 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: Miao Xie To: Qu Wenruo , , Date: 2015年01月20日 11:06 On Tue, 20 Jan 2015 10:53:05 +0800, Qu Wenruo wrote: Add CC to Miao Xie Original Message

RE: [PATCH 10/16] Btrfs: Avoid trustless page-level-repair in dev-replace

2015-01-19 Thread Zhao Lei
Hi, Stefan Behrens Thanks for review these patch. * From: Stefan Behrens [mailto:sbehr...@giantdisaster.de] > Sent: Monday, January 19, 2015 8:26 PM > On Mon, 19 Jan 2015 19:20:59 +0800, Zhaolei wrote: > > From: Zhao Lei > > > > Current code of page level repair for dev-replace can only support

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-19 Thread Miao Xie
On Tue, 20 Jan 2015 10:53:05 +0800, Qu Wenruo wrote: > Add CC to Miao Xie > > Original Message > Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs > to > avoid deadlock. > From: Qu Wenruo > To: dste...@suse.cz, linux-btrfs@vger.kernel.org, Miao Xie

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-19 Thread Qu Wenruo
Add CC to Miao Xie Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: Qu Wenruo To: dste...@suse.cz, linux-btrfs@vger.kernel.org, Miao Xie Date: 2015年01月20日 10:51 Original Message Sub

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-19 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: David Sterba To: Qu Wenruo Date: 2015年01月19日 22:06 On Mon, Jan 19, 2015 at 03:42:41PM +0800, Qu Wenruo wrote: The fix is to check if the fs is frozen, if

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-19 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock. From: Miao Xie To: Qu Wenruo , Date: 2015年01月20日 08:19 On Mon, 19 Jan 2015 15:42:41 +0800, Qu Wenruo wrote: Commit 6b5fe46dfa52 (btrfs: do commit in sync_fs if

Re: [PATCH] btrfs-progs: tests, use non-interactive sudo helper

2015-01-19 Thread Qu Wenruo
Original Message Subject: [PATCH] btrfs-progs: tests, use non-interactive sudo helper From: David Sterba To: Date: 2015年01月20日 02:11 Sudo may not be configured to run without user prompt, in that case the tests would be stuck. Reported-by: Qu Wenruo Signed-off-by: David Ste

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-19 Thread Miao Xie
On Mon, 19 Jan 2015 15:42:41 +0800, Qu Wenruo wrote: > Commit 6b5fe46dfa52 (btrfs: do commit in sync_fs if there are pending > changes) will call btrfs_start_transaction() in sync_fs(), to handle > some operations needed to be done in next transaction. > > However this can cause deadlock if the fi

Re: btrfs convert running out of space

2015-01-19 Thread Gareth Pye
I just tried from a slightly different tack, after doing another -dusage=2 pass I did the following: # btrfs balance start -v -dconvert=raid1 -dsoft -dusage=96 /data Dumping filters: flags 0x1, state 0x0, force is off DATA (flags 0x302): converting, target=16, soft is on, usage=96 Done, had to r

btrfs convert running out of space

2015-01-19 Thread Gareth Pye
Hi, I'm attempting to convert a btrfs filesystem from raid10 to raid1. Things had been going well through a couple of pauses and resumes, but last night it errored with: ERROR: error during balancing '/data' - No space left on device Which is strange because there is around 1.4T spare on the driv

btrfs send multiple subvolumes

2015-01-19 Thread Bjørnar Ness
Looks like this should be supported, but I get an error: btrfs send -e snapshots/ro@20150118 snapshots/ro@20150119 > foo.btrfssend At subvol snapshots/ro@20150118 At subvol snapshots/ro@20150119 ERROR: parent determination failed for 0 Linux 3.18.0-rc5 #1 SMP x86_64 Btrfs v3.18.1 -- Bj(/)r

[PATCH] btrfs-progs: tests, use non-interactive sudo helper

2015-01-19 Thread David Sterba
Sudo may not be configured to run without user prompt, in that case the tests would be stuck. Reported-by: Qu Wenruo Signed-off-by: David Sterba --- tests/common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/common b/tests/common index 84a4b9d335e2..d7d2e9bef050 1006

Re: [PATCH 1/2] btrfs-progs: tests, add support for running commands under root

2015-01-19 Thread David Sterba
> > Most of the checks run fine without root, but some of them may need to > > do a mount test or access the data. Add the support to selectively run > > commands under root, hardcoded to sudo for now. > I'm a little worried about using sudo in such test. Opposed to running the whole testsuite as

[PATCH] btrfs-progs: fix check of running scrub

2015-01-19 Thread David Sterba
Scrub on multiple devices may report wrong status if scrub finishes early on one of them. Reported-by: Sandy McArthur Jr Signed-off-by: David Sterba --- cmds-scrub.c | 6 +- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/cmds-scrub.c b/cmds-scrub.c index eb50d8a65708..06c2a306

Re: btrfs scrub status reports not running when it is

2015-01-19 Thread David Sterba
On Thu, Jan 15, 2015 at 10:02:37AM -0800, Zach Brown wrote: > > > It says that scrub isn't running if any devices have completed. If you > > > drop > > > all those ret < 0 conditional branches that are either noops or wrong, > > > does it > > > work like you'd expect? > > > > Why wrong? The ioc

[PATCH 1/1] Btrfs: move super_kobj and device_dir_kobj from fs_info to btrfs_fs_devices

2015-01-19 Thread Anand Jain
This patch will provide a framework and help to create attributes from the structure btrfs_fs_devices which are available even before fs_info is created. So by moving the parent kobject super_kobj from fs_info to btrfs_fs_devices, it will help to create attributes from the btrfs_fs_devices as well.

Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.

2015-01-19 Thread David Sterba
On Mon, Jan 19, 2015 at 03:42:41PM +0800, Qu Wenruo wrote: > The fix is to check if the fs is frozen, if the fs is frozen, just > return and waiting for the next transaction. > > --- a/fs/btrfs/super.c > +++ b/fs/btrfs/super.c > @@ -1000,6 +1000,14 @@ int btrfs_sync_fs(struct super_block *sb, int

[PATCH 3.19-rc] btrfs: sync ioctl, handle errors after transaction start

2015-01-19 Thread David Sterba
The version merged to 3.19 did not handle errors from start_trancaction and could pass an invalid pointer to commit_transaction. Fixes: 6b5fe46dfa52441f ("btrfs: do commit in sync_fs if there are pending changes") Reported-by: Filipe Manana Signed-off-by: David Sterba --- This is a followup to

Re: [PATCH 10/16] Btrfs: Avoid trustless page-level-repair in dev-replace

2015-01-19 Thread Stefan Behrens
On Mon, 19 Jan 2015 19:20:59 +0800, Zhaolei wrote: > From: Zhao Lei > > Current code of page level repair for dev-replace can only support > io-error, we can't use it in checksum-fail case. > > We can skip this kind of repair in dev-replace just as we in scrub. Why? I see dev-replace as a disk

[PATCH 07/16] Btrfs: Cleanup btrfs_bio_counter_inc_blocked()

2015-01-19 Thread Zhaolei
From: Zhao Lei 1: Remove no-need DEFINE_WAIT(wait) 2: Add likely() for BTRFS_FS_STATE_DEV_REPLACING condition 3: Use while loop instead of goto Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/dev-replace.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff -

[PATCH 08/16] Btrfs: btrfs_rm_dev_replace_blocked(): Use wait_event()

2015-01-19 Thread Zhaolei
From: Zhao Lei Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/dev-replace.c | 13 ++--- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 92109b7..5ec03d9 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs

[PATCH v2 00/16] Btrfs: Cleanup for raid56 scrub

2015-01-19 Thread Zhaolei
From: Zhao Lei Hi, everyone, These are v2 of cleanup patchs for raid56's scrub functions, it is based on review for new-developed raid56's scrub code. Some small typo-fix and cleanup for other functions are also included in this patchset. Changelog v1->v2: 1: scrib -> scrub in description.

[PATCH 05/16] Btrfs: Fix a jump typo of nodatasum_case to avoid wrong WARN_ON()

2015-01-19 Thread Zhaolei
From: Zhao Lei if (sctx->is_dev_replace && !is_metadata && !have_csum) { ... goto nodatasum_case; } ... nodatasum_case: WARN_ON(sctx->is_dev_replace); In above code, nodatasum_case marker should be moved after WARN_ON(). Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrf

[PATCH 06/16] Btrfs: Remove noneed force_write in scrub_write_block_to_dev_replace

2015-01-19 Thread Zhaolei
From: Zhao Lei It is always 1 in this place, because !1 case was already jumped out in previous code. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 19 +++ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub

[PATCH 03/16] Btrfs: Make raid_map array be inlined in btrfs_bio structure

2015-01-19 Thread Zhaolei
From: Zhao Lei It can make code more simple and clear, we need not care about free bbio and raid_map together. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/raid56.c | 77 ++--- fs/btrfs/raid56.h | 11 +++--- fs/btrfs/scrub.c | 31

[PATCH 04/16] Btrfs: add ref_count and free function for btrfs_bio

2015-01-19 Thread Zhaolei
From: Zhao Lei 1: ref_count is simple than current RBIO_HOLD_BBIO_MAP_BIT flag to keep btrfs_bio's memory in raid56 recovery implement. 2: free function for bbio will make code clean and flexible, plus forced data type checking in compile. Changelog v1->v2: Rename following by David Sterb

[PATCH 02/16] Btrfs: sort raid_map before adding tgtdev stripes

2015-01-19 Thread Zhaolei
From: Zhao Lei It can avoid complex calculation of real stripes in sort, moreover, we can clean up code of sorting tgtdev_map because it will be in order initially. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 22 -- 1 file changed, 8 insertions

[PATCH 01/16] Btrfs: fix a out-of-bound access of raid_map

2015-01-19 Thread Zhaolei
From: Zhao Lei We add the number of stripes on target devices into bbio->num_stripes if we are under device replacement, and we just sort the raid_map of those stripes that not on the target devices, so if when we need real raid_map, we need skip the stripes on the target devices. Signed-off-by:

[PATCH 09/16] Btrfs: Break loop when reach BTRFS_MAX_MIRRORS in scrub_setup_recheck_block()

2015-01-19 Thread Zhaolei
From: Zhao Lei Use break instead of useless loop should be more suitable in this case. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index c93c43f..9afc6dd 100644 ---

[PATCH 11/16] Btrfs: Separate finding-right-mirror and writing-to-target's process in scrub_handle_errored_block()

2015-01-19 Thread Zhaolei
From: Zhao Lei In corrent code, code of finding-right-mirror and writing-to-target are mixed in logic, if we find a right mirror but failed in writing to target, it will treat as "hadn't found right block", and fill the target with sblock_bad. Actually, "failed in writing to target" does not mea

[PATCH 16/16] Rename all ref_count to refs in struct

2015-01-19 Thread Zhaolei
From: Zhao Lei refs is better than ref_count to record a struct's ref count. Signed-off-by: Zhao Lei Suggested-by: David Sterba --- fs/btrfs/scrub.c | 26 +- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index 6b5e6

[PATCH 13/16] Btrfs: Simplify scrub_setup_recheck_block()'s argument

2015-01-19 Thread Zhaolei
From: Zhao Lei scrub_setup_recheck_block() have many arguments but most of them can be get from one of them, we can remove them to make code clean. Some other cleanup for that function also included in this patch. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 25 +

[PATCH 15/16] Btrfs: Introduce BTRFS_BLOCK_GROUP_RAID56_MASK to check raid56 simply

2015-01-19 Thread Zhaolei
From: Zhao Lei So we can check raid56 with: (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) instead of long: (map->type & (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID6)) Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/ctree.h | 3 +++ fs/btrfs/inode.c | 3 +-- fs/btrfs/s

[PATCH 14/16] Btrfs: Include map_type in raid_bio

2015-01-19 Thread Zhaolei
From: Zhao Lei Corrent code use many kinds of "clever" way to determine operation target's raid type, as: raid_map != NULL or raid_map[MAX_NR] == RAID[56]_Q_STRIPE To make code easy to maintenance, this patch put raid type into bbio, and we can always get raid type from bbio with a "stupid

[PATCH 12/16] Btrfs: Combine per-page recover in dev-replace and scrub

2015-01-19 Thread Zhaolei
From: Zhao Lei The code are similar, combine them to make code clean and easy to maintenance. Some lost condition are also completed with benefit of this combination. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 110 ---

[PATCH 10/16] Btrfs: Avoid trustless page-level-repair in dev-replace

2015-01-19 Thread Zhaolei
From: Zhao Lei Current code of page level repair for dev-replace can only support io-error, we can't use it in checksum-fail case. We can skip this kind of repair in dev-replace just as we in scrub. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 8 1 file cha