[PATCH v3] btrfs-progs: fi resize: fix false 0.00B sized output

2021-04-19 Thread Su Yue
@new_size to fix this. Issue: #307 Reported-by: Chris Murphy Signed-off-by: Su Yue --- Changelog: v3: Just assign @diff to @new_size. (Boris Burkov) v2: Calculate u64 diff using max() and min(). Calculate mod by comparing new and old size. --- cmds/filesystem.c | 5 - 1 file changed, 4

Re: [PATCH v2] btrfs-progs: fi resize: fix false 0.00B sized output

2021-04-19 Thread Su Yue
On Tue 20 Apr 2021 at 01:08, Boris Burkov wrote: On Mon, Apr 19, 2021 at 09:05:49PM +0800, Su Yue wrote: Resize to nums without sign prefix makes false output: btrfs fi resize 1:150g /srv/extra Resize device id 1 (/dev/sdb1) from 298.09GiB to 0.00B The resize operation would take effect

[PATCH v2] btrfs-progs: fi resize: fix false 0.00B sized output

2021-04-19 Thread Su Yue
Signed-off-by: Su Yue --- cmds/filesystem.c | 10 ++ 1 file changed, 10 insertions(+) --- Changelog: v2: Calculate u64 diff using max() and min(). Calculate mod by comparing new and old size. diff --git a/cmds/filesystem.c b/cmds/filesystem.c index 9e3cce687d6e

Re: [PATCH] btrfs-progs: fi resize: fix false 0.00B sized output

2021-04-19 Thread Su Yue
On Mon 19 Apr 2021 at 20:45, Su Yue wrote: Resize to nums without sign prefix makes false output: Resize device id 1 (/dev/sdb1) from 298.09GiB to 0.00B The resize operation would take effect though. Fix it by handling the case if mod is 0 in check_resize_args(). Issue: #307 Reported-by

[PATCH] btrfs-progs: fi resize: fix false 0.00B sized output

2021-04-19 Thread Su Yue
Resize to nums without sign prefix makes false output: Resize device id 1 (/dev/sdb1) from 298.09GiB to 0.00B The resize operation would take effect though. Fix it by handling the case if mod is 0 in check_resize_args(). Issue: #307 Reported-by: Chris Murphy Signed-off-by: Su Yue --- cmds

Re: [PATCH v4 7/7] fs/xfs: Add dedupe support for fsdax

2021-04-08 Thread Su Yue
On Thu 08 Apr 2021 at 20:04, Shiyang Ruan wrote: Add xfs_break_two_dax_layouts() to break layout for tow dax files. Then call compare range function only when files are both DAX or not. Signed-off-by: Shiyang Ruan Not family with xfs code but reading code make my sleep better :) See b

Re: [PATCH v2] btrfs: fix lockdep warning while mounting sprout fs

2021-04-07 Thread Su Yue
On Wed, Apr 7, 2021 at 3:24 PM David Sterba wrote: > > On Mon, Apr 05, 2021 at 05:18:32PM +0800, Su Yue wrote: > > > > On Mon 05 Apr 2021 at 16:38, Anand Jain > > wrote: > > > > > Ping again. > > > > > It's already queued in misc-next.

Re: [PATCH 00/12] btrfs-progs: refactor and generalize chunk/dev_extent allocation

2021-04-06 Thread Su Yue
On Tue 06 Apr 2021 at 21:24, Naohiro Aota wrote: On Tue, Apr 06, 2021 at 06:54:37PM +0800, Su Yue wrote: On Tue 06 Apr 2021 at 16:05, Naohiro Aota wrote: > This is the userland counterpart of the following series. > > https://lore.kernel.org/linux-btrfs/20200225035626.

Re: [PATCH 00/12] btrfs-progs: refactor and generalize chunk/dev_extent allocation

2021-04-06 Thread Su Yue
On Tue 06 Apr 2021 at 16:05, Naohiro Aota wrote: This is the userland counterpart of the following series. https://lore.kernel.org/linux-btrfs/20200225035626.1049501-1-naohiro.a...@wdc.com/ This series refactors chunk allocation and device_extent allocation functions and make them genera

Re: [PATCH v2] btrfs: fix lockdep warning while mounting sprout fs

2021-04-05 Thread Su Yue
it is read-only. An unmounted seed device can free by btrfs_free_stale_devices(), but it needs uuid_mutex. So this patch removes the unnecessary device_list_mutex in clone_fs_devices(). And adds a lockdep_assert_held(&uuid_mutex) in clone_fs_devices(). Reported-by: Su Yue Signed-off-by: Anand

[PATCH v2] btrfs-progs: check: continue to check space cache if sb cache_generation is 0

2021-03-29 Thread Su Yue
kernel commit mentioned above. Link: https://github.com/kdave/btrfs-progs/issues/338 Signed-off-by: Su Yue Reviewed-by: Boris Burkov --- Changelog: v2: Change the commit description little. Add the reviewed-by. Remove 'RFC' from subject. --- check/main.c | 5 + 1 fil

Re: [PATCH 2/3] btrfs-progs: image: enlarge the output file if no tree modification is needed for restore

2021-03-26 Thread Su Yue
On Fri 26 Mar 2021 at 21:52, Su Yue wrote: On Fri 26 Mar 2021 at 20:50, Qu Wenruo wrote: [BUG] If restoring dumpped image into a new file, under most cases kernel will reject it: # mkfs.btrfs -f /dev/test/test # btrfs-image /dev/test/test /tmp/dump # btrfs-image -r /tmp/dump

Re: [PATCH 1/3] btrfs: image: remove the dead stat() call

2021-03-26 Thread Su Yue
: Su Yue So we can safely remove the call. Signed-off-by: Qu Wenruo --- image/main.c | 8 1 file changed, 8 deletions(-) diff --git a/image/main.c b/image/main.c index 48070e52c21f..24393188e5e3 100644 --- a/image/main.c +++ b/image/main.c @@ -2690,7 +2690,6 @@ static int

Re: [PATCH 2/3] btrfs-progs: image: enlarge the output file if no tree modification is needed for restore

2021-03-26 Thread Su Yue
On Fri 26 Mar 2021 at 20:50, Qu Wenruo wrote: [BUG] If restoring dumpped image into a new file, under most cases kernel will reject it: # mkfs.btrfs -f /dev/test/test # btrfs-image /dev/test/test /tmp/dump # btrfs-image -r /tmp/dump ~/test.img # mount ~/test.img /mnt/btrfs mount: /mnt

Re: [PATCH v2 1/2] btrfs: prevent NULL pointer dereference in compress_file_range() when btrfs_compress_pages() hits default case

2021-03-08 Thread Su Yue
On Tue, Aug 4, 2020 at 3:29 PM Qu Wenruo wrote: > > [BUG] > When running btrfs/071 with inode_need_compress() removed from > compress_file_range(), we got the following crash: > > BUG: kernel NULL pointer dereference, address: 0018 > #PF: supervisor read access in kernel mode > #

Re: [report] lockdep warning when mounting seed device

2021-02-25 Thread Su Yue
On Fri 26 Feb 2021 at 13:01, Anand Jain wrote: On 25/02/2021 12:39, Su Yue wrote: While playing with seed device(misc/next and v5.11), lockdep complains the following: To reproduce: dev1=/dev/sdb1 dev2=/dev/sdb2 umount /mnt mkfs.btrfs -f $dev1 btrfstune -S 1 $dev1 mount $dev1 /mnt btrfs

[report] lockdep warning when mounting seed device

2021-02-24 Thread Su Yue
While playing with seed device(misc/next and v5.11), lockdep complains the following: To reproduce: dev1=/dev/sdb1 dev2=/dev/sdb2 umount /mnt mkfs.btrfs -f $dev1 btrfstune -S 1 $dev1 mount $dev1 /mnt btrfs device add $dev2 /mnt/ -f umount /mnt mount $dev2 /mnt umount /mnt Warning:

Re: [PATCH 03/12] btrfs: disk-io: allow btree_set_page_dirty() to do more sanity check on subpage metadata

2021-02-21 Thread Su Yue
On Mon 22 Feb 2021 at 14:33, Qu Wenruo wrote: For btree_set_page_dirty(), we should also check the extent buffer sanity for subpage support. Unlike the regular sector size case, since one page can contain multiple extent buffers, we need to make sure there is at least one dirty extent bu

Re: [PATCH RFC] btrfs-progs: check: continue to check space cache if sb cache_generation is 0

2021-02-17 Thread Su Yue
On Thu 18 Feb 2021 at 06:18, Boris Burkov wrote: On Mon, Feb 15, 2021 at 05:20:11PM +0800, Su Yue wrote: User reported that test fsck-tests/037-freespacetree-repair fails: # TEST=037\* ./fsck-tests.sh [TEST/fsck] 037-freespacetree-repair btrfs check should have detected corruption

[PATCH RFC] btrfs-progs: check: continue to check space cache if sb cache_generation is 0

2021-02-15 Thread Su Yue
trfs-progs/issues/338 Signed-off-by: Su Yue --- Hi, while reading free space cache v1 related code, I am (still) confused about the value meanings of cache_generation in sb. For outdated space cache v1, cache_gen < sb gen. For valid space cache v1, cache_gen == sb gen. AS for values 0 a

[PATCH] btrfs: initialize btrfs_fs_info::csum_size earlier in open_ctree()

2021-02-11 Thread Su Yue
ues/250 Fixes: 55fc29bed8dd ("btrfs: use cached value of fs_info::csum_size everywhere") Signed-off-by: Su Yue --- fs/btrfs/disk-io.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 6b35b7e88136..07a2b4f69b10 100644 --- a/fs/b

Re: [PATCH] btrfs-progs: remove workaround for setting capabilities in the receive command

2021-02-09 Thread Su Yue
nap1/bar cap_net_raw=p > > Since the kernel fix was backported to all currently supported stable > releases (5.10.x, 5.4.x, 4.19.x, 4.14.x, 4.9.x and 4.4.x), remove the > workaround from receive. Having such a workaround relying on the order > of commands in a send stream is

Re: "bad tree block start" when trying to mount on ARM

2021-01-31 Thread Su Yue
On Mon 01 Feb 2021 at 10:35, Qu Wenruo wrote: On 2021/1/29 下午2:39, Erik Jensen wrote: On Mon, Jan 25, 2021 at 8:54 PM Erik Jensen wrote: On Wed, Jan 20, 2021 at 1:08 AM Erik Jensen wrote: On Wed, Jan 20, 2021 at 12:31 AM Qu Wenruo wrote: On 2021/1/20 下午4:21, Qu Wenruo wrote: On 2021

Re: [PATCH] btrfs: fix lockdep warning due to seqcount_mutex_init() with wrong address

2021-01-21 Thread Su Yue
ain. On Thu, Jan 21, 2021 at 07:39:10PM +0800, Su Yue wrote: while running xfstests on 32 bits test box, many tests failed because of warnings in dmesg. One of those warnings(btrfs/003): [ 66.441305] --

[PATCH] btrfs: fix lockdep warning due to seqcount_mutex_init() with wrong address

2021-01-21 Thread Su Yue
quot;btrfs: convert data_seqcount to seqcount_mutex_t") to fix this. Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=210139 Fixes: d5c8238849e7 ("btrfs: convert data_seqcount to seqcount_mutex_t") CC: sta...@vger.kernel.org # 5.10 Reported-by: Erhard F Signed-off-by: Su Yue ---

Re: [PATCH v11 22/40] btrfs: split ordered extent when bio is sent

2021-01-14 Thread Su Yue
On Tue 22 Dec 2020 at 11:49, Naohiro Aota wrote: For a zone append write, the device decides the location the data is written to. Therefore we cannot ensure that two bios are written consecutively on the device. In order to ensure that a ordered extent maps to a contiguous region on disk,

[PATCH] btrfs-progs: print bytenr of child eb if mismatched level found in read_node_slot()

2021-01-06 Thread Su Yue
ey (16857755648 EXTENT_ITEM 4096) block 177857544192 gen 44416 ... === For easier lookup, print bytenr of child eb if its level is not equal to parent's level - 1 in read_node_slot(). Signed-off-by: Su Yue --- kernel-shared/ctree.

Re: [PATCH] btrfs-progs: properly exclude leaves for lowmem

2021-01-05 Thread Su Yue
gets cleaned up on transaction commit. We're safe for the first modification, but subsequent modifications could blow up in our face. Fix this by properly excluding leaves as well as all of the nodes. Signed-off-by: Josef Bacik LGTM. Reviewed-by: Su Yue --- check/mode-common.c | 8 +

Re: [PATCH v2 2/2] btrfs: tree-checker: check if chunk item end oveflows

2021-01-04 Thread Su Yue
On Mon 04 Jan 2021 at 16:31, Anand Jain wrote: On 3/1/21 5:28 pm, Su Yue wrote: while mounting the poc image user-provided, kernel panics due to the invalid chunk item whose end is less than start. [ 66.387422

Re: [PATCH] btrfs: fix NULL pointer dereference in extent_io_tree_panic()

2021-01-03 Thread Su Yue
On Sun 03 Jan 2021 at 14:52, Qu Wenruo wrote: On 2021/1/3 下午1:01, Su Yue wrote: while mounting the poc image user-provided, kernel panics due to the NULL deference of @tree->inode. [ 109.663024] BTRFS error: fo

[PATCH v2 2/2] btrfs: tree-checker: check if chunk item end oveflows

2021-01-03 Thread Su Yue
te() because of the duplicate end and extent_io_tree_panic() was called. Add overflow check of chunk item end in tree checker then the image will be rejected to be mounted. Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=208929 Signed-off-by: Su Yue --- fs/btrfs/tree-checker.c | 7

[PATCH v2 1/2] btrfs: prevent NULL pointer dereference in extent_io_tree_panic()

2021-01-03 Thread Su Yue
Let it panic as expected at least. Fixes: 05912a3c04eb ("btrfs: drop extent_io_ops::tree_fs_info callback") Signed-off-by: Su Yue --- fs/btrfs/extent_io.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 6e3b72e63e

[PATCH v2 0/2] btrfs: fix issues when mouting the poc image

2021-01-03 Thread Su Yue
The two patches fix issues found by the image which is provided by Insu Yun at SSLab@Gatech. patch 1 fixes a NULL pointer dereference in error handling path. patch 2 enhances tree checker to detect chunk item end overflow. Su Yue (2): btrfs: prevent NULL pointer dereference in

[PATCH] btrfs: fix NULL pointer dereference in extent_io_tree_panic()

2021-01-02 Thread Su Yue
ow_bug.cgi?id=208929 Fixes: 05912a3c04eb ("btrfs: drop extent_io_ops::tree_fs_info callback") Signed-off-by: Su Yue --- I don't know why insert_state() code only warns about the invalid state and extent io operations are used in so many places. So just send the simple patch. --- fs/btrf

Re: [PATCH] btrfs: relocation: fix wrong file extent type check to avoid false -ENOENT error

2020-12-29 Thread Su Yue
EXTENT_PREALLOC. Reported-by: Stéphane Lesimple Fixes: 19b546d7a1b2 ("btrfs: relocation: Use btrfs_find_all_leafs to locate data extent parent tree leaves") Signed-off-by: Qu Wenruo Reviewed-by: Su Yue --- fs/btrfs/relocation.c | 7 ++- 1 file changed, 6 insertions(+

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

2020-12-16 Thread Su Yue
On Thu 17 Dec 2020 at 13:42, Qu Wenruo wrote: On 2020/12/17 下午1:38, Su Yue wrote: On Thu 17 Dec 2020 at 12:57, Qu Wenruo wrote: In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree. Remove such variable shadowing which can be very confusing. Signed-off

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

2020-12-16 Thread Su Yue
On Thu 17 Dec 2020 at 12:57, Qu Wenruo wrote: In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree. Remove such variable shadowing which can be very confusing. Signed-off-by: Qu Wenruo --- fs/btrfs/inode.c | 9 +++-- 1 file changed, 3 insertions(+), 6 del

Re: [PATCH v3 2/2] btrfs-progs: device stats: add json output format

2020-12-16 Thread Su Yue
On Wed, Dec 16, 2020 at 6:52 PM Sidong Yang wrote: > > On Wed, Dec 16, 2020 at 02:30:04PM +0800, Su Yue wrote: > > On Sat, Dec 12, 2020 at 3:04 AM David Sterba wrote: > > > > > > On Fri, Dec 11, 2020 at 06:30:25PM +0100, David Sterba wrote: > > > >

Re: [PATCH v3 2/2] btrfs-progs: device stats: add json output format

2020-12-15 Thread Su Yue
On Sat, Dec 12, 2020 at 3:04 AM David Sterba wrote: > > On Fri, Dec 11, 2020 at 06:30:25PM +0100, David Sterba wrote: > > On Fri, Dec 11, 2020 at 04:48:12PM +, Sidong Yang wrote: > > > Example json format: > > > > > > { > > > "__header": { > > > "version": "1" > > > }, > > > "device-

Re: [PATCH v2 3/6] btrfs-progs: check/common: Make repair_imode_common() to handle inodes in subvolume trees

2019-09-09 Thread Su Yue
On 2019/9/5 3:57 PM, Qu Wenruo wrote: Before this patch, repair_imode_common() can only handle two types of inodes: - Free space cache inodes - ROOT DIR inodes For inodes in subvolume trees, the core complexity is how to determine the correct imode, thus it was not implemented. However there

Re: [RFC PATCH 2/4] btrfs: create structure to encode checksum type and length

2019-07-25 Thread Su Yue
On 2019/7/25 5:33 PM, Johannes Thumshirn wrote: Create a structure to encode the type and length for the known on-disk checksums. Also add a table and a convenience macro for adding the checksum types to the table. This makes it easier to add new checksums later. Signed-off-by: Johannes Thum

Re: [PATCH] btrfs-progs: extent-tree: Unify the parameters of btrfs_inc_extent_ref()

2019-07-24 Thread Su Yue
On 2019/7/24 4:35 PM, Qu Wenruo wrote: The old parameters, @ref_generation and @owner_objectid, are pretty confusing when using auto-completion. Unify the parameters as a quick fix. Signed-off-by: Qu Wenruo Makes sense and looks good. Just to remind, did you notice the next function name

Re: [PATCH 3/9] btrfs-progs: image: Fix a access-beyond-boundary bug when there are 32 online CPUs

2019-06-09 Thread Su Yue
-by: Qu Wenruo This does fix an issue. And as the commit says, why limit the max threads to 32? Does it still make sense in nowadays multiple cores CPU? Can we increase the limit? However, this is another story. For this patch: Reviewed-by: Su Yue --- image/main.c | 1 + 1 file changed, 1 in

Re: 'watch btrfs fi show' crash while 'btrfs device delete'

2019-06-02 Thread Su Yue
On 2019/6/1 8:35 PM, Peter Hjalmarsson wrote: Hi, I was the one reporting the issue to the Red Hat Bugzilla, and was able to reproduce it as well Thanks for the report, reproduced the bug following your steps. I just sent a patch named "btrfs-progs: fix invalid memory write in get_fs_info

Re: [PATCH 1/2] btrfs-progs: add 'btrfs inspect-internal csum-dump' command

2019-04-09 Thread Su Yue
On 2019/4/8 9:31 PM, Johannes Thumshirn wrote: Add a 'btrfs inspect-internal csum-dump' command to dump the on-disk checksums of a file. The dump command first uses the FIEMAP ioctl() to get a map of the file's extents and then uses the BTRFS_TREE_SEARCH_V2 ioctl() to get the checksums for th

Re: [PATCH 5/7] btrfs: check/lowmem: Check and repair free space cache inode mode

2019-03-25 Thread Su Yue
On 2019/3/25 4:22 PM, Qu Wenruo wrote: Unlike inodes in fs roots, we don't really check the inode items in root tree, in fact we just skip everything other than ROOT_ITEM and ROOT_REF. This makes invalid inode items sneak into root tree. For example: item 9 key (256 INODE_ITEM 0) ite

Re: [RFC PATCH 1/2] btrfs-progs: add 'btrfs inspect-internal csum-dump' command

2019-03-15 Thread Su Yue
On 2019/2/28 10:44 PM, Johannes Thumshirn wrote: Add a 'btrfs inspect-internal csum-dump' command to dump the on-disk checksums of a file. Little introduction may help others. One sentence is too much simple... Signed-off-by: Johannes Thumshirn I haven't reviewed BTRFS patch for a long

Re: [PATCH v4] btrfs-progs: dump-tree: add noscan option

2019-03-11 Thread Su Yue
On 2019/2/28 2:54 PM, Anand Jain wrote: From: Anand Jain There is no way to dump tree from a disk in the degraded mode. Such as, when you specify a device for the cli 'btrfs inspect dump-tree /dev/sda' it would invariably scan for its partner devices to built and print the tree. This approac

Re: [PATCH v3 1/9] btrfs: delayed-ref: Introduce better documented delayed ref structures

2019-02-17 Thread Su Yue
On 2/18/19 1:00 PM, Qu Wenruo wrote: On 2019/2/11 下午10:23, Qu Wenruo wrote: [snip] Looking at the dev docs and the description for 'offset' field in btrfs_file_extent_item I can sort of deduce that this field will only be different than null if this reference is for an extent which is shar

Re: [PATCH 3/3] btrfs-progs: Create uuid tree with proper contents

2019-01-02 Thread Su Yue
On 2019/1/3 12:31 AM, David Sterba wrote: On Thu, Dec 27, 2018 at 07:28:40PM +0800, Su Yue wrote: +} + +int create_uuid_tree(struct btrfs_trans_handle *trans) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + struct btrfs_root *uuid_root = fs_info->uuid_root; +

Re: [PATCH 3/3] btrfs-progs: Create uuid tree with proper contents

2018-12-27 Thread Su Yue
On 2018/12/27 3:13 PM, Qu Wenruo wrote: Commit 2a496a5b8b74 ("btrfs-progs: mkfs: precreate the uuid tree") creates uuid tree at mkfs time. However it doesn't populate uuid tree correctly nor just create an empty root. It uses create_tree(), which just copies the content of fs root, containing

Re: [PATCH 2/3] btrfs-progs: uuid: Port btrfs_uuid_tree_add() function

2018-12-26 Thread Su Yue
On 12/27/18 3:13 PM, Qu Wenruo wrote: This function provide the offline ability to add new uuid tree entry. This provides the basis for later proper uuid tree initialization. Signed-off-by: Qu Wenruo Reviewed-by: Su Yue --- ctree.h | 7 +- uuid-tree.c | 65

Re: [PATCH 1/3] btrfs-progs: uuid: Port kernel btrfs_uuid_tree_lookup()

2018-12-26 Thread Su Yue
igned-off-by: Qu Wenruo One nitpick bellow. Reviewed-by: Su Yue --- uuid-tree.c | 71 +++-- 1 file changed, 69 insertions(+), 2 deletions(-) diff --git a/uuid-tree.c b/uuid-tree.c index 320eb67e1404..b9190103c355 100644 --- a/uuid-tree.c +++ b

Re: [PATCH v2] btrfs: add a check for sysfs_create_group

2018-12-25 Thread Su Yue
On 12/26/18 1:37 PM, Kangjie Lu wrote: In case sysfs_create_group fails, let's check its return value and issues an error message. Signed-off-by: Kangjie Lu --- fs/btrfs/sysfs.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index 3717c864ba23..2

Re: [PATCH 00/13] btrfs-progs: check: Fix

2018-12-18 Thread Su Yue
some small bug fixes for: - False alert for file extents gap - Bad data backref -> file extent search in repair mode The patchset is mostly from Fujitsu guys (Lu, Su Yue, Su Yanjun). My work here is: Thanks a lot for your nice work. Su - Solve rebase conflicts Mostly with DIR_ITEM hash

Re: [RFC PATCH 00/17] btrfs: implementation of priority aware allocator

2018-12-01 Thread Su Yue
On 2018/11/28 12:04 PM, Qu Wenruo wrote: On 2018/11/28 上午11:11, Su Yue wrote: This patchset can be fetched from repo: https://github.com/Damenly/btrfs-devel/commits/priority_aware_allocator. Since patchset 'btrfs: Refactor find_free_extent()' does a nice work to simplify find_f

Re: [RFC PATCH 01/17] btrfs: priority alloc: prepare of priority aware allocator

2018-11-28 Thread Su Yue
On 11/28/18 4:24 PM, Nikolay Borisov wrote: On 28.11.18 г. 5:11 ч., Su Yue wrote: To implement priority aware allocator, this patch: Introduces struct btrfs_priority_tree which contains block groups in same level. Adds member priority to struct btrfs_block_group_cache and pointer points to

[PATCH] btrfs: adjust order of unlocks in do_trimming()

2018-11-27 Thread Su Yue
In function do_trimming(), block_group->lock should be unlocked first. Fixes: 7fe1e6415026 ("Btrfs: rewrite btrfs_trim_block_group()") Signed-off-by: Su Yue --- fs/btrfs/free-space-cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/free-space-cache

[RFC PATCH 16/17] btrfs: priority alloc: write bg->priority_tree->groups_sem to avoid race in btrfs_delete_unused_bgs()

2018-11-27 Thread Su Yue
If use priority aware allocator, bg->priority_tree->groups_sem should be written instead of space_info->groups_sem. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 60 +++--- 1 file changed, 44 insertions(+), 16 deletions(-) diff --git a/fs/btr

[RFC PATCH 08/17] btrfs: priority alloc: add btrfs_update_block_group_priority()

2018-11-27 Thread Su Yue
Introduce btrfs_update_block_group_priority() to update block_groups::priority. It will move block group from old tree to new tree if need. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 76 ++ 1 file changed, 76 insertions(+) diff --git a/fs/btrfs

[RFC PATCH 04/17] btrfs: priority alloc: add functions to create/remove priority trees

2018-11-27 Thread Su Yue
Introduce create_priority_trees() to create priority trees in space_info. Introduce remove_priority_trees() to remove priority trees in space_info. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 94 ++ 1 file changed, 94 insertions(+) diff --git a/fs

[RFC PATCH 12/17] btrfs: priority alloc: introduce find_free_extent_search()

2018-11-27 Thread Su Yue
In origin, find_free_extent() just searches block groups in space_info one by one. In priority aware allocator, we first search block groups in higher priority tree than in lower priority tree. This helper unify above two ways for further use. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c

[RFC PATCH 17/17] btrfs: add mount option "priority_alloc=%s"

2018-11-27 Thread Su Yue
Add mount option "priority_alloc=%s", now %s only supports "usage" and "off". The latter is used for remount. "priority_alloc=usage" will active priority aware allocator. This mount option changes the way of find_free_extent to search block groups and ma

[RFC PATCH 13/17] btrfs: priority alloc: modify find_free_extent() to fit priority allocator

2018-11-27 Thread Su Yue
Add member priority_tree to find_free_extent_ctl to represents the tree using. Modify find_free_extent to use find_free_extent_search, so it can work in default mount option and priorit aware allocator. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 114

[RFC PATCH 06/17] btrfs: priority alloc: introduce three macros to mark block group status

2018-11-27 Thread Su Yue
... unlock block_group up_read tree This patch introduce three macros to represents block group is removing /need to updated / busy. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 9 + 1 file changed, 9 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c

[RFC PATCH 03/17] btrfs: priority alloc: introduce compute_block_group_priority/usage

2018-11-27 Thread Su Yue
Introduce compute_block_group_usage() and compute_block_group_usage(). And call the latter in btrfs_make_block_group() and btrfs_read_block_groups(). compute_priority_level use ilog2(free) to compute priority level. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 60

[RFC PATCH 11/17] btrfs: priority alloc: remove block group from priority tree while removing block group

2018-11-27 Thread Su Yue
Export btrfs_remove_block_group_priority() to header ctree.h. Call btrfs_remove_block_group_priority() while deleting transaction->deleted_bgs, btrfs_free_block_groups() and btrfs_remove_block_group(). Signed-off-by: Su Yue --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-tree.c | 3 +++

[RFC PATCH 02/17] btrfs: add mount definition BTRFS_MOUNT_PRIORITY_USAGE

2018-11-27 Thread Su Yue
Now implementation of priority allocator only support usage option. Add BTRFS_MOUNT_PRIORITY_USAGE for further commits. Signed-off-by: Su Yue --- fs/btrfs/ctree.h | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 5c4651d8a524..4c56baf9f7cf 100644

[RFC PATCH 15/17] btrfs: priority alloc: write bg->priority_groups_sem while waiting reservation

2018-11-27 Thread Su Yue
Since if use priority alloc, we should down/up_write() bg->priority_groups_sem. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index f530a4344368..6627bbe56ad5 100

[RFC PATCH 10/17] btrfs: priority alloc: call add_block_group_priority while reading or making block group

2018-11-27 Thread Su Yue
Add block group to priority tree in btrfs_read_block_groups() and btrfs_make_block_groups(). Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 2dec02782df1..fc40901b4772 100644 --- a/fs

[RFC PATCH 14/17] btrfs: priority alloc: introduce btrfs_set_bg_updating and call btrfs_update_block_group_prioriy

2018-11-27 Thread Su Yue
n calls btrfs_update_block_group() to update priority tree if needed. Signed-off-by: Su Yue --- fs/btrfs/ctree.h| 2 ++ fs/btrfs/extent-tree.c | 40 + fs/btrfs/free-space-cache.c | 3 +++ 3 files changed, 45 insertions(+) diff --git a/fs/btrfs/ctree.h b/f

[RFC PATCH 05/17] btrfs: priority alloc: introduce functions to add block group to priority tree

2018-11-27 Thread Su Yue
Introduce compute_priority_level() to compute priority level according priority, now just divides PRIORITY_USAGE_FACOTR. Introduce add_block_group_priority() to add block groups to priority tree. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 76

[RFC PATCH 07/17] btrfs: priority alloc: add functions to remove block group from priority tree

2018-11-27 Thread Su Yue
Introduce btrfs_remove_block_group_priority() to remove block group from priority tree. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 37 + 1 file changed, 37 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 5bae757786dc

[RFC PATCH 09/17] btrfs: priority alloc: call create/remove_priority_trees in space_info

2018-11-27 Thread Su Yue
Call create_priority_trees() in create_space_info(). Call remove_priority_trees() before free of space_info. Signed-off-by: Su Yue --- fs/btrfs/extent-tree.c | 21 + 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent

[RFC PATCH 01/17] btrfs: priority alloc: prepare of priority aware allocator

2018-11-27 Thread Su Yue
fo to represents priority trees in different raid types. Signed-off-by: Su Yue --- fs/btrfs/ctree.h | 24 1 file changed, 24 insertions(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index e62824cae00a..5c4651d8a524 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ct

[RFC PATCH 00/17] btrfs: implementation of priority aware allocator

2018-11-27 Thread Su Yue
reasons includes time and hardware, the use-case is not outstanding enough. And some codes are dirty but I can't found another way. So I named it as RFC. Any comments and suggestions are welcome. Su Yue (17): btrfs: priority alloc: prepare of priority aware allocator btrfs: ad

Re: fsck lowmem mode only: ERROR: errors found in fs roots

2018-11-04 Thread Su Yue
On 11/3/18 5:20 PM, Nikolay Borisov wrote: On 3.11.18 г. 3:34 ч., Su Yue wrote: On 2018/11/2 10:10 PM, Christoph Anton Mitterer wrote: Hey Su. Sorry for the late reply cause I'm busy at other things. Anything further I need to do in this matter or can I consider it "solve

Re: fsck lowmem mode only: ERROR: errors found in fs roots

2018-11-02 Thread Su Yue
On 2018/11/2 10:10 PM, Christoph Anton Mitterer wrote: Hey Su. Sorry for the late reply cause I'm busy at other things. Anything further I need to do in this matter or can I consider it "solved" and you won't need further testing by my side, but just PR the patches of that branch? :-)

Re: [PATCH 01/13] btrfs-progs: lowmem: add argument path to punch_extent_hole()

2018-10-23 Thread Su Yue
On 10/23/18 6:04 PM, Qu Wenruo wrote: On 2018/10/23 下午5:41, Su Yue wrote: Since repair will do CoW, the outer path may be invalid, add an argument path to punch_extent_hole(). When punch_extent_hole() returns, path will still point to the item before calling punch_extent_hole(); Signed

Re: [PATCH 03/13] btrfs-progs: lowmem: fix false alert if extent item has been repaired

2018-10-23 Thread Su Yue
On 10/23/18 6:30 PM, Qu Wenruo wrote: On 2018/10/23 下午5:41, Su Yue wrote: Previously, @err are assigned immediately after check but before repair. repair_extent_item()'s return value also confuses the caller. If error has been repaired and returns 0, check_extent_item() will t

[PATCH 13/13] btrfs-progs: fsck-test: enable lowmem repair for case 001

2018-10-23 Thread Su Yue
Lowmem can repair after commit 'btrfs-progs: lowmem: move nbytes check before isize check', so add the beacon file. Signed-off-by: Su Yue --- tests/fsck-tests/001-bad-file-extent-bytenr/.lowmem_repairable | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100

[PATCH 08/13] btrfs-progs: Revert "btrfs-progs: Add repair and report function for orphan file extent."

2018-10-23 Thread Su Yue
From: Su Yanjun The reason for revert is that according to the existing situation, the probability of problem in the extent tree is higher than in the fs tree. So this feature should be removed. Signed-off-by: Su Yanjun --- check/main.c | 103 +--

[PATCH 11/13] btrfs-progs: check: Delete file extent item with unaligned extent backref

2018-10-23 Thread Su Yue
From: Su Yanjun In original mode, if some file extent item has unaligned extent backref, fixup_extent_refs can't repair it. This patch will check extent alignment then delete file extent with unaligned extent backref. Signed-off-by: Su Yanjun --- check/main.c | 278 +++

[PATCH 10/13] btrfs-progs: check: fix bug in find_possible_backrefs

2018-10-23 Thread Su Yue
From: Su Yanjun It may cost more time to search all extent data of correspond files but should not influence total speed too much cause that only corrupted extent items are participated in. Signed-off-by: Su Yanjun --- check/main.c | 110 ++- 1 f

[PATCH 12/13] btrfs-progs: tests: add case for inode lose one file extent

2018-10-23 Thread Su Yue
From: Lu Fengqi The missing extent will lead to the existence of the gap between adjacent extents. The fsck should can detect the gap correctly and repair by punch a hole. Signed-off-by: Lu Fengqi --- .../.lowmem_repairable | 0 .../038-missing-one-file-extent/defau

[PATCH 02/13] btrfs-progs: lowmem: move nbytes check before isize check

2018-10-23 Thread Su Yue
For files, lowmem repair will try to check nbytes and isize, but isize check depends nbytes. Once bytes has been repaired, then isize should be checked and repaired. So move nbytes check before isize check. Also set nbytes to extent_size once repaired successfully. Signed-off-by: Su Yue

[PATCH 06/13] btrfs-progs: lowmem: rename delete_extent_tree_item() to delete_item()

2018-10-23 Thread Su Yue
The function can delete items in trees besides extent tree. Rename and move it for further use. Signed-off-by: Su Yue --- check/mode-lowmem.c | 99 +++-- 1 file changed, 51 insertions(+), 48 deletions(-) diff --git a/check/mode-lowmem.c b/check/mode

[PATCH 09/13] btrfs-progs: Revert "btrfs-progs: Record orphan data extent ref to corresponding root."

2018-10-23 Thread Su Yue
From: Su Yanjun The reason for revert is that according to the existing situation, the probability of problem in the extent tree is higher than in the fs Tree. So this feature should be removed. Signed-off-by: Su Yanjun --- check/main.c | 120 +-

[PATCH 04/13] btrfs-progs: lowmem: fix false alert about the existence of gaps in the check_file_extent

2018-10-23 Thread Su Yue
From: Lu Fengqi The 'end' parameter of check_file_extent tracks the ending offset of the last checked extent. This is used to detect gaps between adjacent extents. Currently such gaps are wrongly detected since for regular extents only the size of the extent is added to the 'end' parameter. This

[PATCH 01/13] btrfs-progs: lowmem: add argument path to punch_extent_hole()

2018-10-23 Thread Su Yue
Since repair will do CoW, the outer path may be invalid, add an argument path to punch_extent_hole(). When punch_extent_hole() returns, path will still point to the item before calling punch_extent_hole(); Signed-off-by: Su Yue --- check/mode-lowmem.c | 21 - 1 file changed

[PATCH 05/13] btrfs-progs: lowmem: check unaligned disk_bytenr for extent_data

2018-10-23 Thread Su Yue
Add support to check unaligned disk_bytenr for extent_data. Signed-off-by: Su Yue --- check/mode-lowmem.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c index 35fe1adf58e6..e8a2e825c0f3 100644 --- a/check/mode-lowmem.c +++ b

[PATCH 03/13] btrfs-progs: lowmem: fix false alert if extent item has been repaired

2018-10-23 Thread Su Yue
m()'s return codes only represents status of the extent item, error bits are passed by pointer. Move the change of @err after repair. Signed-off-by: Su Yue --- check/mode-lowmem.c | 106 1 file changed, 68 insertions(+), 38 deletions(-) diff --g

[PATCH 00/13] btrfs-progs: fixes of file extent in original and lowmem check

2018-10-23 Thread Su Yue
r and report function for orphan file extent." btrfs-progs: Revert "btrfs-progs: Record orphan data extent ref to corresponding root." btrfs-progs: check: fix bug in find_possible_backrefs btrfs-progs: check: Delete file extent item with unaligned extent backref Su Yu

[PATCH 07/13] btrfs-progs: lowmem: delete unaligned bytes extent data under repair

2018-10-23 Thread Su Yue
If found a extent data item has unaligned part, lowmem repair just deletes it. Signed-off-by: Su Yue --- check/mode-lowmem.c | 8 1 file changed, 8 insertions(+) diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c index 3c9ecff7e498..5381096fa8b2 100644 --- a/check/mode-lowmem.c

Re: fsck lowmem mode only: ERROR: errors found in fs roots

2018-10-18 Thread Su Yue
[Bad format in previous reply, send again] On 10/18/18 10:41 PM, Christoph Anton Mitterer wrote: Hey. So I'm back from a longer vacation and had now the time to try out your patches from below: On Wed, 2018-09-05 at 15:04 +0800, Su Yue wrote: I found the errors should blame to some

Re: fsck lowmem mode only: ERROR: errors found in fs roots

2018-10-18 Thread Su Yue
On 10/18/18 10:41 PM, Christoph Anton Mitterer wrote: Hey. So I'm back from a longer vacation and had now the time to try out your patches from below: On Wed, 2018-09-05 at 15:04 +0800, Su Yue wrote: I found the errors should blame to something about inode_extref check in lowmem mo

Re: [PATCH v3 1/4] btrfs: Introduce find_free_extent_ctl structure for later rework

2018-10-12 Thread Su Yue
during find_free_extent() refactor. Signed-off-by: Qu Wenruo Reviewed-by: Su Yue --- fs/btrfs/extent-tree.c | 253 ++--- 1 file changed, 161 insertions(+), 92 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index de6f75f5547b

Re: [PATCH v2 4/4] btrfs: Refactor find_free_extent() loops update into find_free_extent_update_loop()

2018-10-10 Thread Su Yue
block group | |- Check if the result is valid | | |- If valid, then exit | |- Jump to next block group | |- Push harder to find free extents |- If not found, re-iterate all block groups Clean enough. Signed-off-by: Qu Wenruo Reviewed-by: Su Yue --- fs/btrfs/extent-tree.c | 218

Re: [PATCH v2 3/4] btrfs: Refactor unclustered extent allocation into find_free_extent_unclustered()

2018-10-10 Thread Su Yue
Wenruo Reviewed-by: Su Yue --- fs/btrfs/extent-tree.c | 112 - 1 file changed, 66 insertions(+), 46 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index a603900e0eb8..5bc8919edac2 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs

Re: [PATCH v2 2/4] btrfs: Refactor clustered extent allocation into find_free_extent_clustered()

2018-10-10 Thread Su Yue
jumping between different labels in find_free_extent(), the helper function will use return value to indicate different behavior. LGTM for code. One nit in comment. Reviewed-by: Su Yue Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 239 - 1 file

  1   2   3   4   5   6   >