Re: [PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

On 2019/4/26 上午11:32, robbieko wrote: [snipped] >> >> But now, before creating new snapshot, we will flush all data. >> >> This means at this time, the data will be written back *NOCOW*, just as >> expected. >> >>> will_be_snapshotted will be set, and then flush >> >> will_be_snapshotted will al

Re: [PATCH 00/10] cleanup btrfs_ioctl_setflags and btrfs_set_prop

On 24/4/19 11:14 PM, David Sterba wrote: On Sat, Apr 20, 2019 at 07:48:50PM +0800, Anand Jain wrote: A set of pathes to clean up btrfs_ioctl_setflags(), btrfs_set_prop_trans() and btrfs_set_prop(). So as to avoidi duplicate code in btrfs_ioctl_setflags() and killing of btrfs_set_prop_trans(). A

Re: Some file lost when send back snapshot.

Thank you for your reply. # What does @Host A ("lily" master, new volume) aki@lily /mnt % socat TCP-LISTEN:25500 STDOUT | sudo btrfs receive -vv . @Host B ("daisy" backup, having volumes) aki@daisy /HySlave % sudo btrfs send -v 190315-001000 | socat STDIN TCP:lily.local:25500 ( ...Transfared

Re: [PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

Qu Wenruo 於 2019-04-26 10:53 寫到: On 2019/4/26 上午10:47, robbieko wrote: Qu Wenruo 於 2019-04-26 07:20 寫到: [snip] 1. Serializing snapshots (acquiring write-side of the semaphore) With the new btrfs_start_delalloc_snapshot() in btrfs_commit_transaction(), do we really need to do anything speci

Re: [PATCH v2] btrfs: fix filtering of scratch device in test case 048

On Thu, Apr 25, 2019 at 01:37:09AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > The recent commit 4529b20e1aa8f9 ("btrfs/048: amend property validation > cases"), does not properly filter the scratch device because the error > messages are sent to stderr and not to stdout, and the

[PATCH] btrfs: Check the compression level before getting a workspace

When a file's compression property is set as zlib or zstd but leave the compression mount option not be set, that means btrfs will try to compress the file with default compression level. But in btrfs_compress_pages(), it calls get_workspace() with level = 0. This will return a workspace with a wro

Re: [PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

On 2019/4/26 上午10:47, robbieko wrote: > Qu Wenruo 於 2019-04-26 07:20 寫到: >> [snip] > > 1. Serializing snapshots (acquiring write-side of the semaphore) With the new btrfs_start_delalloc_snapshot() in btrfs_commit_transaction(), do we really need to do anything special

Re: [PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

Qu Wenruo 於 2019-04-26 07:20 寫到: [snip] 1. Serializing snapshots (acquiring write-side of the semaphore) With the new btrfs_start_delalloc_snapshot() in btrfs_commit_transaction(), do we really need to do anything special now? Snapshot creation will only happen in btrfs_commit_transaction(

Re: [PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

[snip] >>> >>> 1. Serializing snapshots (acquiring write-side of the semaphore) >> >> With the new btrfs_start_delalloc_snapshot() in >> btrfs_commit_transaction(), do we really need to do anything special now? >> >> Snapshot creation will only happen in btrfs_commit_transaction(), as >> long as al

Re: fallocate does not prevent ENOSPC on write

On 2019/4/25 下午10:43, Filipe Manana wrote: > On Thu, Apr 25, 2019 at 3:28 PM Qu Wenruo wrote: >> >> >> >> On 2019/4/25 下午10:09, Josef Bacik wrote: >>> On Thu, Apr 25, 2019 at 09:50:25PM +0800, Qu Wenruo wrote: On 2019/4/25 下午9:25, Josef Bacik wrote: [snip] >>> >>> Wha

[PATCH][v2] btrfs: track odirect bytes in flight

When diagnosing a slowdown of generic/224 I noticed we were not doing anything when calling into shrink_delalloc(). This is because all writes in 224 are O_DIRECT, not delalloc, and thus our delalloc_bytes counter is 0, which short circuits most of the work inside of shrink_delalloc(). However O_

Re: Some file lost when send back snapshot.

25.04.2019 16:34, MASAKI Haruka пишет: > I send snapshot from backup volume, and send new snaopshot to backup volume, > and some file lost in received volume. > > In detail: > > 1. Host A has new Btrfs volume and Host B have two backuped readonly > subvolumes (190312-010100, 190315-001000). > 2.

Re: [PATCH] btrfs: fix null pointer dereference in write_pinned_extent_entries

On Fri, Apr 26, 2019 at 01:08:06AM +0800, Anand Jain wrote: > btrfs/049 fails with null pointer dereference > > kernel: BUG: unable to handle kernel NULL pointer dereference at > 0030 > :: > kernel: RIP: 0010:write_pinned_extent_entries+0x27/0x100 [btrfs] > :: > kernel: Call Trace:

[PATCH] btrfs: fix null pointer dereference in write_pinned_extent_entries

btrfs/049 fails with null pointer dereference kernel: BUG: unable to handle kernel NULL pointer dereference at 0030 :: kernel: RIP: 0010:write_pinned_extent_entries+0x27/0x100 [btrfs] :: kernel: Call Trace: kernel: __btrfs_write_out_cache+0x273/0x450 [btrfs] kernel: btrfs_write

Re: [PATCH v3] btrfs: ctree: Dump the leaf before BUG_ON()

On Thu, Apr 25, 2019 at 08:55:53AM +0800, Qu Wenruo wrote: > We have a user reporting BUG_ON() triggered in > btrfs_set_item_key_safe(). > > Let's dump the leaf content before triggering BUG_ON() so that we can > have some clue on what's going wrong. > The output of tree locks should help us to de

Re: [PATCH 0/2] ENOSPC refinements

On Wed, Apr 10, 2019 at 03:56:08PM -0400, Josef Bacik wrote: > I noticed our continuous testing for btrfs started timing out recently, and > dug > down to discover that generic/224 was taking upwards of an hour to run. This > test should take seconds to run, so this is a problem. > > Fix this by

Re: [PATCH] btrfs: tree-checker: Allow error injection for tree-checker

On Thu, Apr 25, 2019 at 10:22:30PM +0800, Qu Wenruo wrote: > > > On 2019/4/25 下午10:13, David Sterba wrote: > > On Thu, Apr 25, 2019 at 08:30:38AM +0800, Qu Wenruo wrote: > >> > >> > >> On 2019/4/25 上午1:45, David Sterba wrote: > >>> On Wed, Apr 24, 2019 at 03:22:53PM +0800, Qu Wenruo wrote: >

Re: [PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

On Thu, Apr 25, 2019 at 2:06 PM Qu Wenruo wrote: > > > > On 2019/4/25 下午5:37, Nikolay Borisov wrote: > > > > > > On 25.04.19 г. 12:29 ч., Qu Wenruo wrote: > >> > >> > >> On 2018/8/6 上午10:30, robbieko wrote: > >>> From: Robbie Ko > >>> > >>> Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writt

Re: fallocate does not prevent ENOSPC on write

On Thu, Apr 25, 2019 at 3:28 PM Qu Wenruo wrote: > > > > On 2019/4/25 下午10:09, Josef Bacik wrote: > > On Thu, Apr 25, 2019 at 09:50:25PM +0800, Qu Wenruo wrote: > >> > >> > >> On 2019/4/25 下午9:25, Josef Bacik wrote: > >> [snip] > > > > What if the commit is reverted, if the problem is othe

Re: fallocate does not prevent ENOSPC on write

On Thu, Apr 25, 2019 at 9:17 AM Qu Wenruo wrote: > > > > On 2019/4/23 下午7:33, David Sterba wrote: > > On Tue, Apr 23, 2019 at 10:16:32AM +0800, Qu Wenruo wrote: > >> On 2019/4/23 上午5:09, Jakob Unterwurzacher wrote: > >>> I have a user who is reporting ENOSPC errors when running gocryptfs on > >>>

Re: [PATCH] btrfs: tree-checker: Allow error injection for tree-checker

On 2019/4/25 下午10:13, David Sterba wrote: > On Thu, Apr 25, 2019 at 08:30:38AM +0800, Qu Wenruo wrote: >> >> >> On 2019/4/25 上午1:45, David Sterba wrote: >>> On Wed, Apr 24, 2019 at 03:22:53PM +0800, Qu Wenruo wrote: Allowing error injection for btrfs_check_leaf_full() and btrfs_check_no

Re: fallocate does not prevent ENOSPC on write

On 2019/4/25 下午10:13, Josef Bacik wrote: > On Thu, Apr 25, 2019 at 10:11:41PM +0800, Qu Wenruo wrote: >> >> >> On 2019/4/25 下午10:09, Josef Bacik wrote: >>> On Thu, Apr 25, 2019 at 09:50:25PM +0800, Qu Wenruo wrote: On 2019/4/25 下午9:25, Josef Bacik wrote: [snip] >>> >>>

Re: fallocate does not prevent ENOSPC on write

On Thu, Apr 25, 2019 at 10:11:41PM +0800, Qu Wenruo wrote: > > > On 2019/4/25 下午10:09, Josef Bacik wrote: > > On Thu, Apr 25, 2019 at 09:50:25PM +0800, Qu Wenruo wrote: > >> > >> > >> On 2019/4/25 下午9:25, Josef Bacik wrote: > >> [snip] > > > > What if the commit is reverted, if the proble

Re: [PATCH] btrfs: tree-checker: Allow error injection for tree-checker

On Thu, Apr 25, 2019 at 08:30:38AM +0800, Qu Wenruo wrote: > > > On 2019/4/25 上午1:45, David Sterba wrote: > > On Wed, Apr 24, 2019 at 03:22:53PM +0800, Qu Wenruo wrote: > >> Allowing error injection for btrfs_check_leaf_full() and > >> btrfs_check_node() is useful to test the failure path of btrf

Re: fallocate does not prevent ENOSPC on write

On 2019/4/25 下午10:09, Josef Bacik wrote: > On Thu, Apr 25, 2019 at 09:50:25PM +0800, Qu Wenruo wrote: >> >> >> On 2019/4/25 下午9:25, Josef Bacik wrote: >> [snip] > > What if the commit is reverted, if the problem is otherwise hard to fix? > This seems to break the semantics of fallocat

Re: fallocate does not prevent ENOSPC on write

On Thu, Apr 25, 2019 at 09:50:25PM +0800, Qu Wenruo wrote: > > > On 2019/4/25 下午9:25, Josef Bacik wrote: > [snip] > >>> > >>> What if the commit is reverted, if the problem is otherwise hard to fix? > >>> This seems to break the semantics of fallocate so the performance should > >>> not the main

Re: fallocate does not prevent ENOSPC on write

On 2019/4/25 下午9:25, Josef Bacik wrote: [snip] >>> >>> What if the commit is reverted, if the problem is otherwise hard to fix? >>> This seems to break the semantics of fallocate so the performance should >>> not the main concern here. >> > > Are we sure the ENOSPC is coming from the data reserv

Re: [PATCH v3] btrfs: ctree: Dump the leaf before BUG_ON()

On Thu, Apr 25, 2019 at 08:55:53AM +0800, Qu Wenruo wrote: > We have a user reporting BUG_ON() triggered in > btrfs_set_item_key_safe(). > > Let's dump the leaf content before triggering BUG_ON() so that we can > have some clue on what's going wrong. > The output of tree locks should help us to de

Some file lost when send back snapshot.

I send snapshot from backup volume, and send new snaopshot to backup volume, and some file lost in received volume. In detail: 1. Host A has new Btrfs volume and Host B have two backuped readonly subvolumes (190312-010100, 190315-001000). 2. Host B send 190315-001000 subvolume to Host A. 3. Host

Re: fallocate does not prevent ENOSPC on write

On Thu, Apr 25, 2019 at 01:49:23PM +0800, Qu Wenruo wrote: > > > On 2019/4/23 下午7:33, David Sterba wrote: > > On Tue, Apr 23, 2019 at 10:16:32AM +0800, Qu Wenruo wrote: > >> On 2019/4/23 上午5:09, Jakob Unterwurzacher wrote: > >>> I have a user who is reporting ENOSPC errors when running gocryptfs

[PATCH 1/2] btrfs-progs: add helper to create xattr name

This is a preparatory patch to add more xattr attributes, care a helper function to alloc xattr name. Signed-off-by: Anand Jain --- props.c | 26 +++--- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/props.c b/props.c index efa11180d4c5..3a498bd9e904 100644 --

[PATCH 2/2] btrfs-progs: add readmirror policy

This sets the readmirror= as a btrfs. extentded attribute. Signed-off-by: Anand Jain --- props.c | 49 + 1 file changed, 49 insertions(+) diff --git a/props.c b/props.c index 3a498bd9e904..1d1a2c7f9d14 100644 --- a/props.c +++ b/props.c @@ -178,6

[PATCH 2/3] btrfs: add readmirror property framework

Function call chain __btrfs_map_block()->find_live_mirror() uses thread %pid to determine the %mirror_num for the read when mirror_num=0 in the argument. This patch introduces a framework so that readmirror is a configurable parameter, with default set to pid. Signed-off-by: Anand Jain --- fs/

[PATCH 3/3] btrfs: add readmirror devid property

Introduces devid readmirror property, to direct read IO to the specified device(s). The readmirror property is stored as extended attribute on the root inode. The readmirror input format is devid1,2,3.. etc. And for the each devid provided, a new flag BTRFS_DEV_STATE_READ_OPTIMISED is set. As of

[PATCH 0/3] readmirror feature

These patches are tested to be working fine. Function call chain __btrfs_map_block()->find_live_mirror() uses thread pid to determine the %mirror_num when the mirror_num=0. This patch introduces a framework so that we can add policies to determine the %mirror_num. And adds the devid as the readm

[PATCH 1/3] btrfs: add inode pointer to prop_handler::validate()

In preparation to add the readmirror property, pass inode in the prop_handler::validate() function vector, so that it can fetch corresponding fs_devices. Signed-off-by: Anand Jain --- fs/btrfs/props.c | 12 +++- fs/btrfs/props.h | 4 ++-- fs/btrfs/xattr.c | 2 +- 3 files changed, 10 in

[PATCH 0/3] readmirror feature

These patches are tested to be working fine. Function call chain __btrfs_map_block()->find_live_mirror() uses thread pid to determine the %mirror_num when the mirror_num=0. This patch introduces a framework so that we can add policies to determine the %mirror_num. And adds the devid as the readm

[PATCH 1/3] btrfs: add inode pointer to prop_handler::validate()

In preparation to add the readmirror property, pass inode in the prop_handler::validate() function vector, so that it can fetch corresponding fs_devices. Signed-off-by: Anand Jain --- fs/btrfs/props.c | 12 +++- fs/btrfs/props.h | 4 ++-- fs/btrfs/xattr.c | 2 +- 3 files changed, 10 in

[PATCH 2/3] btrfs: add readmirror property framework

Function call chain __btrfs_map_block()->find_live_mirror() uses thread %pid to determine the %mirror_num for the read when mirror_num=0 in the argument. This patch introduces a framework so that readmirror is a configurable parameter, with default set to pid. Signed-off-by: Anand Jain --- fs/

[PATCH 3/3] btrfs: add readmirror devid property

From: Anand Jain Introduces devid readmirror property, to direct read IO to the specified device(s). The readmirror property is stored as extended attribute on the root inode. The readmirror input format is devid1,2,3.. etc. And for the each devid provided, a new flag BTRFS_DEV_STATE_READ_OPTIMI

[PATCH 1/3] btrfs: add inode pointer to prop_handler::validate()

In preparation to add the readmirror property, pass inode in the prop_handler::validate() function vector, so that it can fetch corresponding fs_devices. Signed-off-by: Anand Jain --- fs/btrfs/props.c | 12 +++- fs/btrfs/props.h | 4 ++-- fs/btrfs/xattr.c | 2 +- 3 files changed, 10 in

[PATCH 0/3] readmirror feature

These patches are tested to be working fine. Function call chain __btrfs_map_block()->find_live_mirror() uses thread pid to determine the %mirror_num when the mirror_num=0. This patch introduces a framework so that we can add policies to determine the %mirror_num. And adds the devid as the readm

Re: [PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

On 2019/4/25 下午5:37, Nikolay Borisov wrote: > > > On 25.04.19 г. 12:29 ч., Qu Wenruo wrote: >> >> >> On 2018/8/6 上午10:30, robbieko wrote: >>> From: Robbie Ko >>> >>> Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") >>> forced nocow writes to fallback to COW, during writeback, >>> w

Re: [PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

On 25.04.19 г. 12:29 ч., Qu Wenruo wrote: > > > On 2018/8/6 上午10:30, robbieko wrote: >> From: Robbie Ko >> >> Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") >> forced nocow writes to fallback to COW, during writeback, >> when a snapshot is created. This resulted in writes made

Re: [PATCH v3] btrfs: ctree: Dump the leaf before BUG_ON()

On Thu, Apr 25, 2019 at 8:52 AM Qu Wenruo wrote: > > We have a user reporting BUG_ON() triggered in > btrfs_set_item_key_safe(). > > Let's dump the leaf content before triggering BUG_ON() so that we can > have some clue on what's going wrong. > The output of tree locks should help us to debug such

Re: [PATCH] Btrfs: fix unexpected failure of nocow buffered writes after snapshotting when low on space

On 2018/8/6 上午10:30, robbieko wrote: > From: Robbie Ko > > Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") > forced nocow writes to fallback to COW, during writeback, > when a snapshot is created. This resulted in writes made before > creating the snapshot to unexpectedly fail wit