Re: [PATCH 0/2] scop GFP_NOFS api

2016-04-28 Thread NeilBrown
On Tue, Apr 26 2016, Michal Hocko wrote: > Hi, > we have discussed this topic at LSF/MM this year. There was a general > interest in the scope GFP_NOFS allocation context among some FS > developers. For those who are not aware of the discussion or the issue > I am trying to sort out (or at least s

Re: [PATCH v3 3/3] block: avoid to call .bi_end_io() recursively

2016-04-28 Thread Ming Lei
On Fri, Apr 29, 2016 at 12:59 AM, Mikulas Patocka wrote: > > > On Fri, 29 Apr 2016, Ming Lei wrote: > >> On Thu, Apr 28, 2016 at 11:58 PM, Mikulas Patocka >> wrote: >> > >> > >> > On Thu, 28 Apr 2016, Ming Lei wrote: >> > >> >> Hi Mikulas, >> >> >> >> On Thu, Apr 28, 2016 at 11:29 PM, Mikulas Pa

Re: [PATCH RFC 04/16] btrfs-progs: fsck: Introduce function to check referencer of a backref

2016-04-28 Thread Qu Wenruo
Josef Bacik wrote on 2016/04/28 10:17 -0400: On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce a new function check_tree_block_backref() to check if a backref points to correct referencer. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo --- cmds-check.c | 95 ++

Re: [PATCH RFC 03/16] btrfs-progs: fsck: Introduce function to query tree block level

2016-04-28 Thread Qu Wenruo
Josef Bacik wrote on 2016/04/28 10:13 -0400: On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce function query_tree_block_level() to resolve tree block level by reading out the tree block. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo --- This skips the check_tree_bl

Re: [PATCH] Btrfs: fix divide error upon chunk's stripe_len

2016-04-28 Thread Qu Wenruo
Liu Bo wrote on 2016/04/28 10:48 -0700: On Wed, Apr 27, 2016 at 07:33:18PM +0200, David Sterba wrote: On Wed, Apr 27, 2016 at 10:23:35AM -0700, Liu Bo wrote: On Wed, Apr 27, 2016 at 06:39:03PM +0200, David Sterba wrote: On Tue, Apr 26, 2016 at 05:53:31PM -0700, Liu Bo wrote: The struct 'map

Re: [PATCH RFC 00/16] Introduce low memory usage btrfsck mode

2016-04-28 Thread Qu Wenruo
Josef Bacik wrote on 2016/04/28 10:32 -0400: On 04/25/2016 11:48 PM, Qu Wenruo wrote: The branch can be fetched from my github: https://github.com/adam900710/btrfs-progs.git low_mem_fsck_rebasing Original btrfsck checks extent tree in a very efficient method, by recording every checked extent

Re: Question: raid1 behaviour on failure

2016-04-28 Thread Henk Slager
On Thu, Apr 28, 2016 at 7:09 AM, Matthias Bodenbinder wrote: > Am 26.04.2016 um 18:19 schrieb Henk Slager: >> It looks like a JMS567 + SATA port multipliers behaind it are used in >> this drivebay. The command lsusb -v could show that. So your HW >> setup is like JBOD, not RAID. > > Here is the

Re: [PATCH] Btrfs: fix divide error upon chunk's stripe_len

2016-04-28 Thread Liu Bo
On Wed, Apr 27, 2016 at 07:33:18PM +0200, David Sterba wrote: > On Wed, Apr 27, 2016 at 10:23:35AM -0700, Liu Bo wrote: > > On Wed, Apr 27, 2016 at 06:39:03PM +0200, David Sterba wrote: > > > On Tue, Apr 26, 2016 at 05:53:31PM -0700, Liu Bo wrote: > > > > The struct 'map_lookup' uses type int for @

[PATCH v2 0/2] fstests: btrfs: the lost levels

2016-04-28 Thread Omar Sandoval
From: Omar Sandoval Changes since v1: - Fixed _supported_fs and _supported_os in patch 2 - Added Josef's Reviewed-by Original cover letter below: Inspired by a conversation a few of us had at LSF about forgotten fstests, I went back and checked if I had let any tests disappear into the ether.

[PATCH v2 1/2] btrfs: add replace missing and replace RAID 5/6 to profile configs

2016-04-28 Thread Omar Sandoval
From: Omar Sandoval Replacing and scrubbing RAID 5/6 is now supported on Btrfs. Enable it in _btrfs_get_profile_configs while making it more generic to also support replace missing. Reviewed-by: Eryu Guan Signed-off-by: Omar Sandoval --- common/rc | 95

[PATCH v2 2/2] btrfs: add test for replacing a missing device

2016-04-28 Thread Omar Sandoval
From: Omar Sandoval Now that _btrfs_get_profile_configs supports replace missing and the kernel doesn't crash when replacing a missing RAID 5/6 device, test it. Based on an earlier test from Wang Yanfeng. Reviewed-by: Josef Bacik Signed-off-by: Omar Sandoval --- tests/btrfs/027 | 119

Re: [PATCH v3 3/3] block: avoid to call .bi_end_io() recursively

2016-04-28 Thread Mikulas Patocka
On Fri, 29 Apr 2016, Ming Lei wrote: > On Thu, Apr 28, 2016 at 11:58 PM, Mikulas Patocka wrote: > > > > > > On Thu, 28 Apr 2016, Ming Lei wrote: > > > >> Hi Mikulas, > >> > >> On Thu, Apr 28, 2016 at 11:29 PM, Mikulas Patocka > >> wrote: > >> > > >> > > >> > On Thu, 28 Apr 2016, Ming Lei wrot

Re: [PATCH] btrfs: do not return EBUSY on concurrent subvolume mounts

2016-04-28 Thread David Sterba
On Wed, Apr 27, 2016 at 05:14:36PM +0200, David Sterba wrote: > --- a/fs/btrfs/super.c > +++ b/fs/btrfs/super.c > @@ -1588,8 +1588,6 @@ static struct dentry *btrfs_mount(struct > file_system_type *fs_type, int flags, > if (s->s_root) { > btrfs_close_devices(fs_devices); >

Re: [PATCH v3 3/3] block: avoid to call .bi_end_io() recursively

2016-04-28 Thread Ming Lei
On Thu, Apr 28, 2016 at 11:58 PM, Mikulas Patocka wrote: > > > On Thu, 28 Apr 2016, Ming Lei wrote: > >> Hi Mikulas, >> >> On Thu, Apr 28, 2016 at 11:29 PM, Mikulas Patocka >> wrote: >> > >> > >> > On Thu, 28 Apr 2016, Ming Lei wrote: >> > >> >> There were reports about heavy stack use by recurs

Re: [PATCH v3 3/3] block: avoid to call .bi_end_io() recursively

2016-04-28 Thread Mikulas Patocka
On Thu, 28 Apr 2016, Ming Lei wrote: > Hi Mikulas, > > On Thu, Apr 28, 2016 at 11:29 PM, Mikulas Patocka wrote: > > > > > > On Thu, 28 Apr 2016, Ming Lei wrote: > > > >> There were reports about heavy stack use by recursive calling > >> .bi_end_io()([1][2][3]). For example, more than 16K stack

Re: [PATCH v3 3/3] block: avoid to call .bi_end_io() recursively

2016-04-28 Thread Ming Lei
Hi Mikulas, On Thu, Apr 28, 2016 at 11:29 PM, Mikulas Patocka wrote: > > > On Thu, 28 Apr 2016, Ming Lei wrote: > >> There were reports about heavy stack use by recursive calling >> .bi_end_io()([1][2][3]). For example, more than 16K stack is >> consumed in a single bio complete path[3], and in [

Re: [PATCH v3 3/3] block: avoid to call .bi_end_io() recursively

2016-04-28 Thread Mikulas Patocka
On Thu, 28 Apr 2016, Ming Lei wrote: > There were reports about heavy stack use by recursive calling > .bi_end_io()([1][2][3]). For example, more than 16K stack is > consumed in a single bio complete path[3], and in [2] stack > overflow can be triggered if 20 nested dm-crypt is used. > > Also p

Re: [PATCH RFC 04/16] btrfs-progs: fsck: Introduce function to check referencer of a backref

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce a new function check_tree_block_backref() to check if a backref points to correct referencer. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo --- cmds-check.c | 95 ++

Re: [PATCH RFC 00/16] Introduce low memory usage btrfsck mode

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: The branch can be fetched from my github: https://github.com/adam900710/btrfs-progs.git low_mem_fsck_rebasing Original btrfsck checks extent tree in a very efficient method, by recording every checked extent in extent record tree to ensure every extent wi

Checking if two files share extents

2016-04-28 Thread Roman Mamedov
Hello, I'm writing a simple custom deduplication script, and faced a problem with how to make it skip already deduped files. E.g., if I do cp -a --reflink fileA fileB ...after this, how to check if fileA and fileB refer to the same extents on disk? I tried comparing the outputs of "filefrag -e

Re: [PATCH RFC 09/16] btrfs-progs: fsck: Introduce function to check dev extent item

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce function check_dev_extent_item() to find its referencer chunk. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo --- Reviewed-by: Josef Bacik Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe lin

Re: [PATCH RFC 10/16] btrfs-progs: fsck: Introduce function to check dev used space

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce function check_dev_item() to check used space with dev extent items. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo --- cmds-check.c | 64 1 file changed, 64 in

Re: [PATCH RFC 08/16] btrfs-progs: fsck: Introduce function to check an extent

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce function check_extent_item() using previous introduced functions. With previous function to check referencer and backref, this function can be quite easy. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo --- cmds-check.c |

Re: [PATCH RFC 07/16] btrfs-progs: fsck: Introduce function to check shared data backref

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce the function check_shared_data_backref() to check the referencer of a given shared data backref. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef -- To unsubscribe from this list:

Re: [PATCH RFC 06/16] btrfs-progs: fsck: Introduce function to check referencer for data backref

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce new function check_extent_data_backref() to search referencer for a given data backref. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo Reviewed-by: Josef Bacik Thanks, Josef -- To unsubscribe from this list: send the

Re: [PATCH RFC 05/16] btrfs-progs: fsck: Introduce function to check shared block ref

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce function check_shared_block_backref() to check shared block ref. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo --- Reviewed-by: Josef Bacik Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe l

Re: [PATCH RFC 04/16] btrfs-progs: fsck: Introduce function to check referencer of a backref

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce a new function check_tree_block_backref() to check if a backref points to correct referencer. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo --- cmds-check.c | 95 ++

Re: [PATCH RFC 03/16] btrfs-progs: fsck: Introduce function to query tree block level

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce function query_tree_block_level() to resolve tree block level by reading out the tree block. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo --- This skips the check_tree_block if the transid passes, but we could have a

Re: [PATCH RFC v1.1 02/16] btrfs-progs: fsck: Introduce function to check data backref in extent tree

2016-04-28 Thread Josef Bacik
On 04/27/2016 09:43 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce a new function check_data_extent_item() to check if the corresponding data backref exists in extent tree. Signed-off-by: Lu Fengqi Signed-off-by: Qu Wenruo --- Changelog: v1.1: Fix a typo which passed wrong parameter for h

Re: [PATCH RFC 01/16] btrfs-progs: fsck: Introduce function to check tree block backref in extent tree

2016-04-28 Thread Josef Bacik
On 04/25/2016 11:48 PM, Qu Wenruo wrote: From: Lu Fengqi Introduce function check_tree_block_ref() to check whether a tree block has correct backref in extent tree. Unlike old extent tree check method, we only use search_slot() to search reference, no extra structure will be allocated in heap

Re: [PATCH 2/2] btrfs: add test for replacing a missing device

2016-04-28 Thread Josef Bacik
On 04/28/2016 12:31 AM, Omar Sandoval wrote: From: Omar Sandoval Now that _btrfs_get_profile_configs supports replace missing and the kernel doesn't crash when replacing a missing RAID 5/6 device, test it. Based on an earlier test from Wang Yanfeng. Signed-off-by: Omar Sandoval --- tests/btr

Re: [PATCH v5 00/13] Introduce device state 'failed', spare device and auto replace

2016-04-28 Thread Yauhen Kharuzhy
On Mon, Apr 18, 2016 at 07:31:31PM +0800, Anand Jain wrote: > Thanks for various comments, tests and feedback. > > > Changelog: > - > > v4->v5: > Kernel: > a. Originally we had bugs as fixed in the patches below > [PATCH] btrfs: s_bdev is not null after missing replace > [PAT

Re: ...in the matter of partition size

2016-04-28 Thread Austin S. Hemmelgarn
On 2016-04-27 16:18, Juan Alberto Cirez wrote: Quick question: Supposed I have n-number of storage pods (physical servers with n-number of physical hhds). The end deployment will be btrfs at the brick/block level with a distributed file system on top. Keeping in mind that my overriding goal is to

Re: ...in the matter of partition size

2016-04-28 Thread Austin S. Hemmelgarn
On 2016-04-27 22:55, Chris Murphy wrote: On Wed, Apr 27, 2016 at 8:51 PM, Chris Murphy wrote: On Wed, Apr 27, 2016 at 2:18 PM, Juan Alberto Cirez wrote: Quick question: Supposed I have n-number of storage pods (physical servers with n-number of physical hhds). The end deployment will be btrfs

Re: Add device while rebalancing

2016-04-28 Thread Austin S. Hemmelgarn
On 2016-04-27 19:19, Chris Murphy wrote: On Wed, Apr 27, 2016 at 5:22 AM, Austin S. Hemmelgarn wrote: On 2016-04-26 20:58, Chris Murphy wrote: On Tue, Apr 26, 2016 at 5:44 AM, Juan Alberto Cirez wrote: With GlusterFS as a distributed volume, the files are already spread among the servers

Re: [PATCH] btrfs: do not return EBUSY on concurrent subvolume mounts

2016-04-28 Thread David Sterba
On Thu, Apr 28, 2016 at 11:12:20AM +0200, David Sterba wrote: > On Wed, Apr 27, 2016 at 04:22:17PM -0700, Liu Bo wrote: > > On Wed, Apr 27, 2016 at 05:14:36PM +0200, David Sterba wrote: > > > A user reported mount failures with EBUSY during boot, there's root > > > partition and many subvolumes, mo

Re: [PATCH] btrfs: do not return EBUSY on concurrent subvolume mounts

2016-04-28 Thread Filipe Manana
On Wed, Apr 27, 2016 at 4:14 PM, David Sterba wrote: > A user reported mount failures with EBUSY during boot, there's root > partition and many subvolumes, mounted via /etc/fstab. > > The failure depends on timing, when multiple subvolumes reach the code > between superblock creation in RO mode, w

Re: [bug] df returns 100% used for empty mixed block group filesystem

2016-04-28 Thread Luis de Bethencourt
On 27/04/16 23:55, David Sterba wrote: > On Thu, Mar 10, 2016 at 06:40:54PM +0100, David Sterba wrote: >> On Thu, Mar 10, 2016 at 01:33:34PM +0500, Roman Mamedov wrote: >>> On Thu, 10 Mar 2016 01:24:44 -0700 >>> Chris Murphy wrote: >>> kernel 4.5.0 rc7 btrfs-progs-4.4.1-1 Summa

backup and restore consideration of btrfs

2016-04-28 Thread sri
Btrfs supports btrfs send and receive tools to backup subvolumes which is built in and good. However considering the current storage data utilization in the industry, using send/receive would be very slow operation and kills lot of CUP cycles of source machine. As an example, file systems such

Re: [PATCH] btrfs: do not return EBUSY on concurrent subvolume mounts

2016-04-28 Thread David Sterba
On Wed, Apr 27, 2016 at 04:22:17PM -0700, Liu Bo wrote: > On Wed, Apr 27, 2016 at 05:14:36PM +0200, David Sterba wrote: > > A user reported mount failures with EBUSY during boot, there's root > > partition and many subvolumes, mounted via /etc/fstab. > > > > The failure depends on timing, when mul

Re: Question: raid1 behaviour on failure

2016-04-28 Thread Duncan
Gareth Pye posted on Thu, 28 Apr 2016 15:24:51 +1000 as excerpted: > PDF doc info dates it at 23/1/2013, which is the best guess that can > easily be found. Well, "easily" is relative, but motivated by your observation I first confirmed it, then decided to see what google had to say about the a