Re: [PATCH] btrfs: Fix bug for misused dev_t when lookup in dev state hash table.

2017-10-17 Thread Nikolay Borisov
On 18.10.2017 06:43, Gu, Jinxiang wrote: > Hi, > >> -Original Message- >> From: Nikolay Borisov [mailto:nbori...@suse.com] >> Sent: Tuesday, October 17, 2017 9:36 PM >> To: Gu, Jinxiang/顾 金香 ; linux-btrfs@vger.kernel.org; >> h...@lst.de >> Subject: Re: [PATCH] btrfs: Fix bug for misused

Re: Mount failing - unable to find logical

2017-10-17 Thread Roman Mamedov
On Wed, 18 Oct 2017 09:24:01 +0800 Qu Wenruo wrote: > > > On 2017年10月18日 04:43, Cameron Kelley wrote: > > Hey btrfs gurus, > > > > I have a 4 disk btrfs filesystem that has suddenly stopped mounting > > after a recent reboot. The data is in an odd configuration due to > > originally being in a

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-17 Thread Duncan
Austin S. Hemmelgarn posted on Tue, 17 Oct 2017 15:19:09 -0400 as excerpted: >> It's a single-device filesystem, thus disconnects are obviously fatal. >> But, >> they never caused even a single bit of damage (as scrub goes), thus >> proving btrfs handles this kind of disconnects well. Unlike tim

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-17 Thread Duncan
Zoltán Ivánfi posted on Tue, 17 Oct 2017 23:56:56 +0200 as excerpted: > I understand that due to the lack of a write intent bitmap, btrfs does > not know what needs to be resynced exactly, therefore syncing the two > copies of a raid1 can not happen as fast in btrfs as in md. However, I > don't un

Re: Mount failing - unable to find logical

2017-10-17 Thread Qu Wenruo
On 2017年10月18日 11:22, Cameron Kelley wrote: > > > On 10-17-2017 6:24 PM, Qu Wenruo wrote: >> >> >> On 2017-10-18 04:43, Cameron Kelley wrote: >>> Hey btrfs gurus, >>> >>> I have a 4 disk btrfs filesystem that has suddenly stopped mounting >>> after a recent reboot. The data is in an odd configu

RE: [PATCH] btrfs: Fix bug for misused dev_t when lookup in dev state hash table.

2017-10-17 Thread Gu, Jinxiang
Hi, > -Original Message- > From: Nikolay Borisov [mailto:nbori...@suse.com] > Sent: Tuesday, October 17, 2017 9:36 PM > To: Gu, Jinxiang/顾 金香 ; linux-btrfs@vger.kernel.org; > h...@lst.de > Subject: Re: [PATCH] btrfs: Fix bug for misused dev_t when lookup in dev > state hash table. > >

Re: [PATCH] btrfs-progs: qgroup: show subvol path when qgroup show

2017-10-17 Thread Lu Fengqi
On Mon, Oct 16, 2017 at 08:55:11PM +0200, David Sterba wrote: >On Tue, Oct 10, 2017 at 04:37:09PM +0800, Lu Fengqi wrote: >> Show the absolute subvol path for the associated level-0 qgroup. >> >> Signed-off-by: Lu Fengqi >> --- >> Documentation/btrfs-qgroup.asciidoc | 2 + >> cmds-qgroup.c

Re: Mount failing - unable to find logical

2017-10-17 Thread Cameron Kelley
On 10-17-2017 6:24 PM, Qu Wenruo wrote: > > > On 2017-10-18 04:43, Cameron Kelley wrote: >> Hey btrfs gurus, >> >> I have a 4 disk btrfs filesystem that has suddenly stopped mounting >> after a recent reboot. The data is in an odd configuration due to >> originally being in a 3 disk RAID1 before

[PATCH] btrfs-progs: test: add new cli-test for subvol get/set-default

2017-10-17 Thread Misono, Tomohiro
Add new test to check functionality of subvol get/set-default. Signed-off-by: Tomohiro Misono --- .../008-subvolume-get-set-default/test.sh | 45 ++ 1 file changed, 45 insertions(+) create mode 100755 tests/cli-tests/008-subvolume-get-set-default/test.sh diff --git

Re: Mount failing - unable to find logical

2017-10-17 Thread Qu Wenruo
On 2017年10月18日 04:43, Cameron Kelley wrote: > Hey btrfs gurus, > > I have a 4 disk btrfs filesystem that has suddenly stopped mounting > after a recent reboot. The data is in an odd configuration due to > originally being in a 3 disk RAID1 before adding a 4th disk and running > a balance to conv

Re: 4.13: "error in btrfs_run_delayed_refs:3009: errno=-28 No space left" with 1.3TB unallocated / 737G free?

2017-10-17 Thread Duncan
Vladimir Panteleev posted on Tue, 17 Oct 2017 17:19:45 + as excerpted: > I'm experiencing some issues with a btrfs filesystem - mounting and > other operations taking forever, a balance that takes hours to start and > never completes (due to the error in the subject line), and I've also > seen

Re: Best strategie to remove devices from pool

2017-10-17 Thread Duncan
Cloud Admin posted on Tue, 17 Oct 2017 19:58:55 +0200 as excerpted: > Hi, > I want to remove two devices from a BTRFS RAID 1 pool. It should be > enough free space to do it, but what is the best strategie. Remove both > device in one call 'btrfs dev rem /dev/sda1 /dev/sdb1' (for example) or > shou

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-17 Thread Zoltán Ivánfi
Hi, I understand that due to the lack of a write intent bitmap, btrfs does not know what needs to be resynced exactly, therefore syncing the two copies of a raid1 can not happen as fast in btrfs as in md. However, I don't understand why converting from raid1 to single is so slow. After all, btrfs

Mount failing - unable to find logical

2017-10-17 Thread Cameron Kelley
Hey btrfs gurus, I have a 4 disk btrfs filesystem that has suddenly stopped mounting after a recent reboot. The data is in an odd configuration due to originally being in a 3 disk RAID1 before adding a 4th disk and running a balance to convert to RAID10. There wasn't enough free space to compl

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-17 Thread Adam Borowski
On Tue, Oct 17, 2017 at 03:19:09PM -0400, Austin S. Hemmelgarn wrote: > On 2017-10-17 13:06, Adam Borowski wrote: > > On Tue, Oct 17, 2017 at 08:40:20AM -0400, Austin S. Hemmelgarn wrote: > > > On 2017-10-17 07:42, Zoltan wrote: > > > > On Tue, Oct 17, 2017 at 1:26 PM, Austin S. Hemmelgarn > > > >

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-17 Thread Austin S. Hemmelgarn
On 2017-10-17 13:06, Adam Borowski wrote: On Tue, Oct 17, 2017 at 08:40:20AM -0400, Austin S. Hemmelgarn wrote: On 2017-10-17 07:42, Zoltan wrote: On Tue, Oct 17, 2017 at 1:26 PM, Austin S. Hemmelgarn wrote: I forget sometimes that people insist on storing large volumes of data on unreliable

Re: [PATCH RESEND] btrfs-progs: allow "no" to disable compression for convenience

2017-10-17 Thread David Sterba
On Sat, Oct 14, 2017 at 06:19:14PM +0200, Koen Kooi wrote: > Op 14-10-17 om 14:54 schreef Satoru Takeuchi: > > It's messy to use "" to disable compression. Introduce the new value "no" > > which can also be used for this purpose. > > Wouldn't 'none' be a better fit? 'none' will be also accepted -

Re: [PATCH RESEND] btrfs-progs: allow "no" to disable compression for convenience

2017-10-17 Thread David Sterba
On Sat, Oct 14, 2017 at 09:54:54PM +0900, Satoru Takeuchi wrote: > It's messy to use "" to disable compression. Introduce the new value "no" > which can also be used for this purpose. > > Signed-off-by: Satoru Takeuchi Applied, thanks. -- To unsubscribe from this list: send the line "unsubscribe

Re: [PATCH v3] btrfs-progs: change subvol set-default to also accept subvol path

2017-10-17 Thread David Sterba
On Fri, Oct 06, 2017 at 10:05:08AM +0900, Misono, Tomohiro wrote: > This patch changes "subvol set-default" to also accept the subvolume path > for convenience. > > This is the one of the issue on github: > https://github.com/kdave/btrfs-progs/issues/35 > > If there are two args, they are assume

Best strategie to remove devices from pool

2017-10-17 Thread Cloud Admin
Hi, I want to remove two devices from a BTRFS RAID 1 pool. It should be enough free space to do it, but what is the best strategie. Remove both device in one call 'btrfs dev rem /dev/sda1 /dev/sdb1' (for example) or should it be better in two separate calls? What is faster? Are there other constrai

4.13: "error in btrfs_run_delayed_refs:3009: errno=-28 No space left" with 1.3TB unallocated / 737G free?

2017-10-17 Thread Vladimir Panteleev
Hi, I'm experiencing some issues with a btrfs filesystem - mounting and other operations taking forever, a balance that takes hours to start and never completes (due to the error in the subject line), and I've also seen the NULL pointer dereference in free_reloc_roots which Naohiro Aota fixed

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-17 Thread Adam Borowski
On Tue, Oct 17, 2017 at 08:40:20AM -0400, Austin S. Hemmelgarn wrote: > On 2017-10-17 07:42, Zoltan wrote: > > On Tue, Oct 17, 2017 at 1:26 PM, Austin S. Hemmelgarn > > wrote: > > > > > I forget sometimes that people insist on storing large volumes of data on > > > unreliable storage... > > > >

Re: [PATCH v2] Btrfs-progs: do not add stale device into fs_devices

2017-10-17 Thread David Sterba
On Wed, Oct 11, 2017 at 11:57:16AM -0600, Liu Bo wrote: > If one of btrfs's devices was pulled out and we've replaced it with a > new one, then they have the same uuid. > > If that device gets reconnected, 'btrfs filesystem show' will show the > stale one instead of the new one, but on kernel side

Re: [PATCH 3/4] Btrfs: handle unaligned tail of data ranges more efficient

2017-10-17 Thread David Sterba
On Mon, Oct 16, 2017 at 01:09:04AM +0300, Timofey Titovets wrote: > May be then just add a comment at least at one of that functions? > Like: > /* > * Handle unaligned end, end is inclusive, so always unaligned > */ This would be best documented in the data structure definition, so we don't have

Re: [PATCH] btrfs: Fix bug for misused dev_t when lookup in dev state hash table.

2017-10-17 Thread Nikolay Borisov
On 17.10.2017 14:34, Gu Jinxiang wrote: > From: Gu JinXiang > > Fix bug of commit 74d46992e0d9 > ("block: replace bi_bdev with a gendisk pointer and partitions index"). > > In this modify, use bio_dev(bio) to find dev state in function > __btrfsic_submit_bio. But when dev_state added to hashta

Re: next/master build: 214 builds: 29 failed, 185 passed, 29 errors, 68 warnings (next-20171016)

2017-10-17 Thread David Sterba
On Tue, Oct 17, 2017 at 11:00:04AM +0200, Arnd Bergmann wrote: > > 2 fs/btrfs/tree-checker.c:186:4: warning: format '%lu' expects argument of > > type 'long unsigned int', but argument 6 has type 'unsigned int' [-Wformat=] > > 1 fs/btrfs/tree-checker.c:186:70: warning: format '%lu' expects argumen

Re: [PATCH v2 0/6] Btrfs-progs: rescue: To fix device related

2017-10-17 Thread Nikolay Borisov
On 17.10.2017 12:13, Qu Wenruo wrote: > The patchset can be fetched from github: > https://github.com/adam900710/btrfs-progs/tree/check_unaligned_dev > > There are several reports in mail list for btrfs device size related > problems. > > 1) kernel refuse to mount some fs, due to mismatched sup

Re: reproducable oops in btrfs/130 with latests mainline

2017-10-17 Thread Jeff Mahoney
On 10/17/17 8:46 AM, Qu Wenruo wrote: > > > On 2017年10月17日 19:11, Po-Hsu Lin wrote: >> Hello Chris, >> >> I can reproduce this on my side too, with Ubuntu 16.04 + 4.4.0-97 kernel. > > Btrfs/130 is a known bug. > > I submitted it to raise the concern about such situation and purposed > one possi

Re: reproducable oops in btrfs/130 with latests mainline

2017-10-17 Thread Qu Wenruo
On 2017年10月17日 19:11, Po-Hsu Lin wrote: > Hello Chris, > > I can reproduce this on my side too, with Ubuntu 16.04 + 4.4.0-97 kernel. Btrfs/130 is a known bug. I submitted it to raise the concern about such situation and purposed one possible solution (just disable deduped file detection). But

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-17 Thread Austin S. Hemmelgarn
On 2017-10-17 07:42, Zoltan wrote: Hi, On Tue, Oct 17, 2017 at 1:26 PM, Austin S. Hemmelgarn wrote: I forget sometimes that people insist on storing large volumes of data on unreliable storage... In my opinion the unreliability of the storage is the exact reason for wanting to use raid1. An

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-17 Thread Zoltan
Hi, On Tue, Oct 17, 2017 at 1:26 PM, Austin S. Hemmelgarn wrote: > I forget sometimes that people insist on storing large volumes of data on > unreliable storage... In my opinion the unreliability of the storage is the exact reason for wanting to use raid1. And I think any problem one encounter

[PATCH] btrfs: Fix bug for misused dev_t when lookup in dev state hash table.

2017-10-17 Thread Gu Jinxiang
From: Gu JinXiang Fix bug of commit 74d46992e0d9 ("block: replace bi_bdev with a gendisk pointer and partitions index"). In this modify, use bio_dev(bio) to find dev state in function __btrfsic_submit_bio. But when dev_state added to hashtable, it is using dev_t of block_device. Reproduce of th

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-17 Thread Austin S. Hemmelgarn
On 2017-10-16 21:14, Adam Borowski wrote: On Mon, Oct 16, 2017 at 01:27:40PM -0400, Austin S. Hemmelgarn wrote: On 2017-10-16 12:57, Zoltan wrote: On Mon, Oct 16, 2017 at 1:53 PM, Austin S. Hemmelgarn wrote: In an ideal situation, scrubbing should not be an 'only if needed' thing, even for a r

Re: reproducable oops in btrfs/130 with latests mainline

2017-10-17 Thread Po-Hsu Lin
Hello Chris, I can reproduce this on my side too, with Ubuntu 16.04 + 4.4.0-97 kernel. PREEMPT config: $ cat config-4.4.0-97-generic | grep PREEMPT CONFIG_PREEMPT_NOTIFIERS=y # CONFIG_PREEMPT_NONE is not set CONFIG_PREEMPT_VOLUNTARY=y # CONFIG_PREEMPT is not set Bug reports on launchpad: https:/

[PATCH v2 3/6] btrfs-progs: rescue: Introduce fix-device-size

2017-10-17 Thread Qu Wenruo
Introduce fix-device-size rescue subcommand to fix device size alignment related problems. Especially for people unable to mount their fs with super total bytes mismatch, this tool should make their fs live again. Reported-by: Asif Youssuff Reported-by: Rich Rauenzahn Signed-off-by: Qu Wenruo

[PATCH v2 6/6] btrfs-progs: rescue: Fix zero-log mounted branch

2017-10-17 Thread Qu Wenruo
Seems to be a typo that, in (ret > 0) branch of check_mounted(), zero-log set the return value but doesn't return. Fix it by adding back the missing return. Signed-off-by: Qu Wenruo --- cmds-rescue.c | 1 + 1 file changed, 1 insertion(+) diff --git a/cmds-rescue.c b/cmds-rescue.c index 5b51952

[PATCH v2 5/6] btrfs-progs: test/fsck: Add test case image for --fix-dev-size

2017-10-17 Thread Qu Wenruo
From: Qu Wenruo The image has 2 problems mixed: 1) Too small super total_bytes This super total_bytes is manually modified to create such problem. 2) Unaligned dev item total_bytes This is created by v4.12 kernel, with 128M + 2K device added, and original device removed. Then we can

[PATCH v2 2/6] btrfs-progs: Introduce function to fix super block total bytes

2017-10-17 Thread Qu Wenruo
Recent kernel (starting from v4.6) will refuse to mount if super block total bytes is smaller than all devices' size. This makes end user unable to do anything to their otherwise quite healthy fs. To fix such problem, introduce repair function in btrfs-progs to fix it offline. Reported-by: Asif

[PATCH v2 4/6] btrfs-progs: check: Also check and repair unalignment/mismatch device and super size

2017-10-17 Thread Qu Wenruo
Along with the rescue introduced, also introduce check and repair for them. Unlike normal check functions, some of the check is optional, and even if the image failed to pass optional check, kernel can still run fine. (But may cause noisy kernel warning) So some check, mainly for alignment, will

[PATCH v2 1/6] btrfs-progs: Introduce function to fix unaligned device size

2017-10-17 Thread Qu Wenruo
Recent kernel introduced alignment check for dev item, however older kernel doesn't align device size when adding new device or shrinking existing device. This makes noisy kernel warning every time when any DEV_ITEM get updated. Introduce function to fix device size in btrfs-progs to fix the prob

[PATCH v2 0/6] Btrfs-progs: rescue: To fix device related

2017-10-17 Thread Qu Wenruo
The patchset can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/check_unaligned_dev There are several reports in mail list for btrfs device size related problems. 1) kernel refuse to mount some fs, due to mismatched super total_bytes Unmountable if super total_bytes is

Re: next/master build: 214 builds: 29 failed, 185 passed, 29 errors, 68 warnings (next-20171016)

2017-10-17 Thread Arnd Bergmann
On Tue, Oct 17, 2017 at 1:02 AM, kernelci.org bot wrote: > > next/master build: 214 builds: 29 failed, 185 passed, 29 errors, 68 warnings > (next-20171016) > Full Build Summary: > https://kernelci.org/build/next/branch/master/kernel/next-20171016/ > Tree: next > Branch: master > Git Describe: ne

Re: [PATCH 0/4] btrfs-progs repair support for unaligned/mismatched device sizes

2017-10-17 Thread Qu Wenruo
On 2017年10月17日 14:06, Ursul Hempel Egal wrote: > Hi, > > I have encountered the issue where a btrfs filesystem would not mount due to > mismatch of super_total_bytes with fs_devices total_rw_bytes > > I was able to fix it with the new parameter --fix-dev-size > > Below I have listed the steps