Re: User experience issue on btrfs

2020-07-04 Thread Chris Murphy
On Sat, Jul 4, 2020 at 3:29 PM Scott Schmit wrote: > > On Fri, Jul 03, 2020 at 10:37:43AM -0600, Chris Murphy wrote: > > On Thu, Jul 2, 2020 at 10:29 PM Scott Schmit wrote: > > > > > > On Sun, Jun 28, 2020 at 03:40:11PM -0600, Chris Murphy wrote: > > > > Databases and VM images are things btrfs i

Re: User experience issue on btrfs

2020-07-04 Thread Samuel Sieb
On 7/3/20 9:37 AM, Chris Murphy wrote: To give the nodatacow suggestion a try: ## shutdown the database # mkdir /var/lib/mysql2 # chattr +C /var/lib/mysql2 # cp /var/lib/mysql/* /var/lib/mysql2/ # rm /var/lib/mysql/ # mv /var/lib/mysql2/ /var/lib/mysql/ ## resume operation To avoid possible iss

Re: User experience issue on btrfs

2020-07-04 Thread Scott Schmit
On Fri, Jul 03, 2020 at 10:37:43AM -0600, Chris Murphy wrote: > On Thu, Jul 2, 2020 at 10:29 PM Scott Schmit wrote: > > > > On Sun, Jun 28, 2020 at 03:40:11PM -0600, Chris Murphy wrote: > > > Databases and VM images are things btrfs is bad at out of the box. > > > Most of this has to do with fsync

Re: User experience issue on btrfs

2020-07-04 Thread Chris Murphy
On Sat, Jul 4, 2020 at 12:53 PM Randy Barlow wrote: > > On 7/3/20 12:41 PM, Chris Murphy wrote: > > # rm -rf/var/lib/mysql/ > > > >> # mv/var/lib/mysql2/ /var/lib/mysql/ > >> ## resume operation > > BTW this should be proofread/sanity checked, especially because > > there's an rm command (that wil

Re: User experience issue on btrfs

2020-07-04 Thread Randy Barlow
On 7/3/20 12:41 PM, Chris Murphy wrote: # rm -rf/var/lib/mysql/ # mv/var/lib/mysql2/ /var/lib/mysql/ ## resume operation BTW this should be proofread/sanity checked, especially because there's an rm command (that will fail in the original). You also might need a restorecon after this, since

Re: User experience issue on btrfs

2020-07-03 Thread Chris Murphy
On Fri, Jul 3, 2020 at 10:37 AM Chris Murphy wrote: > To give the nodatacow suggestion a try: > ## shutdown the database > # mkdir /var/lib/mysql2 > # chattr +C /var/lib/mysql2 > # cp /var/lib/mysql/* /var/lib/mysql2/ > # rm /var/lib/mysql/ # rm -rf /var/lib/mysql/ > # mv /var/lib/mysql2/ /var/

Re: User experience issue on btrfs

2020-07-03 Thread Chris Murphy
On Thu, Jul 2, 2020 at 10:29 PM Scott Schmit wrote: > > On Sun, Jun 28, 2020 at 03:40:11PM -0600, Chris Murphy wrote: > > Databases and VM images are things btrfs is bad at out of the box. > > Most of this has to do with fsync dependency of other file systems. > > Btrfs is equipped to deal with an

Re: User experience issue on btrfs

2020-07-02 Thread Scott Schmit
On Sun, Jun 28, 2020 at 03:40:11PM -0600, Chris Murphy wrote: > Databases and VM images are things btrfs is bad at out of the box. > Most of this has to do with fsync dependency of other file systems. > Btrfs is equipped to deal with an fsync heavy world out of the box, > using treelog enabled by d

Re: User experience issue on btrfs

2020-06-30 Thread Robert Marcano via devel
On 6/29/20 2:26 AM, John M. Harris Jr wrote: On Sunday, June 28, 2020 5:37:08 PM MST Chris Adams wrote: Once upon a time, John M. Harris Jr said: XFS proved to be troublesome, and still is up to the latest of RHEL7. It's not uncommon to have to run xfs_repair on smaller XFS partitions, espec

Re: User experience issue on btrfs

2020-06-29 Thread Chris Murphy
On Mon, Jun 29, 2020 at 11:56 AM Tom Seewald wrote: > > > I don't want to give the impression that nodatacow (chattr +C) is what > > apps should be doing "to be fast on btrfs". It might be that they can > > reduce their fsync footprint. Or the problem might be lock contention > > related, and an e

Re: User experience issue on btrfs

2020-06-29 Thread Tom Seewald
> I don't want to give the impression that nodatacow (chattr +C) is what > apps should be doing "to be fast on btrfs". It might be that they can > reduce their fsync footprint. Or the problem might be lock contention > related, and an easy optimization for a heavy metadata writing apps > would be f

Re: User experience issue on btrfs

2020-06-29 Thread Chris Murphy
On Sun, Jun 28, 2020 at 7:09 PM Michael Catanzaro wrote: > I'd like to propose a few guidelines: > > 1. If btrfs causes noticeable performance issues for users, that's not > OK. It's understood and expected that it might be slower at many > workloads, but if the difference is large enough that us

Re: User experience issue on btrfs

2020-06-29 Thread Eric Sandeen
On 6/29/20 3:19 AM, John M. Harris Jr wrote: > On Monday, June 29, 2020 1:09:16 AM MST Markus Larsson wrote: >> On 29 June 2020 08:26:21 CEST, "John M. Harris Jr" >> wrote: >>> On Sunday, June 28, 2020 5:37:08 PM MST Chris Adams wrote: >>> Once upon a time, John M. Harris Jr said:

Re: User experience issue on btrfs

2020-06-29 Thread Florian Weimer
* Chris Adams: > Once upon a time, John M. Harris Jr said: >> XFS proved to be troublesome, and still is up to the latest of RHEL7. It's >> not >> uncommon to have to run xfs_repair on smaller XFS partitions, especially / >> boot. I'm not sure if btrfs has the same issue there? > > [citation ne

Re: User experience issue on btrfs

2020-06-29 Thread John M. Harris Jr
On Monday, June 29, 2020 1:09:16 AM MST Markus Larsson wrote: > On 29 June 2020 08:26:21 CEST, "John M. Harris Jr" > wrote: > >On Sunday, June 28, 2020 5:37:08 PM MST Chris Adams wrote: > > > >> Once upon a time, John M. Harris Jr said: > >> > >> > >> > XFS proved to be troublesome, and still i

Re: User experience issue on btrfs

2020-06-29 Thread Markus Larsson
On 29 June 2020 08:26:21 CEST, "John M. Harris Jr" wrote: >On Sunday, June 28, 2020 5:37:08 PM MST Chris Adams wrote: >> Once upon a time, John M. Harris Jr said: >> >> > XFS proved to be troublesome, and still is up to the latest of RHEL7. It's >> > not uncommon to have to run xfs_repair on

Re: User experience issue on btrfs

2020-06-28 Thread John M. Harris Jr
On Sunday, June 28, 2020 5:37:08 PM MST Chris Adams wrote: > Once upon a time, John M. Harris Jr said: > > > XFS proved to be troublesome, and still is up to the latest of RHEL7. It's > > not uncommon to have to run xfs_repair on smaller XFS partitions, > > especially / boot. I'm not sure if btr

Re: User experience issue on btrfs

2020-06-28 Thread John M. Harris Jr
On Sunday, June 28, 2020 6:06:06 PM MST Michael Catanzaro wrote: > 3. Users should not be expected to customize anything or use the > command line, ever, period. So for the purposes of figuring out what's > causing this performance issue, it sounds very useful to test different > mount options. Bu

Re: User experience issue on btrfs

2020-06-28 Thread Tom Seewald
> I'd like to propose a few guidelines: > > 1. If btrfs causes noticeable performance issues for users, that's not > OK. It's understood and expected that it might be slower at many > workloads, but if the difference is large enough that users notice a > significant regression in desktop respon

Re: User experience issue on btrfs

2020-06-28 Thread Neal Gompa
On Sun, Jun 28, 2020 at 9:07 PM Michael Catanzaro wrote: > > On Sun, Jun 28, 2020 at 3:40 pm, Chris Murphy > wrote: > > And yeah, how would anyone know all of this? And is it an opportunity > > for docs (probably) or desktop integration? Detect this workload or > > ask the user? I'm not sure. > >

Re: User experience issue on btrfs

2020-06-28 Thread Michael Catanzaro
On Sun, Jun 28, 2020 at 3:40 pm, Chris Murphy wrote: And yeah, how would anyone know all of this? And is it an opportunity for docs (probably) or desktop integration? Detect this workload or ask the user? I'm not sure. I'd like to propose a few guidelines: 1. If btrfs causes noticeable perfor

Re: User experience issue on btrfs

2020-06-28 Thread Chris Adams
Once upon a time, John M. Harris Jr said: > XFS proved to be troublesome, and still is up to the latest of RHEL7. It's > not > uncommon to have to run xfs_repair on smaller XFS partitions, especially / > boot. I'm not sure if btrfs has the same issue there? [citation needed] I haven't run xfs_

Re: User experience issue on btrfs

2020-06-28 Thread Chris Murphy
On Sun, Jun 28, 2020 at 5:42 PM John M. Harris Jr wrote: > > On Sunday, June 28, 2020 6:45:24 PM MST Alexandre de Farias wrote: > > *snip* > > > > At this point, I'm fine with what I have and BTRFS usage would be > > strictly for testing. Also, is there any reason as to why RHEL went > > with XFS

Re: User experience issue on btrfs

2020-06-28 Thread Chris Murphy
On Sun, Jun 28, 2020 at 4:54 PM Alexandre de Farias wrote: > > On Sun, 2020-06-28 at 15:40 -0600, Chris Murphy wrote: > > On Sun, Jun 28, 2020 at 9:04 AM wrote: > > > > > I'm willing to perform further testing. There shouldn't be anything > > > very special about my workload. I was working mostly

Re: User experience issue on btrfs

2020-06-28 Thread John M. Harris Jr
On Sunday, June 28, 2020 6:45:24 PM MST Alexandre de Farias wrote: > *snip* > > At this point, I'm fine with what I have and BTRFS usage would be > strictly for testing. Also, is there any reason as to why RHEL went > with XFS as a default and Fedora stayed with ext4? I mean, if it was a > consciou

Re: User experience issue on btrfs

2020-06-28 Thread Alexandre de Farias
On Sun, 2020-06-28 at 15:40 -0600, Chris Murphy wrote: > On Sun, Jun 28, 2020 at 9:04 AM wrote: > > > I'm willing to perform further testing. There shouldn't be anything > > very special about my workload. I was working mostly with NodeJS 12 > > and React Native. VS Code (I should mention I make

Re: User experience issue on btrfs

2020-06-28 Thread Chris Murphy
On Sun, Jun 28, 2020 at 9:04 AM wrote: > I'm willing to perform further testing. There shouldn't be anything very > special about my workload. I was working mostly with NodeJS 12 and React > Native. VS Code (I should mention I make use of TabNine, which can be a huge > drain on system resource

Re: User experience issue on btrfs

2020-06-28 Thread Chris Murphy
On Sun, Jun 28, 2020 at 9:04 AM wrote: > > >(if Alexandre is willing to do some further > >testing to put numbers on the problem)? > > I'm willing to perform further testing. There shouldn't be anything very > special about my workload. I was working mostly with NodeJS 12 and React > Native. VS

Re: User experience issue on btrfs

2020-06-28 Thread Alexandre de Farias
On Sun, 2020-06-28 at 09:36 -0600, Chris Murphy wrote: > On Sun, Jun 28, 2020 at 9:04 AM wrote: > Do you recall what mount options you were using? In particular this > sounds like a side effect of using the discard mount option with > certain SSDs. This is a significant motivation of the new > dis

Re: User experience issue on btrfs

2020-06-28 Thread Chris Murphy
On Sun, Jun 28, 2020 at 9:04 AM wrote: > > >(if Alexandre is willing to do some further > >testing to put numbers on the problem)? > > I'm willing to perform further testing. Do you recall what mount options you were using? In particular this sounds like a side effect of using the discard mount o

Re: User experience issue on btrfs

2020-06-28 Thread Chris Murphy
On Sun, Jun 28, 2020 at 8:11 AM Fabio Valentini wrote: > > From what I remember about Android Studio / Simulator, it uses qemu and disk > images under the hood. Setting the nodatacow attribute (chattr +C, I think?) > on VM images should improve performance by a fair bit. chattr +C (nodatacow) i

Re: User experience issue on btrfs

2020-06-28 Thread alexandrebfarias
>(if Alexandre is willing to do some further >testing to put numbers on the problem)? I'm willing to perform further testing. There shouldn't be anything very special about my workload. I was working mostly with NodeJS 12 and React Native. VS Code (I should mention I make use of TabNine, which can

Re: User experience issue on btrfs

2020-06-28 Thread Tomasz Torcz
On Sun, Jun 28, 2020 at 09:20:32AM -0500, Michael Catanzaro wrote: > On Sun, Jun 28, 2020 at 4:11 pm, Fabio Valentini > wrote: > > Maybe libvirt / gnome-boxes / virt-manager should do that by default if > > it detects that the backing storage for an allocated VM image is on > > btrfs (if it doesn'

Re: User experience issue on btrfs

2020-06-28 Thread Michael Catanzaro
On Sun, Jun 28, 2020 at 4:11 pm, Fabio Valentini wrote: Maybe libvirt / gnome-boxes / virt-manager should do that by default if it detects that the backing storage for an allocated VM image is on btrfs (if it doesn't do that already)? Yeah that's the plan: https://gitlab.gnome.org/GNOME/gnom

Re: User experience issue on btrfs

2020-06-28 Thread Fabio Valentini
On Sun, Jun 28, 2020, 16:04 Neal Gompa wrote: > On Sun, Jun 28, 2020 at 9:57 AM Michael Catanzaro > wrote: > > > > (fixing the subject line to not mention nano) > > > > On Sun, Jun 28, 2020 at 5:16 am, alexandrebfar...@gmail.com wrote: > > > Don't expect much love on this, since my opinion has b

Re: User experience issue on btrfs

2020-06-28 Thread Neal Gompa
On Sun, Jun 28, 2020 at 9:57 AM Michael Catanzaro wrote: > > (fixing the subject line to not mention nano) > > On Sun, Jun 28, 2020 at 5:16 am, alexandrebfar...@gmail.com wrote: > > Don't expect much love on this, since my opinion has been downvoted > > on reddit by many of those who don't want to

User experience issue on btrfs

2020-06-28 Thread Michael Catanzaro
(fixing the subject line to not mention nano) On Sun, Jun 28, 2020 at 5:16 am, alexandrebfar...@gmail.com wrote: Don't expect much love on this, since my opinion has been downvoted on reddit by many of those who don't want to hear bad news about btrfs. And no, I don't have any benchmarks and di