> On 17 May 2016, at 11:09, Steven Hartland <kill...@multiplay.co.uk> wrote: > >> I understand that, but I don’t think it’s a good that ZFS depends blindly on >> a driver feature such >> as that. Of course, it’s great to exploit it. >> >> I have also noticed that ZFS has a good throttling mechanism for write >> operations. A similar >> mechanism should throttle trim requests so that trim requests don’t clog the >> whole system. > It already does.
I see that there’s a limit to the number of active TRIM requests, but not an explicit delay such as it’s applied to write requests. So, even with a single maximum active TRIM request, it seems that TRIM wins. >> >>> I’d be extremely hesitant to tossing away TRIMs. They are actually quite >>> important for >>> the FTL in the drive’s firmware to proper manage the NAND wear. More free >>> space always >>> reduces write amplification. It tends to go as 1 / freespace, so simply >>> dropping them on >>> the floor should be done with great reluctance. >> I understand. I was wondering about choosing the lesser between two evils. A >> 15 minute >> I/O stall (I deleted 2 TB of data, that’s a lot, but not so unrealistic) or >> settings trims aside >> during the peak activity. >> >> I see that I was wrong on that, as a throttling mechanism would be more than >> enough probably, >> unless the system is close to running out of space. >> >> I’ve filed a bug report anyway. And copying to -stable. >> >> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209571 >> > TBH it sounds like you may have badly behaved HW, we've used ZFS + TRIM and > for years on large production boxes and while we're seen slow down we haven't > experienced the total lockups you're describing. I am using ZFS+TRIM on SATA SSD disks for a very long time. Actually, a single SSD I tried at home can TRIM at around 2 GB/s. Warner Losh told me that the nvd driver is not currently coalescing the TRIMs, which is a disadvantage compared to the ada driver, which does. > The graphs on you're ticket seem to indicate peak throughput of 250MB/s which > is extremely slow for standard SSD's let alone NVMe ones and when you add in > the fact you have 10 well it seems like something is VERY wrong. The pool is a raidz2 vdev with 10 P3500 NVMe disks. That graph is the throughput of just one of the disks (the other 9 graphs are identical). Bonnie++ reports around 1.7 Gigabytes/s writing “intelligently”, 1 GB/s “rewriting” and almost 2 GB/s “reading intelligently” which, as far as I know, is more or less reasonable. The really slow part are the TRIM requests. When destroying the files (it’s four concurrent bonnie++ tasks writing a total of 2 Terabytes) > I just did a quick test on our DB box here creating and then deleting a 2G > file as you describe and I couldn't even spot the delete in the general noise > it was so quick to process and that's a 6 disk machine with P3700’s. Totalling 2 TB? In my case it was FOUR files, 512 GB each. I’m realy puzzled, Borja. _______________________________________________ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
