I might give this a shot. What is the best way tell if geli ought to use direct dispatch? Is there a generic "are crypto instructions available" macro that would cover aesni as well as other platform-specific instructions?
-Alan On Sat, Jul 4, 2020, 2:55 PM Paweł Jakub Dawidek <pa...@dawidek.net> wrote: > Direct dispatch would be great for geli, especially that geli can use own > (multiple) threads when necessary (eg. using crypto cards). With AES-NI you > could go straight to the disk. > > -- > Paweł Jakub Dawidek > > > > > On Jul 3, 2020, at 13:22, Alan Somers <asom...@freebsd.org> wrote: > > > > I don't. What I meant was that a single thread (geom) is limiting the > > performance of the system overall. I'm certain, based on top, gstat, and > > zpool iostat, that geom is the limiting factor on this system. > > -Alan > > > >> On Fri, Jul 3, 2020 at 2:18 PM Paweł Jakub Dawidek <pa...@dawidek.net> > >> wrote: > >> > >> Hi Alan, > >> > >> why do you think it will hurt single-threaded performance? > >> > >> -- > >> Paweł Jakub Dawidek > >> > >> > >> > >>>> On Jul 3, 2020, at 12:30, Alan Somers <asom...@freebsd.org> wrote: > >>> > >>> I'm using geli, gmultipath, and ZFS on a large system, with hundreds > of > >>> drives. What I'm seeing is that under at least some workloads, the > >> overall > >>> performance is limited by the single geom kernel process. procstat and > >>> kgdb aren't much help in telling exactly why this process is using so > >> much > >>> CPU, but it certainly must be related to the fact that over 15,000 IOPs > >> are > >>> going through that thread. What can I do to improve this situation? > >> Would > >>> it make sense to enable direct dispatch for geli? That would hurt > >>> single-threaded performance, but probably improve performance for > highly > >>> multithreaded workloads like mine. > >>> > >>> Example top output: > >>> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU > COMMAND > >>> 13 root -8 - 0B 96K CPU46 46 82.7H 70.54% > >>> geom{g_down} > >>> 13 root -8 - 0B 96K - 9 35.5H 25.32% > >>> geom{g_up} > >>> > >>> -Alan > >>> _______________________________________________ > >>> freebsd-geom@freebsd.org mailing list > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-geom > >>> To unsubscribe, send any mail to "freebsd-geom-unsubscr...@freebsd.org > " > >> > >> > > _______________________________________________ > > freebsd-geom@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-geom > > To unsubscribe, send any mail to "freebsd-geom-unsubscr...@freebsd.org" > > _______________________________________________ freebsd-geom@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-geom To unsubscribe, send any mail to "freebsd-geom-unsubscr...@freebsd.org"