On Thu, Apr 21, 2022 at 03:44:02PM +0200, Alexander Leidinger wrote:
| Quoting Mateusz Guzik <[email protected]> (from Thu, 21 Apr 2022
| 14:50:42 +0200):
|
| > On 4/21/22, Alexander Leidinger <[email protected]> wrote:
| >> I tried nocache on a system with a lot of jails which use nullfs,
| >> which showed very slow behavior in the daily periodic runs (12h runs
| >> in the night after boot, 24h or more in subsequent nights). Now the
| >> first nightly run after boot was finished after 4h.
| >>
| >> What is the benefit of not disabling the cache in nullfs? I would
| >> expect zfs (or ufs) to cache the (meta)data anyway.
| >>
| >
| > does the poor performance show up with
| > https://people.freebsd.org/~mjg/vnlru_free_pick.diff ?
|
| I would like to have all the 22 jails run the periodic scripts a
| second night in a row before trying this.
|
| > if the long runs are still there, can you get some profiling from it?
| > sysctl -a before and after would be a start.
| >
| > My guess is that you are the vnode limit and bumping into the 1 second
sleep.
|
| That would explain the behavior I see since I added the last jail
| which seems to have crossed a threshold which triggers the slow
| behavior.
|
| Current status (with the 112 nullfs mounts with nocache):
| kern.maxvnodes: 10485760
| kern.numvnodes: 3791064
| kern.freevnodes: 3613694
| kern.cache.stats.heldvnodes: 151707
| kern.vnodes_created: 260288639
|
| The maxvnodes value is already increased by 10 times compared to the
| default value on this system.
With the patch, you shouldn't mount with nocache! However, you might
want to tune:
vfs.zfs.arc.meta_prune
vfs.zfs.arc.meta_adjust_restarts
Since the code on restart will increment the prune amount by
vfs.zfs.arc.meta_prune and submit that amount to the vnode reclaim
code. So then it will end up reclaiming a lot of vnodes. The
defaults of 10000 * 4096 and submitting it each loop can most of
the cache to be freed. With relative small values of them, then
the cache didn't shrink to much.
Doug A.
