On Tue, 2026-10-27 at 15:49 -0700, Matthew Brost wrote: > Backing up a BO costs a GPU blit plus a page copy, and if the BO is > part > of an active working set it is faulted straight back in on the next > exec. > Evicting a BO that was used moments ago is therefore almost always a > loss, yet the shrinker has no notion of recency beyond LRU order. > > Timestamp BOs wherever their LRU position is updated, and read the > stamp > back from whatever owns that position. A BO in a VM's bulk move is > anchored inside its range, so it is aged by the VM-wide stamp taken > in > xe_vm_bump_lru() alongside the bulk move itself. Any other BO carries > its own, stamped there too and on a successful move into TT in > xe_bo_move(), where ttm_resource_alloc() has already placed the new > resource at the LRU tail. > > Add a per-device shrinker_min_age_ms sysfs attribute, defaulting to > CONFIG_DRM_XE_SHRINKER_MIN_AGE_MS (0, disabled). While set, the > shrinker > won't back up BOs younger than the delay. Each LRU priority is > roughly > age ordered, so the gate is applied per priority: xe_shrinker_count() > snoops the first shrinkable BO of each priority and reports nothing > only > if all of them are young, and the backup walk skips to the next > priority > at the first young BO, both as a backstop for the count sample and so > a > hot priority doesn't shield colder BOs in another one. > > Only BOs this shrinker would actually back up have a say. BOs in > XE_PL_SYSTEM, which are not GPU mapped and hence not part of a > working > set, and BOs xe_bo_shrink() rejects outright are never young. > Purgeable > BOs and those advised DONTNEED are neither: they are dropped rather > than > backed up and accounted to the purgeable shrinker, so they are never > young and the count snoop skips their XE_BO_PRIORITY_LOW band > entirely. > > The gate is hard and applies at every reclaim priority, so a working > set > kept younger than the delay is never backed up and can lead to OOM. > This > is deliberate, and why it is off by default.
For completeness, Perhaps add to the commit message that the plan is to follow up with an OOM notifier? I also wonder what happens during the PM notifier preparing for suspend on DGFX, Does that move mark bos accessed, hence possibly blocked by the shrinker at suspend time? > > Assisted-by: LLM > Signed-off-by: Matthew Brost <[email protected]> > --- > drivers/gpu/drm/xe/Kconfig.profile | 15 +++ > drivers/gpu/drm/xe/xe_bo.c | 83 ++++++++++++++ > drivers/gpu/drm/xe/xe_bo.h | 5 + > drivers/gpu/drm/xe/xe_bo_types.h | 8 ++ > drivers/gpu/drm/xe/xe_device_sysfs.c | 35 ++++++ > drivers/gpu/drm/xe/xe_pagefault.c | 2 +- > drivers/gpu/drm/xe/xe_shrinker.c | 164 > +++++++++++++++++++++++++-- > drivers/gpu/drm/xe/xe_shrinker.h | 3 + > drivers/gpu/drm/xe/xe_vm.c | 4 +- > drivers/gpu/drm/xe/xe_vm_types.h | 8 ++ > 10 files changed, 316 insertions(+), 11 deletions(-) > > diff --git a/drivers/gpu/drm/xe/Kconfig.profile > b/drivers/gpu/drm/xe/Kconfig.profile > index 23045ed2aa94..dd3985e502b8 100644 > --- a/drivers/gpu/drm/xe/Kconfig.profile > +++ b/drivers/gpu/drm/xe/Kconfig.profile > @@ -74,6 +74,7 @@ config DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT > to apply to applicable user. For elevated user, all above > MIN > and MAX values will apply when this configuration is > enable to > apply limitation. By default limitation is applied. > + > config DRM_XE_SHRINKER_NON_PURGEABLE_SEEKS > int "Default seeks value of the non-purgeable bo shrinker" > default 8 > @@ -86,3 +87,17 @@ config DRM_XE_SHRINKER_NON_PURGEABLE_SEEKS > reclaim pressure on non-purgeable buffer objects, which > need > GPU idling and writeback to be reclaimed, in favour of > purgeable > ones and other kernel caches. > + > +config DRM_XE_SHRINKER_MIN_AGE_MS > + int "Default minimum BO age before shrinker backup (ms)" > + default 0 > + range 0 1000000 > + help > + Default for the per-device shrinker_min_age_ms sysfs > attribute. The > + shrinker won't back up a BO that was moved into TT, or > bumped on the > + LRU by an exec, less than this many milliseconds ago. BOs > already > + evicted to system memory are not part of a working set and > are > + never protected. This applies at every reclaim priority: a > working > + set kept younger than this is never backed up and can > cause OOM. > + Only set if that risk is preferred over thrashing a hot > working > + set. 0 disables. > diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c > index 42a6aa1a2e4a..a4c0c6284a24 100644 > --- a/drivers/gpu/drm/xe/xe_bo.c > +++ b/drivers/gpu/drm/xe/xe_bo.c > @@ -1357,9 +1357,92 @@ static int xe_bo_move(struct ttm_buffer_object > *ttm_bo, bool evict, > xe_tt_unmap_sg(xe, ttm_bo->ttm); > } > > + /* > + * ttm_resource_alloc() already placed the new resource at > the tail of > + * its LRU, or of @bo's bulk move, so only record when that > happened. > + */ > + if (!ret && ttm_bo->resource && ttm_bo->resource->mem_type > == XE_PL_TT) > + WRITE_ONCE(bo->lru_stamp, jiffies); > + > return ret; > } > > +/** > + * xe_bo_bump_lru_locked() - Bump a BO on the LRU and stamp its last > use > + * @bo: The buffer object, reserved by the caller. > + * > + * Move @bo to the tail of its LRU and record the time for the > shrinker's > + * minimum age gate. Only meaningful for BOs outside a bulk move, > since a > + * BO in one is anchored inside its range and is bumped, and aged, > with it. > + * The caller must hold the device LRU lock. > + */ > +void xe_bo_bump_lru_locked(struct xe_bo *bo) > +{ > + WRITE_ONCE(bo->lru_stamp, jiffies); > + ttm_bo_move_to_lru_tail(&bo->ttm); > +} > + > +/** > + * xe_bo_shrink_backup_candidate() - Whether the shrinker may back > up a BO > + * @bo: The TTM buffer object, reserved by the caller. > + * > + * A BO is a backup candidate if backing it up is the only way to > reclaim it. > + * BOs advised DONTNEED, which sit at XE_BO_PRIORITY_LOW, and > purgeable ones > + * are not: their pages are simply dropped, which is cheap, and they > are > + * accounted to the purgeable shrinker. Neither are the BOs > xe_bo_shrink() > + * hands to xe_bo_shrink_purge() outright: ghost BOs, which aren't > xe BOs, > + * and BOs already freed and waiting to be reaped. Nor are BOs whose > tt > + * isn't TTM_TT_FLAG_EXTERNAL_MAPPABLE, which xe_bo_shrink() > rejects. > + * > + * Return: true if @bo would be backed up by a non-purge shrink > pass. > + */ > +bool xe_bo_shrink_backup_candidate(struct ttm_buffer_object *bo) > +{ > + struct xe_ttm_tt *xe_tt; > + > + if (!xe_bo_is_xe_bo(bo) || !kref_read(&bo->base.refcount) || > !bo->ttm || > + !(bo->ttm->page_flags & TTM_TT_FLAG_EXTERNAL_MAPPABLE) > || > + bo->priority <= XE_BO_PRIORITY_LOW) > + return false; > + > + xe_tt = container_of(bo->ttm, struct xe_ttm_tt, ttm); > + > + return !xe_tt->purgeable; > +} > + > +/** > + * xe_bo_shrink_too_young() - Whether a BO was used too recently to > shrink > + * @bo: The TTM buffer object, reserved by the caller. > + * @min_age: Minimum age in jiffies. 0 disables the check. > + * > + * The age is taken from whatever owns @bo's LRU position: > &xe_vm.lru_stamp > + * if @bo rides along with a VM's bulk move, its own > &xe_bo.lru_stamp if not. > + * > + * BOs that the shrink walk would not back up anyway are never > considered > + * young, see xe_bo_shrink_backup_candidate(). Neither are BOs in > + * XE_PL_SYSTEM: they are not GPU mapped, hence not part of a > working set, > + * and must be moved back to TT, refreshing the stamp, before use. > + * > + * Return: true if @bo was touched less than @min_age jiffies ago. > + */ > +bool xe_bo_shrink_too_young(struct ttm_buffer_object *bo, unsigned > long min_age) > +{ > + struct xe_bo *xe_bo; > + unsigned long stamp; > + > + if (!min_age || bo->resource->mem_type == XE_PL_SYSTEM || > + !xe_bo_shrink_backup_candidate(bo)) > + return false; > + > + xe_bo = ttm_to_xe_bo(bo); > + if (bo->bulk_move && xe_bo->vm) > + stamp = READ_ONCE(xe_bo->vm->lru_stamp); > + else > + stamp = READ_ONCE(xe_bo->lru_stamp); > + > + return stamp && time_before(jiffies, stamp + min_age); > +} > + > static long xe_bo_shrink_purge(struct ttm_operation_ctx *ctx, > struct ttm_buffer_object *bo, > unsigned long *scanned) > diff --git a/drivers/gpu/drm/xe/xe_bo.h b/drivers/gpu/drm/xe/xe_bo.h > index 77b3a02e9b8e..65d4c638e8d8 100644 > --- a/drivers/gpu/drm/xe/xe_bo.h > +++ b/drivers/gpu/drm/xe/xe_bo.h > @@ -623,6 +623,11 @@ struct xe_bo_shrink_flags { > u32 writeback : 1; > }; > > +void xe_bo_bump_lru_locked(struct xe_bo *bo); > + > +bool xe_bo_shrink_backup_candidate(struct ttm_buffer_object *bo); > +bool xe_bo_shrink_too_young(struct ttm_buffer_object *bo, unsigned > long min_age); > + > long xe_bo_shrink(struct ttm_operation_ctx *ctx, struct > ttm_buffer_object *bo, > const struct xe_bo_shrink_flags flags, > unsigned long *scanned); > diff --git a/drivers/gpu/drm/xe/xe_bo_types.h > b/drivers/gpu/drm/xe/xe_bo_types.h > index 0265fb314c77..f63b668c7134 100644 > --- a/drivers/gpu/drm/xe/xe_bo_types.h > +++ b/drivers/gpu/drm/xe/xe_bo_types.h > @@ -49,6 +49,14 @@ struct xe_bo { > u32 flags; > /** @vm: VM this BO is attached to, for extobj this will be > NULL */ > struct xe_vm *vm; > + /** > + * @lru_stamp: jiffies at the last LRU bump, done on a move > into TT > + * and at exec time. Only describes the LRU position of a BO > outside > + * a bulk move; BOs in one are aged by &xe_vm.lru_stamp > instead. > + * Consulted by the shrinker's minimum age gate. Accessed > with > + * READ_ONCE()/WRITE_ONCE(). > + */ > + unsigned long lru_stamp; > /** > * @q: Queue this BO is attached to, mostly for LRC BO, NULL > otherwise. > * Protected by the BO dma_resv: readers must hold it across > both the > diff --git a/drivers/gpu/drm/xe/xe_device_sysfs.c > b/drivers/gpu/drm/xe/xe_device_sysfs.c > index ff5d25dd867c..26b1f71ed522 100644 > --- a/drivers/gpu/drm/xe/xe_device_sysfs.c > +++ b/drivers/gpu/drm/xe/xe_device_sysfs.c > @@ -38,6 +38,12 @@ > * reclaimed by the shrinker. Higher values reduce reclaim pressure > on > * non-purgeable buffer objects. Valid values are 1 to 64. Defaults > to > * CONFIG_DRM_XE_SHRINKER_NON_PURGEABLE_SEEKS. > + * > + * shrinker_min_age_ms - Report/change the minimum time, in ms, > since a BO was > + * last moved to TT or bumped on the LRU by an exec before the > shrinker may > + * back it up. BOs evicted to system memory are never protected. > Defaults to > + * CONFIG_DRM_XE_SHRINKER_MIN_AGE_MS, 0 disables. This applies at > every > + * reclaim priority, so a non-zero value can lead to OOM. > */ > > static ssize_t > @@ -331,8 +337,37 @@ shrinker_non_purgeable_seeks_store(struct device > *dev, > > static DEVICE_ATTR_RW(shrinker_non_purgeable_seeks); > > +static ssize_t > +shrinker_min_age_ms_show(struct device *dev, struct device_attribute > *attr, > + char *buf) > +{ > + struct xe_device *xe = pdev_to_xe_device(to_pci_dev(dev)); > + > + return sysfs_emit(buf, "%u\n", xe_shrinker_min_age_ms(xe- > >mem.shrinker)); > +} > + > +static ssize_t > +shrinker_min_age_ms_store(struct device *dev, struct > device_attribute *attr, > + const char *buff, size_t count) > +{ > + struct xe_device *xe = pdev_to_xe_device(to_pci_dev(dev)); > + unsigned int ms; > + int ret; > + > + ret = kstrtouint(buff, 0, &ms); > + if (ret) > + return ret; > + > + xe_shrinker_set_min_age_ms(xe->mem.shrinker, ms); > + > + return count; > +} > + > +static DEVICE_ATTR_RW(shrinker_min_age_ms); > + > static struct attribute *shrinker_attrs[] = { > &dev_attr_shrinker_non_purgeable_seeks.attr, > + &dev_attr_shrinker_min_age_ms.attr, > NULL > }; > > diff --git a/drivers/gpu/drm/xe/xe_pagefault.c > b/drivers/gpu/drm/xe/xe_pagefault.c > index 6828f81df50e..895866d639f8 100644 > --- a/drivers/gpu/drm/xe/xe_pagefault.c > +++ b/drivers/gpu/drm/xe/xe_pagefault.c > @@ -226,7 +226,7 @@ static int xe_pagefault_handle_vma(struct xe_gt > *gt, struct xe_vma *vma, > > if (bo) { > scoped_guard(spinlock, &vm->xe- > >ttm.lru_lock) > - ttm_bo_move_to_lru_tail(&bo- > >ttm); > + xe_bo_bump_lru_locked(bo); > } > } > xe_validation_ctx_fini(&ctx); > diff --git a/drivers/gpu/drm/xe/xe_shrinker.c > b/drivers/gpu/drm/xe/xe_shrinker.c > index c86455ea08d1..64e5e684f591 100644 > --- a/drivers/gpu/drm/xe/xe_shrinker.c > +++ b/drivers/gpu/drm/xe/xe_shrinker.c > @@ -39,6 +39,8 @@ > * @shrink: Pointer to the mm shrinker handling non-purgeable > (shrinkable) objects. > * @purge_shrink: Pointer to the mm shrinker handling purgeable > objects. > * @pm_worker: Worker to wake up the device if required. > + * @min_age_ms: Minimum BO age before backup, in ms. 0 disables. > Defaults to > + * CONFIG_DRM_XE_SHRINKER_MIN_AGE_MS, overridable through sysfs. > */ > struct xe_shrinker { > struct xe_device *xe; > @@ -50,6 +52,7 @@ struct xe_shrinker { > struct shrinker *shrink; > struct shrinker *purge_shrink; > struct work_struct pm_worker; > + unsigned int min_age_ms; > }; > > static struct xe_shrinker *to_xe_shrinker(struct shrinker *shrink) > @@ -104,6 +107,7 @@ static void xe_shrinker_runtime_pm_put(struct > xe_shrinker *shrinker, bool runtim > static int __xe_shrinker_walk(struct xe_shrinker *shrinker, > struct ttm_operation_ctx *ctx, > const struct xe_bo_shrink_flags flags, > + unsigned long min_age, > unsigned long to_scan, unsigned long > *scanned, > unsigned long *freed) > { > @@ -136,6 +140,16 @@ static int __xe_shrinker_walk(struct xe_shrinker > *shrinker, > if (flags.purge && ttm_bo->priority > > XE_BO_PRIORITY_LOW) > break; > > + /* > + * Each LRU priority is roughly age ordered, > so the > + * rest of this priority is most likely > young too. > + * Move on to the next priority. > + */ > + if (xe_bo_shrink_too_young(ttm_bo, min_age)) > { > + ttm_bo_lru_cursor_skip_priority(&cur > s); > + continue; > + } > + > if (!ttm_bo_shrink_suitable(ttm_bo, ctx)) > continue; > > @@ -171,6 +185,7 @@ static int __xe_shrinker_walk(struct xe_shrinker > *shrinker, > static int xe_shrinker_walk(struct xe_shrinker *shrinker, > struct ttm_operation_ctx *ctx, > const struct xe_bo_shrink_flags flags, > + unsigned long min_age, > unsigned long to_scan, unsigned long > *scanned, > unsigned long *freed) > { > @@ -180,26 +195,126 @@ static int xe_shrinker_walk(struct xe_shrinker > *shrinker, > > swap(no_wait_gpu, ctx->no_wait_gpu); > save_flags.writeback = false; > - ret = __xe_shrinker_walk(shrinker, ctx, save_flags, to_scan, > scanned, > - freed); > + ret = __xe_shrinker_walk(shrinker, ctx, save_flags, min_age, > + to_scan, scanned, freed); > swap(no_wait_gpu, ctx->no_wait_gpu); > if (ret || *scanned >= to_scan) > return ret; > > if (!ctx->no_wait_gpu) { > - ret = __xe_shrinker_walk(shrinker, ctx, save_flags, > to_scan, scanned, > - freed); > + ret = __xe_shrinker_walk(shrinker, ctx, save_flags, > min_age, > + to_scan, scanned, freed); > if (ret || *scanned >= to_scan) > return ret; > } > > if (flags.writeback) > - ret = __xe_shrinker_walk(shrinker, ctx, flags, > to_scan, scanned, > - freed); > + ret = __xe_shrinker_walk(shrinker, ctx, flags, > min_age, > + to_scan, scanned, freed); > > return ret; > } > > +/** > + * DOC: Shrinker minimum age > + * > + * The per-device shrinker_min_age_ms sysfs attribute, defaulting to > + * CONFIG_DRM_XE_SHRINKER_MIN_AGE_MS, protects recently used BOs > from backup. > + * A BO's age is taken from whatever owns its position on the LRU, > and is > + * stamped wherever that position is updated: the VM-wide stamp for > a BO > + * riding along with a VM's bulk move, bumped on exec in > xe_vm_bump_lru(), > + * its own stamp otherwise, bumped there as well and on a move into > TT in > + * xe_bo_move(). BOs in XE_PL_SYSTEM are never young: they are not > GPU > + * mapped, so not part of a working set, and will be stamped again > when > + * moved back to TT for use. > + * > + * The gate is applied per LRU priority, each of which is roughly > age > + * ordered. xe_shrinker_count() snoops the first shrinkable BO of > each > + * priority on each TT-backed LRU. Only if all of them are young > does it stop > + * reporting shrinkable pages; anything else, including reaching no > verdict > + * at all, lets it report. The backup walk in xe_shrinker_scan() > skips to the > + * next priority at the first young BO, as a backstop since the > count snoop > + * is only a sample. A hot priority thus doesn't shield colder BOs > in another > + * priority. BOs that aren't backed up anyway have no say either > way: those > + * in XE_PL_SYSTEM are never young, and purgeable and DONTNEED ones, > which > + * belong to the purgeable shrinker, are ignored by the snoop > entirely. > + * > + * The gate is hard and applies at every reclaim priority. A working > set kept > + * younger than the delay is never backed up, which can lead to OOM. > This is > + * intended for users who prefer that risk over thrashing a hot > working set, > + * hence it is disabled by default. > + */ > +static unsigned long xe_shrinker_min_age(struct xe_shrinker > *shrinker) > +{ > + unsigned int ms = READ_ONCE(shrinker->min_age_ms); > + > + return ms ? msecs_to_jiffies(ms) : 0; > +} > + > +/* Max shrink-unsuitable BOs to skip per LRU before giving up the > snoop. */ > +#define XE_SHRINKER_SNOOP_MAX 16 > + > +/* > + * Whether the first BO of every priority on every TT-backed LRU > that this > + * shrinker would back up is younger than @min_age. BOs are > trylocked through > + * the reserved LRU walker since their tt may be torn down while > still on the > + * LRU. Contended BOs are skipped by the walker, and may be all of > them. If no > + * verdict can be reached, because nothing was inspected or the > snoop budget > + * ran out, report not young and leave it to the scan side check. > + */ > +static bool xe_shrinker_lru_young(struct xe_shrinker *shrinker, > + unsigned long min_age) > +{ > + struct ttm_operation_ctx ctx = { > + .interruptible = false, > + .no_wait_gpu = false, > + }; > + struct xe_device *xe = shrinker->xe; > + unsigned int mem_type; > + bool young = false; > + > + for (mem_type = XE_PL_SYSTEM; mem_type <= XE_PL_TT; > ++mem_type) { > + struct ttm_resource_manager *man = > ttm_manager_type(&xe->ttm, mem_type); > + struct ttm_bo_lru_cursor curs; > + struct ttm_buffer_object *ttm_bo; > + struct ttm_lru_walk_arg arg = { > + .ctx = &ctx, > + .trylock_only = true, > + }; > + unsigned int budget = XE_SHRINKER_SNOOP_MAX; > + > + if (!man || !man->use_tt) > + continue; > + > + ttm_bo_lru_for_each_reserved_guarded(&curs, man, > &arg, ttm_bo) { > + /* > + * Nothing in the DONTNEED band is backed > up, and it > + * is accounted to the purgeable shrinker, > so it has > + * no say here. Skip the whole band in one > go. > + */ > + if (ttm_bo->priority <= XE_BO_PRIORITY_LOW) > { > + ttm_bo_lru_cursor_skip_priority(&cur > s); > + continue; > + } > + > + if (!ttm_bo_shrink_suitable(ttm_bo, &ctx) || > + !xe_bo_shrink_backup_candidate(ttm_bo)) > { > + if (!--budget) > + return false; > + continue; > + } > + > + if (!xe_bo_shrink_too_young(ttm_bo, > min_age)) > + return false; > + > + young = true; > + ttm_bo_lru_cursor_skip_priority(&curs); > + } > + } > + > + return young; > +} > + > static unsigned long > xe_shrinker_count_purgeable(struct shrinker *shrink, struct > shrink_control *sc) > { > @@ -239,7 +354,7 @@ static unsigned long > xe_shrinker_count(struct shrinker *shrink, struct shrink_control > *sc) > { > struct xe_shrinker *shrinker = to_xe_shrinker(shrink); > - unsigned long num_pages; > + unsigned long num_pages, min_age; > bool can_backup = !!(sc->gfp_mask & __GFP_FS); > > if (!can_backup) > @@ -263,6 +378,10 @@ xe_shrinker_count(struct shrinker *shrink, > struct shrink_control *sc) > > write_unlock(&shrinker->lock); > > + min_age = xe_shrinker_min_age(shrinker); > + if (num_pages && min_age && xe_shrinker_lru_young(shrinker, > min_age)) > + return SHRINK_EMPTY; > + > return num_pages ? num_pages : SHRINK_EMPTY; > } > > @@ -297,7 +416,8 @@ static unsigned long > xe_shrinker_scan_purgeable(struct shrinker *shrink, > .purge = true, > }; > > - xe_shrinker_walk(shrinker, &ctx, shrink_flags, nr_to_scan, > &nr_scanned, &freed); > + xe_shrinker_walk(shrinker, &ctx, shrink_flags, 0, > nr_to_scan, &nr_scanned, > + &freed); > > sc->nr_scanned = nr_scanned; > > @@ -322,7 +442,9 @@ static unsigned long xe_shrinker_scan(struct > shrinker *shrink, struct shrink_con > /* Might need runtime PM to access backed-up or CCS- > compressed data. */ > runtime_pm = xe_shrinker_runtime_pm_get(shrinker); > > - xe_shrinker_walk(shrinker, &ctx, shrink_flags, nr_to_scan, > &nr_scanned, &freed); > + xe_shrinker_walk(shrinker, &ctx, shrink_flags, > + xe_shrinker_min_age(shrinker), nr_to_scan, > &nr_scanned, > + &freed); > > sc->nr_scanned = nr_scanned; > xe_shrinker_runtime_pm_put(shrinker, runtime_pm); > @@ -387,6 +509,29 @@ int xe_shrinker_set_non_purgeable_seeks(struct > xe_shrinker *shrinker, > return 0; > } > > +/** > + * xe_shrinker_min_age_ms() - Get the shrinker minimum BO age > + * @shrinker: Pointer to the struct xe_shrinker. > + * > + * Return: Minimum age, in ms, a BO must reach before it may be > backed up. > + */ > +unsigned int xe_shrinker_min_age_ms(struct xe_shrinker *shrinker) > +{ > + return READ_ONCE(shrinker->min_age_ms); > +} > + > +/** > + * xe_shrinker_set_min_age_ms() - Set the shrinker minimum BO age > + * @shrinker: Pointer to the struct xe_shrinker. > + * @ms: Minimum age, in ms. 0 disables the gate. > + * > + * See DOC: Shrinker minimum age. A non-zero value can lead to OOM. > + */ > +void xe_shrinker_set_min_age_ms(struct xe_shrinker *shrinker, > unsigned int ms) > +{ > + WRITE_ONCE(shrinker->min_age_ms, ms); > +} > + > /** > * xe_shrinker_create() - Create an xe per-device shrinker > * @xe: Pointer to the xe device. > @@ -415,6 +560,7 @@ int xe_shrinker_create(struct xe_device *xe) > > INIT_WORK(&shrinker->pm_worker, xe_shrinker_pm); > shrinker->xe = xe; > + shrinker->min_age_ms = CONFIG_DRM_XE_SHRINKER_MIN_AGE_MS; > rwlock_init(&shrinker->lock); > > shrinker->purge_shrink->count_objects = > xe_shrinker_count_purgeable; > diff --git a/drivers/gpu/drm/xe/xe_shrinker.h > b/drivers/gpu/drm/xe/xe_shrinker.h > index 4b3c73689619..586d7772bd58 100644 > --- a/drivers/gpu/drm/xe/xe_shrinker.h > +++ b/drivers/gpu/drm/xe/xe_shrinker.h > @@ -18,4 +18,7 @@ int xe_shrinker_non_purgeable_seeks(struct > xe_shrinker *shrinker); > int xe_shrinker_set_non_purgeable_seeks(struct xe_shrinker > *shrinker, > unsigned int seeks); > > +unsigned int xe_shrinker_min_age_ms(struct xe_shrinker *shrinker); > +void xe_shrinker_set_min_age_ms(struct xe_shrinker *shrinker, > unsigned int ms); > + > #endif > diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c > index 86c3b4fd232f..583c55b7f97d 100644 > --- a/drivers/gpu/drm/xe/xe_vm.c > +++ b/drivers/gpu/drm/xe/xe_vm.c > @@ -463,13 +463,15 @@ void xe_vm_bump_lru(struct xe_vm *vm, struct > drm_exec *exec) > spinlock_t *lru_lock = &vm->xe->ttm.lru_lock; > struct drm_gem_object *obj; > > + WRITE_ONCE(vm->lru_stamp, jiffies); > + > spin_lock(lru_lock); > ttm_lru_bulk_move_tail(&vm->lru_bulk_move); > drm_exec_for_each_locked_object(exec, obj) { > if (!drm_gpuvm_is_extobj(&vm->gpuvm, obj)) > continue; > > - ttm_bo_move_to_lru_tail(&gem_to_xe_bo(obj)->ttm); > + xe_bo_bump_lru_locked(gem_to_xe_bo(obj)); > cond_resched_lock(lru_lock); > } > spin_unlock(lru_lock); > diff --git a/drivers/gpu/drm/xe/xe_vm_types.h > b/drivers/gpu/drm/xe/xe_vm_types.h > index fb9305124679..046505a285e9 100644 > --- a/drivers/gpu/drm/xe/xe_vm_types.h > +++ b/drivers/gpu/drm/xe/xe_vm_types.h > @@ -267,6 +267,14 @@ struct xe_vm { > /** @lru_bulk_move: Bulk LRU move list for this VM's BOs */ > struct ttm_lru_bulk_move lru_bulk_move; > > + /** > + * @lru_stamp: jiffies at the last xe_vm_bump_lru(), which > is where > + * @lru_bulk_move is moved to the LRU tail. Ages every BO in > that > + * bulk move for the shrinker's minimum age gate. Accessed > with > + * READ_ONCE()/WRITE_ONCE(). > + */ > + unsigned long lru_stamp; Isn't the bo resv / vm resv always held then this is accessed? Thanks, Thomas > + > u64 size; > > struct xe_pt *pt_root[XE_MAX_TILES_PER_DEVICE];
