Hi, v4 of the TLB invalidation ack stall fix for ARL, rebased on today's drm-tip. Tracked in:
https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8678 Recap: on the standalone media GT of MTL/ARL the CPU reads stale cache lines for data the GuC has already written. The visible symptom is TLB invalidation acks appearing to stall for a near-constant ~2.3s. i915 works around this as Wa_22016122933; xe never inherited it. Patch 3 implements it, scoped like i915. In v3 I asked two open questions in this cover letter. Since there was no preference expressed, v4 makes the conservative choice in both cases and documents the reasoning in the commit message, so the series is not blocked on a decision: 1. CPU mapping: keeping XE_BO_FLAG_NEEDS_UC (uncached on both sides). It is the tested configuration and no throughput difference against the CPU-WC variant was measurable. Matching i915's exact CPU-WC + GGTT-UC combination needs either a new BO flag or decoupling the GGTT cache-mode selection from XE_BO_FLAG_NEEDS_UC; happy to add that plumbing if parity is preferred. 2. Scope: covering the GuC-shared allocations (CTBs, log, ADS, SLPC, engine activity), which is where the failures were observed. i915 additionally covers media-GT LRC/ring state; that can be a follow-up if wanted. Fixes:/Cc: stable are still left out, since MTL/ARL is require_force_probe in xe. Also happy to add them. Validation of patch 3 is now six weeks on two ARL machines (7d51 and 7dd1), across kernels 7.1.6, 7.1.8 and 7.2, with over 10M TLB invalidations processed and zero ack stalls. Before the fix both machines reproduced 20-60 stalls/day, every day, on two GuC firmware versions. The 7dd1 machine, which could not survive a day of media workloads on xe without a platform freeze, has been running xe full time since 11 August with zero incidents. Patches 1-2 are the diagnostics that made the investigation possible and are unchanged since v3. checkpatch is clean on the series. v3 -> v4: - Rebased on drm-tip. - Patch 3: resolved both open questions in the commit message instead of leaving them for discussion; refreshed validation data. Thanks, Tales Tales A. Mendonça (3): drm/xe: Capture devcoredump on TLB invalidation timeout drm/xe: Log when a timed out TLB invalidation ack finally arrives drm/xe: Implement Wa_22016122933 drivers/gpu/drm/xe/xe_devcoredump.c | 46 +++++++++++---------- drivers/gpu/drm/xe/xe_devcoredump.h | 15 +++++-- drivers/gpu/drm/xe/xe_guc.c | 16 +++++++ drivers/gpu/drm/xe/xe_guc.h | 2 + drivers/gpu/drm/xe/xe_guc_ads.c | 3 +- drivers/gpu/drm/xe/xe_guc_ct.c | 6 ++- drivers/gpu/drm/xe/xe_guc_engine_activity.c | 6 ++- drivers/gpu/drm/xe/xe_guc_log.c | 7 +++- drivers/gpu/drm/xe/xe_guc_pc.c | 3 +- drivers/gpu/drm/xe/xe_tlb_inval.c | 39 +++++++++++++++++ drivers/gpu/drm/xe/xe_tlb_inval_types.h | 17 ++++++++ drivers/gpu/drm/xe/xe_wa_oob.rules | 1 + 12 files changed, 128 insertions(+), 33 deletions(-) -- 2.55.0
