gnguralnick opened a new pull request, #18961: URL: https://github.com/apache/tvm/pull/18961
## Summary - Hoists `Int32Array`/`Uint32Array`/`Float32Array` allocation out of the per-dispatch `submitShader` closure into the per-shader `createShaderFunc` scope, eliminating 3 typed array allocations + 1 `ArrayBuffer` per GPU kernel dispatch. - `podArgIndices.length` is fixed per shader, so the cached views have the correct size for every invocation. Every slot `0..podArgIndices.length` is unconditionally written before `writeBuffer` copies the data out, so no stale values can leak between dispatches. - Builds on top of the batched dispatch architecture from #18871 — the uniform buffer pool already gives each dispatch its own GPU-side buffer, so reusing the CPU-side staging array is safe. ## Motivation In workloads with many small dispatches (e.g. LLM token generation), the per-dispatch typed array allocations become a measurable source of GC pressure. Pre-allocating and reusing the views avoids this overhead. ## Test plan - [ ] Verify `npm run lint` passes in `web/` - [ ] Run WebGPU model inference (e.g. via MLC-LLM web demo) and confirm correct output - [ ] Profile dispatch-heavy workload to confirm reduced allocation rate -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
