gnguralnick commented on code in PR #18961:
URL: https://github.com/apache/tvm/pull/18961#discussion_r3024084532
##########
web/src/webgpu.ts:
##########
@@ -697,6 +697,13 @@ export class WebGPUContext {
bindGroupLayouts: [bindGroupLayout]
});
+ // Pre-allocate typed array views for pod args (reused across dispatches)
+ const maxPodArgs = podArgIndices.length + 1; // +1 for packGridDimX
+ const podArgsArrayBuffer = new ArrayBuffer(maxPodArgs * 4);
+ const i32ViewCached = new Int32Array(podArgsArrayBuffer);
+ const u32ViewCached = new Uint32Array(podArgsArrayBuffer);
+ const f32ViewCached = new Float32Array(podArgsArrayBuffer);
Review Comment:
Good catch — applied this suggestion. Also hoisted `podArgBytes` so the
dispatch path reuses it directly instead of recomputing. See ccb9b77.
##########
web/src/webgpu.ts:
##########
@@ -759,32 +766,28 @@ export class WebGPUContext {
const sizeOfI32 = 4;
const bufBytes = (podArgIndices.length + 1) * sizeOfI32;
const podArgBuffer = this.getUniformFromPool(bufBytes);
- const i32View = new Int32Array(podArgIndices.length + 1);
- const u32View = new Uint32Array(i32View.buffer);
- const f32View = new Float32Array(i32View.buffer);
for (let i = 0; i < podArgIndices.length; ++i) {
const value = args[podArgIndices[i]];
const dtype = finfo.arg_types[podArgIndices[i]];
if (dtype.startsWith("int")) {
- i32View[i] = value;
+ i32ViewCached[i] = value;
} else if (dtype.startsWith("uint")) {
- u32View[i] = value;
+ u32ViewCached[i] = value;
} else if (dtype.startsWith("float")) {
- f32View[i] = value;
+ f32ViewCached[i] = value;
Review Comment:
Agreed this would be a nice follow-up optimization — pre-computing a typed
enum array per shader to replace the `startsWith` checks. I'd prefer to keep
this PR focused on the allocation reduction and tackle the dtype dispatch in a
separate change, since it touches different logic and deserves its own
benchmarking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
