[quote="comaniac, post:3, topic:11362"] This mechanism is used for the case that a BYOC backend attempts to manage the constant values with certain processes, such as layout transform [/quote]
CUTLASS does seem to support specialized layouts for gemm / conv2d. If we want to make use of them and if the layout transform cannot be expressed by `relay.transform`, then I think we really need to take in Constants. [quote="comaniac, post:3, topic:11362"] IMHO, we could have a specialized mechanism for C codegen to manage constants. For example, we could let C codegen serialize the constants to a separate artifact file, and encapsulate it along with the generated/compiled engines, and load them to the memory at the first execution [/quote] This makes sense, the problem I imagine would be that C-codegen doesn't really have a proper runtime. So I guess all logic for managing constants will be written in a big string and compiled together with the actual offload calls. I don't want to write code like that :sweat_smile: So I think we need to think about switching to Json runtime if a need for a proper handling of Constants ever comes up. The difficulty I see is that unlike C-codegen based BYOC or TensorRT, we have to manage compilation and loading of the compiled lib ourselves (call `nvcc` directly from the Json runtime and use `dlopen` etc to retrieve a handle to a compiled function). Maybe what we need is something like NVRTC for cutlass. --- [Visit Topic](https://discuss.tvm.apache.org/t/byoc-cutlass-dealing-with-constants-in-c-source-gen-based-byoc/11362/4) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/b41dcfe97914ee9495aad954aee3679a98750ffa4633ec35410cef726bde8666).
