@junrushao1994 , For A.)
The reason is constants represent something intimate to the compute and requiring space. Moreover, in the scheduling passes where we want to do slicing in loops where the weights get sliced and need to undergo transformations (e.g. compression), it will need to keep on reading attributes and writing attributes throughout such passes. That seems like we are abusing attributes for something that should have been able to express in TIR. I guess the main difference is unlike the existing passes, the constant will be treated as a primary entity in TIR rather than being treated as another input. Therefore, I would argue against putting constants as attributes of IRModule. For B.) I believe this support is already implemented in the PR : https://github.com/apache/tvm/pull/8472 https://github.com/apache/tvm/blob/7458e2ddd22e14ba9f58f3c8dd3b5842b03a9aab/tests/python/unittest/test_tvmscript_roundtrip.py#L2895-L2911 For C.) I feel if the constants do not get embedded in the compilation artifact of TVM, we should avoid 'linking' the constant into the device PrimFunc, however, the host PrimFunc could codegen it in an appropriate section and pass a pointer to device PrimFunc if the non-volatile memory is accessible by the device. Otherwise, the host PrimFunc would need a cache_read stage to copy it to local storage before passing it onto the device PrimFunc. For D.) By default, the data could be generated into .rodata section, however USMP could allow user to place it in a different section that goes to different memory in the case where SoC have multiple non-volatile memories. See U3 here : https://github.com/apache/tvm-rfcs/blob/c520a3912279bcd3c432dafd4070661f614882cf/rfcs/0009_Unified_Static_Memory_Planning.md For E.) As I replied the original discussion, I think after the initial stage we could revisit to insert a new pass to do this optimization. I believe I have answered all your questions. Let me know what you think. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/tvm-rfcs/pull/22#issuecomment-921211511
