Oh wow, I've been away for a few days and really appreciate the amount of discussion that's arrived :smile_cat: Thanks @mbs-octoml, @zxybazh, @tqchen, @comaniac, @junrushao1994 and @manupa-arm!
Firstly let's address a few specifics which may help narrow the discussion slightly: [quote="junrushao1994, post:10, topic:11372"] Last, there are cases where multiple executors working together. For example, if we want to offload some fragments to TensorRT executor, some to CUDA graph, while keep the rest in the VM, then the Relay function could potentially be partitioned into 3 Relay functions that targets to different executors. With composite target, we are able to attach different executors in the Target object in a more explicit way. [/quote] There's an unfortunate overloading of the terms `Executor` and `Runtime` which is the inherent risk with a diverse heterogeneous compiler :smile_cat:. In this RFC, let's define the `Executor` and `Runtime` as specific to the TVMs `Executor` and `Runtime` rather than the implementation of a `Target`. How a `Target` gets generated and linked is outside the immediate scope of the TVM `Runtime` and `Executor` as they're designed to invoke the generated `Target` code. [quote="mbs-octoml, post:3, topic:11372"] * I think you want your tvmc cmdline flags to be interpreted as rewrites on CompilerConfig in some uniform way, right? [/quote] [quote="junrushao1994, post:10, topic:11372"] The RFC text doesn’t mention how it could improve the UX in TVMC command line. However, I would argue that the UX could be improved simply with target tags. For example, on CUDA GPUs, our target tag system supports creating CUDA targets with a single short string. [/quote] Thanks @mbs-octoml, I missed some Prior Art here! In `tvmc`, we have the concept of a configuration as defined in [Command Line Configuration Files](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0030-tvmc-comand-line-configuration-files.md). `CompilationConfig` would allow this to be a standard way of defining such as configuration with `Target`s within it - this meets the needs of the Target tagging which @junrushao1994 and @tqchen are discussing by instead wrapping them into a `CompilationConfig` that represents the system. Within the [Command Line Configuration Files RFC](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0030-tvmc-comand-line-configuration-files.md), it defines the `<TYPE>` and indicates the use of `--config` for cloud instances. The terminology would shift from a tagged `Target` to a `CompilationConfig` here to represent they exist at two different levels of the hierarchy? [quote="tqchen, post:12, topic:11372"] * I do not think there is a strong contention on C1, the main point is that the target can be recursive. So a target like the follows is totally OK. [/quote] As defined in [Migrating Target Attributes to IRModule](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0029-migrating-to-irmodule-attributes.md), splitting the TVM concepts of `Target`, `Runtime` and `Executor` means we can more clearly see what is most relevant to a specific `Target`. Which means that call-site annotations for `Target` are limited to only options that are relevant to a specific `Target` rather than to an `IRModule`. By virtue of working on this RFCs implementation, although we should still land the implementation agreed in the RFC, it does illustrate how we can better manage the representation of this configuration internally to TVM. One reason not to motivate this as purely a `tvmc` concern is that `tvmc` is the CLI interface to TVM, if a user attempts to use `tvmc` and then moves to a Python script they should not be re-learning the interface to TVM. [quote="tqchen, post:5, topic:11372"] G1: Ability to enable incremental customization (via python API), attach constraints(such as BYOC) and then send back to the build function for further lowering. [/quote] This sounds sensible, and also a feature of `CompilationConfig` is the ability to specify the complete picture of the system which TVM is being built for including all `Target`s which can be used by all `Pass`es. Specific annotations of storage and execution make sense to be defined at call-sites within the IR rather than at the top level with the `IRModule` - what `CompilationConfig` provides is a frame of reference to do those annotations and pick from a variety of `Target`s and `Device`s which the `IRModule` is constrained to. As we continue with [`Target` registered compiler flow customisation](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0010-target-registered-compiler-flow-customisation.md), annotating a call-site with a `Target` will become standardised with the BYOC flow whether partitioned or otherwise to match the expectation you described with partitioned `Target`s. [quote="zxybazh, post:4, topic:11372"] The design of targets list and target host seems really similar to a composite target, can we actually use a single composite target (where each target can be a composite target) with executor and runtime field as compilation config? [/quote] This doesn't rule out the possibility of using a composite `Target` as a `Target` in the `targets` list as we're not redefining how that works here - rather defining a bounding box for system level configuration within TVM. [quote="zxybazh, post:4, topic:11372"] `CheckAndUpdateHostConsistency` is designed to make sure target.host and target host are consistent along the way. If we have a better machanism to do that since we have target host as a field in compilation configuration, we may further reconsider the usage of target inside of compilation flow. [/quote] The end state for this configuration update would be to run a single pass over the `CompilationConfig` early on to ensure the internal state was correct using `CheckAndUpdateHostConsistency` which guarantees that subsequent `Pass`es such as device or memory planning are safe making assumptions about the state of the used `Target`s. Hopefully that clarifies it's less of a replacement, but more of a consolidation of the logic to early in the compilation flow if these checks are still required :smile_cat: We'd still need to have `Target` annotations within the IR and that `Target` will therefore have to be stable during compilation. ## Where we're at Going over this thread a few times, the discussion revolves around: M0. Split the CompilationConfig from Target ``` (CompilationConfig) -> (Target), (Target), (Target) -> (Executor) -> (Runtime) ``` M1. Recursively allowing Target to represent any system ``` (Tagged Target) -> (Target), (Target), (Target) -> (Executor) -> (Runtime) ``` It is my opinion, and the motivation behind this RFC, that better defining the `CompilationConfig` would relieve cognitive load on the user and provide definitions which can be bounded easily. By continuing to use M1 the term `Target` is increasingly overloaded and difficult for both developers of TVM and more importantly users of TVM. This hierarchical terminology has prior art in large scale cloud frameworks, such as Kubernetes which uses different terminology for `Cluster`, `Deployment`, `Service`, `Pod` and `Container` which are all different levels of granularity of computing resources; the decision here is both a UX decision and a practical separation of concerns for both users and developers of Kubernetes. --- [Visit Topic](https://discuss.tvm.apache.org/t/pre-rfc-compilation-configuration-representation/11372/13) to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/245bfd5f645af4809aa78d336679fadf06da4095f7be61d0e044fb31bef619a2).
