guest2180 commented on issue #19887: URL: https://github.com/apache/tvm/issues/19887#issuecomment-4923343891
@tlopex Update from our side after testing a newer local TVM checkout around commit `ad87f9b`: The unsupported-op / naming mismatch part appears to be improved. In particular, the earlier TensorRT BYOC failures around `permute_dims` / `image.resize2d` reaching BYOC but then failing as unsupported are no longer the first blocking issue in our setup. However, when we switch to the official Relax TensorRT partition flow on a YOLO11 segmentation model, we now hit a different blocker earlier in the BYOC pipeline: Model: https://github.com/ultralytics/assets/releases/download/v8.3.0/yolo11n-seg.onnx Test script: ``` from __future__ import annotations import argparse import traceback from collections import Counter from pathlib import Path import onnx from tvm.relax.backend.contrib.tensorrt import partition_for_tensorrt from tvm.relax.frontend.onnx import from_onnx DEFAULT_ONNX = Path("/home/perception/yolo11n_repro/yolo11n-seg.onnx") DEFAULT_INPUT_NAME = "images" DEFAULT_INPUT_SHAPE = [1, 3, 640, 640] def inspect_onnx_model(onnx_path: Path) -> dict[str, object]: model = onnx.load(str(onnx_path)) return { "model": model, "inputs": [ (value.name, [dim.dim_value or dim.dim_param for dim in value.type.tensor_type.shape.dim]) for value in model.graph.input ], "outputs": [ (value.name, [dim.dim_value or dim.dim_param for dim in value.type.tensor_type.shape.dim]) for value in model.graph.output ], "node_count": len(model.graph.node), "opset": [(item.domain or "ai.onnx", item.version) for item in model.opset_import], "ops": Counter(node.op_type for node in model.graph.node), } def parse_shape(text: str) -> list[int]: return [int(part.strip()) for part in text.split(",") if part.strip()] def main() -> int: parser = argparse.ArgumentParser( description="Reproduce the official Relax TensorRT partition cycle on yolo11n-seg.onnx" ) parser.add_argument("--onnx", type=Path, default=DEFAULT_ONNX) parser.add_argument("--input-name", default=DEFAULT_INPUT_NAME) parser.add_argument("--input-shape", default="1,3,640,640") parser.add_argument("--show-trace", action="store_true") args = parser.parse_args() input_shape = parse_shape(args.input_shape) info = inspect_onnx_model(args.onnx) print(f"onnx={args.onnx}") print(f"inputs={info['inputs']}") print(f"outputs={info['outputs']}") print(f"nodes={info['node_count']}") print(f"opset={info['opset']}") print(f"top_ops={info['ops'].most_common(12)}") print("step=from_onnx") mod = from_onnx(info["model"], shape_dict={args.input_name: input_shape}) print("from_onnx=ok") print("step=partition_for_tensorrt") try: _ = partition_for_tensorrt(mod) except Exception as err: print(f"partition=FAIL type={type(err).__name__}") print(f"error={err}") if args.show_trace: print("traceback_begin") print(traceback.format_exc()) print("traceback_end") return 1 print("partition=OK") return 0 if __name__ == "__main__": raise SystemExit(main()) ``` python repro_yolo11seg_native_cuda.py --opt-level 0 --show-trace What we see: 1. `from_onnx(...)` succeeds 2. native TVM CUDA compilation/execution succeeds 3. but `tvm.relax.backend.contrib.tensorrt.partition_for_tensorrt(mod)` fails in `MergeCompositeFunctions()` with a cyclic-dependency error Error: `InternalError: Check failed: (depgroup != cur_group) is false: A cyclic dependency detected between the groups lv13 and lv77 are in.` So from our current testing, this looks like progress: the BYOC operator-name / runtime-converter mismatch is no longer the main blocker, but larger YOLO-style graphs now run into a Relax partition/merge issue (likely around `concat` / `split`-heavy subgraph formation) before codegen. Separately, the plain detection model also exposes another front-end issue: https://github.com/ultralytics/assets/releases/download/v8.3.0/yolo11n.onnx This one currently does not get past `from_onnx(...)` in our environment. It fails while converting `Range` into Relax `arange`, with a type mismatch on the dtype argument (`Expected DataType but got ir.PrimType`). So at this point we are seeing three stages of progress / remaining issues: - the original BYOC unsupported-op naming problem is improved - `yolo11n-seg.onnx` now reaches official TensorRT partitioning but fails with a cyclic dependency during `MergeCompositeFunctions` - `yolo11n.onnx` still fails earlier in the ONNX importer on `Range` One additional note from our side: for relatively simple unsupported ops such as `resize2d` or some small shape/dim-related glue handling, it is often still practical for downstream users to do small model-specific or project-specific hand-written workarounds. Those cases may not always require a full official TVM-side update immediately. The cyclic-dependency problem feels different. Once the graph reaches `MergeCompositeFunctions()` and fails with a group-dependency cycle on official partitioning, that is beyond what we can reasonably patch around at the model/project level. It likely needs a more formal fix in the Relax partition / merge pipeline itself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
