Nanmur opened a new issue, #19965:
URL: https://github.com/apache/tvm/issues/19965
## Summary
During compatibility testing, two industrial-style ONNX graphs consistently
fail at the TVM Relax ONNX importer stage, before tuning can start:
1. `PP-OCRv6_tiny.onnx`, exported from a PaddleOCR-style model, fails around
`Squeeze -> Transpose`.
2. `FasterRCNN-12.onnx`, a detection graph with dynamic post-processing,
fails around shape `Gather` and then dynamic `TopK`.
I would like to ask whether these are expected importer limitations in the
current Relax ONNX frontend, and whether the preprocessing workarounds
described below are recommended, or if there is a better official way to handle
these graphs.
## Environment
```text
TVM version: 0.24.0
Python: 3.11.15
ONNX: 1.21.0
OS: Linux
Frontend used: tvm.relax.frontend.onnx.from_onnx
```
## Case 1: PP-OCRv6_tiny Squeeze axes are lost before Transpose
The model is `PP-OCRv6_tiny.onnx`, opset 11. The failing pattern in the ONNX
graph is:
```text
Squeeze.0
inputs = ['p2o.pd_op.pool2d.0.0']
outputs = ['p2o.pd_op.squeeze.0.0']
attrs = {'axes': [2]}
Transpose.0
inputs = ['p2o.pd_op.squeeze.0.0']
outputs = ['p2o.pd_op.transpose.0.0']
attrs = {'perm': [0, 2, 1]}
```
Using either OCR runtime shape below gives the same failure:
```python
from tvm.relax.frontend.onnx import from_onnx
import onnx
model = onnx.load("PP-OCRv6_tiny.onnx")
from_onnx(
model,
shape_dict={"x": [1, 3, 48, 128]},
dtype_dict={"x": "float32"},
opset=11,
keep_params_in_input=False,
)
```
Observed error:
```text
Error converting operator Transpose, with inputs: [R.squeeze(lv151,
axis=None)]
ValueError: Transpose: number of axes in perm attribute (3) must equal the
number of input tensor dimensions (2)
```
The ONNX node has `Squeeze axes=[2]`, so the expected output rank should be
3 and `Transpose perm=[0,2,1]` should be valid. The TVM error shows
`R.squeeze(..., axis=None)`, which suggests that the importer may be ignoring
the ONNX `axes` attribute for this opset/node form.
### Related PaddleOCR Conv pattern
The same model also contains Paddle-style grouped/depthwise `1xK` conv
patterns:
```text
node = Conv.35
inputs = ['p2o.pd_op.unsqueeze.1.0', 'p2o.pd_op.unsqueeze.0.0']
outputs = ['p2o.pd_op.depthwise_conv2d.9.0']
attrs = {
'dilations': [1, 1],
'kernel_shape': [1, 5],
'strides': [1, 1],
'group': 160,
'pads': [0, 2, 0, 2]
}
```
I experimented with two graph rewrites:
1. Normalize symmetric ONNX Conv pads from `[top,left,bottom,right]` to
`[h,w]` for TVM.
2. Rewrite `Unsqueeze(axis=2) -> Conv(1xK) -> Squeeze(axis=2)` into an
equivalent Conv1D form.
However, the first rewrite is not ONNX-spec-safe as a persisted ONNX model,
because ONNX Runtime validation reports:
```text
Node (Conv.0) Op (Conv) [ShapeInferenceError] Attribute pads has incorrect
size
```
So I currently treat this as a TVM-only workaround and avoid saving it as a
general runtime ONNX graph.
## Case 2: FasterRCNN-12 dynamic detection post-processing
The model is `FasterRCNN-12.onnx`, opset 12. It contains a full detection
post-processing graph:
```text
TopK: 7
NonMaxSuppression: 85
RoiAlign: 4
Resize: 3
Shape: 117
Gather: 798
```
Raw import fails first at shape `Gather`:
```text
Error converting operator Gather, with inputs: [R.shape([1, 3, 1, 1]), 2034]
AssertionError: Only constant indices supported for shape gather.
```
After freezing/stabilizing static shape subgraphs and importing the prepared
runtime ONNX with:
```python
from_onnx(
model,
shape_dict={"image": [3, 224, 224]},
dtype_dict={"image": "float32"},
opset=12,
keep_params_in_input=False,
)
```
the next importer failure is dynamic `TopK`:
```text
Error converting operator TopK, with inputs: [R.sigmoid(lv242), v_5635]
ValueError: TopK k must be a constant
```
## Current workarounds in my pipeline
The pipeline currently does the following:
- Fold static shape subgraphs into initializers when this preserves ONNX
Runtime validation.
- Fix missing or empty `Resize` ROI inputs.
- Rewrite static `Split` tensor inputs to `Slice` where TVM treats the
second input as dynamic.
- Remove or bypass inference-time `Dropout`.
- Skip full detection post-processing graphs containing `NonMaxSuppression +
dynamic TopK`.
- Skip Paddle-style degenerate grouped `1xK` conv graphs when the rewrite
would make the persisted ONNX invalid.
## Questions
1. For opset 11 `Squeeze` with an `axes` attribute, should the Relax ONNX
importer preserve the axes and emit `R.squeeze(..., axis=[2])` instead of
`axis=None`?
2. Is the shape `Gather` failure expected when the input is `R.shape(...)`
and the index is a scalar constant-like value?
3. Is dynamic `TopK k` unsupported by design in Relax ONNX import, or is
there a recommended way to keep it symbolic?
4. For full detection graphs with `NonMaxSuppression + TopK`, does the TVM
team recommend splitting the graph before import, or should users expect
full-graph import to work eventually?
5. Are graph-level rewrites such as static shape folding, static
Split-to-Slice, and `Unsqueeze-Conv-Squeeze -> Conv1D` considered reasonable
preprocessing for TVM, or is there a more official path?
## Expected behavior
Ideally, TVM should import the valid ONNX graph or report a precise
unsupported-pattern diagnostic. In the PP-OCRv6 case, the `Squeeze axes=[2]`
attribute appears to be valid and should not reduce the tensor with `axis=None`.
## Attachments
log:
```text
[11:04:07] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66:
Warning: BlockBuilder destroyed with remaining blocks!
[11:04:07] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66:
Warning: BlockBuilder destroyed with remaining blocks!
[11:04:07] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66:
Warning: BlockBuilder destroyed with remaining blocks!
[11:04:08] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66:
Warning: BlockBuilder destroyed with remaining blocks!
==========================================================================================
PP-OCRv6_tiny raw ONNX -> TVM Relax from_onnx
==========================================================================================
shape_dict = {'x': [1, 3, 48, 1]}
dtype_dict = {'x': 'float32'}
Error converting operator Transpose, with inputs: [R.squeeze(lv151,
axis=None)]
IMPORT_FAILED
ValueError: Transpose: number of axes in perm attribute (3) must equal the
number of input tensor dimensions (2)
^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5173, in from_onnx
self._construct_nodes(graph)
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5382, in _construct_nodes
raise err
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5376, in _construct_nodes
op = self._convert_operator(op_name, inputs, attr, self.opset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5476, in _convert_operator
sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 931, in _impl_v13
raise ValueError(
ValueError: Transpose: number of axes in perm attribute (3) must equal the
number of input tensor dimensions (2)
[Preprocess] Normalized 37 symmetric Conv pads from 4D to 2D for TVM.
==========================================================================================
PP-OCRv6_tiny after symmetric Conv pads normalization
==========================================================================================
shape_dict = {'x': [1, 3, 48, 1]}
dtype_dict = {'x': 'float32'}
Error converting operator Transpose, with inputs: [R.squeeze(lv151,
axis=None)]
IMPORT_FAILED
ValueError: Transpose: number of axes in perm attribute (3) must equal the
number of input tensor dimensions (2)
^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5173, in from_onnx
self._construct_nodes(graph)
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5382, in _construct_nodes
raise err
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5376, in _construct_nodes
op = self._convert_operator(op_name, inputs, attr, self.opset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5476, in _convert_operator
sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 931, in _impl_v13
raise ValueError(
ValueError: Transpose: number of axes in perm attribute (3) must equal the
number of input tensor dimensions (2)
[Preprocess] Normalized 37 symmetric Conv pads from 4D to 2D for TVM.
==========================================================================================
PP-OCRv6_tiny after pads normalization + Unsqueeze-Conv-Squeeze to Conv1D
rewrite
==========================================================================================
shape_dict = {'x': [1, 3, 48, 1]}
dtype_dict = {'x': 'float32'}
Error converting operator Transpose, with inputs: [R.squeeze(lv151,
axis=None)]
IMPORT_FAILED
ValueError: Transpose: number of axes in perm attribute (3) must equal the
number of input tensor dimensions (2)
^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5173, in from_onnx
self._construct_nodes(graph)
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5382, in _construct_nodes
raise err
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5376, in _construct_nodes
op = self._convert_operator(op_name, inputs, attr, self.opset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5476, in _convert_operator
sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 931, in _impl_v13
raise ValueError(
ValueError: Transpose: number of axes in perm attribute (3) must equal the
number of input tensor dimensions (2)
==========================================================================================
PP-OCRv6_tiny representative grouped 1xK Conv nodes
==========================================================================================
node= Conv.35 inputs= ['p2o.pd_op.unsqueeze.1.0', 'p2o.pd_op.unsqueeze.0.0']
outputs= ['p2o.pd_op.depthwise_conv2d.9.0'] attrs= {'dilations': [1, 1],
'kernel_shape': [1, 5], 'strides': [1, 1], 'group': 160, 'pads': [0, 2, 0, 2]}
==========================================================================================
FasterRCNN-12 raw ONNX -> TVM Relax from_onnx
==========================================================================================
shape_dict = {'image': [3, 1, 1]}
dtype_dict = {'image': 'float32'}
Error converting operator Gather, with inputs: [R.shape([1, 3, 1, 1]), 2034]
IMPORT_FAILED
AssertionError: Only constant indices supported for shape gather.
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5173, in from_onnx
self._construct_nodes(graph)
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5382, in _construct_nodes
raise err
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5376, in _construct_nodes
op = self._convert_operator(op_name, inputs, attr, self.opset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5476, in _convert_operator
sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 1086, in _impl_v13
assert isinstance(indices, relax.Constant), (
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Only constant indices supported for shape gather.
==========================================================================================
FasterRCNN-12 prepared runtime_fp32.onnx -> TVM Relax from_onnx
==========================================================================================
shape_dict = {'image': [3, 224, 224]}
dtype_dict = {'image': 'float32'}
Error converting operator TopK, with inputs: [R.sigmoid(lv242), v_5635]
IMPORT_FAILED
ValueError: TopK k must be a constant
^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5173, in from_onnx
self._construct_nodes(graph)
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5382, in _construct_nodes
raise err
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5376, in _construct_nodes
op = self._convert_operator(op_name, inputs, attr, self.opset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 5476, in _convert_operator
sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py",
line 4154, in _impl_v11[11:04:08]
/home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning:
BlockBuilder destroyed with remaining blocks!
raise ValueError("TopK k must be a constant")
ValueError: TopK k must be a constant
==========================================================================================
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
