[Apache TVM Discuss] [Development/RFC] [RFC] Building a new reproducible benchmark for TVM

2020-11-20 Thread Yao Wang via Apache TVM Discuss
Yeah. A performance regression test would be very nice. There are a lot of times we need to do binary search to find the commit causing regression. --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-building-a-new-reproducible-benchmark-for-tvm/8496/3) to respond. You are receiving t

[Apache TVM Discuss] [Development/RFC] [RFC] Building a new reproducible benchmark for TVM

2020-11-20 Thread Ziheng Jiang via Apache TVM Discuss
Great suggestion! Can we make it as a nightly/weekly regression test utils and also consider adding accuracy evaluation for quantization model into this loop? --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-building-a-new-reproducible-benchmark-for-tvm/8496/2) to respond. You are

[Apache TVM Discuss] [Development/RFC] [RFC] Building a new reproducible benchmark for TVM

2020-11-20 Thread Lianmin Zheng via Apache TVM Discuss
## Motivation Currently, TVM lacks an up-to-date and reproducible benchmark. The only benchmark is hosted at [tvm/apps/benchmark](https://github.com/apache/incubator-tvm/tree/main/apps/benchmark). However, this benchmark is too old and has several flaws. 1. The results were obtained 2 years ag

[Apache TVM Discuss] [Development] Dynamic Model Support

2020-11-20 Thread Zhao Wu via Apache TVM Discuss
I enjoy the reading of https://arxiv.org/abs/2006.03031 which supports dyn model support in the TVM using relay vm. However, i want to ask some quick questions: 1. Do we have uploaded completely of Nimble code on the mainstream? Especially about the memory performance issue like this : https