[Apache TVM Discuss] [Development/RFC] [RFC] CSE Optimization

2020-10-17 Thread M1k3 via Apache TVM Discuss
I suppose CSE would solve my question earlier where 2 identical adds with the same tensor shapes where not simplified as 1 add with both tensors added? --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-cse-optimization/8130/7) to respond. You are receiving this because you enabled ma

[Apache TVM Discuss] [Development/RFC] [RFC] CSE Optimization

2020-10-16 Thread Junru Shao via Apache TVM Discuss
I agree with @tqchen. CSE is not hard to implement, and CSE on both sides provides different benefits. So we can potentially support both variants. --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-cse-optimization/8130/6) to respond. You are receiving this because you enabled mailin

[Apache TVM Discuss] [Development/RFC] [RFC] CSE Optimization

2020-10-16 Thread tqchen via Apache TVM Discuss
I think there are two potential ways to think about it. We can either try to do CSE in the Expr level, or we can do CSE in the TE level. I think both will bring some of the benefit, so it would be helpful to support both variants --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-cse-

[Apache TVM Discuss] [Development/RFC] [RFC] CSE Optimization

2020-10-16 Thread Yizhi Liu via Apache TVM Discuss
@tqchen @junrushao1994 how much of the work do you think we can reuse once we move to https://discuss.tvm.apache.org/t/rfc-tensorir-a-schedulable-ir-for-tvm? --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-cse-optimization/8130/4) to respond. You are receiving this because you ena

[Apache TVM Discuss] [Development/RFC] [RFC] CSE Optimization

2020-10-16 Thread Jack Zheng via Apache TVM Discuss
- The user API looks like the following: ```C++ /*! * \brief Eliminate common subexpressions among \p in_args and between them and \p output . * * \param output The output tensor. * \param input_grads The gradients of input tensors. */ std::pair > CSE(const Tensor& output,

[Apache TVM Discuss] [Development/RFC] [RFC] CSE Optimization

2020-10-15 Thread Yizhi Liu via Apache TVM Discuss
Thanks for the RFC. Nice finding and it looks good to me overall. Could you also provide an example of how the user api looks like? Besides, what information does Tensor Expression Tree provide in addition to Tensor? --- [Visit Topic](https://discuss.tvm.apache.org/t/rfc-cse-optimization/8

[Apache TVM Discuss] [Development/RFC] [RFC] CSE Optimization

2020-10-09 Thread Jack Zheng via Apache TVM Discuss
# [RFC] CSE ## Motivation **C**ommon **S**ubexpression **E**limination is a compiler optimization that effectively avoids repeated computation of the same expression. It is needed in the AutoDiff submodule to improve its performance. As an example, consider a compute definition that is as simpl