> -----Original Message----- > From: Guo, Yejun > Sent: Tuesday, June 04, 2019 3:10 PM > To: ffmpeg-devel@ffmpeg.org > Cc: Guo, Yejun <yejun....@intel.com> > Subject: [PATCH V2 1/2] libavfilter/dnn: add script to convert TensorFlow > model > (.pb) to native model (.model) > > For example, given TensorFlow model file espcn.pb, > to generate native model file espcn.model, just run: > python convert.py espcn.pb > > In current implementation, the native model file is generated for > specific dnn network with hard-code python scripts maintained out of ffmpeg. > For example, srcnn network used by vf_sr is generated with > https://github.com/HighVoltageRocknRoll/sr/blob/master/generate_header_a > nd_model.py#L85 > > In this patch, the script is designed as a general solution which > converts general TensorFlow model .pb file into .model file. The script > now has some tricky to be compatible with current implemention, will > be refined step by step. > > The script is also added into ffmpeg source tree. It is expected there > will be many more patches and community needs the ownership of it. > > Another technical direction is to do the conversion in c/c++ code within > ffmpeg source tree. While .pb file is organized with protocol buffers, > it is not easy to do such work with tiny c/c++ code, see more discussion > at http://ffmpeg.org/pipermail/ffmpeg-devel/2019-May/244496.html. So, > choose the python script. > > Signed-off-by: Guo, Yejun <yejun....@intel.com> > --- > .gitignore | 1 + > libavfilter/dnn/python/convert.py | 52 ++++++ > libavfilter/dnn/python/convert_from_tensorflow.py | 201 > ++++++++++++++++++++++ > 3 files changed, 254 insertions(+) > create mode 100644 libavfilter/dnn/python/convert.py > create mode 100644 libavfilter/dnn/python/convert_from_tensorflow.py
ping for review, thanks. Here is my rough plan after this patch. - move dnn relative .h/.c from libavfilter to libavfilter/dnn, it is expected there will be more files for dnn module (code for both model loading and execution). - add a layer for padding (tf.pad) for native mode and its fate test. - change the script to add tf.pad support, and so the native model and the tf model of vf_sr will be the same. in current implementation, the two models have a little difference, it makes the script not a general solution to convert tf model to native model. - add layer maximum and fate test. This layer appears in tf model, but not in native model, of vf_sr. - introduce operand concept in native mode (both execution and model), to support data split and merge/concat in the network, such split/concat is very common. it also makes possible to reuse memory for the intermediate data as the output of the hidden layers. - tune conv2d layer performance (it is very slow now) or add more layers for native mode. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
