> -----Original Message----- > From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf > Of Liu Steven > Sent: Wednesday, February 20, 2019 7:18 PM > To: FFmpeg development discussions and patches <ffmpeg- > de...@ffmpeg.org> > Cc: Thilo Borgmann <thilo.borgm...@mail.de>; Liu Steven > <l...@chinaffmpeg.org> > Subject: Re: [FFmpeg-devel] GSoC mentored project: derain filter > > > > > 在 2019年2月20日，下午6:35，孟学苇 <xwm...@pku.edu.cn> 写道： > > > > Hi Dev-Community, > > > > > > > > > > I am Iris Meng from China. I’m a PhD student in Institute of Digital Media, > Peking University. I wish to contribute as a GSoC applicant this year. > > > > I am interested in Deep Learning. I want to add a derain filter in ffmpeg. > > If > you have any suggestion or question, we can contact by email. My > motivation and plans are as follows. > > > > > > > > > > Motivation > > > > Rain and fog are very common weather in actual life. However, it can affect > the visibility. Especially in heavy rain, rain streaks from various directions > accumulate and make the background scene misty, which will seriously > influence the accuracy of many computer vision systems, including video > surveillance, object detection and tracking in autonomous driving, etc. > Therefore, it is an important task to remove the rain and fog, and recover the > background from rain images. It can be used for image and video processing > to make them clearer and it can be a preprocessing method for many > computer vision systems. > > > > > > > > > > Proposed Idea > > > > We propose to implement this technology in ffmpeg. For video [1][2], we > can utilize the relationship between frames to remove rain and fog. For > single image [3], we can use traditional methods, such as discriminative > sparse coding, low rank representation and the Gaussian mixture model. We > can also use some deep learning methods. We should investigate these > methods, and ultimately consider the effect of rain/fog removal and the > complexity of the algorithm, and choose the optimal scheme. > > > > > > > > > > Practical application > > > > The derain and dehaze method can improve the subjective quality of > videos and images. > > > > > > > > > > Development plan > > > > I would like to start working on my qualification task and try to solve my > problems. Overall, I will follow the following steps to complete the project. > > > > (1) Literature and algorithm investigation > > > > (2) Data sets preparation > > > > (3) Coding: Implement network, training code, inference code and so on > > > > (4) Select the best method and transplantation it into ffmpeg > > > > > > > > > > Reference > > > > [1] Zhang X, Li H, Qi Y, et al. Rain removal in video by combining > > temporal > and chromatic properties[C]//2006 IEEE International Conference on > Multimedia and Expo. IEEE, 2006: 461-464. > > > > [2] Tripathi A K, Mukhopadhyay S. Removal of rain from videos: a > review[J]. Signal, Image and Video Processing, 2014, 8(8): 1421-1430. > > > > [3] Li X, Wu J, Lin Z, et al. Recurrent squeeze-and-excitation context > aggregation net for single image deraining[C]//Proceedings of the European > Conference on Computer Vision (ECCV). 2018: 254-269. > > > > > > > I think this can reference libavflter/sr.c to implementation, maybe you can > try two ways to implement it, one is native and the other is model. >
and currently, only TensorFlow model is supported via tensorflow C API, you can easily save the model file in python with function tf.graph_util.convert_variables_to_constants and tf.train.write_graph. For the native mode (executed with CPU), two operations (CONV and DEPTH_TO_SPACE) are supported now, you might add more. > > Thanks > Steven > > > > > > > > > > > Thanks, > > > > Regards, > > > > Iris Meng > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
