> -----Original Message----- > From: ffmpeg-devel [mailto:[email protected]] On Behalf > Of Liu Steven > Sent: Wednesday, February 20, 2019 7:18 PM > To: FFmpeg development discussions and patches <ffmpeg- > [email protected]> > Cc: Thilo Borgmann <[email protected]>; Liu Steven > <[email protected]> > Subject: Re: [FFmpeg-devel] GSoC mentored project: derain filter > > > > > 在 2019年2月20日，下午6:35，孟学苇 <[email protected]> 写道： > > > > Hi Dev-Community, > > > > > > > > > > I am Iris Meng from China. I’m a PhD student in Institute of Digital Media, > Peking University. I wish to contribute as a GSoC applicant this year. > > > > I am interested in Deep Learning. I want to add a derain filter in ffmpeg. > > If > you have any suggestion or question, we can contact by email. My > motivation and plans are as follows. > > > > > > > > > > Motivation > > > > Rain and fog are very common weather in actual life. However, it can affect > the visibility. Especially in heavy rain, rain streaks from various directions > accumulate and make the background scene misty, which will seriously > influence the accuracy of many computer vision systems, including video > surveillance, object detection and tracking in autonomous driving, etc. > Therefore, it is an important task to remove the rain and fog, and recover the > background from rain images. It can be used for image and video processing > to make them clearer and it can be a preprocessing method for many > computer vision systems. > > > > > > > > > > Proposed Idea > > > > We propose to implement this technology in ffmpeg. For video [1][2], we > can utilize the relationship between frames to remove rain and fog. For > single image [3], we can use traditional methods, such as discriminative > sparse coding, low rank representation and the Gaussian mixture model. We > can also use some deep learning methods. We should investigate these > methods, and ultimately consider the effect of rain/fog removal and the > complexity of the algorithm, and choose the optimal scheme. > > > > > > > > > > Practical application > > > > The derain and dehaze method can improve the subjective quality of > videos and images. > > > > > > > > > > Development plan > > > > I would like to start working on my qualification task and try to solve my > problems. Overall, I will follow the following steps to complete the project. > > > > (1) Literature and algorithm investigation > > > > (2) Data sets preparation > > > > (3) Coding: Implement network, training code, inference code and so on > > > > (4) Select the best method and transplantation it into ffmpeg > > > > > > > > > > Reference > > > > [1] Zhang X, Li H, Qi Y, et al. Rain removal in video by combining > > temporal > and chromatic properties[C]//2006 IEEE International Conference on > Multimedia and Expo. IEEE, 2006: 461-464. > > > > [2] Tripathi A K, Mukhopadhyay S. Removal of rain from videos: a > review[J]. Signal, Image and Video Processing, 2014, 8(8): 1421-1430. > > > > [3] Li X, Wu J, Lin Z, et al. Recurrent squeeze-and-excitation context > aggregation net for single image deraining[C]//Proceedings of the European > Conference on Computer Vision (ECCV). 2018: 254-269. > > > > > > > I think this can reference libavflter/sr.c to implementation, maybe you can > try two ways to implement it, one is native and the other is model. >
and currently, only TensorFlow model is supported via tensorflow C API, you can easily save the model file in python with function tf.graph_util.convert_variables_to_constants and tf.train.write_graph. For the native mode (executed with CPU), two operations (CONV and DEPTH_TO_SPACE) are supported now, you might add more. > > Thanks > Steven > > > > > > > > > > > Thanks, > > > > Regards, > > > > Iris Meng > > _______________________________________________ > > ffmpeg-devel mailing list > > [email protected] > > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > _______________________________________________ > ffmpeg-devel mailing list > [email protected] > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel _______________________________________________ ffmpeg-devel mailing list [email protected] https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
