Hi Dev-Community,
I am Iris Meng from China. I’m a PhD student in Institute of Digital Media,
Peking University. I wish to contribute as a GSoC applicant this year.
I am interested in Deep Learning. I want to add a derain filter in ffmpeg. If
you have any suggestion or question, we can contact by email. My motivation and
plans are as follows.
Motivation
Rain and fog are very common weather in actual life. However, it can affect the
visibility. Especially in heavy rain, rain streaks from various directions
accumulate and make the background scene misty, which will seriously influence
the accuracy of many computer vision systems, including video surveillance,
object detection and tracking in autonomous driving, etc. Therefore, it is an
important task to remove the rain and fog, and recover the background from rain
images. It can be used for image and video processing to make them clearer and
it can be a preprocessing method for many computer vision systems.
Proposed Idea
We propose to implement this technology in ffmpeg. For video [1][2], we can
utilize the relationship between frames to remove rain and fog. For single
image [3], we can use traditional methods, such as discriminative sparse
coding, low rank representation and the Gaussian mixture model. We can also use
some deep learning methods. We should investigate these methods, and ultimately
consider the effect of rain/fog removal and the complexity of the algorithm,
and choose the optimal scheme.
Practical application
The derain and dehaze method can improve the subjective quality of videos and
images.
Development plan
I would like to start working on my qualification task and try to solve my
problems. Overall, I will follow the following steps to complete the project.
(1) Literature and algorithm investigation
(2) Data sets preparation
(3) Coding: Implement network, training code, inference code and so on
(4) Select the best method and transplantation it into ffmpeg
Reference
[1] Zhang X, Li H, Qi Y, et al. Rain removal in video by combining
temporal and chromatic properties[C]//2006 IEEE International Conference on
Multimedia and Expo. IEEE, 2006: 461-464.
[2] Tripathi A K, Mukhopadhyay S. Removal of rain from videos: a
review[J]. Signal, Image and Video Processing, 2014, 8(8): 1421-1430.
[3] Li X, Wu J, Lin Z, et al. Recurrent squeeze-and-excitation context
aggregation net for single image deraining[C]//Proceedings of the European
Conference on Computer Vision (ECCV). 2018: 254-269.
Thanks,
Regards,
Iris Meng
_______________________________________________
ffmpeg-devel mailing list
[email protected]
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
