AVFrame already has some support for embedded 608/708 subtitles in a form of side data (AV_FRAME_DATA_A53_CC tag), and some decoders/encoders already can handle that. Main issue with this approach is that video frames can be modified, removed or duplicated (mainly by video filters but also by ffmpeg's video sync code).
2016-09-04 6:14 GMT+03:00 Jonathan Campbell <jonat...@impactstudiopro.com>: > While I'm finishing up the CNG patch, I'd like to start developing another > feature that I think FFMPEG should be able to at least expose through the > API. > > I'm aware that FFMPEG supports subtitle tracks, but as far as I know, > doesn't support caption data embedded in the video stream itself. > > Some scenarios especially in broadcast TV transmit MPEG-2 or H.264 video > with caption data embedded in the encoded frame. I would like to write a > patch that can read the caption data embedded in MPEG-2 user packets and > expose it in the AVFrame. > > There are two common forms I work with in software: One carries EIA 608 > closed caption packets per keyframe (typically seen on DVDs here in > America), the other carries CEA 708 closed caption data (Caption > Description Packets) every frame (or every I/P frame) with a side channel > for 608 (typically seen in terrestrial HDTV broadcast here in America). How > could I modify the MPEG-2 decoder and/or bitstream parser to read and > expose this data through the API for software that needs it? I'm also > interested in modding the MPEG-2 encoder to take this data through the API > and encode it into the frame. If done right, the extension could allow > FFMPEG to transcode DVD or HDTV broadcasts while keeping the caption data > intact. > > Similar standards exist to carry caption data packets in H.264 (as SEI > packets) and DV video (as DIF packets). > > Jonathan Campbell > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
