> On Jun 22, 2016, at 10:20 AM, Aman Gupta <ffm...@tmm1.net> wrote: > > This patch should fix the assertion failure you see: > https://github.com/FFmpeg/FFmpeg/commit/1087f0dc172a9adf779e41bf2dc82639fb04ebd4 > > <https://github.com/FFmpeg/FFmpeg/commit/1087f0dc172a9adf779e41bf2dc82639fb04ebd4> > > Aman > On Sat, Jun 18, 2016 at 8:43 AM Amancio Hasty <aha...@gmail.com > <mailto:aha...@gmail.com>> wrote: > >> >>> On Jun 16, 2016, at 11:16 AM, Aman Gupta <ffm...@tmm1.net> wrote: >>> >>> The patchset that was merged into libav is now available in ffmpeg as >> well: >>> https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/omx.c >>> >>> You can compile ffmpeg from the master branch >>> with --enable-omx --enable-omx-rpi >>> >>> Aman >>> >>> On Thu, Jun 16, 2016 at 2:16 AM, Amancio Hasty <aha...@gmail.com> wrote: >>> >>>> >>>>> On May 9, 2016, at 7:55 AM, Amancio Hasty <aha...@gmail.com> wrote: >>>>> >>>>> Hi, >>>>> >>>>> So what is the next step? >>>>> >>>>> If you want testers I suggest posting to ccrisan’s motionpie mailing >>>> list. >>>>> Cheers >>>>> Amancio >>>>> >>>>> >>>>>> On Mar 31, 2016, at 7:27 PM, Amancio Hasty <aha...@gmail.com> wrote: >>>>>> >>>>>> I am not a lawyer… >>>>>> >>>>>> >>>>>> I updated the patch. vc264.c now has a the copyright notice embedded >> in >>>>>> a volatile global so if a binary is compiled against vc264.o , the >>>> copyright notice >>>>>> can be displayed by: >>>>>> strings ffmpeg | grep -i copyright >>>>>> >>>>>> LICENSE.md has been updated to include Broadcom’s copyright notice. >>>>>> >>>>>> A distribution of a binary that includes vc264.o should include >>>> LICENSE.md and if >>>>>> that is missing, the copyright notice can be displayed via the shell >>>>>> command ‘strings’ . >>>>>> >>>>>> Amancio >>>>>> <c-0001-added-support-for-hardware-assist-H264-video-encodin.patch> >>>>>>> On Mar 22, 2016, at 12:12 PM, Lou Logan <l...@lrcd.com> wrote: >>>>>>> >>>>>>> On Mon, 21 Mar 2016 20:07:01 -0700, Amancio Hasty wrote: >>>>>>> >>>>>>>> From 874a72eec2a78f4935fea091003e534b5f8d5413 Mon Sep 17 00:00:00 >> 2001 >>>>>>>> From: Amancio Hasty <aha...@gmail.com> >>>>>>>> Date: Mon, 21 Mar 2016 18:56:05 -0700 >>>>>>>> Subject: [PATCH] added support for hardware assist H264 video >>>> encoding for >>>>>>>> the Raspberry Pi >>>>>>>> >>>>>>>> --- >>>>>>>> configure | 12 ++ >>>>>>>> libavcodec/Makefile | 1 + >>>>>>>> libavcodec/allcodecs.c | 2 + >>>>>>>> libavcodec/vc264.c | 387 >>>> +++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>>> 4 files changed, 402 insertions(+) >>>>>>>> create mode 100644 libavcodec/vc264.c >>>>>>>> >>>>>>> [...] >>>>>>>> diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c >>>>>>>> index 2a25d66..3c7bd9b 100644 >>>>>>>> --- a/libavcodec/allcodecs.c >>>>>>>> +++ b/libavcodec/allcodecs.c >>>>>>>> @@ -74,6 +74,7 @@ void avcodec_register_all(void) >>>>>>>> initialized = 1; >>>>>>>> >>>>>>> >>>>>>> Nit: Whitespace on the line above should be removed. >>>>>>> >>>>>>> [...] >>>>>>>> --- /dev/null >>>>>>>> +++ b/libavcodec/vc264.c >>>>>>>> @@ -0,0 +1,387 @@ >>>>>>>> +/* H.264 hardware assist video encoding code taken from >>>>>>>> + * raspberry's os : >>>>>>>> + * /opt/vc/src/hello_pi/hello_encode/encode.c >>>>>>>> + */ >>>>>>>> + >>>>>>>> +/* >>>>>>>> +Copyright (c) 2012, Broadcom Europe Ltd >>>>>>>> +Copyright (c) 2012, Kalle Vahlman <zuh@iki> >>>>>>>> + Tuomas Kulve <tuo...@kulve.fi> >>>>>>>> +All rights reserved. >>>>>>>> + >>>>>>>> +Redistribution and use in source and binary forms, with or without >>>>>>>> +modification, are permitted provided that the following conditions >>>> are met: >>>>>>>> +* Redistributions of source code must retain the above copyright >>>>>>>> + notice, this list of conditions and the following disclaimer. >>>>>>>> + * Redistributions in binary form must reproduce the above >>>> copyright >>>>>>>> + notice, this list of conditions and the following disclaimer >>>> in the >>>>>>>> + documentation and/or other materials provided with the >>>> distribution. >>>>>>>> + * Neither the name of the copyright holder nor the >>>>>>>> + names of its contributors may be used to endorse or promote >>>> products >>>>>>>> + derived from this software without specific prior written >>>> permission. >>>>>>>> + >>>>>>>> +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS >>>> "AS IS" AND >>>>>>>> +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >>>> THE IMPLIED >>>>>>>> +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE >>>> ARE >>>>>>>> +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS >>>> BE LIABLE FOR ANY >>>>>>>> +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL >>>> DAMAGES >>>>>>>> +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR >>>> SERVICES; >>>>>>>> +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER >>>> CAUSED AND >>>>>>>> +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, >>>> OR TORT >>>>>>>> +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE >>>> USE OF THIS >>>>>>>> +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >>>>>>> >>>>>>> I wonder if any of the above legalese is compatible. Granted, I see a >>>>>>> similar paragraph in "libavformat/aadec.c". >>>>>>> >>>>>>>> + * ffmpeg driver for hardware assist video H.264 encoding using >>>> Broadcom's GPU >>>>>>>> + * Copyright (C) 2016 Amancio Hasty aha...@gmail.com >>>>>>>> + * >>>>>>>> + * >>>>>>>> + * This file is part of FFmpeg. >>>>>>>> + * >>>>>>>> + * FFmpeg is free software; you can redistribute it and/or >>>>>>>> + * modify it under the terms of the GNU Lesser General Public >>>>>>>> + * License as published by the Free Software Foundation; either >>>>>>>> + * version 2.1 of the License, or (at your option) any later >> version. >>>>>>>> + * >>>>>>>> + * FFmpeg is distributed in the hope that it will be useful, >>>>>>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of >>>>>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >> GNU >>>>>>>> + * Lesser General Public License for more details. >>>>>>>> + * >>>>>>>> + * You should have received a copy of the GNU Lesser General Public >>>>>>>> + * License along with FFmpeg; if not, write to the Free Software >>>>>>>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA >>>> 02110-1301 USA >>>>>>>> + * >>>>>>>> + */ >>>>>>>> + >>>>>>>> + >>>>>>>> +/** >>>>>>>> + * @ file vc264.c >>>>>>>> + * Broadcom bm2865's Visual Core hardware assist h264 using >>>>>>>> + openMax interface to the GPU. >>>>>>>> + >>>>>>>> +*/ >>>>>>>> + >>>>>>>> +#include <stdio.h> >>>>>>>> +#include <stdlib.h> >>>>>>>> +#include <string.h> >>>>>>>> +#define OMX_SKIP64BIT >>>>>>>> +#include "bcm_host.h" >>>>>>>> +#include "ilclient.h" >>>>>>>> +#include "avcodec.h" >>>>>>>> +#include "internal.h" >>>>>>>> + >>>>>>>> +typedef struct VC264Context { >>>>>>>> + OMX_VIDEO_PARAM_PORTFORMATTYPE format; >>>>>>>> + OMX_PARAM_PORTDEFINITIONTYPE def; >>>>>>>> + COMPONENT_T *video_encode; >>>>>>>> + COMPONENT_T *list[5]; >>>>>>>> + OMX_BUFFERHEADERTYPE *buf; >>>>>>>> + OMX_BUFFERHEADERTYPE *out; >>>>>>>> + ILCLIENT_T *client; >>>>>>>> + OMX_VIDEO_PARAM_BITRATETYPE bitrateType; >>>>>>>> + int width; >>>>>>>> + int height; >>>>>>>> + int bit_rate; >>>>>>>> +} VC264Context; >>>>>>>> + >>>>>>>> + >>>>>>>> +static int vc264_init(AVCodecContext *avctx) { >>>>>>>> + >>>>>>>> + >>>>>>>> + >>>>>>>> + OMX_ERRORTYPE r; >>>>>>>> + int error; >>>>>>>> + >>>>>>>> + >>>>>>>> + >>>>>>>> + VC264Context *vc = avctx->priv_data; >>>>>>>> + >>>>>>>> + vc->width = avctx->width; >>>>>>>> + vc->height = avctx->height; >>>>>>>> + vc->bit_rate = avctx->bit_rate; >>>>>>>> + printf("vc264: bit rate %d \n", avctx->bit_rate); >>>>>>>> +#if FF_API_CODED_FRAME >>>>>>>> +FF_DISABLE_DEPRECATION_WARNINGS >>>>>>>> + >>>>>>>> + avctx->coded_frame = av_frame_alloc(); >>>>>>>> + avctx->coded_frame->pict_type = AV_PICTURE_TYPE_I; >>>>>>>> +FF_ENABLE_DEPRECATION_WARNINGS >>>>>>>> +#endif >>>>>>>> + >>>>>>>> + >>>>>>>> + memset(&vc->list, 0, sizeof(vc->list)); >>>>>>>> + bcm_host_init(); >>>>>>>> + if ((vc->client = ilclient_init()) == NULL) { >>>>>>>> + return -3; >>>>>>>> + } >>>>>>>> + error = OMX_Init(); >>>>>>>> + >>>>>>>> + if (error != OMX_ErrorNone) { >>>>>>>> + ilclient_destroy(vc->client); >>>>>>>> + av_log(avctx,AV_LOG_ERROR,"in vc264_init OMX_Init failed "); >>>>>>>> + return -4; >>>>>>>> + } >>>>>>>> + >>>>>>>> + // create video_encode >>>>>>>> + r = ilclient_create_component(vc->client, &vc->video_encode, >>>> (char *) "video_encode", >>>>>>>> + ILCLIENT_DISABLE_ALL_PORTS | >>>>>>>> + ILCLIENT_ENABLE_INPUT_BUFFERS | >>>>>>>> + ILCLIENT_ENABLE_OUTPUT_BUFFERS); >>>>>>> >>>>>>> Tabs should be converted to spaces. There are many instances of tabs >>>>>>> being used in this patch. >>>>>>> >>>>>>> Others will have to provide a more technical review (not to mention >>>>>>> possible additions docs, Changelog, MAINTAINERS, and >>>>>>> libavcodec/version.h). >>>>>>> _______________________________________________ >>>>>>> ffmpeg-devel mailing list >>>>>>> ffmpeg-devel@ffmpeg.org >>>>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>>>>> >>>>> >>>> >>>> Any thoughts on how we may proceed forward? >>>> >>>> Thank You, >>>> Amancio >>>> >>>> >>>> _______________________________________________ >>>> ffmpeg-devel mailing list >>>> ffmpeg-devel@ffmpeg.org >>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>>> >>> _______________________________________________ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel < >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> <http://ffmpeg.org/mailman/listinfo/ffmpeg-devel>> >> >> >> This is what I getting over here with RPI3: >> >> ./ffmpeg_g -loglevel debug -f v4l2 -input_format yuv420p -framerate 25 >> -video_size 640x480 -i /dev/video0 output.h264 >> ffmpeg version N-80363-g403a53c Copyright (c) 2000-2016 the FFmpeg >> developers >> built with gcc 4.9.2 (Raspbian 4.9.2-10) >> configuration: --enable-gpl --enable-omx --enable-omx-rpi >> --enable-nonfree --extra-libs=-ldl --extra-cflags=' -pipe ' >> libavutil 55. 24.100 / 55. 24.100 >> libavcodec 57. 46.100 / 57. 46.100 >> libavformat 57. 38.100 / 57. 38.100 >> libavdevice 57. 0.101 / 57. 0.101 >> libavfilter 6. 46.101 / 6. 46.101 >> libswscale 4. 1.100 / 4. 1.100 >> libswresample 2. 1.100 / 2. 1.100 >> libpostproc 54. 0.100 / 54. 0.100 >> Splitting the commandline. >> Reading option '-loglevel' ... matched as option 'loglevel' (set logging >> level) with argument 'debug'. >> Reading option '-f' ... matched as option 'f' (force format) with argument >> 'v4l2'. >> Reading option '-input_format' ... matched as AVOption 'input_format' with >> argument 'yuv420p'. >> Reading option '-framerate' ... matched as AVOption 'framerate' with >> argument '25'. >> Reading option '-video_size' ... matched as AVOption 'video_size' with >> argument '640x480'. >> Reading option '-i' ... matched as input file with argument '/dev/video0'. >> Reading option 'output.h264' ... matched as output file. >> Finished splitting the commandline. >> Parsing a group of options: global . >> Applying option loglevel (set logging level) with argument debug. >> Successfully parsed a group of options. >> Parsing a group of options: input file /dev/video0. >> Applying option f (force format) with argument v4l2. >> Successfully parsed a group of options. >> Opening an input file: /dev/video0. >> [video4linux2,v4l2 @ 0x2189260] fd:3 capabilities:85200005 >> [video4linux2,v4l2 @ 0x2189260] Current input_channel: 0, input_name: >> Camera 0, input_std: 0 >> [video4linux2,v4l2 @ 0x2189260] Setting time per frame to 1/25 >> [video4linux2,v4l2 @ 0x2189260] All info found >> Input #0, video4linux2,v4l2, from '/dev/video0': >> Duration: N/A, start: -140423796.316413, bitrate: 92160 kb/s >> Stream #0:0, 1, 1/1000000: Video: rawvideo, 1 reference frame (I420 / >> 0x30323449), yuv420p, 640x480, 0/1, 92160 kb/s, 25 fps, 25 tbr, 1000k tbn, >> 1000k tbc >> Successfully opened the file. >> Parsing a group of options: output file output.h264. >> Successfully parsed a group of options. >> Opening an output file: output.h264. >> [file @ 0x218c930] Setting default whitelist 'file,crypto' >> Successfully opened the file. >> detected 4 logical cores >> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'video_size' to value >> '640x480' >> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'pix_fmt' to value '0' >> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'time_base' to value >> '1/1000000' >> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'pixel_aspect' to >> value '0/1' >> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'sws_param' to value >> 'flags=2' >> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'frame_rate' to value >> '25/1' >> [graph 0 input from stream 0:0 @ 0x2195270] w:640 h:480 pixfmt:yuv420p >> tb:1/1000000 fr:25/1 sar:0/1 sws_param:flags=2 >> [format @ 0x2195530] compat: called with args=[yuv420p] >> [format @ 0x2195530] Setting 'pix_fmts' to value 'yuv420p' >> [AVFilterGraph @ 0x2194b40] query_formats: 4 queried, 3 merged, 0 already >> done, 0 delayed >> [h264_omx @ 0x218c2c0] Using OMX.broadcom.video_encode >> [h264_omx @ 0x218c2c0] OMX state changed to 2 >> [h264_omx @ 0x218c2c0] OMX state changed to 3 >> [h264_omx @ 0x218c2c0] OMX port 201 settings changed >> [h264 @ 0x218b010] Using AVStream.codec to pass codec parameters to muxers >> is deprecated, use AVStream.codecpar instead. >> Output #0, h264, to 'output.h264': >> Metadata: >> encoder : Lavf57.38.100 >> Stream #0:0, 0, 1/25: Video: h264 (h264_omx), 1 reference frame, >> yuv420p, 640x480, 0/1, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc >> Metadata: >> encoder : Lavc57.46.100 h264_omx >> Stream mapping: >> Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_omx)) >> Press [q] to stop, [?] for help >> cur_dts is invalid (this is harmless if it occurs once at the start per >> stream) >> [rawvideo @ 0x218ac30] PACKET SIZE: 460800, STRIDE: 960 >> Clipping frame in rate conversion by 0.000008 >> Assertion ret <= 0 failed at libavcodec/utils.c:1956 >> Aborted >> >> How did you test h264 encoding? >> >> Thanks! >> Amancio >> >> >> >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org> >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> <http://ffmpeg.org/mailman/listinfo/ffmpeg-devel> >> > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org <mailto:ffmpeg-devel@ffmpeg.org> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > <http://ffmpeg.org/mailman/listinfo/ffmpeg-devel> Hello Aman,
Thanks for the patch. I noticed that in libav/utils.c: ret = avctx->codec->encode2(avctx, avpkt, frame, got_packet_ptr); 1325 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1325> if (!ret) { 1326 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1326> if (*got_packet_ptr) { 1327 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1327> if (!(avctx->codec->capabilities & AV_CODEC_CAP_DELAY)) { 1328 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1328> if (avpkt->pts == AV_NOPTS_VALUE) 1329 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1329> avpkt->pts = frame->pts; 1330 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1330> if (!avpkt->duration) 1331 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1331> avpkt->duration = ff_samples_to_time_base(avctx, 1332 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1332> frame->nb_samples); 1333 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1333> } 1334 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1334> avpkt->dts = avpkt->pts; 1335 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1335> } else { 1336 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1336> avpkt->size = 0; 1337 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1337> } Which is different than ffmpeg’s libavcodec/utils.c …. Last but not least I get a bunch or warnings in ffmpeg when I run it with your patch. ./ffmpeg_g -f v4l2 -input_format yuv420p -framerate 25 -video_size 640x480 -i /dev/video0 a.h264 DTS 140462612887087, next:1760000 st:0 invalid dropping PTS 140462612887087, next:1760000 invalid dropping st:0 …. Thanks again! Amancio _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
