On June 22, 2016 at 12:12:02 PM, Amancio Hasty (aha...@gmail.com) wrote:
On Jun 22, 2016, at 10:20 AM, Aman Gupta <ffm...@tmm1.net> wrote: This patch should fix the assertion failure you see: https://github.com/FFmpeg/FFmpeg/commit/1087f0dc172a9adf779e41bf2dc82639fb04ebd4 Aman On Sat, Jun 18, 2016 at 8:43 AM Amancio Hasty <aha...@gmail.com> wrote: On Jun 16, 2016, at 11:16 AM, Aman Gupta <ffm...@tmm1.net> wrote: The patchset that was merged into libav is now available in ffmpeg as well: https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/omx.c You can compile ffmpeg from the master branch with --enable-omx --enable-omx-rpi Aman On Thu, Jun 16, 2016 at 2:16 AM, Amancio Hasty <aha...@gmail.com> wrote: On May 9, 2016, at 7:55 AM, Amancio Hasty <aha...@gmail.com> wrote: Hi, So what is the next step? If you want testers I suggest posting to ccrisan’s motionpie mailing list. Cheers Amancio On Mar 31, 2016, at 7:27 PM, Amancio Hasty <aha...@gmail.com> wrote: I am not a lawyer… I updated the patch. vc264.c now has a the copyright notice embedded in a volatile global so if a binary is compiled against vc264.o , the copyright notice can be displayed by: strings ffmpeg | grep -i copyright LICENSE.md has been updated to include Broadcom’s copyright notice. A distribution of a binary that includes vc264.o should include LICENSE.md and if that is missing, the copyright notice can be displayed via the shell command ‘strings’ . Amancio <c-0001-added-support-for-hardware-assist-H264-video-encodin.patch> On Mar 22, 2016, at 12:12 PM, Lou Logan <l...@lrcd.com> wrote: On Mon, 21 Mar 2016 20:07:01 -0700, Amancio Hasty wrote: From 874a72eec2a78f4935fea091003e534b5f8d5413 Mon Sep 17 00:00:00 2001 From: Amancio Hasty <aha...@gmail.com> Date: Mon, 21 Mar 2016 18:56:05 -0700 Subject: [PATCH] added support for hardware assist H264 video encoding for the Raspberry Pi --- configure | 12 ++ libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 2 + libavcodec/vc264.c | 387 +++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 402 insertions(+) create mode 100644 libavcodec/vc264.c [...] diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 2a25d66..3c7bd9b 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -74,6 +74,7 @@ void avcodec_register_all(void) initialized = 1; Nit: Whitespace on the line above should be removed. [...] --- /dev/null +++ b/libavcodec/vc264.c @@ -0,0 +1,387 @@ +/* H.264 hardware assist video encoding code taken from + * raspberry's os : + * /opt/vc/src/hello_pi/hello_encode/encode.c + */ + +/* +Copyright (c) 2012, Broadcom Europe Ltd +Copyright (c) 2012, Kalle Vahlman <zuh@iki> + Tuomas Kulve <tuo...@kulve.fi> +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: +* Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of the copyright holder nor the + names of its contributors may be used to endorse or promote products + derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. I wonder if any of the above legalese is compatible. Granted, I see a similar paragraph in "libavformat/aadec.c". + * ffmpeg driver for hardware assist video H.264 encoding using Broadcom's GPU + * Copyright (C) 2016 Amancio Hasty aha...@gmail.com + * + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + * + */ + + +/** + * @ file vc264.c + * Broadcom bm2865's Visual Core hardware assist h264 using + openMax interface to the GPU. + +*/ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#define OMX_SKIP64BIT +#include "bcm_host.h" +#include "ilclient.h" +#include "avcodec.h" +#include "internal.h" + +typedef struct VC264Context { + OMX_VIDEO_PARAM_PORTFORMATTYPE format; + OMX_PARAM_PORTDEFINITIONTYPE def; + COMPONENT_T *video_encode; + COMPONENT_T *list[5]; + OMX_BUFFERHEADERTYPE *buf; + OMX_BUFFERHEADERTYPE *out; + ILCLIENT_T *client; + OMX_VIDEO_PARAM_BITRATETYPE bitrateType; + int width; + int height; + int bit_rate; +} VC264Context; + + +static int vc264_init(AVCodecContext *avctx) { + + + + OMX_ERRORTYPE r; + int error; + + + + VC264Context *vc = avctx->priv_data; + + vc->width = avctx->width; + vc->height = avctx->height; + vc->bit_rate = avctx->bit_rate; + printf("vc264: bit rate %d \n", avctx->bit_rate); +#if FF_API_CODED_FRAME +FF_DISABLE_DEPRECATION_WARNINGS + + avctx->coded_frame = av_frame_alloc(); + avctx->coded_frame->pict_type = AV_PICTURE_TYPE_I; +FF_ENABLE_DEPRECATION_WARNINGS +#endif + + + memset(&vc->list, 0, sizeof(vc->list)); + bcm_host_init(); + if ((vc->client = ilclient_init()) == NULL) { + return -3; + } + error = OMX_Init(); + + if (error != OMX_ErrorNone) { + ilclient_destroy(vc->client); + av_log(avctx,AV_LOG_ERROR,"in vc264_init OMX_Init failed "); + return -4; + } + + // create video_encode + r = ilclient_create_component(vc->client, &vc->video_encode, (char *) "video_encode", + ILCLIENT_DISABLE_ALL_PORTS | + ILCLIENT_ENABLE_INPUT_BUFFERS | + ILCLIENT_ENABLE_OUTPUT_BUFFERS); Tabs should be converted to spaces. There are many instances of tabs being used in this patch. Others will have to provide a more technical review (not to mention possible additions docs, Changelog, MAINTAINERS, and libavcodec/version.h). _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel Any thoughts on how we may proceed forward? Thank You, Amancio _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel < http://ffmpeg.org/mailman/listinfo/ffmpeg-devel> This is what I getting over here with RPI3: ./ffmpeg_g -loglevel debug -f v4l2 -input_format yuv420p -framerate 25 -video_size 640x480 -i /dev/video0 output.h264 ffmpeg version N-80363-g403a53c Copyright (c) 2000-2016 the FFmpeg developers built with gcc 4.9.2 (Raspbian 4.9.2-10) configuration: --enable-gpl --enable-omx --enable-omx-rpi --enable-nonfree --extra-libs=-ldl --extra-cflags=' -pipe ' libavutil 55. 24.100 / 55. 24.100 libavcodec 57. 46.100 / 57. 46.100 libavformat 57. 38.100 / 57. 38.100 libavdevice 57. 0.101 / 57. 0.101 libavfilter 6. 46.101 / 6. 46.101 libswscale 4. 1.100 / 4. 1.100 libswresample 2. 1.100 / 2. 1.100 libpostproc 54. 0.100 / 54. 0.100 Splitting the commandline. Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'. Reading option '-f' ... matched as option 'f' (force format) with argument 'v4l2'. Reading option '-input_format' ... matched as AVOption 'input_format' with argument 'yuv420p'. Reading option '-framerate' ... matched as AVOption 'framerate' with argument '25'. Reading option '-video_size' ... matched as AVOption 'video_size' with argument '640x480'. Reading option '-i' ... matched as input file with argument '/dev/video0'. Reading option 'output.h264' ... matched as output file. Finished splitting the commandline. Parsing a group of options: global . Applying option loglevel (set logging level) with argument debug. Successfully parsed a group of options. Parsing a group of options: input file /dev/video0. Applying option f (force format) with argument v4l2. Successfully parsed a group of options. Opening an input file: /dev/video0. [video4linux2,v4l2 @ 0x2189260] fd:3 capabilities:85200005 [video4linux2,v4l2 @ 0x2189260] Current input_channel: 0, input_name: Camera 0, input_std: 0 [video4linux2,v4l2 @ 0x2189260] Setting time per frame to 1/25 [video4linux2,v4l2 @ 0x2189260] All info found Input #0, video4linux2,v4l2, from '/dev/video0': Duration: N/A, start: -140423796.316413, bitrate: 92160 kb/s Stream #0:0, 1, 1/1000000: Video: rawvideo, 1 reference frame (I420 / 0x30323449), yuv420p, 640x480, 0/1, 92160 kb/s, 25 fps, 25 tbr, 1000k tbn, 1000k tbc Successfully opened the file. Parsing a group of options: output file output.h264. Successfully parsed a group of options. Opening an output file: output.h264. [file @ 0x218c930] Setting default whitelist 'file,crypto' Successfully opened the file. detected 4 logical cores [graph 0 input from stream 0:0 @ 0x2195270] Setting 'video_size' to value '640x480' [graph 0 input from stream 0:0 @ 0x2195270] Setting 'pix_fmt' to value '0' [graph 0 input from stream 0:0 @ 0x2195270] Setting 'time_base' to value '1/1000000' [graph 0 input from stream 0:0 @ 0x2195270] Setting 'pixel_aspect' to value '0/1' [graph 0 input from stream 0:0 @ 0x2195270] Setting 'sws_param' to value 'flags=2' [graph 0 input from stream 0:0 @ 0x2195270] Setting 'frame_rate' to value '25/1' [graph 0 input from stream 0:0 @ 0x2195270] w:640 h:480 pixfmt:yuv420p tb:1/1000000 fr:25/1 sar:0/1 sws_param:flags=2 [format @ 0x2195530] compat: called with args=[yuv420p] [format @ 0x2195530] Setting 'pix_fmts' to value 'yuv420p' [AVFilterGraph @ 0x2194b40] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed [h264_omx @ 0x218c2c0] Using OMX.broadcom.video_encode [h264_omx @ 0x218c2c0] OMX state changed to 2 [h264_omx @ 0x218c2c0] OMX state changed to 3 [h264_omx @ 0x218c2c0] OMX port 201 settings changed [h264 @ 0x218b010] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead. Output #0, h264, to 'output.h264': Metadata: encoder : Lavf57.38.100 Stream #0:0, 0, 1/25: Video: h264 (h264_omx), 1 reference frame, yuv420p, 640x480, 0/1, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc Metadata: encoder : Lavc57.46.100 h264_omx Stream mapping: Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_omx)) Press [q] to stop, [?] for help cur_dts is invalid (this is harmless if it occurs once at the start per stream) [rawvideo @ 0x218ac30] PACKET SIZE: 460800, STRIDE: 960 Clipping frame in rate conversion by 0.000008 Assertion ret <= 0 failed at libavcodec/utils.c:1956 Aborted How did you test h264 encoding? Thanks! Amancio _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel Hello Aman, Thanks for the patch. I noticed that in libav/utils.c: ret = avctx->codec->encode2(avctx, avpkt, frame, got_packet_ptr); 1325 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1325> if (!ret) { 1326 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1326> if (*got_packet_ptr) { 1327 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1327> if (!(avctx->codec->capabilities & AV_CODEC_CAP_DELAY)) { 1328 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1328> if (avpkt->pts == AV_NOPTS_VALUE) 1329 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1329> avpkt->pts = frame->pts; 1330 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1330> if (!avpkt->duration) 1331 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1331> avpkt->duration = ff_samples_to_time_base(avctx, 1332 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1332> frame->nb_samples); 1333 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1333> } 1334 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1334> avpkt->dts = avpkt->pts; 1335 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1335> } else { 1336 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1336> avpkt->size = 0; 1337 <https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1337> } Which is different than ffmpeg’s libavcodec/utils.c …. Last but not least I get a bunch or warnings in ffmpeg when I run it with your patch. ./ffmpeg_g -f v4l2 -input_format yuv420p -framerate 25 -video_size 640x480 -i /dev/video0 a.h264 DTS 140462612887087, next:1760000 st:0 invalid dropping PTS 140462612887087, next:1760000 invalid dropping st:0 …. Thanks again! Amancio With respect to your patch this was posted : https://git.libav.org/?p=libav.git;a=commit;h=0c9c4004ed57de210b4d83c7b39bbfb00b86b9af It just deletes setting “ret”. I tried it over here and it seems to work. Thanks! Amancio _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
