On 11/15/23 22:45, Michael Niedermayer wrote: > On Tue, Nov 14, 2023 at 01:59:48PM +0100, Michael Riedl wrote: >> On 11/14/23 11:05, Tomas Härdin wrote: >>> This patchset is missing tests. I know that we for some reason don't >>> really have tests for protocols, but I feel the issue is worthwhile to >>> bring up. I've worked a bit with WebRTC recently and it's fiddly, so >>> it'd be nice to have some automated thing that keeps track of which >>> WebRTC implementations this works with. Or maybe this is better handled >>> with an external project, testing which implementations interoperate? >> >> I agree that having automated tests would be useful for stability in the >> future. >> I tested the patchset with both SRS and Millicast, and it worked well. >> >> For automated testing, we could use FATE, but it needs an extra server with >> protection and someone to keep it updated. Testing with paid services like >> Millicast is tricky. >> >> Another option is an external project for WebRTC testing, but the challenge >> is >> keeping it maintained and compatible with changes in implementations. >> >> External services might update their software, causing issues with tests. We >> would need a plan for dealing with that. >> >> For paid services like Millicast, we need to figure out who pays for the >> tests. >> Understanding the costs is essential if we want to use paid services for >> testing. >> >> I'd like to hear your thoughts on these points and how you would propose to >> proceed with adding tests for protocols like WebRTC. > simple, add server support for this to FFmpeg. > > FATE is run in cases without network access (for example googles ossfuzz setup > i belives does not permit the fuzzed code to access external things IIRC) > > The practice of implementing only one end of a protocol is honestly wrong. > And if there is no usable free server, then even more so. > > thx
Server support was planned for later, but maybe it's better to do it now. Multiple solutions are possible, but one solution requires adding 2 more (de)muxers for server support to this patch series. It would also be possible to split the patch series into 2 parts, one for WHIP client and server support and one for WHEP client and server support. But I'm not sure which solution is better. My idea is to add 2 more (de)muxers for server support. This would allow us to test the following: - WHIP muxer (client) --> WHIP demuxer (server) - WHEP muxer (server) --> WHEP demuxer (client) Regardless, this will only test the implementation against itself. If that makes sense and sounds reasonable, I will add the server support to this patch series. Please let me know what you think. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
