[...] On Wed, Jul 8, 2020 at 7:42 PM James Almer <jamr...@gmail.com> wrote: > The sum of all sizes should be size_t if each size is a size_t. But if > you do that you can't return error values, so i'm not sure what to > suggest other than just stick to ints for both (ptrdiff_t linesizes > should be ok). > I'd like to hear Michael's opinion about it. > > For that matter, as it is right now i think this would be the first > function that returns ptrdiff_t to signal AVERROR values.
Yeah, I wasn't sure what the best thing to do here would be since it seemed like switching back to ints for the outputs would make the ptrdiff_t linesizes less useful. We could have each size be a ptrdiff_t as well, but that seems a bit weird too. There is one function in libavformat (ff_subtitles_read_line) that returns a negative error ptrdiff_t to signal errors. [...] > > - if (linesizes[0] > (INT_MAX - 1024) / height) > > + if (linesizes[0] > (PTRDIFF_MAX - 1024) / height) > > This check would need to ensure the calculation below fits in a size_t > instead. If the return type is ptrdiff_t, then I guess we would need to check both. I had thought ptrdiff_t was the corresponding signed type for size_t (so it would be guaranteed to have a lower max), but I guess that isn't guaranteed? _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
