Hello, I'm experimenting with an integration between OpenCV and Pharo based on uFFI. My native C library opens a video camera and captures frames for detecting certain points (aka blobs) of interest for my use case. The data for these frames are available to Pharo as uchar* inside a C struct.
My issues came when I tried to render the video camera frames inside Pharo. The C examples I developed for testing the shared library show frames around 50-60 fps. However, the best I have got in Pharo is around 15-20 fps. My first try consisted of writing an specific ImageReadWriter for the OpenCV image format, so I can build a Form for each frame and render it with Morphic. This way I can render the video camera in GTInspector, but at a terrible 5-10 fps. The second try was to use Bloc to render the Forms. I though the Moz2D backend will provider better performance. Indeed, I got around 15-20 fps, but it is still insufficient for playing video smoothly. My current try consist of using BlExternalForm to avoid all the frame data processing inside Pharo, but I'm finding problems to render this kind of "shallow" Forms using Bloc. I've also find some old mails (2017) about Clément Bera working in games based on Cairo + SDL. Should I explore this approach for video rendering? In summary, I don't know what is the recommended approach (if any) to show a sequence of video frames with decent bit-rate. Thank you!
