Need help from ubuntu-sponsors. ** Description changed:
+ [SRU Justification] + + [Impact] + + On a MIPI camera through Intel IPU6 platform that its raw V4L2 loopback + interface is preserved for Intel Camera HAL libraries, a relay daemon + + v4l2loopback is used to allow the usage of legacy V4L2 based apps. + + By design, the relayd will open v4l2sink to v4l2loopback OUTPUT deivce, + and it will only open libcamhal based GStreamer source element, and + therefore underlying camera hardware, until received new client + notifications via V4L2 Event API that is introduced in this SRU. + + Besides, frame sizes/intervals enumeration is also fixed to meet better + compliance with user apps. + + [Test Plan] + + v4l2loopback doesn't support V4L2 Event API until recently, so requests + for the usage will always fail: + + struct v4l2_event_subscription sub; + int fd; + + memset (&sub, 0, sizeof (sub)); + sub.type = ...; + if (ioctl (fd, VIDIOC_SUBSCRIBE_EVENT, &sub) == 0) // fail + + With this fix, it shall support Event API operations, e.g. + VIDIOC_SUBSCRIBE_EVENT, VIDIOC_UNSUBSCRIBE_EVENT, VIDIOC_DQEVENT. + + [Where problems could occur] + + While a custom type of V4L2 event is now registered for OEM projects, + software expected same ID (numerically equivalent to + V4L2_EVENT_PRIVATE_START) may get confused. While this falls in the + private usage section, it's not supposed to be used in general, and when + it does, it should be under some presumptions, e.g. selected hardware, + so it's unlikely to happen on OEM projects. + + For generic Ubuntu, programs may begin to take advantage of this new + capability to update its UI, or to take other actions after receiving + desired events. There might be behavior changes, but should be under the + original design if was done carefully. + + [Other Info] + + For Focal backports, 0.12.5-1 is equivalent to 0.12.3-1ubuntu0.3 plus + micro version updates, so there should be little risk backport a new + release. + + ========= original bug report ========== + In addition to kernel/firmware proposed in bug 1921345, several user space daemons/libraries, as well as fixes/features in v4l2loopback-dkms, are also required to have seamless support for legacy/existing Linux kernel V4L2 API based applications to adopt libcamera or Intel libcamhal provided camera interfaces. The idea is to add a v4l2 streaming relay daemon (currently developed in [1] and packaged in [2]) that helps redirecting V4L2 buffer streams into v4l2loopback output device, and then legacy apps open v4l2loopback capture device in the ordinary way. - hw -> libcamera/libcamhal -> v4l2-relayd -> v4l2loopback -> v4l2-based + hw -> libcamera/libcamhal -> v4l2-relayd -> v4l2loopback -> v4l2-based apps To achieve this, we'd like v4l2-relayd to only open GStreamer icamerasrc pipeline (provided by [3],[4]) when there is actual usage request from v4l2-based apps for privacy and power consumption's concerns. However, current (0.12.5 and therefore Ubuntu/Debian 0.12.5-1) doesn't have any available mechanism for this, therefore a V4L2 Event API based proposal has been sent and accepted by upstream[5][6]. This is needed for Ubuntu OEM projects with MIPI cameras through Intel IPU6 (Imaging Processing Unit version 6). [1]: https://gitlab.com/vicamo/v4l2-relayd [2]: https://code.launchpad.net/~oem-solutions-engineers/v4l2-relayd/+git/packaging [3]: https://github.com/intel/ipu6-camera-hal [4]: https://github.com/intel/icamerasrc [5]: https://github.com/umlaeute/v4l2loopback/pull/345 [6]: https://github.com/umlaeute/v4l2loopback/pull/352 ** Description changed: [SRU Justification] [Impact] On a MIPI camera through Intel IPU6 platform that its raw V4L2 loopback interface is preserved for Intel Camera HAL libraries, a relay daemon + v4l2loopback is used to allow the usage of legacy V4L2 based apps. By design, the relayd will open v4l2sink to v4l2loopback OUTPUT deivce, and it will only open libcamhal based GStreamer source element, and therefore underlying camera hardware, until received new client notifications via V4L2 Event API that is introduced in this SRU. Besides, frame sizes/intervals enumeration is also fixed to meet better compliance with user apps. [Test Plan] v4l2loopback doesn't support V4L2 Event API until recently, so requests for the usage will always fail: - struct v4l2_event_subscription sub; - int fd; + struct v4l2_event_subscription sub; + int fd; - memset (&sub, 0, sizeof (sub)); - sub.type = ...; - if (ioctl (fd, VIDIOC_SUBSCRIBE_EVENT, &sub) == 0) // fail + memset (&sub, 0, sizeof (sub)); + sub.type = ...; + if (ioctl (fd, VIDIOC_SUBSCRIBE_EVENT, &sub) == 0) // fail With this fix, it shall support Event API operations, e.g. VIDIOC_SUBSCRIBE_EVENT, VIDIOC_UNSUBSCRIBE_EVENT, VIDIOC_DQEVENT. [Where problems could occur] While a custom type of V4L2 event is now registered for OEM projects, software expected same ID (numerically equivalent to V4L2_EVENT_PRIVATE_START) may get confused. While this falls in the private usage section, it's not supposed to be used in general, and when it does, it should be under some presumptions, e.g. selected hardware, so it's unlikely to happen on OEM projects. For generic Ubuntu, programs may begin to take advantage of this new capability to update its UI, or to take other actions after receiving desired events. There might be behavior changes, but should be under the original design if was done carefully. [Other Info] For Focal backports, 0.12.5-1 is equivalent to 0.12.3-1ubuntu0.3 plus micro version updates, so there should be little risk backport a new - release. + release, and we can drop additional patches carried. This implies bug + 1905613. ========= original bug report ========== In addition to kernel/firmware proposed in bug 1921345, several user space daemons/libraries, as well as fixes/features in v4l2loopback-dkms, are also required to have seamless support for legacy/existing Linux kernel V4L2 API based applications to adopt libcamera or Intel libcamhal provided camera interfaces. The idea is to add a v4l2 streaming relay daemon (currently developed in [1] and packaged in [2]) that helps redirecting V4L2 buffer streams into v4l2loopback output device, and then legacy apps open v4l2loopback capture device in the ordinary way. hw -> libcamera/libcamhal -> v4l2-relayd -> v4l2loopback -> v4l2-based apps To achieve this, we'd like v4l2-relayd to only open GStreamer icamerasrc pipeline (provided by [3],[4]) when there is actual usage request from v4l2-based apps for privacy and power consumption's concerns. However, current (0.12.5 and therefore Ubuntu/Debian 0.12.5-1) doesn't have any available mechanism for this, therefore a V4L2 Event API based proposal has been sent and accepted by upstream[5][6]. This is needed for Ubuntu OEM projects with MIPI cameras through Intel IPU6 (Imaging Processing Unit version 6). [1]: https://gitlab.com/vicamo/v4l2-relayd [2]: https://code.launchpad.net/~oem-solutions-engineers/v4l2-relayd/+git/packaging [3]: https://github.com/intel/ipu6-camera-hal [4]: https://github.com/intel/icamerasrc [5]: https://github.com/umlaeute/v4l2loopback/pull/345 [6]: https://github.com/umlaeute/v4l2loopback/pull/352 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1921474 Title: Support client usage notification via V4l2 Event API To manage notifications about this bug go to: https://bugs.launchpad.net/hwe-next/+bug/1921474/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
