This series introduces support for the Qualcomm CAMSS Offline Processing
Engine (OPE), as found on Agatti-based platforms. Boards such as Arduino
UNO-Q use this SoC family and will benefit from hardware-assisted image
processing enabled by this work.
This represents the first step toward enabling image processing beyond
raw capture on Qualcomm platforms by using hardware blocks for
operations such as debayering, color correction, and scaling.
The OPE sits outside the live capture pipeline. It operates on frames
fetched from system memory and writes processed results back to memory.
Because of this design, the OPE is not tied to any specific capture
interface: frames may come from CAMSS RDI or PIX paths, or from any
other producer capable of providing memory-backed buffers.
The hardware can sustain up to 580 megapixels per second, which is
sufficient to process a 9MPix (4K) stream at 60 fps or to handle
four parallel 2MPix (HD) streams at 60 fps.
In order to enable reuse across future CAMSS ISP implementations,
a new camss-isp-common module is introduced, providing common
helpers for:
- buffer queue management
- job scheduling
- pipeline construction
- parameter parsing
While currently tailored for CAMSS ISP drivers, these helpers are
designed in a way that allows further generalization and extension
into the V4L2 ISP framework.
The driver is designed to support multiple processing contexts. At
present, however, only a single context is instantiated, which is
created on the first media-pipeline open. This keeps the implementation
simple while ensuring the design is ready for forthcoming multi-context
support at the V4L2/media framework level.
Since the OPE is a child node of the CAMSS device, the CAMSS driver
is reworked to properly share and manage power-related resources,
including power domains and common clocks.
Graph:
ope_input ----+
+--> ope_proc --> ope_disp --> ope_disp_output
ope_params ---+
Compliance: v4l2-compliance has been executed without any error on
the media and video devices.
Testing: A camss-isp-m2m tool is available at
https://github.com/loicpoulain/camss-isp-m2m-test
The tool can be used to exercise and stress-test the OPE engine,
as well as to observe performance and latency behavior under
various operating conditions (resolution, format, buf count, etc).
./camss-isp-m2m -s 3840x2160 -n 5
Name Device Dir Type
--------------------------------------------------------------
ope_input /dev/video10 output video
ope_params /dev/video11 output meta
ope_disp_output /dev/video12 capture video
ope_proc /dev/v4l-subdev13 - subdev
ope_disp /dev/v4l-subdev14 - subdev
Test: 3840x2160 RGGB -> 3840x2160 NV12 [5 frames]
Input /dev/video10 3840x2160 RGGB bpl=3840 size=8294400
Output /dev/video12 3840x2160 NV12 bpl=5760 size=12441600
Buffers:
/dev/video10 buf[0] offset=0x00000000 length=8294400 VA=0xffffbcb87000
/dev/video11 buf[0] offset=0x00000000 length=12441600 VA=0xffffbbfa9000
Streaming 5 frames...
seq=0 buf: in=0 out=0 14.833 ms (67.4 fps)
seq=1 buf: in=0 out=0 14.874 ms (67.2 fps)
seq=2 buf: in=0 out=0 14.902 ms (67.1 fps)
seq=3 buf: in=0 out=0 14.960 ms (66.8 fps)
seq=4 buf: in=0 out=0 14.896 ms (67.1 fps)
Changes in v3:
- Simplified camss_init_pm_clks()- no clk_get, no rate setting;
- Added Assigned clock rate in devicetree for CAMSS AXI clock
- Add Kernel Doc for camss-isp-bufq exported functions
- Added description explaining the 2-cell for CAMSS bus
- Reworked OPE media topology with intermediate ope_disp subdev
- Support for crop and compose in ope_proc sink/source
- Fixed Write Engine violation with some scaling config
- Fixed input/output custom-stride issue (Hans)
- Split uapi header (camss-config.h) into its own dedicated commit
- Patch reordering to respect dependency in regards to ABI/dt-bindings
introduction
- Link to v2:
https://lore.kernel.org/r/[email protected]
Changes in v2:
- Not an RFC anymore
- Move from v4l2-m2m to multi-devices design
- Support for parameters
- Helpers for pipeline, job scheduling, buf-queues
- OPE as a child node of CAMSS
---
Bryan O'Donoghue (1):
media: qcom: camss: Populate CAMSS child devices via DT
Loic Poulain (14):
media: qcom: camss: Add PM clock support and integrate with runtime PM
media: qcom: camss: Add PM clock definitions for QCM2290
media: qcom: camss: Drop top_ahb/axi from QCM2290 subdevice clocks
media: qcom: camss: Add camss-isp-bufq helper
media: qcom: camss: Add camss-isp-sched helper
media: qcom: camss: Add camss-isp-pipeline helper
media: qcom: camss: Add V4L2 meta format for CAMSS ISP parameters
media: qcom: camss: Add camss-isp-params helper
dt-bindings: media: qcom: Add CAMSS Offline Processing Engine (OPE)
dt-bindings: media: qcom,qcm2290-camss: Add OPE ISP subnode
media: uapi: Add CAMSS ISP configuration definition
media: qcom: camss: Add CAMSS Offline Processing Engine driver
arm64: dts: qcom: agatti: Assigned clock rate for CAMSS AXI
arm64: dts: qcom: agatti: Add OPE node
.../bindings/media/qcom,qcm2290-camss-ope.yaml | 131 +
.../bindings/media/qcom,qcm2290-camss.yaml | 13 +
arch/arm64/boot/dts/qcom/agatti.dtsi | 73 +
drivers/media/platform/qcom/camss/Kconfig | 32 +
drivers/media/platform/qcom/camss/Makefile | 11 +
drivers/media/platform/qcom/camss/camss-isp-bufq.c | 101 +
drivers/media/platform/qcom/camss/camss-isp-bufq.h | 122 +
drivers/media/platform/qcom/camss/camss-isp-ope.c | 3025 ++++++++++++++++++++
.../media/platform/qcom/camss/camss-isp-params.c | 67 +
.../media/platform/qcom/camss/camss-isp-params.h | 62 +
.../media/platform/qcom/camss/camss-isp-pipeline.c | 372 +++
.../media/platform/qcom/camss/camss-isp-pipeline.h | 230 ++
.../media/platform/qcom/camss/camss-isp-sched.c | 223 ++
.../media/platform/qcom/camss/camss-isp-sched.h | 174 ++
drivers/media/platform/qcom/camss/camss.c | 67 +-
drivers/media/platform/qcom/camss/camss.h | 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/camss-config.h | 115 +
include/uapi/linux/videodev2.h | 3 +
19 files changed, 4807 insertions(+), 16 deletions(-)
---
base-commit: cb49dcae0241fb3ea59f42ce2edd69367784b51c
change-id: 20260427-camss-isp-ope-4b575ffec2ed
Best regards,
--
Loic Poulain <[email protected]>
