From: Harry Wentland <harry.wentl...@amd.com>
Add documentation for color pipeline API.
Signed-off-by: Alex Hung <alex.h...@amd.com>
Signed-off-by: Harry Wentland <harry.wentl...@amd.com>
---
v8:
- Fix typo "definint" -> "defining"
v7:
- Add a commit messages
v5:
- Don't require BYPASS to succeed (Sebastian)
- use DATA for 1D and 3D LUT types (Sebastian)
- update 3DLUT ops to use 3DLUT_MODES and 3DLUT_MODE_INDEX
- Add section on drm_colorop extensibility
- Add color_pipeline.rst to RFC toc tree
v4:
- Drop IOCTL docs since we dropped the IOCTLs (Pekka)
- Clarify reading and setting of COLOR_PIPELINE prop (Pekka)
- Add blurb about not requiring to reject a pipeline due to
incompatible ops, as long as op can be bypassed (Pekka)
- Dropped informational strings (such as input CSC) as they're
not actually intended to be advertised (Pekka)
v3:
- Describe DRM_CLIENT_CAP_PLANE_COLOR_PIPELINE (Sebastian)
- Ask for clear documentation of colorop behavior (Sebastian)
v2:
- Update colorop visualizations to match reality (Sebastian, Alex Hung)
- Updated wording (Pekka)
- Change BYPASS wording to make it non-mandatory (Sebastian)
- Drop cover-letter-like paragraph from COLOR_PIPELINE Plane Property
section (Pekka)
- Use PQ EOTF instead of its inverse in Pipeline Programming example
(Melissa)
- Add "Driver Implementer's Guide" section (Pekka)
- Add "Driver Forward/Backward Compatibility" section (Sebastian,
Pekka)
Documentation/gpu/rfc/color_pipeline.rst | 378 +++++++++++++++++++++++
Documentation/gpu/rfc/index.rst | 3 +
2 files changed, 381 insertions(+)
create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
diff --git a/Documentation/gpu/rfc/color_pipeline.rst b/Documentation/
gpu/rfc/color_pipeline.rst
new file mode 100644
index 000000000000..58bcc2a5ffd8
--- /dev/null
+++ b/Documentation/gpu/rfc/color_pipeline.rst
@@ -0,0 +1,378 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+========================
+Linux Color Pipeline API
+========================
+
+What problem are we solving?
+============================
+
+We would like to support pre-, and post-blending complex color
+transformations in display controller hardware in order to allow for
+HW-supported HDR use-cases, as well as to provide support to
+color-managed applications, such as video or image editors.
+
+It is possible to support an HDR output on HW supporting the Colorspace
+and HDR Metadata drm_connector properties, but that requires the
+compositor or application to render and compose the content into one
+final buffer intended for display. Doing so is costly.
+
+Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and
other
+operations to support color transformations. These operations are often
+implemented in fixed-function HW and therefore much more power
efficient than
+performing similar operations via shaders or CPU.
+
+We would like to make use of this HW functionality to support complex
color
+transformations with no, or minimal CPU or shader load.
+
+
+How are other OSes solving this problem?
+========================================
+
+The most widely supported use-cases regard HDR content, whether video or
+gaming.
+
+Most OSes will specify the source content format (color gamut,
encoding transfer
+function, and other metadata, such as max and average light levels)
to a driver.
+Drivers will then program their fixed-function HW accordingly to map
from a
+source content buffer's space to a display's space.
+
+When fixed-function HW is not available the compositor will assemble
a shader to
+ask the GPU to perform the transformation from the source content
format to the
+display's format.
+
+A compositor's mapping function and a driver's mapping function are
usually
+entirely separate concepts. On OSes where a HW vendor has no insight
into
+closed-source compositor code such a vendor will tune their color
management
+code to visually match the compositor's. On other OSes, where both
mapping
+functions are open to an implementer they will ensure both mappings
match.
+
+This results in mapping algorithm lock-in, meaning that no-one alone can
+experiment with or introduce new mapping algorithms and achieve
+consistent results regardless of which implementation path is taken.
+
+Why is Linux different?
+=======================
+
+Unlike other OSes, where there is one compositor for one or more
drivers, on
+Linux we have a many-to-many relationship. Many compositors; many
drivers.
+In addition each compositor vendor or community has their own view of
how
+color management should be done. This is what makes Linux so beautiful.
+
+This means that a HW vendor can now no longer tune their driver to one
+compositor, as tuning it to one could make it look fairly different from
+another compositor's color mapping.
+
+We need a better solution.
+
+
+Descriptive API
+===============
+
+An API that describes the source and destination colorspaces is a
descriptive
+API. It describes the input and output color spaces but does not
describe
+how precisely they should be mapped. Such a mapping includes many minute
+design decision that can greatly affect the look of the final result.
+
+It is not feasible to describe such mapping with enough detail to
ensure the
+same result from each implementation. In fact, these mappings are a
very active
+research area.
+
+
+Prescriptive API
+================
+
+A prescriptive API describes not the source and destination
colorspaces. It
+instead prescribes a recipe for how to manipulate pixel values to
arrive at the
+desired outcome.
+
+This recipe is generally an ordered list of straight-forward operations,
+with clear mathematical definitions, such as 1D LUTs, 3D LUTs, matrices,
+or other operations that can be described in a precise manner.
+
+
+The Color Pipeline API
+======================
+
+HW color management pipelines can significantly differ between HW
+vendors in terms of availability, ordering, and capabilities of HW
+blocks. This makes a common definition of color management blocks and
+their ordering nigh impossible. Instead we are defining an API that
+allows user space to discover the HW capabilities in a generic manner,
+agnostic of specific drivers and hardware.
+
+
+drm_colorop Object
+==================
+
+To support the definition of color pipelines we define the DRM core
+object type drm_colorop. Individual drm_colorop objects will be chained
+via the NEXT property of a drm_colorop to constitute a color pipeline.
+Each drm_colorop object is unique, i.e., even if multiple color
+pipelines have the same operation they won't share the same drm_colorop
+object to describe that operation.
+
+Note that drivers are not expected to map drm_colorop objects statically
+to specific HW blocks. The mapping of drm_colorop objects is entirely a
+driver-internal detail and can be as dynamic or static as a driver needs
+it to be. See more in the Driver Implementation Guide section below.
+
+Each drm_colorop has three core properties:
+
+TYPE: An enumeration property, defining the type of transformation,
such as
+* enumerated curve
+* custom (uniform) 1D LUT
+* 3x3 matrix
+* 3x4 matrix
+* 3D LUT
+* etc.
+
+Depending on the type of transformation other properties will describe
+more details.
+
+BYPASS: A boolean property that can be used to easily put a block into
+bypass mode. The BYPASS property is not mandatory for a colorop, as long
+as the entire pipeline can get bypassed by setting the COLOR_PIPELINE on
+a plane to '0'.
+
+NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
+drm_colorop is the last in the chain.
+
+An example of a drm_colorop object might look like one of these::
+
+ /* 1D enumerated curve */
+ Color operation 42
+ ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3
matrix, 3x4 matrix, 3D LUT, etc.} = 1D enumerated curve
+ ├─ "BYPASS": bool {true, false}
+ ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, sRGB inverse EOTF, PQ EOTF,
PQ inverse EOTF, …}
+ └─ "NEXT": immutable color operation ID = 43
+
+ /* custom 4k entry 1D LUT */
+ Color operation 52
+ ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3
matrix, 3x4 matrix, 3D LUT, etc.} = 1D LUT
+ ├─ "BYPASS": bool {true, false}
+ ├─ "SIZE": immutable range = 4096
+ ├─ "DATA": blob
+ └─ "NEXT": immutable color operation ID = 0
+
+ /* 17^3 3D LUT */
+ Color operation 72
+ ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3
matrix, 3x4 matrix, 3D LUT, etc.} = 3D LUT
+ ├─ "BYPASS": bool {true, false}
+ ├─ "3DLUT_MODES": read-only blob of supported 3DLUT modes
+ ├─ "3DLUT_MODE_INDEX": index of selected 3DLUT mode
+ ├─ "DATA": blob
+ └─ "NEXT": immutable color operation ID = 73
+
+drm_colorop extensibility
+-------------------------
+
+Unlike existing DRM core objects, like &drm_plane, drm_colorop is not
+extensible. This simplifies implementations and keeps all functionality
+for managing &drm_colorop objects in the DRM core.
+
+If there is a need one may introduce a simple &drm_colorop_funcs
+function table in the future, for example to support an IN_FORMATS
+property on a &drm_colorop.
+
+If a driver requires the ability to create a driver-specific colorop
+object they will need to add &drm_colorop func table support with
+support for the usual functions, like destroy, atomic_duplicate_state,
+and atomic_destroy_state.
