On Mon, Nov 25, 2013 at 7:51 AM, Christopher James Halse Rogers <r...@ubuntu.com> wrote: > One of the architectural things that I want to get done at the sprint > next week is a solid idea of how we want to do nested > compositing/out-of-process plugins/subsurfaces - which all seem to me to > be aspects of the same problem. >
There is a security relevant issue here, too. Trusted helpers as well as input methods should have a way to "compose" UIs across process boundaries, too. See https://wiki.ubuntu.com/Security/TrustStoreAndSessions for an introduction to the problem. > In order to prime the discussion - and to invite outside contributions - > I thought I'd lay out the usecases as I see them before we get to > London. > > There are two conceptual use-cases here - > 1) “I want to delegate some of my UI to a third party”, and > 2) “I need to do some compositing, and want to do this efficiently” > > A Unity8 session running under unity-system-compositor falls under (2). > > A video player playing a YUV stream that might also want to throw some > RGB-rendered UI over it is also (2); a video player that has some chrome > around a video widget is (1) and (2). > > The “embed bits of other applications in our window” requested on > https://bugs.launchpad.net/mir/+bug/1230091 is firmly in (2). > > As I see it, there are also two classes of problem: > a) How is the rendering loop coordinated between parent and child - does > the parent need to swap each time the child has new rendering, or can > the child swap independently, or are both modes available? What happens > if a child also wishes to embed children of its own? > > This is the only concern for the type (2) use-case. > > b) How is input handled? Does the parent need to proxy input events and > forward them on? How does enter/leave notification work for the parent > and child? Can the child return events to the parent? Etc. > > This is necessary for the type (1) use-cases, and also seems to be the > hairy bit. > > Weston (but not yet Wayland) currently partially solves (2) with the > subsurfaces protocol, which has chosen the “no child rendering appears > until the parent swaps” approach, and doesn't handle out of process > renderers at all. > > For full out-of-process rendering, and for type-1 usecases, the my > understanding of the current state of the art is that the parent should > become a Wayland compositor itself. This seems a bit of a co-out to me, > and doesn't really solve case 2; however, this area is gnarly, so it > might prove to be the best solution. > >From my pov, for out-of-process rendering approaches, the rendering and input handling should be opaque to top/higher-level compositors, with a compositor only requiring knowledge of its direct children. However, I would think that we should avoid forcing a compositor model on applications like chrome. They already have app-local compositors running, together with event handling. What we should offer (and commit to) is a way to stream pixel data across process boundaries. Candidate solutions could be: (1.) http://www.khronos.org/registry/gles/extensions/OES/OES_EGL_image_external.txt: We use this extension today on Ubuntu Touch for streaming data from decoded video and from the builtin cameras. (2.) EGLStream with a texture sink (somewhat in the future). For convenience, we could see if we could offer applications a way to become a Mir compositor (ideally by leveraging Gerry's work on scenegraph integration). But as a bare minimum, I would think that a well-defined of streaming textures solves the immediate use-cases for applications that already have an app-local compositor. > For X, the relevant prior-art is XEMBED¹, in all of its > map-an-invisible-1x1-window-to-work-around-X11's-focus-model glory. > Ouch, that hurts :) And this is one of the reason why the compositor hierarchy should be opaque from my POV. Cheers, Thomas > ¹: http://standards.freedesktop.org/xembed-spec/xembed-spec-latest.html > > -- > Mir-devel mailing list > Mir-devel@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/mir-devel > -- Mir-devel mailing list Mir-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/mir-devel
