diff options
author | Vinod Ahuja <vahuja@unomaha.edu> | 2022-11-19 13:59:20 -0600 |
---|---|---|
committer | Jan-Simon Moeller <jsmoeller@linuxfoundation.org> | 2022-11-21 11:21:04 +0000 |
commit | 120a2677992ea299eea5fb5cb0ed1081f76bb92c (patch) | |
tree | f9cc1835cc14536f0397d3cfbbae744695d42bf1 /docs/6_Component_Documentation/2_agl-compositor.md | |
parent | 33727f4e9619f9da65fdfc608a10a92887c7257c (diff) |
Updating File name and links in the documentationoctopus_14.92.0octopus_14.91.0octopus/14.92.0octopus/14.91.014.92.014.91.0
Updating documentation:
1. Renamed all the files and folder following 01-99 index numbering
2. Matched title name and file name
3. Inserted title in the files where ever missing
4. Updated all the reference links in the documentation
Bug-AGL: [SPEC-4470]
Signed-off-by: Vinod Ahuja <vahuja@unomaha.edu>
Change-Id: Ib73e1f91107646e11b5b19ab6ead4080add506e4
Reviewed-on: https://gerrit.automotivelinux.org/gerrit/c/AGL/documentation/+/28170
Reviewed-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Tested-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Diffstat (limited to 'docs/6_Component_Documentation/2_agl-compositor.md')
-rw-r--r-- | docs/6_Component_Documentation/2_agl-compositor.md | 498 |
1 files changed, 0 insertions, 498 deletions
diff --git a/docs/6_Component_Documentation/2_agl-compositor.md b/docs/6_Component_Documentation/2_agl-compositor.md deleted file mode 100644 index 437e6a7..0000000 --- a/docs/6_Component_Documentation/2_agl-compositor.md +++ /dev/null @@ -1,498 +0,0 @@ ---- -title: agl-compositor ---- - -# Wayland compositor - -When the AGL project was started, weston was chosen as the compositor, which is -the reference implementation of a Wayland compositor, while for window management -functionality it relied on *ivi-shell* (In-Vehicle Infotainment) together -with an extension, called [wayland-ivi-exension](https://github.com/GENIVI/wayland-ivi-extension). - -A demo platform image of AGL comes with a handful of demo applications, done -with the Qt, which abstracts the protocol communication between the client and -the compositor. Additional functionality was in place under the form of -library, to control and signal back to the compositor when applications were -started, among other things. - -Management of applications, starting, running and stopping them is done in AGL -with AppFW [Application Framework Management](../3_Developer_Guides/1_Application_Framework/1_Introduction.md), -which is an umbrella name to denote the suite of tools and daemons that handle -all of that. It is integrated with systemd and with the current security model. -Applications can use AppFW to hang off data, and to pass it down to -other services. Together with AppFW, applications could tell the compositor -which application to activate or to switch to. - - -## Simplifying the graphical stack - -Trimming down these abstractions, simplifying the way clients interact with the -compositor, and avoid using modules that aren't really maintained upstream were -the reasons behind looking at alternatives to ivi-shell. On the desktop, -[xdg-shell](https://gitlab.freedesktop.org/wayland/wayland-protocols/-/raw/master/stable/xdg-shell/xdg-shell.xml) -is currently de-facto protocol for handling all window management related -functionality. - -Wayland protocol has a window-like interface embedded into its protocol (called -wl_shell), but *xdg-shell* has long time ago deprecated it and instead -of adding it in the wayland protocol namespace, it was integrated -together with some other useful protocols, into -[wayland-protocols](https://gitlab.freedesktop.org/wayland/wayland-protocols) -project. The whole purpose of wayland-protocols is to enhance the Wayland -protocol with new functionality and bring new extensions entirely. Compositors -are free to implement, modify, enhance, and add new extensions to -wayland-protocols but they need to do so in consensus. - -Besides the core wayland protocol and extended functionality from -wayland-protocols, a compositor can provide and implement additional protocol -extensions (custom to that compositor). By using such private extensions we -align with the AGL project and its requirements, without compromising specific -functionality and allows to add or improve the current ones. With that in mind, -the approach was to create a new compositor, called -[agl-compositor](https://gerrit.automotivelinux.org/gerrit/admin/repos/src/agl-compositor) -and implement dedicated private extensions, rather than trying to modify weston -itself, which AGL project would have been required to keep and maintain for -itself, as a fork. - -## A compositor based on libweston - -The compositor used currently in AGL, just like weston, is built on top of -*libweston* and *libweston-desktop*. The latter, among other things, is required -as it provides the server side implementation of the xdg-shell protocol which -underlying toolkits (like Qt/Chromium project) makes use of to deliver -desktop-like functionality. The former is used to provide back-ends and -rendering support, effectively managing the HW, besides implementing the -wayland protocol. - -The high-level goal of [libweston](https://wayland.pages.freedesktop.org/weston/toc/libweston.html) is -to decouple the compositor from the shell implementation. - -Traditionally, clients were entirely separated from the window manager, the -desktop environment and the display server. In wayland all these are -conceptually under the same entity though they are implemented as different -(UNIX) processes, or as a different namespaces with front and back-end APIs, -exposed by libraries. The compositor and the shell driving the UI should be -seen as one and the same, and in practice, this happens on desktop -environments. For AGL, the shell client can be represented under different -forms, as well as the fact that the process management has another layer -baked-in to handle MAC (Mandatory Access Control) labels and use the -above-mentioned Application Framework. These are all tightly -integrated and therefore, the AGL compositor will not automatically start the -shell client, although there's code to handle that. - -## Specifying a shell client to be started by the compositor - -Nevertheless, one can modify the configuration file, add the shell client path, and the -compositor will attempt to start it. - -``` -[shell-client] -command=/path/to/your/client/shell -``` - - - -## Private extensions - -Compositors can define and implement custom extensions to further control -application behaviour. For AGL, we have two private extensions defined. -One targeted at defining surface roles commonly found in desktop environments -(like panels, and backgrounds), which a shell client would bind to, and one -targeted at regular application(s) that might require additional functionality: -being able to display/activate its own surface or other's application surface, -implement some kind of split screen management of windows, or -dialog/pop-ups that exhibit always-on-top property even if the active -surface has been changed. - -![Layers_And_Extensions](images/agl-compositor/drawing_shell.png) - -Clients can make use of these private extensions to define other kind of roles -for instance dialog/pop-ups or full-screen roles, and split windows vertically or -horizontally. It includes the ability to activate other applications, assuming -that the surfaces have been created, and the capability of delaying -presentation for the client shell. Doing so, all the information is displayed -at once, rather than waiting for the toolkit to map/show the surface. - -An application identification mechanism was required to be able to activate -other clients windows/surfaces. A string-based identifier name was chosen -which can be used by the client to set an application-based identifier using -the xdg-shell protocol. While there's nothing stopping the client to avoid -doing that, specifically, to avoid assigning an application identifier, -the compositor won't be able to find which surfaces matches to a particular -client, if one would want to activate/display it at some time in the future. - -### agl-shell - -Client shellls can make use of this protocol to define panels and background -roles for different surfaces. It includes to ability to activate other -applications, assuming that those are already running. Activation happens by -using using the app_id, respectively using set_app_id request as defined by the -xdg-shell protocol. Established client-side implementation of the xdg-shelll -protocol will have a function exposed which can be used to set an application -identifier. Further more, the compositor will not present/display anything to -the user as long the `ready()` is not requested. So, after creating the surfaces -assigning them panel and/or background roles, and they're fully loaded, -the client can then issue `ready()` request and the compositor will start -presenting. - -Please consult the [protocol file](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-compositor.git;a=blob_plain;f=protocol/agl-shell.xml;hb=refs/heads/master) -as that is the authoritative way of getting the latest version. - -#### V2 updates - -Version 2 of the agl-shell protocol, while it is is not obligatory to be -supported by the shell client, it adds two new events: bound_ok and bound_fail -events. - -It has been observed that in some cases where we do not explicitly have a knob -in the toolkit to tell whether the application is a regular one (which doesn't -need to bind to the agl-shell protocol) or a one that needs to implement -agl-shell protocol might result in terminating the wayland connection. - -That happens because we can't have multiple clients bind to the agl-shell -protocol interface and was particularly visible when using regular -flutter applications with other shell clients (Qt homescreen, or WAM/chromum), -basically mashing together different kind of toolkits in the same image. Once -a client has already bound to the agl-shell protocol interface any other client -that attempts to do same will get its wayland connection severed and the -application will be terminated. - -These two events provide a race-free method in which the clients can tell if -they're in charge (of being the shell client) or their just regular -applications. Explicitly implementing this protocol if you have other means to -specify which type of application it is running wouldn't be required nor -necessary. But by using the protocol we can provide the same thing, -programmatically, without fearing that the wayland connection might be -severed, and implicitly taking down the application. - -#### V3 updates - -Version 3 of the agl-shell protocol adds 4 more events to signal out when the -application state was changed: started, activated, deactivated and terminated. - -Version 3 update was mostly prompted by an issue with start-up of applications -but also is part of the first steps to reduce and simplify a bit more -activation handling in the compositor. Specifically with this protocol update, -we can correctly orchestrate start-up and activation of applications. - -At the moment of adding this protocol update, the default compositor behaviour -is to display/activate applications as soon they're started, a feature which -we've called activate-by-default (and which is turned on by default). -But users can choose to disable that in such a way that activation is entirely -driven the shell client. - -Implicitly having this activate-by-default papered over various -issue when don't have that activation by default turned on. Supporting both -use-cases (activate-by-default, on and off) turned out to be cluster of -problems and regression over time. Not only that the amount of complexity in -the compositor is unwarranted and can simplified by telling the shell client -handle any window management interaction on its own. - -Further more, with this protocol update we also includes some events already -present in the agl-shell-desktop protocol like deactivate and terminate. - -### agl-shell-desktop - -This extension is targeted at keeping some of the functionally already -established in AGL as to a) allow applications display/activate other -surfaces/application window, and b) set further roles, specially dialog/window -pop-ups and split-type of surfaces. - -Clients can make use of this protocol to set further roles, like independently -positioned pop-up dialog windows, split type of surfaces or fullscreen ones. -Additional roles, and implicitly functionality can be added by extending the -protocol. These roles serve as hints for the compositor and should be used -before the actual surface creation takes place, such that the compositor can -take the necessary steps to satisfy those requirements. - -Please consult the [protocol file](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-compositor.git;a=blob_plain;f=protocol/agl-shell-desktop.xml;hb=refs/heads/master) -as that is the authoritative way of getting the latest version. - -#### Additional surface roles in agl-shell-desktop - -Like mentioned earlier, the compositor is already making use of some (internal) -roles, and with this extension we add some further ones. These are: - -* split (there's vertical and a horizontal one) -* fullscreen -* dialog/pop-up - -Internally these are encoded with different values such that there's a -translation needed, between the protocol values and the internal ones. Besides -the roles, additional data can to be passed on, but only depending on the role. -It is highly recommend **to avoid** using the protocol to pass down information -between different applications using this communication channel. It is only -intended to help out with demo applications. Other sharing mechanism are -available in the AGL project that can satisfy those requirements. - -#### Receiving application state events from (other) applications - -agl-shell-desktop exposes two events which client can install handlers for, one -that signals when regular xdg application have been created, and one that -signals state changes (active/hidden) as well as destroyed/no longer present -surfaces. These events can be useful to add additional functionality if -needed. - -#### Activating (other) applications - -Both agl-shell and agl-shell-desktop have requests to activate other -application based on their xdg-shell app_id. In case the application is -present/running, it will attempt to make the surface backing that application -the current activate one, with each output having independently active -surfaces. - -## Explicit output - -The activation and setting surface roles requires passing a Wayland output -(wl_output). The output is the wayland interface representation of an output -and is **mandatory** to pass it down to the compositor when activating a surface. -Clients can retrieve it (the output) if they wish to place the surface on other -outputs by using the toolkits that expose the Wayland objects. A human-like -representation is provided by either the toolkit, or by using other extensions -implemented by the client, for instance [xdg-output](https://gitlab.freedesktop.org/wayland/wayland-protocols/-/raw/master/unstable/xdg-output/xdg-output-unstable-v1.xml) -is the one recommended way and provides a mapping between a human -representation of the output and the wayland one. - -One can also choose the output where the application can start, by configuring -directly the AGL compositor. Under the `[output]` section one can use -`agl-shell-app-id=appid` restart the AGL compositor unitd systemd service and -start the application. Currently this *only* applies to regular applications, the -client shell having to handle it in the code. - -## Available toolkits, application conversions and available eco-systems - -Users and OEM vendors alike have the possibility, depending on their use-cases, -to either use some kind of a toolkit (Qt/GTK) for writing application, -or use plain C and the wayland library to communicate with the compositor -directly, without any toolkit abstraction. - -Currently, the demo applications in AGL use the Qt platform with Chromium being -at this phase, a second-class citizen, being currently in the works of -achieving the same level of integration as QtWayland (the underlying library that -abstracts the Wayland interaction) has at this moment. The Qt platform has -long been favoured in embedded systems so it feels natural why AGL project -chose it over other alternatives. In the same time, as web applications are -for quite some time now permeating the application development scene, it also -felt natural to add support for a runtime that gives that option, which in AGL -was achieved with the help of the Chromium project. - -For normal applications, not needing the ability to activate or displaying -other's application surface, would basically mean that it would use what the -toolkit has to offer, simplifying the application handling even more. Under -Qt, the client shell can use QPA (Qt Platform Abstraction) to gain access to -Wayland primitives, and implicitly is being able use the private extensions. - -![Architecture Diagram](images/agl-compositor/arch_diagram.png) - -On the Chromium side of things, that happens indirectly, as Chromium doesn't -expose the Wayland primitives. Not only that, but on the Chromium platform, -there's another mid-layer component, called [WAM](https://github.com/webosose/wam) -(WebApplicationManager) with the purpose of handling web applications life-cycle. - -So, controlling and passing information from a web application, that resembles -that of a shell client, has to travel more than a few levels in the software -stack, until it reaches the lower layers in Chromium where the Wayland -communication and interaction takes place. Support for the private extension -was done at the Ozone interface abstraction, which Chromium projects uses now -to handle the display/graphical interaction with the lower stack levels. - -## Streaming buffers and receiving events to and from remote outputs - -Quite a common feature, in the infotainment market, is the ability to stream -out buffers/data to remote outputs. For instance, super-imposing the navigation -application, between the speedometer and tachometer, in the IC (Instrument -Cluster) of a car is such a scenario. Just like weston, the AGL compositor is -capable of loading up libweston modules and make use of them. And just like -weston, the AGL compositor loads up the remoting-plugin to achieve the same -thing. - -The remoting-plugin uses the DRM virtual output API from libweston together -with gstreamer pipeline to capture, using DMA buffers, the DRM output and to -stream it, remotely to another machine. They can be over the network, or -locally. - -Further more, to cope with situations where the output is just a -panel/display, without some kind of compositor driving it, the necessity of -handling input events is an important feature to have, giving the user to -possibility to manipulate the application/environment as he or she seems fit. -The compositor loads a plug-in that streams out the buffers to an output -remotely, with [another plug-in](2_waltham-receiver_waltham-transmitter.md) -handling the input events. The events, which are sent back from the display to -the compositor, are generated with the help of wayland-eque protocol that works -over the network, called [Waltham](https://github.com/waltham/waltham). - -Together, they provide proper means to achieve a seamless integration with -other display devices in the car cabin. - -## Policies and Role Base Arbitration - -The compositor contains an API useful for implementing user-defined policies. -It contains a policy engine, and installs by default an allow-all kind of -policy. The policy engine controls if the client using the private extensions -is permitted to perform those calls. Not only that, but with some policies, it -won't allow the client to bind to the interfaces in the first place. That -happens with the deny-all policy, which is able to retrieve the client's -SMACK label and compares it with the ones statically defined. - -In the works, there's a new policy model, called [Role Based -Arbitration](https://gerrit.automotivelinux.org/gerrit/admin/repos/staging/rba). -Internally, how it works, should be found at [RBA](3_rba.md). -While the other two policies are embedded into the compositor, the RBA policy -model is an off the-shell policy. Obviously, vendors and users can hook up -their own policies, just like RBA did. These all work towards satisfying -the driver distraction mitigation requirement for the AGL project, as to avoid -overwhelming the driver with too much information. - -Users wanting to create their own policy should create a specialized version -of the callbacks defined in `struct ivi_policy_api`. - -As there's no dynamic loading of policies you'll need to recompile the compositor -with that policy in mind, specifically like the following: - - $ meson -Dprefix=/path/to/install-compositor/ -Dpolicy-default=my_policy build_directory - -The default policy found in src/policy-default.c should more than sufficient to -get started on creating new ones. Users can either re-purpose the default -policy or create a new one entirely different, based on their needs. - -These are hooks in place by the policy engine control the creation, committing -and activation of surfaces (`ivi_policy_api::surface_create()`, -`ivi_policy_api::surface_commited()`, `ivi_policy_api::surface_activate()`), -among other situations. - -Users can customize the hooks by using some sort of database to retrieve the -application name to compare against, or incorporate some kind of policy rule -engine. Alternatively, one can use the deny-all policy engine which allows the -top panel applications to be used/displayed as permitted applications. - -### Reactive rules - -The policy engine is stateful, and allows the ability to inject back events, -such that it allows the user to add custom rules into a policy and, depending -on the event received by the policy engine, to execute a rule match for that -event. Further more, the framework allows adding new states and events and the -default implementation has code for handling events like showing or hiding the -application specified in the policy rule. The most common example to exemplify -this feature is the ability to show a custom application, like displaying the -rear view camera application, when the automobile has been put in reverse. - -For deadling with these kind of rules, `ivi_policy_api::policy_rule_allow_to_add()` -can be used to control if policy rules could be added or not. Finally, we have -`ivi_policy_api::policy_rule_try_event()` which is executed for each policy -rule added, by using the policy API `ivi_policy_add()` function. - -By default the policy framework it will add the 'show', and 'hide' events and -the 'start', 'stop' and 'reverse' states. An special type, assigned by default -is 'invalid'. A **state change** has to be propagated to the compositor, which can -happen by using `ivi_policy_state_change()` function, and which signals the -compositor the state change took place, in order to apply the policy rules, and -implicitly to call the event handler `ivi_policy_api::policy_rule_try_event()`. - -## Back-ends and specific options for agl-compositor - -The compositor has support for the following back-ends: - -* **DRM/KMS** - runs a stand-alone back-end, uses Direct Rendering Manager/Kernel - Modesetting and evdev, that is utilizes and runs on real or virtualized HW - (qemu/Vbox/etc). -* **Wayland** - runs as a Wayland application, nested in another Wayland compositor - instance -* **X11** - run as a x11 application, nested in a X11 display server instance - -### Building and running the compositor on different platforms - -The compositor can run on desktop machines as easily as it does on AGL -platform. It should infer, depending on the environment, if it is being -compiled with the AGL SDK, or with the host build system. Running would also -be inferred from the environment. - -The compositor has some additional configuration options like: - -* `--debug` - enables the screenshooter interface, useful if one would want to - take a screenshot using `agl-screenshooter` client. This might be seen as a - security risk to it only be enabled in the AGL platform if built with agl-devel - DISTRO FEATURES. - -Additional configuration ini options have been added to help with the CI -integration. Worth mentioning are: - -* `activate-by-default=[true]` - if the surface of the client should be - displayed when the application started. Present in the `[core]` section. - By default set to `true`. Setting it to `false` will not activate, - by default, the client's surface when started. -* `hide-cursor=[false]` - do not advertise pointer/cursor to clients. Present - in the `[core]` section. - -## Running with software rendering - -By default the compositor will attempt to use the GL-renderer, and implicitly -the GPU. One could instead use the CPU, by making use of the Pixman library. To -use it in the compositor append `--use-pixman` to the command line. This purely -software approach has the benefit that would not rely at all on any GL -implementatation or library. In constrast, even if the GL-renderer is used, -in some situations it won't be able to use the GPU supported implementation -and fallback to sofware based one, and for instance that might happen when -running in virtualized environments. - -Both approaches could end up not actually using the GPU, but the latter does -actually use the GL library and perform the operations in software, while the -former does not use any GL whatsover. All back-ends support disabling the -GL-render to make sure it does not interfere with the composing process. - -## Multiple output set-up and touch input devices - -There's no deterministic way in which the compositor enables the outputs and -depending on the input devices, specifically touch input devices, and the way -the connectors are wired, a touch input device might be associated with a -different output than the one intended. - -A consistent way, that survives a reboot, is to use -[udev rules](https://man7.org/linux/man-pages/man7/udev.7.html), which -libweston would be able to use such that a particular output is tied/associated -to a particular touch input device. - -For instance, assuming that you have a set-up consisting of 4 outputs, a 4 -touch input devices, when the outputs are being enabled the compositor -front-end will associate all 4 touch input device -- if they haven't been -previously being associated to a particular output, to the first enabled -output. - -In order to avoid that, and associate each touch input device to -their respective output an udev rule can be installed, for the default -seat (named `seat0`). - -Example of a udev rule: - -``` -SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004a", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-1" -SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004b", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-2" -SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004c", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-3" -SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004d", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-4" -``` - -Add the following under `/etc/udev/rules.d/91-output.rules` and reload udev -rules for these changes to take effect: - - $ udevadm control --reload-rules && udevadm trigger - -Note that in the above example, we use physical seat, named `seat0` which is -the default physical seat. You can verify that these changes have been applied by -checking the compositor logs (under `/run/platform/display/compositor.log` file) -You should be seeing `CONNECTOR-NO by udev` message like the following: - -``` -associating input device event0 with output HDMI-A-1 (HDMI-A-1 by udev) -``` - -vs - -``` -associating input device event0 with output HDMI-A-2 (none by udev) -``` - -where the rules are either incorrect or badly written. - -Retrieving device attributes could be done archaically using `lsusb` or `lspci` -or using `udevadm info -a /dev/input/event*` which can provide with a multitude -of attributes to use. In our above example we only relied `idVendor` and -`idProduct` but potentially other attributes might be used. |