diff options
| author |   Vinod Ahuja <vahuja@unomaha.edu> | 2022-11-07 16:18:44 -0600 |
|---|---|---|
| committer |   Jan-Simon Moeller <jsmoeller@linuxfoundation.org> | 2022-11-08 14:47:43 +0000 |
| commit | 8be9db6f309e1e1b547e187c5db6ceac15f85a50 (patch) | |
| tree | ca3a6179b37b381eaee1bf948aebf78c809883b4 /docs/6_Component_Documentation | |
| parent | e660399f8b909146a699e44eb340b8c0b7e7f12f (diff) | |
Fixing the index numbering
Fixing the index numbering for all documentation
Bug-AGL: [SPEC-4470]
Signed-off-by: Vinod Ahuja <vahuja@unomaha.edu>
Change-Id: I96b482a3ab598f0739c692e301de66c0553ba0e4
Reviewed-on: https://gerrit.automotivelinux.org/gerrit/c/AGL/documentation/+/28118
Reviewed-by: Walt Miner <wminer@linuxfoundation.org>
Reviewed-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Tested-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Diffstat (limited to 'docs/6_Component_Documentation')
15 files changed, 2222 insertions, 0 deletions
diff --git a/docs/6_Component_Documentation/1_AGL_components.md b/docs/6_Component_Documentation/1_AGL_components.md new file mode 100644 index 0000000..a251b05 --- /dev/null +++ b/docs/6_Component_Documentation/1_AGL_components.md @@ -0,0 +1,27 @@ +--- +title: AGL Components +--- + +## Components under development within AGL + +### Graphics + +- [The AGL compositor](1_agl-compositor.md) +- [Waltham receiver/transmitter configuration](2_waltham-receiver_waltham-transmitter.md) +- [DRM lease manager](4_drm-leasemanager.md) + + +### Sound + +- [Pipewire & Wireplumber](6_pipewire_wireplumber.md) +- [IC and Sound Manager](7_ic-sound-manager.md) + + +### Policies + +- [Rule based arbitrator](3_rba.md) + + +### Lifecycle management + +- [Application Framework](../3_Developer_Guides/1_Application_Framework/1_Introduction.md) diff --git a/docs/6_Component_Documentation/2_agl-compositor.md b/docs/6_Component_Documentation/2_agl-compositor.md new file mode 100644 index 0000000..437e6a7 --- /dev/null +++ b/docs/6_Component_Documentation/2_agl-compositor.md @@ -0,0 +1,498 @@ +--- +title: agl-compositor +--- + +# Wayland compositor + +When the AGL project was started, weston was chosen as the compositor, which is +the reference implementation of a Wayland compositor, while for window management +functionality it relied on *ivi-shell* (In-Vehicle Infotainment) together +with an extension, called [wayland-ivi-exension](https://github.com/GENIVI/wayland-ivi-extension). + +A demo platform image of AGL comes with a handful of demo applications, done +with the Qt, which abstracts the protocol communication between the client and +the compositor. Additional functionality was in place under the form of +library, to control and signal back to the compositor when applications were +started, among other things. + +Management of applications, starting, running and stopping them is done in AGL +with AppFW [Application Framework Management](../3_Developer_Guides/1_Application_Framework/1_Introduction.md), +which is an umbrella name to denote the suite of tools and daemons that handle +all of that. It is integrated with systemd and with the current security model. +Applications can use AppFW to hang off data, and to pass it down to +other services. Together with AppFW, applications could tell the compositor +which application to activate or to switch to. + + +## Simplifying the graphical stack + +Trimming down these abstractions, simplifying the way clients interact with the +compositor, and avoid using modules that aren't really maintained upstream were +the reasons behind looking at alternatives to ivi-shell. On the desktop, +[xdg-shell](https://gitlab.freedesktop.org/wayland/wayland-protocols/-/raw/master/stable/xdg-shell/xdg-shell.xml) +is currently de-facto protocol for handling all window management related +functionality. + +Wayland protocol has a window-like interface embedded into its protocol (called +wl_shell), but *xdg-shell* has long time ago deprecated it and instead +of adding it in the wayland protocol namespace, it was integrated +together with some other useful protocols, into +[wayland-protocols](https://gitlab.freedesktop.org/wayland/wayland-protocols) +project. The whole purpose of wayland-protocols is to enhance the Wayland +protocol with new functionality and bring new extensions entirely. Compositors +are free to implement, modify, enhance, and add new extensions to +wayland-protocols but they need to do so in consensus. + +Besides the core wayland protocol and extended functionality from +wayland-protocols, a compositor can provide and implement additional protocol +extensions (custom to that compositor). By using such private extensions we +align with the AGL project and its requirements, without compromising specific +functionality and allows to add or improve the current ones. With that in mind, +the approach was to create a new compositor, called +[agl-compositor](https://gerrit.automotivelinux.org/gerrit/admin/repos/src/agl-compositor) +and implement dedicated private extensions, rather than trying to modify weston +itself, which AGL project would have been required to keep and maintain for +itself, as a fork. + +## A compositor based on libweston + +The compositor used currently in AGL, just like weston, is built on top of +*libweston* and *libweston-desktop*. The latter, among other things, is required +as it provides the server side implementation of the xdg-shell protocol which +underlying toolkits (like Qt/Chromium project) makes use of to deliver +desktop-like functionality. The former is used to provide back-ends and +rendering support, effectively managing the HW, besides implementing the +wayland protocol. + +The high-level goal of [libweston](https://wayland.pages.freedesktop.org/weston/toc/libweston.html) is +to decouple the compositor from the shell implementation. + +Traditionally, clients were entirely separated from the window manager, the +desktop environment and the display server. In wayland all these are +conceptually under the same entity though they are implemented as different +(UNIX) processes, or as a different namespaces with front and back-end APIs, +exposed by libraries. The compositor and the shell driving the UI should be +seen as one and the same, and in practice, this happens on desktop +environments. For AGL, the shell client can be represented under different +forms, as well as the fact that the process management has another layer +baked-in to handle MAC (Mandatory Access Control) labels and use the +above-mentioned Application Framework. These are all tightly +integrated and therefore, the AGL compositor will not automatically start the +shell client, although there's code to handle that. + +## Specifying a shell client to be started by the compositor + +Nevertheless, one can modify the configuration file, add the shell client path, and the +compositor will attempt to start it. + +``` +[shell-client] +command=/path/to/your/client/shell +``` + + + +## Private extensions + +Compositors can define and implement custom extensions to further control +application behaviour. For AGL, we have two private extensions defined. +One targeted at defining surface roles commonly found in desktop environments +(like panels, and backgrounds), which a shell client would bind to, and one +targeted at regular application(s) that might require additional functionality: +being able to display/activate its own surface or other's application surface, +implement some kind of split screen management of windows, or +dialog/pop-ups that exhibit always-on-top property even if the active +surface has been changed. + + + +Clients can make use of these private extensions to define other kind of roles +for instance dialog/pop-ups or full-screen roles, and split windows vertically or +horizontally. It includes the ability to activate other applications, assuming +that the surfaces have been created, and the capability of delaying +presentation for the client shell. Doing so, all the information is displayed +at once, rather than waiting for the toolkit to map/show the surface. + +An application identification mechanism was required to be able to activate +other clients windows/surfaces. A string-based identifier name was chosen +which can be used by the client to set an application-based identifier using +the xdg-shell protocol. While there's nothing stopping the client to avoid +doing that, specifically, to avoid assigning an application identifier, +the compositor won't be able to find which surfaces matches to a particular +client, if one would want to activate/display it at some time in the future. + +### agl-shell + +Client shellls can make use of this protocol to define panels and background +roles for different surfaces. It includes to ability to activate other +applications, assuming that those are already running. Activation happens by +using using the app_id, respectively using set_app_id request as defined by the +xdg-shell protocol. Established client-side implementation of the xdg-shelll +protocol will have a function exposed which can be used to set an application +identifier. Further more, the compositor will not present/display anything to +the user as long the `ready()` is not requested. So, after creating the surfaces +assigning them panel and/or background roles, and they're fully loaded, +the client can then issue `ready()` request and the compositor will start +presenting. + +Please consult the [protocol file](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-compositor.git;a=blob_plain;f=protocol/agl-shell.xml;hb=refs/heads/master) +as that is the authoritative way of getting the latest version. + +#### V2 updates + +Version 2 of the agl-shell protocol, while it is is not obligatory to be +supported by the shell client, it adds two new events: bound_ok and bound_fail +events. + +It has been observed that in some cases where we do not explicitly have a knob +in the toolkit to tell whether the application is a regular one (which doesn't +need to bind to the agl-shell protocol) or a one that needs to implement +agl-shell protocol might result in terminating the wayland connection. + +That happens because we can't have multiple clients bind to the agl-shell +protocol interface and was particularly visible when using regular +flutter applications with other shell clients (Qt homescreen, or WAM/chromum), +basically mashing together different kind of toolkits in the same image. Once +a client has already bound to the agl-shell protocol interface any other client +that attempts to do same will get its wayland connection severed and the +application will be terminated. + +These two events provide a race-free method in which the clients can tell if +they're in charge (of being the shell client) or their just regular +applications. Explicitly implementing this protocol if you have other means to +specify which type of application it is running wouldn't be required nor +necessary. But by using the protocol we can provide the same thing, +programmatically, without fearing that the wayland connection might be +severed, and implicitly taking down the application. + +#### V3 updates + +Version 3 of the agl-shell protocol adds 4 more events to signal out when the +application state was changed: started, activated, deactivated and terminated. + +Version 3 update was mostly prompted by an issue with start-up of applications +but also is part of the first steps to reduce and simplify a bit more +activation handling in the compositor. Specifically with this protocol update, +we can correctly orchestrate start-up and activation of applications. + +At the moment of adding this protocol update, the default compositor behaviour +is to display/activate applications as soon they're started, a feature which +we've called activate-by-default (and which is turned on by default). +But users can choose to disable that in such a way that activation is entirely +driven the shell client. + +Implicitly having this activate-by-default papered over various +issue when don't have that activation by default turned on. Supporting both +use-cases (activate-by-default, on and off) turned out to be cluster of +problems and regression over time. Not only that the amount of complexity in +the compositor is unwarranted and can simplified by telling the shell client +handle any window management interaction on its own. + +Further more, with this protocol update we also includes some events already +present in the agl-shell-desktop protocol like deactivate and terminate. + +### agl-shell-desktop + +This extension is targeted at keeping some of the functionally already +established in AGL as to a) allow applications display/activate other +surfaces/application window, and b) set further roles, specially dialog/window +pop-ups and split-type of surfaces. + +Clients can make use of this protocol to set further roles, like independently +positioned pop-up dialog windows, split type of surfaces or fullscreen ones. +Additional roles, and implicitly functionality can be added by extending the +protocol. These roles serve as hints for the compositor and should be used +before the actual surface creation takes place, such that the compositor can +take the necessary steps to satisfy those requirements. + +Please consult the [protocol file](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/agl-compositor.git;a=blob_plain;f=protocol/agl-shell-desktop.xml;hb=refs/heads/master) +as that is the authoritative way of getting the latest version. + +#### Additional surface roles in agl-shell-desktop + +Like mentioned earlier, the compositor is already making use of some (internal) +roles, and with this extension we add some further ones. These are: + +* split (there's vertical and a horizontal one) +* fullscreen +* dialog/pop-up + +Internally these are encoded with different values such that there's a +translation needed, between the protocol values and the internal ones. Besides +the roles, additional data can to be passed on, but only depending on the role. +It is highly recommend **to avoid** using the protocol to pass down information +between different applications using this communication channel. It is only +intended to help out with demo applications. Other sharing mechanism are +available in the AGL project that can satisfy those requirements. + +#### Receiving application state events from (other) applications + +agl-shell-desktop exposes two events which client can install handlers for, one +that signals when regular xdg application have been created, and one that +signals state changes (active/hidden) as well as destroyed/no longer present +surfaces. These events can be useful to add additional functionality if +needed. + +#### Activating (other) applications + +Both agl-shell and agl-shell-desktop have requests to activate other +application based on their xdg-shell app_id. In case the application is +present/running, it will attempt to make the surface backing that application +the current activate one, with each output having independently active +surfaces. + +## Explicit output + +The activation and setting surface roles requires passing a Wayland output +(wl_output). The output is the wayland interface representation of an output +and is **mandatory** to pass it down to the compositor when activating a surface. +Clients can retrieve it (the output) if they wish to place the surface on other +outputs by using the toolkits that expose the Wayland objects. A human-like +representation is provided by either the toolkit, or by using other extensions +implemented by the client, for instance [xdg-output](https://gitlab.freedesktop.org/wayland/wayland-protocols/-/raw/master/unstable/xdg-output/xdg-output-unstable-v1.xml) +is the one recommended way and provides a mapping between a human +representation of the output and the wayland one. + +One can also choose the output where the application can start, by configuring +directly the AGL compositor. Under the `[output]` section one can use +`agl-shell-app-id=appid` restart the AGL compositor unitd systemd service and +start the application. Currently this *only* applies to regular applications, the +client shell having to handle it in the code. + +## Available toolkits, application conversions and available eco-systems + +Users and OEM vendors alike have the possibility, depending on their use-cases, +to either use some kind of a toolkit (Qt/GTK) for writing application, +or use plain C and the wayland library to communicate with the compositor +directly, without any toolkit abstraction. + +Currently, the demo applications in AGL use the Qt platform with Chromium being +at this phase, a second-class citizen, being currently in the works of +achieving the same level of integration as QtWayland (the underlying library that +abstracts the Wayland interaction) has at this moment. The Qt platform has +long been favoured in embedded systems so it feels natural why AGL project +chose it over other alternatives. In the same time, as web applications are +for quite some time now permeating the application development scene, it also +felt natural to add support for a runtime that gives that option, which in AGL +was achieved with the help of the Chromium project. + +For normal applications, not needing the ability to activate or displaying +other's application surface, would basically mean that it would use what the +toolkit has to offer, simplifying the application handling even more. Under +Qt, the client shell can use QPA (Qt Platform Abstraction) to gain access to +Wayland primitives, and implicitly is being able use the private extensions. + + + +On the Chromium side of things, that happens indirectly, as Chromium doesn't +expose the Wayland primitives. Not only that, but on the Chromium platform, +there's another mid-layer component, called [WAM](https://github.com/webosose/wam) +(WebApplicationManager) with the purpose of handling web applications life-cycle. + +So, controlling and passing information from a web application, that resembles +that of a shell client, has to travel more than a few levels in the software +stack, until it reaches the lower layers in Chromium where the Wayland +communication and interaction takes place. Support for the private extension +was done at the Ozone interface abstraction, which Chromium projects uses now +to handle the display/graphical interaction with the lower stack levels. + +## Streaming buffers and receiving events to and from remote outputs + +Quite a common feature, in the infotainment market, is the ability to stream +out buffers/data to remote outputs. For instance, super-imposing the navigation +application, between the speedometer and tachometer, in the IC (Instrument +Cluster) of a car is such a scenario. Just like weston, the AGL compositor is +capable of loading up libweston modules and make use of them. And just like +weston, the AGL compositor loads up the remoting-plugin to achieve the same +thing. + +The remoting-plugin uses the DRM virtual output API from libweston together +with gstreamer pipeline to capture, using DMA buffers, the DRM output and to +stream it, remotely to another machine. They can be over the network, or +locally. + +Further more, to cope with situations where the output is just a +panel/display, without some kind of compositor driving it, the necessity of +handling input events is an important feature to have, giving the user to +possibility to manipulate the application/environment as he or she seems fit. +The compositor loads a plug-in that streams out the buffers to an output +remotely, with [another plug-in](2_waltham-receiver_waltham-transmitter.md) +handling the input events. The events, which are sent back from the display to +the compositor, are generated with the help of wayland-eque protocol that works +over the network, called [Waltham](https://github.com/waltham/waltham). + +Together, they provide proper means to achieve a seamless integration with +other display devices in the car cabin. + +## Policies and Role Base Arbitration + +The compositor contains an API useful for implementing user-defined policies. +It contains a policy engine, and installs by default an allow-all kind of +policy. The policy engine controls if the client using the private extensions +is permitted to perform those calls. Not only that, but with some policies, it +won't allow the client to bind to the interfaces in the first place. That +happens with the deny-all policy, which is able to retrieve the client's +SMACK label and compares it with the ones statically defined. + +In the works, there's a new policy model, called [Role Based +Arbitration](https://gerrit.automotivelinux.org/gerrit/admin/repos/staging/rba). +Internally, how it works, should be found at [RBA](3_rba.md). +While the other two policies are embedded into the compositor, the RBA policy +model is an off the-shell policy. Obviously, vendors and users can hook up +their own policies, just like RBA did. These all work towards satisfying +the driver distraction mitigation requirement for the AGL project, as to avoid +overwhelming the driver with too much information. + +Users wanting to create their own policy should create a specialized version +of the callbacks defined in `struct ivi_policy_api`. + +As there's no dynamic loading of policies you'll need to recompile the compositor +with that policy in mind, specifically like the following: + + $ meson -Dprefix=/path/to/install-compositor/ -Dpolicy-default=my_policy build_directory + +The default policy found in src/policy-default.c should more than sufficient to +get started on creating new ones. Users can either re-purpose the default +policy or create a new one entirely different, based on their needs. + +These are hooks in place by the policy engine control the creation, committing +and activation of surfaces (`ivi_policy_api::surface_create()`, +`ivi_policy_api::surface_commited()`, `ivi_policy_api::surface_activate()`), +among other situations. + +Users can customize the hooks by using some sort of database to retrieve the +application name to compare against, or incorporate some kind of policy rule +engine. Alternatively, one can use the deny-all policy engine which allows the +top panel applications to be used/displayed as permitted applications. + +### Reactive rules + +The policy engine is stateful, and allows the ability to inject back events, +such that it allows the user to add custom rules into a policy and, depending +on the event received by the policy engine, to execute a rule match for that +event. Further more, the framework allows adding new states and events and the +default implementation has code for handling events like showing or hiding the +application specified in the policy rule. The most common example to exemplify +this feature is the ability to show a custom application, like displaying the +rear view camera application, when the automobile has been put in reverse. + +For deadling with these kind of rules, `ivi_policy_api::policy_rule_allow_to_add()` +can be used to control if policy rules could be added or not. Finally, we have +`ivi_policy_api::policy_rule_try_event()` which is executed for each policy +rule added, by using the policy API `ivi_policy_add()` function. + +By default the policy framework it will add the 'show', and 'hide' events and +the 'start', 'stop' and 'reverse' states. An special type, assigned by default +is 'invalid'. A **state change** has to be propagated to the compositor, which can +happen by using `ivi_policy_state_change()` function, and which signals the +compositor the state change took place, in order to apply the policy rules, and +implicitly to call the event handler `ivi_policy_api::policy_rule_try_event()`. + +## Back-ends and specific options for agl-compositor + +The compositor has support for the following back-ends: + +* **DRM/KMS** - runs a stand-alone back-end, uses Direct Rendering Manager/Kernel + Modesetting and evdev, that is utilizes and runs on real or virtualized HW + (qemu/Vbox/etc). +* **Wayland** - runs as a Wayland application, nested in another Wayland compositor + instance +* **X11** - run as a x11 application, nested in a X11 display server instance + +### Building and running the compositor on different platforms + +The compositor can run on desktop machines as easily as it does on AGL +platform. It should infer, depending on the environment, if it is being +compiled with the AGL SDK, or with the host build system. Running would also +be inferred from the environment. + +The compositor has some additional configuration options like: + +* `--debug` - enables the screenshooter interface, useful if one would want to + take a screenshot using `agl-screenshooter` client. This might be seen as a + security risk to it only be enabled in the AGL platform if built with agl-devel + DISTRO FEATURES. + +Additional configuration ini options have been added to help with the CI +integration. Worth mentioning are: + +* `activate-by-default=[true]` - if the surface of the client should be + displayed when the application started. Present in the `[core]` section. + By default set to `true`. Setting it to `false` will not activate, + by default, the client's surface when started. +* `hide-cursor=[false]` - do not advertise pointer/cursor to clients. Present + in the `[core]` section. + +## Running with software rendering + +By default the compositor will attempt to use the GL-renderer, and implicitly +the GPU. One could instead use the CPU, by making use of the Pixman library. To +use it in the compositor append `--use-pixman` to the command line. This purely +software approach has the benefit that would not rely at all on any GL +implementatation or library. In constrast, even if the GL-renderer is used, +in some situations it won't be able to use the GPU supported implementation +and fallback to sofware based one, and for instance that might happen when +running in virtualized environments. + +Both approaches could end up not actually using the GPU, but the latter does +actually use the GL library and perform the operations in software, while the +former does not use any GL whatsover. All back-ends support disabling the +GL-render to make sure it does not interfere with the composing process. + +## Multiple output set-up and touch input devices + +There's no deterministic way in which the compositor enables the outputs and +depending on the input devices, specifically touch input devices, and the way +the connectors are wired, a touch input device might be associated with a +different output than the one intended. + +A consistent way, that survives a reboot, is to use +[udev rules](https://man7.org/linux/man-pages/man7/udev.7.html), which +libweston would be able to use such that a particular output is tied/associated +to a particular touch input device. + +For instance, assuming that you have a set-up consisting of 4 outputs, a 4 +touch input devices, when the outputs are being enabled the compositor +front-end will associate all 4 touch input device -- if they haven't been +previously being associated to a particular output, to the first enabled +output. + +In order to avoid that, and associate each touch input device to +their respective output an udev rule can be installed, for the default +seat (named `seat0`). + +Example of a udev rule: + +``` +SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004a", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-1" +SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004b", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-2" +SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004c", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-3" +SUBSYSTEM=="input", ATTRS{idVendor}=="222a", ATTRS{idProduct}=="004d", OWNER="display", ENV{ID_SEAT}="seat0", ENV{WL_OUTPUT}="HDMI-A-4" +``` + +Add the following under `/etc/udev/rules.d/91-output.rules` and reload udev +rules for these changes to take effect: + + $ udevadm control --reload-rules && udevadm trigger + +Note that in the above example, we use physical seat, named `seat0` which is +the default physical seat. You can verify that these changes have been applied by +checking the compositor logs (under `/run/platform/display/compositor.log` file) +You should be seeing `CONNECTOR-NO by udev` message like the following: + +``` +associating input device event0 with output HDMI-A-1 (HDMI-A-1 by udev) +``` + +vs + +``` +associating input device event0 with output HDMI-A-2 (none by udev) +``` + +where the rules are either incorrect or badly written. + +Retrieving device attributes could be done archaically using `lsusb` or `lspci` +or using `udevadm info -a /dev/input/event*` which can provide with a multitude +of attributes to use. In our above example we only relied `idVendor` and +`idProduct` but potentially other attributes might be used. diff --git a/docs/6_Component_Documentation/3_waltham-receiver_waltham-transmitter.md b/docs/6_Component_Documentation/3_waltham-receiver_waltham-transmitter.md new file mode 100644 index 0000000..b98de32 --- /dev/null +++ b/docs/6_Component_Documentation/3_waltham-receiver_waltham-transmitter.md @@ -0,0 +1,96 @@ +--- +title: Waltham receiver/transmitter +--- + +# Waltham + +[Waltham protocol](https://github.com/waltham/waltham) is a IPC library similar +to [Wayland](https://wayland.freedesktop.org), developed with networking in +mind. It operates over TCP sockets, while the wayland protocol only works +locally over a UNIX socket. It retains wayland-esque paradigm, making use of +XMLs to describe the protocol, and it follows an object-oriented design with an +asynchronous architecture. + +## Differences from Wayland to Waltham + +Some of the differences between Wayland and Waltham are: + +* Waltham uses TCP sockets for communication +* Waltham cannot send file descriptors +* Waltham API is minimal and symmetric between server and client sides +* Waltham does not provide an event loop implementation +* The registry implementation is left out of the library, only the interface is + provided +* No multi-threading support for sharing objects between threads + +## Waltham-transmitter and remoting plugin + +Surface sharing is not part of Waltham protocol, each system needs to implement +the most efficient way of passing by the buffers from one side to another. On +AGL, make use of remoting-plugin to enable surface sharing which uses GStreamer +as encoder/decoder. It uses libweston DRM virtual API to grab the buffers, and +then to stream them over the network. The gstreamer pipeline uses UDP while the +input events are communicated with Waltham protocol. The input part is handled +by *waltham-transmitter* plugin which provides an API to create remote +connections and push surfaces over the network and handles remote input. The +act of pushing surface is a misnomer, kept from the older, previous +implementation, and acts a notification mechanism from the transmitter side to +the remote side. + +## The receiver client + +waltham-receiver application is a wayland demo implementation which should be +running at the remote side. It is using Waltham protocol to obtain and process +remote input events which handled the transmitter side by the waltham-transmitter +plugin. It creates a similar gstreamer pipeline to process the incoming buffers +and draw and displaying them into a subsurface created by waylandsink. + +Contrary to expectations, the waltham receiver is the one that implements the +server side of the Waltham protocol and is capable of displaying the incoming +buffers but also process input events locally and forward them with the help of +the Waltham protocol back at the transmitter side, which in turn will update +the image contents and stream to the receiver, showing the changes caused by +that input. + + + ECU 1 ECU 2 + +---------------------------------------------+ +--------------------------------------+ + | +-----------------+ | | | + | | Application | | | +-----------+-----------+ | + | +-----------------+ | | | Gstreamer | | | + | ^ | Buffer ---------------> (Decode) | | | + | wayland | +-------------------/ | +-----------+ | | + | v | | (Ethernet) | | Waltham-receiver | | + | +----+---------------------+ | | --------------------> | | + | | | Transmitter plugin |<---------------------/ | +-----------------------+ | + | | | | | | Waltham-Protocol | ^ | + | | |---------------------| | | | wayland | | + | | | Remoting plugin |-------+ | | v | + | | | | | | +---------------------+ | + | | +---------------------+ | | | | | + | | | | | | compositor | | + | | compositor | | | | | | + | +------+-------------------+ | | +----------------+----+ | + | | | | | | + | v | | v | + | +------------+ | | +----------+ | + | | Display | | | | Display | | + | | | | | | | | + | +------------+ | | +----------+ | + +---------------------------------------------+ +--------------------------------------+ + +## Migrating/Placing applications on other outputs + +In order to start or place an application on the remoting-ouput output, we can +use `agl-shell-app-id` ini entry for that particular output. + + [transmitter-output] + name=transmitter-1 + mode=640x720@30 + host=192.168.20.99 + port=5005 + agl-shell-app-id=<APP_ID> + +Alternatively, and programmatically, one can use the +[agl-shell-desktop](1_agl-compositor.md#private-extensions) protocol and inform +the compositor that it should migrate it to other, remote outputs. diff --git a/docs/6_Component_Documentation/4_rba.md b/docs/6_Component_Documentation/4_rba.md new file mode 100644 index 0000000..9661923 --- /dev/null +++ b/docs/6_Component_Documentation/4_rba.md @@ -0,0 +1,1207 @@ +--- +title: Rule Based Arbitrator (RBA) +--- + +# RBA + +Rule Based Arbitrator decides which of the content to display when a large number of contents to be displayed on the cockpit display device (CID, meter, HUD, etc.) occur simultaneously under a certain rule (arbitration). + +### 1. Overview + +#### 1.1 Purpose of this document + +This document describes the syntax of the Rule-based Arbitration Model. + +#### 1.2 Basic syntax + +The basic syntax of the Rule-based Arbitration Model is as follows. + + + +Define the properties of the model element in {} after declaration of Model element keyword, Model element ID. Each property depends on the element. The properties of the model element may also define other model elements. + +#### 1.3 Relationship between files and Model definitions + +The Rule-based Arbitration Model can be defined in multiple files. (The extension of the file will be ".rba") Firstly, define the Package model element in the file. + +Areas.rba +```shell +Package AreasPackage { + +} +``` +Content.rba +```shell +Package ContentsPackage { + +} +``` + +#### 1.4 Structure of Rule-based Arbitration Model + +The elements of the Rule-based Arbitration Model are as follows. Each inheritance relationship between elements is defined and expresses elements that can be described in PackagableElement. + + + +#### 1.5 Notation of Syntax Definition + +For Model element + +syntax: +```shell +Package [ID] { + description: [String] + [PackagableElement] +} +``` + +Description: + +| Syntax element | Description | +| :--- | :---- | +| description: 0..1 | Description | +| [PackagableElement] * | Child element Package,Display,Size,Constraint,PostConstraint,Scene,Event,Rule,Area,AreaSet,ViewContent,ViewContentSet| + +The multiplicity is expressed as follows: + +| Expression of multiplicity | Description | +| :--- | :--- | +| * | 0 or more | +| 1..* | 1 or more | +| 1 | 1 | +| 0..1 | 0 or 1 | + +Description of [] notation is as follows: + +| Syntax element | Description | +| :--- | :---- | +| [ID] | ID. A character string in which the first character is not a number. Only _ can be used for symbols, Space cannot be used. +| [String] | An arbitrary character string. specify it by enclosing with "". | +| [Number] | An integer that is greater than equal 0. | +| [expression] | An expression whose return value is a property type. | +| [<Model element keyword\>] | Definition of other Model elements. Sometimes an abstract class is specified. | +| [ID of A] | Reference to other Model elements. A represents an element. | +| [X\|Y] | Indicates that you can describe X or Y. | + +### 2. Common + +#### 2.1 Package + +The Package element is the root element in the file. It has PackagableElement as a Child element and groups PackagableElements in arbitrary units. + +syntax: +```shell +Package [ID] { + description: [String] + [PackagableElement] +} +``` + +| Syntax element | Description | +| :--- | :---- | +| description: 0..1 | Description | +| [PackagableElement] * | Child element Package,Display,Size,Constraint,PostConstraint,Scene,Event,Rule,Area,AreaSet,ViewContent,ViewContentSet| + +Description example: +```shell +Package SamplePackage { + description: "This is a sample Packege" + Area SampleArea { + arbitrationPolicy: DEFAULT + sizeReference: SampleSize1 + visibility: NONE_VALUE + zorder: 3 + } + ViewContent SampleContent { + allocatable: [SampleArea1] + State NORMAL { + priority: STANDARD_VALUE + } + } +} +``` + +#### 2.2 Size + +Size is the size of Area and ViewContent. + +Syntax: + +```shell +Size [ID] { + description: [String] + width: [Number] + height: [Number] +} +``` + +| Syntax element | Description | +| :--- | :---- | +| description: 0..1 | Description of Size element | +| width: 1 | width | +| height: 1 | height | + +Description example: +```shell +Size ScreenSize { + description: " Screen size" + width: 200 + height: 200 +} +``` + +#### 2.3 Project + +The Project element is an element that can be defined only once in one project. +Like The Package element, it can be defined directly under the file. + +Syntax: +```shell +Project { + version: [String] +} +``` + +| Syntax element | Description | +| :--- | :---- | +|version: 1| Version of the Project| + +Description example: +```shell +Project { + version: "version 1.0" +} +``` + + +### 3. Area + +Area is a frame to display the ViewContent on the screen. Only one ViewContent is allocated to one Area at most. Since Arbitration is executed for each Area, the Arbitration Policy is defined. + +Syntax: + +```shell +Area [ID] { + description: [String] + arbitrationPolicy: [DEFAULT | FIRST_COME_FIRST | LAST_COME_FIRST | PRIORITY_FIRST_COME_FIRST | PRIORITY_LAST_COME_FIRST] + [[Size]| sizeReference:[ID of Size]] + [visibility|priority]:[Number|Expression] + zorder:[Number|Expression] +} +``` + +| Syntax element | Description | +| :--- | :---- | +| description: 0..1 | Description | +| arbitrationPolicy: 0..1 | Arbitration Policy for Area. Refer to "Types of Arbitration Policy" for the policies which can be specified. If not specified, consider it as "DEFAULT". +| [Size\|sizeReference:] 1..* | Size: Size definition.| +|| sizeReference: Reference to Size definition.| +|| ※One or more definitions of either Size or sizeReference are needed. Multiple definitions should be defined consecutively.| +| [visibility\|priority]: 1 | Value of Area. Describe the relative value for other Areas as an expression or set a numerical value.| +|| The following predefined value can be set as a String. NONE_VALUE, STANDARD_VALUE, MIN_VALUE, MAX_VALUE See " Value / Z order definition" for details. | +| zorder: 1 | Z order of Area. Describe the relative Z order for other Areas as an expression or set a numerical value. The following predefined value can be set as a String. | +||NONE_VALUE, STANDARD_VALUE, MIN_VALUE, MAX_VALUE See " Value / Z order definition" for details. | + + + +Types of Arbitration Policy + +| Arbitration Policy | Description | +| :--- | :---- | +| DEFAULT | Same as Priority and First Come First Arbitration. If the Arbitration Policy is not specified, it will be this type.| +| FIRST_COME_FIRST | First Come First Arbitration. Give priority to the request which occurred first. | +| LAST_COME_FIRST | Last Come First Arbitration. Give priority to the request which occurred later. | +| PRIORITY_FIRST_COME_FIRST | Priority and First Come First Arbitration. It follows the priority set to the Content. If the priority is the same, give priority to the request which occurred first. | +| PRIORITY_LAST_COME_FIRST | Priority and Last Come First Arbitration. It follows the priority set to the Content. If the priority is the same, give priority to the request which occurred later.| + +Description example: +```shell +Area SampleArea { + description: "This is a sample Area." + arbitrationPolicy: DEFAULT + Size Default { + width: 200 + height: 150 + } + sizeReference: SampleSize1 + visibility: NONE_VALUE + zorder: 3 +} +``` + +### 4. Value / Z order definition + +Values (visibility or priority) of area is the order of arbitrating of the area. The higher is arbitrated first. You can describe only one of the visibility or priority. +For the Z order (zorder), the higher one will be on the front of the screen. You can define values absolutely by number or define relatively to other areas by expressions. You can use comparison operators (> and <), equality operator (=) and That-of operator for the expressions. + +- Description example of value : +Visibility of SampleArea2 is 10, zorder is 5. +```shell +Area SampleArea2 { + visibility: 10 + zorder: 5 +} +``` +- Description example of comparison operator: +Priority of SampleArea1 is greater than SampleArea2. Zorder of SampleArea1 is less than SampleArea3. +```shell +Area SampleArea1 { + priority: > That-of SampleArea1 + zorder: < That-of SampleArea3 +} +``` +- Description example of range: +Visibility of SampleArea1 is greater than 1 and less than SampleArea2. Zorder of SampleArea1 is greater than SampleArea3 and less than SampleArea4. +```shell +Area SampleArea1 { + visibility: (> 1 AND < That-of SampleArea2) + zorder: (> That-of SampleArea3 AND < That-of SampleArea4) +} +``` + +- Description example of equality operator: +```shell +Area SampleArea1 { + visibility: = That-of SampleArea2 + zorder: = That-of SampleArea3 + That-of SampleArea2 +} +``` + +- Equality operator can be omitted. +```shell +Area SampleArea1 { + visibility: That-of SampleArea2 + zorder: That-of SampleArea3 + That-of SampleArea2 +} +``` + +### 5. ViewContent definition + +#### 5.1 ViewContent + +ViewContent is an object to be displayed in the Area. ViewContent has multiple states. When ViewContent is allocated to an Area, active state of theViewContent is displayed. Define the Area which can display ViewContent to the ViewContent. You can define several Areas. + +Syntax: +```shell +ViewContent [ID] { + description: [String] + allocatable: [ [ID of Area] | [ID of Set] ] + [ViewContentState] + [[Size]|sizeReference: [ID of Size]] + loserType: [GOOD_LOSER | DO_NOT_GIVEUP_UNTIL_WIN | NEVER_GIVEUP] +} +``` + +|Syntax element | Description| +| :--- | :---- | +|description: 0..1 | Description| +|allocatable: 0..* | Displayable Area. Specify the ID of displayable Areas or Sets with comma-separated.If don’t specify allocatable, should specify allocatable in the ViewContentSet that includes this ViewContent. | +| [ViewContentState] 1..* |Define the State of ViewContent.| +|[Size\|sizeReference:] 1..*|Size: Size definition| +||sizeReference: Reference to Size definition.| +||※One or more definitions of either Size or sizeReference are needed. Multiple definitions should be defined consecutively.| +|loserType: 0..1| Action on lost. Specify whether to cancel a request, if ViewContent is not displayed after arbitration. See "Types of Action on lost" for details. If not defined, consider it as "NEVER_GIVEUP".| + +Types of Action on lost + +|loserType | Description| +| :--- | :---- | +|GOOD_LOSER |When losing arbitration, cancel a request.| +|DO_NOT_GIVEUP_UNTIL_WIN |When the state is changed visible to invisible , cancel a request.| +|NEVER_GIVEUP|Keep a request.| + +Description example: +```shell +ViewContent Power { +description: " Warning from power management" + allocatable: [ + MessageArea, HUDMessageArea + ] + State Warning { + priority: > That-of Power.Attention + } + State Attention { + priority: > That-of TEL.Incoming + } +Size Default { +description: "Default size" +width: 200 +height: 200 +} + sizeReference: InterruptMessageSize + loserType: GOOD_LOSER +} +``` +#### 5.2 State(ViewContent) + +State is the state of the ViewContent. + +Syntax: +```shell +State [ID] { + description: [String] + [priority|value]: [Number|Expression] +} +``` + +|Syntax element | Description| +| :--- | :---- | +|description: 0..1 | Description +|[priority\|value]: 1 | Priority of ViewContent. | +||Describe the relative value for other State under ViewContent as an expression, or set a numerical value. | +||The following predefined value can be set as a String.NONE_VALUE, STANDARD_VALUE, MIN_VALUE, MAX_VALUE See "Priority definition"for details. | + +Description example: +```shell +State Warning { + description: "Display warning message" +priority: > That-of Attention +} +State Attention { + description: " Display attention message " +priority: > That-of TEL.Incoming +} +``` +### 6. Priority definition + +Define the priority of Content to the State under ViewContent or SoundContent (hereinafter, Content). The priority is used when the area / zone arbitration policy is "Priority and First Come First Arbitration" or "Priority and Last Come First Arbitration", the higher one is allocated first. Values can be defined absolutely by number or relative to other areas by expressions. You can describe only one of the visibility or priority. You can use comparison operators (> and <), equality operator (=) and That-of operator for the expressions. + +- Description example of value: +Priority of Warning is 10. +```shell +State Warning { + description: " Display warning message " +priority: 10 +} +``` +- Description example of comparison operator: +Value of Warning is greater than Attention. +<br>Value of Attention is greater than State Incoming of TEL of other Content. +<br>Value of Notice is less than Attention. +<br>※When referring to the State of another Content, refer to the Content ID and State ID by connecting with “.”. +```shell +State Warning { + value: > That-of Attention +} + +State Attention { + value: > That-of TEL.Incoming +} + +State Notice { + value: < That-of Attention +} +``` + +- Description example of range: Priority of Attention is greater than Incoming of TEL and less than Warning. +<br>※When referring to the State of another Content, refer to the Content ID and State ID by connecting with “.”. +```shell +State Attention { + priority: (> That-of TEL.Incoming AND < That-of Warning) +} +``` + +- Description example of equality operator:Value of Attention is equal to State Incoming of TEL of other Content. +```shell +State Attention { + priority: = That-of TEL.Incoming +} +``` +- Equality operator can be omitted. +```shell +State Attention { + priority: That-of TEL.Incoming +} +``` + +### 7. Screen layout definition + +#### 7.1. Display + +Display represents one screen. When defined more than one, it will be Multi-screen layout. +The Display defines the set of Areas to be allocated on the screen. + +Syntax: +```shell +Display [ID] { + description: [String] + [CompositeArea] + [[Size]|sizeReference:[ID of Size]] +} +``` + +|Syntax element | Description| +| :--- | :---- | +| description: 0..1 | Description| +|[CompositeArea] 1 | Component of the screen| +|[Size\|sizeReference:] 1 | Size: Size definition| +|| sizeReference: Reference to Size definition| +|| ※One or more definitions of either Size or sizeReference are needed. | + +Description example: +```shell +Display METER { + description:"Meter display definition" + Size FULLSCREEN { + width: 500 + height: 400 + } + CompositeArea METER_Root { + layout: FixedPositionLayout { + PositionContainer { + x: 100 + y: 100 + basePoint: LEFT_TOP + areaReference: SpeedMeterArea + } + } + } +} +``` + +#### 7.2. CompositeArea + +CompositeArea specifies an Area that becomes a component of Display. + +Syntax: +```shell +CompositeArea [ID] { + description: [String] + layout: [FixedPositionLayout] +} +``` + +|Syntax element | Description| +| :--- | :---- | +|description: 0..1 | Description | +|layout: [FixedPositionLayout] 1| Layout type of Area| + +Description example: +```shell +CompositeArea METER_Root { + description:" Definition of Area’s layout method" + layout: FixedPositionLayout { + PositionContainer { + x: 100 + y: 100 + basePoint: LEFT_TOP + areaReference: SpeedMeterArea + } + } +} +``` + +#### 7.3. FixedPositionLayout + +FixedPositionLayout declares that areas are laid out with fixed values. The specific position define by PositionContainer. + +Syntax: +```shell +FixedPositionLayout { + [PositionContainer] +} +``` + +|Syntax element | Description| +| :--- | :---- | +|[PositionContainer] 1..*| Position information.| + +```shell +FixedPositionLayout { + PositionContainer { + x: 100 + y: 100 + basePoint: LEFT_TOP + areaReference: SpeedMeterArea + } +} +``` +#### 7.4. PositionContainer + +PositionContainer specifies the display position of the Area. + +syntax: +```shell +PositionContainer { + x: [Number] + y: [Number] + basePoint: [Value] + areaReference: [ID of Area] + [Offset] +} +``` + +|Syntax element | Description| +| :--- | :---- | +|x: 1 |x position| +|y: 1 |y position| +|basePoint: 1 | Defined x, y position. The following predefined value can be set.| +|| CENTER_BOTTOM (Define x, y as the lower center position)| +|| CENTER_MIDDLE(Define x, y as center position)| +|| CENTER_TOP (Define x, y as the upper center position)| +||LEFT_BOTTOM (Define x, y as the lower left position)| +||LEFT_MIDDLE (Define x, y as the left center position)| +||LEFT_TOP (Define x, y as the upper left position)| +||RIGHT_BOTTOM (Define x, y as the lower right position)| +||RIGHT_MIDDLE (Define x, y as right center position)| +||RIGHT_TOP (Define x, y as the upper right position)| +|areaReference: 1| Area to be placed in Display. Specify ID of the Area.| +|[Offset] 0..*| Display position offset of AreaSpecify the position of the Area for each size.| + + +Description example: +```shell +FixedPositionLayout { + PositionContainer { + x: 100 + y: 100 + basePoint: LEFT_TOP + areaReference: SpeedMeterArea + Offset { x:-50 y:20 sizeReference: SpeedMeterArea } + } +} +``` +#### 7.5. Offset + +Offset is the offset position for each size. + +Syntax: +```shell +Offset { + description: [String] + x: [Number] + y: [Number] + sizeReference: [ID of Size] +} +``` + +|Syntax element | Description| +| :--- | :---- | +|description: 0..1| Description| +|x: 1| Offset from x position of PositionContainer.| +|y: 1| Offset from y position of PositionContainer.| +|sizeReference: 1| Size to apply the offset. Specify ID of Size.| +||※ This Size must be specified in Area.| + +Description example: +```shell +Offset { +description:"Offset" + x: 100 + y: -50 + sizeReference: SpeedMeterSize +} +``` + +### 8. Constraint defination + +Constraints can be defined according to the state of the Area/Zone and the state of the ViewContent/SoundContent. The syntax can be used for Constraints is shown below. + +- State reference of Area: + +|Name | Notation |Meaning| +| :--- | :---- | :---- | +|IsDisplayed | <Area\>.isDisplayed()| The Area is displayed. (At this time, the ViewContent is necessarily allocated to the Area) True if the Area is displayed.| +|DisplayingContent| <Area\>.displayingContent()| ViewContent which is displayed in the Area. When the Area is hidden or ViewContent is not allocated, it is not evaluated. +|AllocatedContent| <Area\>.allocatedContent()| ViewContent which is allocated to the Area. Even if the Area is hidden after allocation, it is possible to refer to the allocated ViewContent.| +|IsHidden| <Area\>.isHidden() |The Area is hidden. Regardless of whether the ViewContent is allocated to the Area or not,true if the Area is hidden.| +|ContentValue| <Area\>.contentValue()| The value of the ViewContentState allocated to the Area. If ViewContent is not allocated to the Area, it is not evaluated.| +|ContentsList| <Area\>.contentsList()| A set of ViewContent which is allocatable to the Area.| +|activeContents| <Area\>.activeContents() |The set of active content amang the ViewContent which is allocatable to the Area| + +- State reference of ViewContent: + +|Name | Notation |Meaning| +| :--- | :---- | :---- | +|IsActive| <ViewContent\>.isActive()| True if there is a Content request of ViewContent.| +|IsVisible| <ViewContent\>.isVisible()| The ViewContent is displayed. True if the ViewContent is allocated to any Area.| +|StateValue |<ViewContent\>.stateValue() |Priority/value that is defined in active state of the ViewContent. When the ViewContent has no active state, it does not evaluate.| +|HasComeEarlierThan| <ViewContentA\>.hasComeEarlierThan(<ViewContentB\>)| True if the ViewContentA’s request has come earlier than the ViewContentB’s. When the either ViewContent has been not requested, it does not evaluate.| +|HasComeLaterThan| <ViewContentA\>.hasComeLaterThan(<ViewContentB\>) |True if the ViewContentA’s request has come later than the ViewContentB’s. When the either ViewContent has been not requested, it does not evaluate.| +|Allocatables| <ViewContent\>.allocatables()| A set of Areas where the ViewContent can be displayed.| + +- Scene reference: + +|Name | Notation |Meaning| +| :--- | :---- | :---- | +|IsOn| <Scene\>.isOn()| True if the Scene is valid.| +|Get| <Scene\>.<Property\>.get()| Get scene property value.| + +- Stereotyope: + +|Name | Notation |Meaning| +| :--- | :---- | :---- | +|isTypeOf| <Model\>.isTypeOf(“<ID of Stereotype\>”)| Whether or not “<ID of Stereotype\>” is used in the <Model\> | +|||It can be applied to the following models Area,ViewContent| + +- Operator: + +|Name | Notation |Meaning| +| :--- | :---- | :---- | +|AND| <Bool expression\> AND <Bool expression\>| True if all Bool expressions are true.| +|OR| <Bool expression\> OR <Bool expression\> |True if any Bool expression is true.| +|Negation| ! <Bool expression\>| True if Bool expression is false.| +|Implication| <Bool expression\> -> <Bool expression\> | A -> B is equivalent to ((A AND B) OR !A).| +|Equal sign (Comparison of values)| <Expression\> = <Expression\>| True if the values shown on the left-hand side and the right-hand side are identical.The type of left-hand side and the right-hand side expressions must match.| +|Equal sign (Comparison of objects) |<Expression\> == <Expression\>| True if the values shown on the left-hand side and the right-hand side are identical.The type of left-hand side and the right-hand side expressions must match.| +|Comparison (greater than) |<Number\> \> <Number\>|True if the Number on the left-hand side is greater than the Number on the right-hand side.| +|Comparison (greater than)| <Number\> \>= <Number\>|True if the Number on the left-hand side is greater than equal to the Number on the right-hand side.| +|Comparison (less than)| <Number\> < <Number\>| True if the Number on the left-hand side is less than the Number on the right-hand side.| +|Comparison (less than)| <Number\> <= <Number\>| True if the Number on the left-hand side is less than the equal to Number on the right-hand side.| + +- Quantization symbol: + +|Name | Notation |Meaning| +| :--- | :---- | :---- | +|∀(For All) |For-All<Set\>{ <x\> \| <Expression\> } |<Set\> contains the ViewContentSet or the SoundContentSet, the AreaSet, the ZoneSet, and the One-time Set. True if <Expression\> is true for all set element <x\>.| +|∃(Exists)| Exists< Set\>{ <x\> \| <Expression\> } |<Set\> contains the ViewContentSet or the SoundContentSet, the AreaSet, the ZoneSet, and the One-time Set. True if < Set\> has one or more elements that satisfy <Expression\>.| + +- Built-in defined Expression: + +|Name | Notation |Meaning| +| :--- | :---- | :---- | +|All Area| ALL_AREAS| A set of all the Areas defined in the Model.| +|All ViewContent| ALL_VIEWCONTENTS| A set of all the ViewContents defined in the Model.| +|IF-THEN-ELSE Syntax| IF (<Bool expression\>)THEN <Expression\>ELSE <Expression\>| The type of the expression must be identical, for the THEN clause, if the Bool expression is true, for THEN and ELSE clause ,if the Bool expression is false.| +|Let Expression| let <Variable name\> = <Object expression\>| Evaluate <Object expression\> as variable <variabl name>, which can then be referenced in subsequent expressions| +|||Can only be used inside the expression. For-All, Exists, Max, Min, Select| +|Pre-arbitration value specifier| (pre)<Area \| ViewContent \| Scene\| Scene.Property\> | Refere to the state before arbitration of Area, ViewContent, Property.| + +**The syntax precedence and associativity are shown below:** + +|Priority| Name| Notation| Connectivity| +| :--- | :---- | :---- | :---- | +|1 |Parentheses| (<Expression\>) |-| +|2| ∃(Exists)| Exists<Set\>{ <x\> \| <Expression\> }|-| +|| ∀(For All)| For-All<Set\>{ <x\> \| <Expression\> }| -| +|| IF-THEN-ELSE Syntax| IF (<Bool expressinon\>) THEN <Expressinon\> ELSE <Expressinon\>| -| +||SetOf operator| { <Expression\>, <Expression\> }| -| +|| Pre-arbitration value specifier| (pre)<Area\| ViewContent\| Scene\| Scene.Property\>| -| +|3|Predicate connector |<Object\>.<Predicate expression\>| Left| +|4 |Negation| ! <Bool expression\>| Right| +|5| Comparison (less than)| <Number\> < <Number\>| Left| +|| Comparison (greater than)|<Number\> \> <Number\>|Left| +|| Comparison (less than)| <Number\> <= <Number\>| Left| +|| Comparison (greater than)| <Number\> \>= <Number\>| Left| +|6| Equal sign (Comparison of values)| <Expressinon\> = <Expressinon\>| Left| +|| Equal sign (Comparison of objects)| <Expressinon\> == <Expressinon\>| Left| +|7| AND| <BoolExpressinon\> AND <BoolExpressinon\> |Left| +|8 |OR| <BoolExpressinon\> OR <BoolExpressinon\>| Left| +|9 |Implication| <BoolExpressinon\> -\> <BoolExpressinon\>| Left| + + +#### 8.1 Constraint + +Constraint describes constraint expressions. There are two types of constraint expressions Runtime constraints and Offline constraints. A Runtime constraint is a constraint expression that should be be true at the time of arbitration and controls the behavior of arbitration. An Offline constraint is a constraint expression that should be satisfied after arbitration and tests the arbitration result. + +Syntax: +```shell +Constraint [ID] { + description: [String] + runtime: [true|false] + [expression] +} +``` + +|Syntax element |Description| +| :--- | :---- | +|description: 0..1| Description| +|runtime: 1 |true: Runtime constraint| +||Arbitrate each Area / Zone to be true this constraint.| +||false: Offline constraint| +||Verify that this constraint is true after all Area arbitration.| +|[expression] 1| Constraint expression| + +Description example + +- AND/ OR/ Negation +<br>In the constraint expression, it is possible to express logical AND, logical OR, negation, which is a general logical operator, with AND, OR, and !. By using () you can also use in combination. + +Example: Content1 is displayed or Content2 and Content3 are not displayed at the same time. +```shell +Constraint SampleConstraint1 { +runtime: true +Content1.isVisible() OR !(Content2.isVisible() AND Content3.isVisible()) +} +``` +Example: SampleContent1 is displayed or SampleContent2 is not displayed. +```shell +Constraint SampleConstraint1 { + description: "Sample Constraint Expression" + runtime: true + SampleContent1.isVisible() OR !SampleContent2.isVisible() +} +``` + +- Implication + +Implications are false only if the left-hand side is true and the right-hand side is false, otherwise it is true. + +Example: SampleContent4 is displayed if there is a request of SampleContent4. +```shell +Constraint SampleConstraint3 { + runtime: true + SampleContent4.isActive() -> SampleContent4.isVisible() +} +``` + +Example: If SampleArea1 displays SampleContent3, SampleArea2 does not display SampleContent3. +```shell +Constraint SampleConstraint2 { + runtime: true + (SampleArea1.displayingContent() == SampleContent3) +-> !(SampleArea2.displayingContent() == SampleContent3) +} +``` + +Example: If the value of the content displayed on SampleArea1 is higher than the value of the content displayed in SampleArea2, hide SampleArea2. +```shell +Constraint SampleConstraint2 { + runtime: true + (SampleArea1.contentValue() > SampleArea2.contentValue()) +-> SampleArea2.isHidden()) +} +``` + +Example: If the value of property1 of the information received from other RBAModel (Project) is 1, Content10 is not displayed. +```shell +Constraint SampleConstraint { + (SampleScene1.isOn() AND SampleScene1.property1.get() = 1) + -> ! Content10.isVisible() +} +``` + +- For-All + +For-All targets a set, and it is true if all element of the set satisfies a lambda expression ({element name declaration | element condition}). +For the set, you can specify a defined set or an One-time set. (Refer to “Group definition” for more informations.) +<br>Example: If SampleContent1 is displayed, all Areas of AreaGroup1 is not displayed. +<br> It is assumed that "AreaGroup1" is defined as AreaSet. + +```shell +Constraint SampleConstraint { + runtime: true + SampleContent1.isVisible() -> +For-All AreaGroup1 { x | x.isHidden() } +} +``` + +- Exists + +Exists targets a set, and it is true if even one element of the set satisfies a Lambda expression ({element name declaration | element condition}). +<br>For the set, you can specify a defined set or an One-time set. (Refer to “Group definition” for more informations.) +<br>Example: If any Content of ContentGroup1 is displayed, all Areas of AreaGroup1 is not displayed. +<br> It is assumed that "ContentGroup1" is defined as ContentSet. +<br> It is assumed that "AreaGroup1" is defined as AreaSet. +```shell +Constraint SampleConstraint { + runtime: true + Exists ContentGroup1{ x | x.isVisible() } -> +For-All AreaGroup1 { x | x.isHidden() } +} +``` + +- IF-THEN-ELSE +Example: If the scene is SampleScene1, SampleContent4 is displayed, otherwise SampleContent4 is not displayed. + +```shell +Constraint SampleConstraint4 { + runtime: true + IF(SampleScene1.isOn()) + THEN + SampleContent4.isVisible() + ELSE + !SampleContent4.isVisible() +} +``` + +- (pre) +Example: If SampleContent 1 is displayed (before arbitration), SampleContent 2 is not displayed. +```shell +Constraint SampleConstraint { + runtime: true +(pre)SampleContent1.isVisible() -> !SampleContent2.isVisible() +} +``` + +- HasComeEarlierThan/HasComeLaterThan +Example: If request SampleContent 1 has come earlier then SampleContent 2, SampleContent 2 does not displayed. +```shell +Constraint SampleConstraint { + runtime: true +SampleContent1.hasComeEarlierThan(SampleContent2) -> !SampleContent2.isVisible() +} +``` +Below constraint behave as same as above. +```shell +Constraint SampleConstraint { + runtime: true +SampleContent2.hasComeLaterThan(SampleContent1) -> !SampleContent2.isVisible() +} +``` + +#### 8.2 Syntax sugar + +This syntax sugar simplifies the constraint expressions and improves their readability. You can use them like the existing constraint expressions. +Below are the syntax sugars that can be used. + +- Inequality（!=） +<br>It means that the left side value and the right side value are not equal. You can use this to compare objects. +True if the left side value and the right side value are not equal. +The types of expressions on the left and the right sides must match. + +|Type| Operator| +| :--- | :---- | +|Notation| <Expression1\> != <Expression2\>| +|ECE*| !(<Expression1\> == <Expression2\>)| + +*ECE: Equivalent constraint expression + +Description example: The ViewContent allocated to AreaA is not Content1. + +||| +| :--- | :---- | +|Notation| AreaA.allocatedContent() != Content1| +|ECE| !( AreaA.allocatedContent() == Content1)| + +- Allocation of Area/Zone +<br>It indicates that ViewContent/SoundContent is allocated to the Area/Zone. +True if theViewConten/SoundContentt is allocated to the Area/Zone. + +|Type| State reference of ViewContent| +| :--- | :---- | +|Notation| <ViewContent\>.isAllocatedTo(<Area\>)| +|ECE| <Area\>.allocatedContent() == <ViewContent\>| + +Description example: The Content1 is allocated to the AreaA. + +||| +| :--- | :---- | +|Notation| Content1.isAllocatedTo(AreaA)| +|ECE| AreaA.allocatedContent() == Content1| + +- Allocation changing of Area/Zone +<br>It indicates a changing of the ViewContent’s/SoundContent’s allocated to the Area/Zone. +True if the changing happens. + +|Type |State reference of Area| +| :--- | :---- | +|Notation| <Area\>.isChanged()| +|ECE| !((pre)<Area\>.allocatedContent() == <Area\>.allocatedContent())| + +Description example: The ViewContent allocated to AreaA has changed. + +||| +| :--- | :---- | +|Notation| AreaA.isChanged()| +|ECE| !((pre)AreaA.allocatedContent() == AreaA.allocatedContent())| + +- Changing of a content allocated to Area/Zone +<br>It indicates whether a ViewContent/SoundContent allocated to the Area/Zone has changed to the specified ViewContent/SoundContent.True if the changing happens + +|Type |State reference of Area| +| :--- | :---- | +|Notation| <Area\>.isTranslatedTo(<ViewContent\>)| +|ECE| !((pre)<Area\>.allocatedContent() == <ViewContent\>) AND (<Area\>.allocatedContent() == <ViewContent\>)| + +Description example: A ViewContent allocated to the AreaA has changed to Content1. + +||| +| :--- | :---- | +|Notation| AreaA.isTranslatedTo(Content1)| +|ECE |!((pre)AreaA.allocatedContent() == Content1) AND (AreaA.allocatedContent() == Content1)| + +- Displaying in the Area +<br>It indicates whether the ViewContent displayed or not in the Area. +True if the ViewContent is displayed in the Area. + +|Type| State reference of ViewContent| +| :--- | :---- | +|Notation| <ViewContent\>.isDisplayedOn(<Area\>)| +|ECE| <Area\>.isDisplayed() AND (<Area\>.displayingContent() == <ViewContent\>)| + +Description example: The Content1 is displayed in the AreaA. + +||| +| :--- | :---- | +|Notation| Content1.isDisplayedOn(AreaA)| +|ECE| AreaA.isDisplayed() AND (AreaA.displayingContent() == Content1)| + +- Display changing of the Area +<br>It indicates whether the display of the Area changes or not. +True if the change happens. + +|Type |State reference of Area| +| :--- | :---- | +|Notation| <Area\>.isChangedDisplay()| +|ECE |!((pre)<Area\>.displayingContent() == <Area\>.displayingContent()) OR ((pre)<Area\>.isDisplayed() AND !<Area\>.isDisplayed()) OR (!(pre)<Area\>.isDisplayed() AND <Area\>.isDisplayed())| + +Description example: The display of the AreaA has changed. + +||| +| :--- | :---- | +|Notation |AreaA.isChangedDisplay()| +|ECE |!((pre)AreaA.displayingContent() == AreaA.displayingContent()) OR((pre)AreaA.isDisplayed() AND !AreaA.isDisplayed()) OR(!(pre)AreaA.isDisplayed() AND AreaA.isDisplayed())| + +- Changing of the displayed ViewContent to the specified ViewContent +<br>It indicates whether the ViewContent displayed in the Area has changed to anther specified ViewContent. +True if the change happens. + +|Type |State reference of Area| +| :--- | :---- | +|Notation| <Area\>.isTranslatedViewTo(<ViewContent\>)| +|ECE |(<Area\>.isDisplayed()) AND (<Area\>.displayingContent() == <ViewContent\>) AND (!((pre)<Area\>.displayingContent() == <ViewContent\>) OR !(pre) <Area\>.isDisplayed())| + +Description example: A ViewContent displayed in the AreaA has changed to another Content1. + +||| +| :--- | :---- | +|Notation| AreaA.isTranslatedViewTo(Content1)| +|ECE| (AreaA.isDisplayed()) AND (AreaA.displayingContent() == Content1) AND (!((pre)AreaA.displayingContent() == Content1) OR !(pre)AreaA.isDisplayed())| + +- Hide a lower priority Area +<br>Compare the two Areas and hide the Area displaying a ViewContent with lower priority. + +|Type| State reference of Area| +| :--- | :---- | +|Notation| HideLowerPriority(<Area1\>, <Area2\>)| +|ECE| ((<Area1\>.contentValue() < <Area2\>.contentValue() -\> <Area1\>.isHidden()) AND (<Area1\>.contentValue() \> <Area2\>.contentValue() -\> <Area2\>.isHidden()))| + +Description example: AreaA and AreaB are compared to hide the Area which displays a ViewContent with lower priority. + +||| +| :--- | :---- | +|Notation| HideLowerPriority(AreaA, AreaB)| +|ECE| ((AreaA.contentValue() < AreaB.contentValue() -\> AreaA.isHidden()) AND (AreaA.contentValue() \> AreaB.contentValue() -\> AreaB.isHidden()))| + +### 9. Group definition + +#### 9.1 AreaSet + +When dealing with multiple Areas in the Constraint expression, in order to describe it simply, you can define multiple Areas together in one group. + +Syntax: +```shell +AreaSet [ID] { + description: [String] + target: [ [ID of Area] ] +} +``` + +|Syntax element | Description| +| :--- | :---- | +|description: | 0..1 Description| +|target: 0..* | Areas or AreaSets which compose the group. Specify the ID of the Area or the AreaSet with comma-separated.| + +Description example: +```shell +AreaSet MainScreen { + description: "Area constituting the main screen" + target: [SampleArea, SampleArea1, OtherAreaSet] +} +``` +#### 9.2 ViewContentSet + +When dealing with multiple ViewContents in the Constraint expression, in order to describe it simply, you can define multiple ViewContents together in one group. + +Syntax: +```shell +ViewContentSet [ID] { + description: [String] + allocatable: [ [ID of Area] ] + target: [ [ID of ViewContent] ] +} +``` + +|Syntax element | Description| +| :--- | :---- | +|description: | 0..1 Description| +|allocatable: 0..*| Displayable Areas or AreaSets.Specify the ID of the Areas or the AreaSets with comma-separated.| +|target: 0..*| ViewContents or ViewContentSets which compose the group. Specify the ID of the ViewContents or ViewContentSets with comma-separated.| + +Description example: +```shell +ViewContentSet InterruptContent{ + allocatable: [ + MessageArea, HUDMessageArea, InterruptAreaGroup + ] + target: [ + TEL,Power, ViewContentGroup + ] +} +``` + +#### 9.3 One-time Set + +Specify an set in the For-All or the Exists constraint, you can define the One-time set. + +Syntax: +```shell +{ [ID] } +``` + +|Syntax element | Description| +| :--- | :---- | +|[ID] 1..* |Element of the set. Specify the IDs of elements with comma-separated. +||The type of elements should be same.| +||At this time, Area and AreaSet regard it as the same type.( ViewContent and ViewContentSet are treat similarly.) + +- Description example: Define the One-time set as AreaSet. +```shell +Constraint SampleConstraint { + runtime: true + SampleContent1.isVisible() -> +For-All {Area1, Area2, Area3} { x | x.isHidden() } +} +``` + +- Description example: Define the One-time set as ViewContentSet. +```shell +Constraint SampleConstraint { + runtime: true + Exists {ViewContent1, ViewContentSet1} { x | x.isActive() -> x.isVisible() } +} +``` +### 10. Scene definition +#### 10.1 Scene +Scene comprehensively expresses the state at the time including the system, and it is used to switch the status of Area/Zone, View/Sound Content by Scene. You can also define the Global scene and share the results of arbitration among multiple RBA Models (Projects). + +Syntax: +```shell +Scene [ID] { + description: [String] + global: [true|false] + [Property] +} +``` + +|Syntax element | Description| +| :--- | :---- | +|description: 0..1| Description| +|global: 0..1| true: Global Scene for sharing arbitration results with other RBA Models (Projects).| +||false: Local Scene.| +||If not defined, consider it as "false".| +|[Property] 0..*| Information for sharing arbitration results with other RBA models (Projects).| +||Can define only if global is true.| + +Description example: +```shell +Scene AutoDrivingScene { + description: "In auto driving mode" +} + +Scene DisplayEventNotification { + global: true + int OverallPriority: 6 + int RiskSeverity: 3 + int RiskMargin: 1 +} +``` +#### 10.2 Property + +Property is arbitrary properties which is defined in the Scene. + +Syntax: +```shell +int [String]: [Number] +``` + +|Syntax element | Description| +| :--- | :---- | +|[String] 1 |Arbitrary Property name| +|[Number] 1 |Default value of Property. Positive integer can be specified.| + +Description example: +```shell +int RiskSeverity: 3 +``` + +### 11. Stereotype + +Syntax: +```shell +StereoType<[Target Model Name]> [ID] ([valiable]) @BEGIN@ +description: [String] + [Propaties of Element] +@END@ +``` + +|Syntax element | Description| +| :--- | :---- | +|description: 0..1| Description| +|[Target Model Name] 1| Identify the applicable model by describing the model element keywords| +||It can be applied to the Area, ViewContent models| +|[valiable] 0..*| Describe an arbitrary variable as the parameter to be replaced, which will be set in the property of the model element specified in [Target Model Name].If there are multiple variables, separate them with commas.| +|[Properties of Element] 1..*| Describe the properties of the model element specified in [Target Model Name].In the property, the value to be replaced is written in @{[variable]} .| +||The variables must match the variables described in ().| + +Description example: +```shell +Stereotype<ViewContent> display_warnning (name,allocatable,priority) @BEGIN@ + description: “@{name}” + allocatable: [@{allocatable}] + Size WarnSize { + width: 300 + height: 100 + } + State NORMAL { piority: @{priority} } + +@END@ +``` + +### 12. Tag + +Syntax: +```shell +<<[Stereotype ID]>> | <<[Stereotype ID]>>{“[String]”} +``` + +|Syntax element | Description| +| :--- | :---- | +|[ID of Stereotype] 0..*| Can be assigned to Area, ViewContent| +||If StereoType is defined, then the child elements and attributes defined there will be expanded as child elements and attributes of this object| +||Multiple definitions can be made for a single model. To define more than one, define them consecutively.| +|[String] 0..* |Argument value| +||If there is no argument, it can be omitted| + +Description example: +```shell +ViewContent Warning_1 { + <<display_warning>>{ “Warning_01”,”Area1”,“1” } +} + +ViewContent Warning_2 { + <<display_warning>>{ “Warning_02”,”Area2”,”2” } +} + +Stereotype<ViewContent> display_warnning (name,allocatable,priority) @BEGIN@ + description: “@{name}” + allocatable: [@{allocatable}] + Size WarnSize { + width: 300 + height: 100 + } + State NORMAL { piority: @{priority} } +@END@ +``` + +## Generate .json from .rba file +Download [prebuilt package](https://git.automotivelinux.org/staging/rba-tool/tree/tool_bin) + +- If some errors occur, RBAModel.json is not generated (exit code 1). +```` +java -cp ./ -jar JSONGenerator.jar "[path to model directory]" "[path to output directory]" +```` + +Example: + +- RBAModel.json file is generated in directory same as JSONGenerator.jar +```` +java -cp ./ -jar JSONGenerator.jar "./RBAModel.rba" +```` + +- RBAModel.json file is generated in ~/ directory +```` +java -cp ./ -jar JSONGenerator.jar "./RBAModel.rba" "~/" +```` +Note: For the reference .rba file is given under the path [prebuilt package/sample_model](https://git.automotivelinux.org/staging/rba-tool/tree/tool_bin/sample_model) diff --git a/docs/6_Component_Documentation/5_drm-leasemanager.md b/docs/6_Component_Documentation/5_drm-leasemanager.md new file mode 100644 index 0000000..3b7b2e9 --- /dev/null +++ b/docs/6_Component_Documentation/5_drm-leasemanager.md @@ -0,0 +1,108 @@ +--- +title: DRM lease manager +--- + +# DRM Lease Manager + +## Overview + +The DRM Lease Manager is used in AGL to allocate display controller outputs to different processes. Each process has direct access to its allocated output via the kernel DRM interface, and is prevented from accessing any other outputs managed by the display controller. + +The display controller partitioning is made possible by using the kernel DRM Lease feature. For more information on the DRM lease functionality, please see the Linux kernel [DRM userspace API documentation](https://www.kernel.org/doc/html/latest/gpu/drm-uapi.html). + + +## Usage + +### Building DRM Lease manager and sample clients + +Enable the `agl-drm-lease` AGL feature when setting up your build environment +with aglsetup.sh. + +This will add the drm-lease-manager package to the image, and will add DRM +lease support to the following packages: + +* weston +* kmscube + +kmscube is not included in the image by default. To add the package to the +image, add the following to your local.conf + +``` +IMAGE_INSTALL_append = " kmscube" +``` + +### Starting the DRM lease manager + +The drm-lease-manager must be the only process to directly open the DRM device. +Shut down any running window systems (eg. weston or agl-compositor) and run: + +```shell + systemctl start drm-lease-manager +``` + +This will create a lease for each output connection on the platform. +The name of each lease will be in the form of `card0-<output name>` + (eg. `card0-LVDS-1` or `card0-HDMI-A-1`) + +**Note**: `drm-lease-manager` can be started on a different display controller (i.e. not `card0`) by modifying the command line in the systemd unit file. + +Run the help command for details. + +```shell + drm-lease-manager --help +``` + +### Running weston + +weston can be started on any available DRM lease by running with the +`--drm-lease=<lease name>` option. + +```shell + weston --drm-lease=card0-HDMI-A-1 +``` + +### Running kmscube sample +With the `drm-lease-manager` running, `kmscube` can display on any available +lease by launching it with the `-L -D<lease name>` options. + +```shell + kmscube -L -Dcard0-HDMI-A-1 +``` + +Multiple weston or kmscube instances (one per DRM lease) can be started at the same time. + +## Adding support to your application + +The DRM Lease Manager packages includes a client library (libdlmclient) to request access to the DRM leases it creates. The client library provides file descriptors that can be used as if they were created by directly opening the underlying DRM device. + +Clients that want _direct access_ to the DRM device can use this library to do so. Compositor (agl-compositor or weston) client applications do not need to use this interface. + + +To use the DRM leases, clients only need to replace their calls to +`drmOpen()` and `drmClose()` with the appropriate libdlmclient API calls. + +### libdlmclient API usage + +_Error handling has been omitted for brevity and clarity of examples._ + +#### Requesting a lease from the DRM Lease Manager + +```c + struct dlm_lease *lease = dlm_get_lease("card0-HDMI-A-1"); + int drm_device_fd = dlm_lease_fd(lease); +``` + +`drm_device_fd` can now be used to access the DRM device + +#### Releasing a lease + +```c + dlm_release_lease(lease); +``` + +**Note: `drm_device_fd` is not usable after calling `dlm_release_lease()`** + +## Runtime directory +A runtime directory under `/var` is used to keep the Unix Domain sockets that the drm-lease-manager and clients to communicate with each other. + +The default path is `/var/run/drm-lease-manager`, but can be changed by setting the `DLM_RUNTIME_PATH` environment variable. diff --git a/docs/6_Component_Documentation/6_application_framework.md b/docs/6_Component_Documentation/6_application_framework.md new file mode 100644 index 0000000..1af8796 --- /dev/null +++ b/docs/6_Component_Documentation/6_application_framework.md @@ -0,0 +1,7 @@ +--- +title: Application Framework +--- + +# AppFW + +FIXME. diff --git a/docs/6_Component_Documentation/7_pipewire_wireplumber.md b/docs/6_Component_Documentation/7_pipewire_wireplumber.md new file mode 100644 index 0000000..e58b97d --- /dev/null +++ b/docs/6_Component_Documentation/7_pipewire_wireplumber.md @@ -0,0 +1,159 @@ +--- +title: PipeWire / WirePlumber +--- + +# PipeWire / WirePlumber + +## Overview + +AGL uses the PipeWire daemon service to provide audio playback and capture capabilities. +PipeWire is accompanied by a secondary service, WirePlumber (also referred to as the +*session manager*), which provides policy management, device discovery, configuration and more. + +Applications can connect to the PipeWire service through its UNIX socket, by using either the +*libpipewire* or *libwireplumber* libraries as a front-end to that socket. + +Upstream documentation for these components can be found at the links below: + +- [PipeWire documentation](https://docs.pipewire.org/) + +- [WirePlumber documentation](https://pipewire.pages.freedesktop.org/wireplumber/) + +- [PipeWire Wiki](https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/home) + +## APIs + +### libpipewire + +The main entry point for applications to access the audio system is the API offered by +*libpipewire*. The functionality offered by *libpipewire* is vast and it is beyond the +scope of this document to describe it all. + +For playback and capture, applications should use *struct pw_stream* and its associated methods. +See [PipeWire: Tutorial - Part 4: Playing a tone](https://docs.pipewire.org/page_tutorial4.html) +for a starting point. + +### GStreamer (Recommended) + +For convenience, applications that use GStreamer can use the PipeWire GStreamer elements to +plug the functionality offered by *struct pw_stream* directly in the GStreamer pipeline. These +elements are called *pipewiresrc* and *pipewiresink* + +Example: + +```shell +> gst-launch-1.0 audiotestsrc ! pipewiresink +``` + +Through these elements, it is possible to specify the application role by setting it in the +*stream-properties* property of the element, as shown below: + +```shell +> gst-launch-1.0 audiotestsrc ! pipewiresink stream-properties="p,media.role=Multimedia"" +``` + +or, in the C API: + +```c +gst_util_set_object_arg (sink, "stream-properties", "p,media.role=Multimedia"); +``` + +Of course, it is also possible to use *alsasink* and *alsasrc* and route audio through the +virtual ALSA device that is mentioned below. This is also the default behavior of *playbin* +and similar auto-plugging elements, because the PipeWire GStreamer elements are not autoplugged +(this may change in a future version). + +### ALSA + +PipeWire offers a virtual ALSA device (called *pipewire*) that redirects audio to PipeWire +through an ALSA PCM plugin. This device is the default one, so unless you explicitly specify +a device in your ALSA client application, audio will go through PipeWire instead. + +Example: + +```shell +> aplay sound.wav # the default device is 'pipewire' +> aplay -D pipewire sound.wav +``` + +In order to specify the application role while using the ALSA compatibility device, pass the role +as a device parameter like this: + +```shell +> aplay -D pipewire:ROLE=Navigation turnleft.wav +``` + +### Audiomixer service + +See the separate [agl-service-audiomixer](https://git.automotivelinux.org/apps/agl-service-audiomixer/about/) documentation. + +### libwireplumber + +The WirePlumber library provides API that wraps libpipewire and makes it easier to work with +when you are writing control applications, such as a volume mixer. The audiomixer service is in +fact implemented using *libwireplumber*. + +WirePlumber also provides support for lua-based scripting. Standalone scripts, executed with the +*wpexec* tool, may be used as a means to rapidly make use of the API provided by *libwireplumber* + +## Tools + +* **wpctl**: allows inspecting the devices, choosing which source & sink are the default ones + and allows volume/mute adjustments to be made on the command line. Try `wpctl status` and + `wpctl help` to get started with it + +* **wpexec**: allows running wireplumber lua scripts standalone, which is useful to implement + custom scripts to interact with PipeWire + +* **pw-cli**: this is the main tool for interacting with pipewire directly + +* **pw-dump**: dumps all the objects in the pipewire graph to a big JSON. The output of this + tool is very useful to include in bug reports. It is also suitable for implementing scripts + that parse information with jq + +* **pw-dot** is a useful debug tool that dumps the objects in a dot graph for easy visualization + +* **pw-cat / pw-play / pw-record**: This is a set of tools similar to aplay/arecord, for simple + audio operations + +* **pw-top**: This is a performance measurement tool that shows realtime timing information + about the audio pipeline. Before running this tool, you will need to uncomment the loading + of "libpipewire-module-profiler" in /etc/pipewire/pipewire.conf and restart pipewire + +## Systemd Integration + +The PipeWire service, `pipewire.service`, is activated on demand, via systemd socket activation, +by `pipewire.socket`. The WirePlumber service, `wireplumber.service`, is bound to the pipewire +service and therefore started and stopped together with the PipeWire service. + +If you wish to manually stop or restart both services, you can do so by using *systemctl*, +operating on the *.socket* unit: + +```shell +> systemctl restart pipewire.socket +> systemctl stop pipewire.socket +``` + +## Debugging + +The PipeWire daemon can be configured to be more verbose by editing +**/etc/pipewire/pipewire.conf** and setting `log.level=n` (n=0-5) in section +`context.properties`. + +Similarly, the WirePlumber daemon can be configured to be more verbose by editing +**/etc/wireplumber/wireplumber.conf** and setting `log.level=n` (n=0-5) in section +`context.properties`. + +All messages will be available in the systemd journal, inspectable with journalctl. + +For applications, at runtime, `PIPEWIRE_DEBUG` can be set to a value between 0 and 5, +with 5 being the most verbose and 0 the least verbose. + +For applications that use *libwireplumber* the equivalent environment variable is +`WIREPLUMBER_DEBUG`, which also takes values between 0 and 5. + +The default debug level for the daemons is 2, while for applications it's 0 +(with few exceptions). + +More information is also available on +[WirePlumber's documentation on debug logging](https://pipewire.pages.freedesktop.org/wireplumber/daemon-logging.html) diff --git a/docs/6_Component_Documentation/8_ic-sound-manager.md b/docs/6_Component_Documentation/8_ic-sound-manager.md new file mode 100644 index 0000000..75e163e --- /dev/null +++ b/docs/6_Component_Documentation/8_ic-sound-manager.md @@ -0,0 +1,120 @@ +# Instrument Cluster Sound Management + +## Introduction + +This document describes the design of the software setup which enables the integration +of AGL’s sound system with applications running in the Instrument Cluster domain. +This software setup is specific to the case where a single system is used to implement +both the Instrument Cluster and some other domain of the vehicle, typically the +In-Vehicle-Infotainment domain, using container technology to separate them. + +Applications running in the Instrument Cluster need a way to safely play important +sounds to alert the driver of conditions that need the driver’s attention. At the same +time, in a containerized environment that serves multiple vehicle domains, applications +running in other containers may be using the sound hardware to play less important sounds, +such as music, which conflicts with the IC’s need to play sound on the same hardware. + +The solution developed here, for safety reasons, relies on the operating system and the +hardware itself to allow the IC applications to stream sounds to the speakers using a +dedicated device handle, while applications from other domains are all routed through a +sound server that runs on the host container and operates on a different sound device handle. + +However, to achieve good inter-operation, there is need for additional software mechanisms +that will work in combination with this hardware-based solution. First of all, it is necessary +to have a mechanism that allows IC applications to pause all sounds that are being routed via +the sound server while there is an important IC sound playing and resume them afterwards. +This is so that other domain applications can be informed of this temporary pause and offer +the appropriate user experience. Secondly, it is desirable to have separation of duties +between the host and the other domain’s (non-IC) container. It should be the responsibility +of the other domain’s container to implement the sound system policy, so that the host does +not need to be aware of the exact applications that are running on this container. + +## Requirements + +- Single system shared between IC and at least one secondary domain (IVI, other ...) + +- The domains are separated using containers + +- All the containers, including the host, are running a variant of AGL + +- The host OS and the secondary domain container use PipeWire and WirePlumber + to implement the sound system + +- The sound hardware offers, on the Linux kernel driver side, a separate ALSA + device for sounds that belong to the IC and a separate ALSA device for other sounds + +## Architectural design + + + +The core of the sound system consists of the PipeWire daemon, which is responsible for routing +audio between the kernel and applications running in the “Other Container”. + +The PipeWire session is orchestrated by a secondary daemon, WirePlumber. WirePlumber is +designed in such a way so that it can have multiple instances, for task separation. +One instance shall be running in the host container and it shall be responsible for +managing the devices that PipeWire handles as well as the security isolation between +different applications and different containers. At least one more instance shall be +running in the “Other Container” and be responsible for implementing policy mechanisms +related to the applications that are running in that container. + +Further WirePlumber instances are possible to run as well. For instance, it may be desirable +to have another “policy” instance in a third container that implements another vehicle system +and shares the main PipeWire daemon from the host. Additionally, the “Other Container” may +be running a separate WirePlumber instance to manage bluetooth audio devices, which shall be +the responsibility of that container instead of the host. + +To implement communication between the IC and the host, a third daemon is used: pipewire-ic-ipc. +This daemon listens on a UNIX domain socket for messages from the IC applications and offers +them the ability to pause or resume sounds that are being routed via PipeWire. + +Finally, IC applications are given a library (icipc library) that allows them to send messages +to pipewire-ic-ipc on the host. This library is minimal and has no external dependencies, +for safety reasons. + +For sound playback, IC applications are expected to use the ALSA API directly and communicate +with the dedicated ALSA device that is meant for IC sounds. Arbitration of this device between +different IC applications is out of scope for this document and it is assumed to be a solved +problem. + +### PipeWire-IC-IPC + +This component acts as the server-side component for the UNIX socket that is used for +communication between the IC applications and the host. It is implemented as a pipewire module, +therefore it needs the `/usr/bin/pipewire` process in order to be launched. Launching happens +with a special configuration file (`pipewire-ic-ipc.conf`) which instructs this PipeWire process +to be launched as a client (`core.daemon = false`) and to load only `module-ic-ipc` together +with `module-protocol-native`. The latter enables communication with the daemon instance of +PipeWire (`core.daemon = true`), which implements the sound server. + + + +### icipc library + +The IC Application is given a library (‘libicipc’) that implements the client side of +pipewire-ic-ipc. This library allows sending two commands: + +- SUSPEND + - Asks WirePlumber (via PipeWire) to cork applications and mute the ALSA device used by PipeWire +- RESUME + - Reverts the effects of SUSPEND + +IC Applications are expected to send the SUSPEND command before starting playback of a sound +to their dedicated ALSA device. The RESUME command should be sent after playback of this IC +sound has finished. + +It should be noted that the RESUME command is also issued automatically when the IC application +disconnects from the pipewire-ic-ipc UNIX socket. + +If multiple IC application issue SUSPEND to the pipewire-ic-ipc server, then only the first +SUSPEND generates actions for WirePlumber. The rest are counted and the pipewire-ic-ipc +server expects an equal number of RESUME commands before generating resume actions for +WirePlumber. + +The implementation of the SUSPEND/RESUME mechanism uses PipeWire’s metadata to signal +WirePlumber. PipeWire-IC-IPC will look for the “default” metadata object in PipeWire’s list +of objects and will write the “suspend.playback” key with a value of “true” on id 0. +The metadata change is then notified to all clients. WirePlumber, being a client, gets +notified of this change and takes actions. All actions are defined in Lua scripts. + + diff --git a/docs/6_Component_Documentation/images/agl-compositor/arch_diagram.png b/docs/6_Component_Documentation/images/agl-compositor/arch_diagram.png Binary files differnew file mode 100644 index 0000000..88a4381 --- /dev/null +++ b/docs/6_Component_Documentation/images/agl-compositor/arch_diagram.png diff --git a/docs/6_Component_Documentation/images/agl-compositor/drawing_shell.png b/docs/6_Component_Documentation/images/agl-compositor/drawing_shell.png Binary files differnew file mode 100644 index 0000000..bcd0a98 --- /dev/null +++ b/docs/6_Component_Documentation/images/agl-compositor/drawing_shell.png diff --git a/docs/6_Component_Documentation/images/ic-sound-manager/architecture.png b/docs/6_Component_Documentation/images/ic-sound-manager/architecture.png Binary files differnew file mode 100644 index 0000000..3d0820f --- /dev/null +++ b/docs/6_Component_Documentation/images/ic-sound-manager/architecture.png diff --git a/docs/6_Component_Documentation/images/ic-sound-manager/pipewire-ic-ipc-calls.png b/docs/6_Component_Documentation/images/ic-sound-manager/pipewire-ic-ipc-calls.png Binary files differnew file mode 100644 index 0000000..9003e60 --- /dev/null +++ b/docs/6_Component_Documentation/images/ic-sound-manager/pipewire-ic-ipc-calls.png diff --git a/docs/6_Component_Documentation/images/ic-sound-manager/pipewire-ic-ipc-processes.png b/docs/6_Component_Documentation/images/ic-sound-manager/pipewire-ic-ipc-processes.png Binary files differnew file mode 100644 index 0000000..494b760 --- /dev/null +++ b/docs/6_Component_Documentation/images/ic-sound-manager/pipewire-ic-ipc-processes.png diff --git a/docs/6_Component_Documentation/images/rba/Basic_syntax.png b/docs/6_Component_Documentation/images/rba/Basic_syntax.png Binary files differnew file mode 100644 index 0000000..6704abd --- /dev/null +++ b/docs/6_Component_Documentation/images/rba/Basic_syntax.png diff --git a/docs/6_Component_Documentation/images/rba/model.png b/docs/6_Component_Documentation/images/rba/model.png Binary files differnew file mode 100644 index 0000000..e43289e --- /dev/null +++ b/docs/6_Component_Documentation/images/rba/model.png |
