3_Developer_Guides: add new app framework documentation

Starting with `marlin`, the old AppFW has been phased out and replaced
by a new implementation, relying mainly on standard technologies and
software. This commit documents the current state of the new Application
Framework so developers can create compliant applications.

Bug-AGL: SPEC-4236
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
Change-Id: I4cf52799426efbc2401b8785aaa57cc925a4968f
Reviewed-on: https://gerrit.automotivelinux.org/gerrit/c/AGL/documentation/+/27166
Reviewed-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Tested-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>

4 files changed, 591 insertions, 2 deletions

diff --git a/docs/3_Developer_Guides/1_Application_Framework/1_Introduction.md b/docs/3_Developer_Guides/1_Application_Framework/1_Introduction.md
new file mode 100644
index 0000000..9087254
--- /dev/null
+++ b/docs/3_Developer_Guides/1_Application_Framework/1_Introduction.md
@@ -0,0 +1,137 @@
+---
+title: Introduction
+---
+
+# Foreword
+
+The AGL Application Framework is nothing new. However, the implementation used up until
+the `lamprey` release has been retired starting with the `marlin` release and replaced
+by a redesigned Application Framework one. However, this new implementation isn't a 1:1
+replacement, and as such it doesn't provide all of the features of the previous
+Application Framework. Some of those will be added back over time, others have been
+discarded in favor of more modern and/or widely-used alternatives.
+
+# Introduction
+
+As a provider of an integrated solution to build up on, AGL needs to define a reliable
+and well-specified method for managing the deployment and integration of applications
+and services, as well as the way they can interact with the rest of the system.
+
+This is achieved by providing a common set of rules and components, known as the
+Application Framework. By ensuring conformity to those rules, application developers
+can have a good understanding of the requirements for creating and packaging
+applications targeting AGL-based systems. Likewise, system developers and integrators
+have a clear path for including such applications in AGL-based products.
+
+The Application Framework's scope extends to the following areas:
+- system services integration and lifecycle management
+- user session management, including user-level applications and services lifecycle
+  management
+- inter-process communication
+
+In order to be as simple as possible and avoid any unneded custom implementation, the
+Application Framework relies mainly on third-party technologies and/or software
+components, most of those being maintained under the
+[freedesktop.org](https://www.freedesktop.org) umbrella. Those include:
+- [systemd](https://www.freedesktop.org/wiki/Software/systemd/): system and services
+  management
+- [D-Bus](https://www.freedesktop.org/wiki/Software/dbus/): inter-process communication
+- [Desktop Entry specification](https://www.freedesktop.org/wiki/Specifications/desktop-entry-spec/):
+  application enumeration and startup
+
+AGL also provides reference implementations whenever possible and relevant, located in
+the [meta-agl](../6_AGL_Layers/2_meta-agl/) layer under `meta-app-framework`. At the
+moment, the Application Framework contains 2 such components:
+- `agl-session`: `systemd` unit files for user sessions management
+- `applaunchd`: application launcher service
+
+# Services management
+
+Both system and user services are managed by `systemd`, which provides a number of
+important features, such as dependency management or service monitoring: when starting
+a service, `systemd` will ensure any other units this service depends on are available,
+and otherwise start those dependencies. Similarly, `systemd` can automatically restart
+a crashed service, ensuring minimal downtime.
+
+`systemd` also provides an efficient first layer of security through its
+[sandboxing](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Sandboxing)
+and other security-related options.
+
+It is also well integrated with D-Bus and can be used for a more fine-grained control
+over D-Bus activated services: by delegating the actual service startup to `systemd`,
+developers can take advantage of some of its advanced features, allowing for improved
+reliability and security.
+
+Each service should be represented by a `systemd` unit file installed to the appropriate
+location. More details can be obtained from the [Creating a New Service](../2_Creating_a_New_Service/)
+document.
+
+# User session management
+
+Similarly, user sessions and the services they rely on are also managed by `systemd`.
+
+AGL provides 2 `systemd` units:
+
+1. `agl-session@.service` is a template system service for managing user sessions; it
+takes a username or UID as a parameter, creating a session for the desired user.
+Instanciating this service can be achieved by enabling `agl-session@USER.service`, for
+example by executing the following command on a running system:
+
+```
+$ systemctl enable agl-session@USER.service
+```
+
+By default, AGL enables this service as `agl-session@agl-driver.service`, running as
+user `agl-driver`.
+
+*Note: while you can create sessions for as many users as needed, only one instance of
+`agl-session@.service` is allowed per user.*
+
+2. `agl-session.target` is a user target for managing user services and their
+dependencies. It is started by `agl-session@.service`.
+
+By default, `agl-compositor` is part of this target. It is therefore automatically
+started for user `agl-driver`.
+
+Any other service needed as part of the user session should similarly depend on this
+target by appending the following lines to their unit file:
+
+```
+[Install]
+WantedBy=agl-session.target
+```
+
+# Inter-process communication
+
+In order to provide a "standard", language-independent IPC mechanism and avoid the need
+for maintaining custom bindings for each programming language to be used on top of AGL,
+the Application Framework promotes the use of [D-Bus](https://www.freedesktop.org/wiki/Software/dbus/)
+as the preferred way for applications to interact with services.
+
+Most services already included in AGL provide one or several D-Bus interfaces, and can
+therefore interact with D-Bus capable applications and services without requiring any
+additional component. Those services include, among others:
+- [ConnMan](https://git.kernel.org/pub/scm/network/connman/connman.git/): network connectivity
+- [BlueZ](http://www.bluez.org/): Bluetooth connectivity
+- [oFono](https://git.kernel.org/pub/scm/network/ofono/ofono.git): telephony and modem
+  management
+- [GeoClue](https://gitlab.freedesktop.org/geoclue/geoclue/-/wikis/home): geolocation
+
+# Application launcher service
+
+As mentioned above, the Application Framework follows the guidelines of the
+[Desktop Entry specification](https://www.freedesktop.org/wiki/Specifications/desktop-entry-spec/)
+for application enumeration and startup.
+
+As no simple reference implementation exists for this part of the specification, AGL
+provides an application launcher service named `applaunchd`. This service is part of the
+default user session, and as such is automatically started on session startup. It can
+therefore be considered always available.
+
+`applaunchd` enumerates applications installed on the system and provides a D-bus
+interface for services and applications to:
+- query the list of available applications
+- request the startup and/or activation of a specific application
+- be notified when applications are started or terminated
+
+`applaunchd` is described with more details in [the following document](2_Application_Startup/).
diff --git a/docs/3_Developer_Guides/1_Application_Framework/2_Application_Startup.md b/docs/3_Developer_Guides/1_Application_Framework/2_Application_Startup.md
new file mode 100644
index 0000000..4841ce5
--- /dev/null
+++ b/docs/3_Developer_Guides/1_Application_Framework/2_Application_Startup.md
@@ -0,0 +1,184 @@
+---
+title: Application Startup
+---
+
+# Introduction
+
+At system runtime, it may be necessary for applications to start other applications
+on demand. Such actions can be executed in reaction to a user request, or they may
+be needed to perform a specific task.
+
+In order to do so, running applications and services need an established way of
+discovering installed applications and executing those. The
+[Desktop Entry specification](https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html)
+defines how applications can be discovered by using `.desktop` files, but there's no
+simple reference implementation for this function.
+
+In order to provide a language-independent interface for applications and service to
+use, AGL includes `applaunchd`, a user service part of the default session.
+
+*Note: as mentioned [previously](1_Introduction/), services are managed using `systemd`
+and are therefore not in the scope of this document.*
+
+# Application launcher service
+
+The purpose of `applaunchd` is to enumerate applications available on the system and
+provide a way for other applications to query this list and start those on demand.
+It is also able to notify clients of the startup and termination of applications it
+manages.
+
+To that effect, `applaunchd` provides a D-Bus interface other applications can use
+in order to execute those actions.
+
+*Note: `applaunchd` will only send notifications for applications it started; it isn't
+aware of applications started by other means (`systemd`, direct executable call...),
+and therefore can't send notifications for those.*
+
+## Application discovery
+
+On startup, `applaunchd` inspects all `.desktop` files present under the `applications/`
+subfolder of any element of the `XDG_DATA_DIRS` environment variable, ignoring all entries
+containing either the `NoDisplay=true` or `Hidden=true` lines.
+
+It then looks for the following keys:
+- `Terminal`
+- `DBusActivatable`
+
+If the desktop entry file contains the `Terminal` key set to `true`, then the application
+is marked as a non-graphical one. As such, it won't be included in the applications list
+if the client requests only graphical applications.
+
+If `DBusActivatable` is set to `true`, then the application is marked as D-Bus activated.
+Additionally, `applaunchd` will search for a corresponding D-Bus service file in case this
+line is missing. This is a workaround allowing D-Bus activated applications providing
+an incomplete desktop entry file (i.e missing the `DBusActivatable` key) to be
+identified as such.
+
+### Requirements for D-Bus activation
+
+`applaunchd` will always start D-Bus activatable applications using D-Bus activation
+instead of executing the command line stated in the desktop entry file.
+
+This is handled by calling the `Activate` method of the
+[org.freedesktop.Application](https://specifications.freedesktop.org/desktop-entry-spec/1.1/ar01s07.html)
+interface with an empty argument.
+
+As a consequence, all D-Bus activatable applications **must** implement this D-Bus
+interface.
+
+## Application identifiers
+
+Each application is identified by a unique Application ID. Although this ID can be
+any valid string, it is highly recommended to use the "reverse DNS" convention in order
+to avoid potential name collisions and ease D-Bus integration.
+
+The application ID is set in the desktop entry file itself for
+[graphical applications](../3_Creating_a_New_Application/#graphical-applications):
+it is the value of the `StartupWMClass` field, which must be identical to the `app-id`
+advertised through the Wayland XDG toplevel protocol. In case this field is missing
+(as is usually the case for non-graphical application), the application ID will be the
+desktop entry file name, stripped from its `.desktop` extension.
+
+## D-Bus interface
+
+The `applaunchd` D-Bus interface is named `org.automotivelinux.AppLaunch`. The object
+path for `applaunchd` is `/org/automotivelinux/AppLaunch`. The interface provides methods
+for the following actions:
+- retrieve the list of available applications; the client can choose to retrieve all
+  available applications, or only those suitable for a graphical environment
+- request an application to be started
+
+Moreover, signals are available for clients to be notified when an application has
+successfully started or its execution terminated.
+
+### Applications list
+
+The `listApplications` method allows clients to retrieve the list of available applications.
+It takes one boolean argument named `graphical`:
+- if set to `true`, only applications suitable for graphical environments are returned
+- otherwise, the list contains all applications
+
+This method returns an array of variants (type `av`), each element being a structure made up
+of 3 strings (type `(sss)`):
+- the application ID
+- the application's displayed name
+- the full path to the application icon file (or an empty string if no icon was specified in
+  the application's desktop entry file)
+
+### Application startup request
+
+Applications can be started by using the `start` method, passing the corresponding application
+ID as the only argument. This method doesn't return any data.
+
+If the application is already running, `applaunchd` won't start another instance, but instead
+emit a `started` signal to notify clients the application is ready.
+
+### Status notifications
+
+The `org.automotivelinux.AppLaunch` interface provides 2 signals clients can connect to:
+- `started` indicates an application has started
+    - for D-Bus activated applications, it is emitted upon successful completion of the
+      call to the `Activate` method of the `org.freedesktop.Application` interface
+    - for other applications, this signal is emitted as soon as the child process has been
+      successfully created
+- `terminated` is emitted when an application quits
+
+Both signals have an additional argument named `appid`, containing the ID of the application
+affected by the event.
+
+As mentioned above, the `started` signal is also emitted if `applaunchd` receives a request to
+start an already running application. This can be useful, for example, when switching between
+graphical applications:
+- the application switcher doesn't need to track the state of each application; instead, it can
+  simply send a `start` request to `applaunchd` every time the user requests to switch to another
+  application
+- the desktop environment then receives the `started` signal, indicating it should activate the
+  window with the corresponding `app-id`
+
+## Testing
+
+`applaunchd` can be manually tested using the `gdbus` command-line tool:
+
+1. Query the application list (graphical applications only):
+
+```
+$ gdbus call --session --dest "org.automotivelinux.AppLaunch" \
+                       --object-path "/org/automotivelinux/AppLaunch" \
+                       --method "org.automotivelinux.AppLaunch.listApplications" \
+                       true
+```
+
+This command will output something similar to what follows:
+
+```
+([<('navigation', 'Navigation', '/usr/share/icons/hicolor/scalable/navigation.svg')>,
+ <('settings', 'Settings', '/usr/share/icons/hicolor/scalable/settings.svg')>,
+ <('dashboard', 'Dashboard', '/usr/share/icons/hicolor/scalable/dashboard.svg')>,
+ <('hvac', 'HVAC', '/usr/share/icons/hicolor/scalable/hvac.svg')>,
+ <('org.freedesktop.weston.wayland-terminal', 'Weston Terminal', '/usr/share/icons/Adwaita/scalable/apps/utilities-terminal-symbolic.svg')>],)
+```
+
+2. Request startup of the `org.freedesktop.weston.wayland-terminal` application:
+
+```
+$ gdbus call --session --dest "org.automotivelinux.AppLaunch" \
+                       --object-path "/org/automotivelinux/AppLaunch" \
+                       --method "org.automotivelinux.AppLaunch.start" \
+                       "org.freedesktop.weston.wayland-terminal"
+```
+
+3. Monitor signals emitted by `applaunchd`:
+
+```
+$ gdbus monitor --session --dest "org.automotivelinux.AppLaunch"
+```
+
+This results in the following output when starting, then exiting, the
+`org.freedesktop.weston.wayland-terminal` application:
+
+```
+Monitoring signals from all objects owned by org.automotivelinux.AppLaunch
+The name org.automotivelinux.AppLaunch is owned by :1.4
+/org/automotivelinux/AppLaunch: org.automotivelinux.AppLaunch.started ('org.freedesktop.weston.wayland-terminal',)
+/org/automotivelinux/AppLaunch: org.automotivelinux.AppLaunch.terminated ('org.freedesktop.weston.wayland-terminal',)
+```
diff --git a/docs/3_Developer_Guides/2_Creating_a_New_Service.md b/docs/3_Developer_Guides/2_Creating_a_New_Service.md
index 33a3fb5..7974d00 100644
--- a/docs/3_Developer_Guides/2_Creating_a_New_Service.md
+++ b/docs/3_Developer_Guides/2_Creating_a_New_Service.md
@@ -2,4 +2,144 @@
 title: Creating a New Service
 ---
 
-TBD
\ No newline at end of file
+Services are software running in the background and providing, as their name suggests,
+various services to other software: access to specific system hardware, connectivity
+management, network servers... They can be split into 2 categories:
+- system services: those usually run as a privileged user and make use of shared system
+  resources which they should have exclusive access to
+- user services: such services run as part of an unprivileged user's session and can
+  only be called by said user
+
+# Basic requirements
+
+The only mandatory requirement is that service packages provide a `.service` file
+so they can be properly managed by `systemd`. This file must be installed to a specific
+location, determined by the service type (system or user):
+- `/usr/lib/systemd/system/` for system services
+- `/usr/lib/systemd/user/` for user services
+
+Below is an example of a simple user service, running in a graphical session and
+therefore requiring a compositor to be already running before the service starts:
+
+```
+[Unit]
+Requires=agl-compositor.service
+After=agl-compositor.service
+
+[Service]
+Type=simple
+ExecStart=/usr/bin/homescreen
+Restart=on-failure
+
+[Install]
+WantedBy=agl-session.target
+```
+
+The `WantedBy=agl-session.target` indicates the service is part of the default AGL
+user session, as mentioned in the [Application Framework](1_Application_Framework/1_Introduction/#user-session-management)
+documentation.
+
+The `Restart=on-failure` directive ensures the service will be automatically
+restarted by `systemd` in case it crashes.
+
+More details about `systemd` service files can be found in the
+[systemd documentation](https://www.freedesktop.org/software/systemd/man/systemd.service.html).
+
+# D-Bus activation
+
+Services can also provide a D-Bus interface. In this case, they need not be started
+on system boot (or user session startup in the case of user services) but can be
+automatically started only when a client sends a request to the D-Bus name the service
+registers.
+
+D-Bus activated services must name their `systemd` service file `dbus-NAME.service`
+where `NAME` is the D-Bus name registered by the service. This file must include the
+following lines:
+
+```
+[Service]
+Type=dbus
+BusName=NAME
+ExecStart=/path/to/executable
+```
+
+In addition, they must provide a D-Bus service file named `NAME.service` and installed
+to one of the following locations:
+- `/usr/share/dbus-1/system-services` for system services
+- `/usr/share/dbus-1/services` for user services
+
+The contents of the D-Bus service file must be the following:
+
+```
+[D-BUS Service]
+Name=NAME
+Exec=/path/to/executable
+SystemdService=dbus-NAME.service
+```
+
+This ensures the service can be safely activated through D-Bus and no conflict will occur
+between `systemd` and the D-Bus daemon.
+
+More details about D-Bus activation can be found in the
+[D-Bus specification](https://dbus.freedesktop.org/doc/dbus-specification.html#message-bus-starting-services),
+under the "Message Bus Starting Services (Activation)" section.
+
+# Services startup
+
+For D-Bus activated services, no additional action is required as those will be automatically
+started whenever needed. Other services, however, need a few more steps in order to be
+executed on system or session startup.
+
+## System services
+
+System services can take advantage of the Yocto `systemd` class which automates the process of
+enabling such services.
+
+1. Ensure the recipe inherits from the `systemd` class:
+
+```
+inherit systemd
+```
+
+2. Declare the system services that needs to be enabled on boot:
+
+```
+SYSTEMD_SERVICE:${PN} = "NAME.service"
+```
+
+3. Ensure the `FILES` variable includes the systemd service directory the corresponding
+file will be installed to:
+
+```
+FILES:${PN} = "\
+    ...
+    ${systemd_system_unitdir}/* \
+"
+```
+
+## User services
+
+The `systemd` class doesn't provide an equivalent mechanism for user services. This must
+therefore be done manually as part of the package's install process.
+
+1. Make the service a part of the user session:
+
+```
+do_install:append() {
+    install -d ${D}${systemd_user_unitdir}/agl-session.target.wants
+    ln -s ../NAME.service ${D}${systemd_user_unitdir}/agl-session.target.wants/NAME.service
+}
+```
+
+This ensures `agl-session.target` depends on `NAME.service`, the latter being therefore
+automatically started on session creation.
+
+2. Ensure the `FILES` variable includes the systemd service directory the corresponding
+file will be installed to:
+
+```
+FILES:${PN} = "\
+    ...
+    ${systemd_user_unitdir}/* \
+"
+```
diff --git a/docs/3_Developer_Guides/3_Creating_a_New_Application.md b/docs/3_Developer_Guides/3_Creating_a_New_Application.md
index 6c382fd..c98cf1c 100644
--- a/docs/3_Developer_Guides/3_Creating_a_New_Application.md
+++ b/docs/3_Developer_Guides/3_Creating_a_New_Application.md
@@ -2,4 +2,132 @@
 title: Creating a New Application
 ---
 
-TBD
\ No newline at end of file
+Applications are either software designed to perform a specific task during a limited amount of
+time, or graphical applications presenting the user with an interface they can interact with.
+
+Such applications are executed by `applaunchd`, the AGL
+[application launcher service](1_Application_Framework/2_Application_Startup/).
+
+# Basic requirements
+
+In order to be enumerated by `applaunchd`, applications must provide the a `.desktop` file, as
+defined by the [Desktop Entry specification](https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html).
+
+The desktop entry file should be installed to `/usr/share/applications` (or the `applications`
+sub-directory of any entry present in the `XDG_DATA_DIRS` environment variable) and have a
+meaningful name. It is considered good practice to use reverse-DNS notation for the desktop
+entry file name, following the recommendations of the [D-Bus specification](https://dbus.freedesktop.org/doc/dbus-specification.html#message-protocol-names):
+- this avoids potential name collisions with other desktop entry files
+- it makes it easier to enable [D-Bus activation](#d-bus-activation) during the application
+  development cycle if needed
+- for [graphical applications](#graphical-applications), it ensures the chosen Wayland `app-id`
+  will be unique
+
+Such a file must contain at least the following keys:
+- `Type`: desktop entry type, must be set to `Application`
+- `Name`: application name, as it should be displayed in menus and application launchers
+- `Exec`: full path to the main executable
+
+Below is an example of a minimal desktop entry file:
+
+```
+[Desktop Entry]
+Type=Application
+Name=Example Application
+Exec=/usr/bin/example-app
+```
+
+Graphical applications must also provide an `Icon` entry pointing to the application icon.
+The value for this entry must either be the full path to the icon's file or, for icons part
+of an existing [icon theme](https://specifications.freedesktop.org/icon-theme-spec/icon-theme-spec-latest.html),
+the name of an element of this theme.
+
+In addition, a number of optional fields can be used to change how `applaunchd` will consider the
+application:
+- `Version`: version of the Desktop Entry Specification the file conforms to, must be `1.5`
+- `Hidden`: boolean value, if `true` the application is always ignored by `applaunchd` and
+  won't be listed nor executed
+- `Terminal`: boolean value, if `true` the application is excluded when requesting the list of
+  graphical applications from `applaunchd`
+- `DBusActivatable`: boolean value, must be `true` for [D-Bus activated applications](#d-bus-activation)
+- `Implements`: list of D-Bus interfaces the application implements, only used for D-Bus activated
+  applications.
+
+Finally, graphical applications may also define the `StartupWMClass` key in some cases. Please
+refer to the [graphical applications](#graphical-applications) section for more information.
+
+# D-Bus activation
+
+Similarly to [services](2_Creating_a_New_Service.md/#d-bus-activation), applications can
+also be activated through D-Bus.
+
+Such applications must name their `.desktop` file after the D-Bus name they register. In addition,
+this file must contain the following entries:
+
+```
+DBusActivatable=true
+Implements=IFACE1;IFACE2;...
+```
+
+Where `IFACEn` are the names of the D-Bus interfaces the application implements.
+
+In addition, they must provide a D-Bus service file named `NAME.service` and installed
+to `/usr/share/dbus-1/services`.
+
+The contents of the D-Bus service file must be the following:
+
+```
+[D-BUS Service]
+Name=NAME
+Exec=/path/to/executable
+```
+
+For example, an application registering the `org.automotivelinux.Example` D-Bus name
+and implementing the `org.automotivelinux.Example.Search1` and `org.automotivelinux.Example.Execute1`
+interfaces would provide the following files:
+
+* Desktop entry (`/usr/share/applications/org.automotivelinux.Example.desktop`):
+
+```
+[Desktop Entry]
+Type=Application
+Version=1.5
+Name=Example Application
+Exec=/usr/bin/example-app
+Icon=example-icon
+Terminal=false
+DBusActivatable=true
+Implements=org.automotivelinux.Example.Search1;org.automotivelinux.Example.Execute1
+```
+
+* D-Bus service file (`/usr/share/dbus-1/services/org.automotivelinux.Example.service`):
+
+```
+[D-BUS Service]
+Name=org.automotivelinux.Example
+Exec=/usr/bin/example-app
+```
+
+*Note: in addition to their own D-Bus interface, D-Bus activated applications must also
+implement the `org.freedesktop.Application` interface as defined in the
+[Desktop Entry specification](https://specifications.freedesktop.org/desktop-entry-spec/1.1/ar01s07.html).*
+
+# Graphical applications
+
+In addition, graphical applications need to comply with a few more requirements:
+
+1. Each application must set a Wayland application ID appropriately as soon as its main window
+is created.
+
+2. The `app-id` must be specified in the desktop entry file by adding the following line:
+
+```
+StartupWMClass=APP_ID
+```
+
+3. The desktop entry file must be named `APP_ID.desktop`.
+
+Doing so will ensure other software can associate the actual `app-id` to the proper application.
+
+*Note: application ID's are set using the [XDG toplevel](https://wayland-book.com/xdg-shell-basics/xdg-toplevel.html)
+Wayland interface.*