diff options
author | Marius Vlad <marius.vlad@collabora.com> | 2020-10-19 23:58:48 +0300 |
---|---|---|
committer | Marius Vlad <marius.vlad@collabora.com> | 2020-10-20 00:20:31 +0300 |
commit | c9d23a045e6cb46c023c15f8189ef3ee8b1ddf20 (patch) | |
tree | dfaab199379ca17c25e7f4f630ed7995435e0085 /docs/1-waltham-client-and-receiver.md | |
parent | 0e79ab59165bf925f1288476dad66109aa01b3fa (diff) |
docs/: Added docsjellyfish_10.0.1jellyfish/10.0.110.0.1
This adds the documentations bits, most, if not all, extracted from
the original wayland-ivi-plugins repository.
It describes the waltham-transmiter-plugin, and the receiver side, how
to configure and run them. It also removes any ivi-shell and with it,
any menton about LayoutManagerControls.
Bug-AGL: SPEC-3601
Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
Change-Id: I2f900f7eee34c431cc8d601131c5925c31280425
Diffstat (limited to 'docs/1-waltham-client-and-receiver.md')
-rw-r--r-- | docs/1-waltham-client-and-receiver.md | 156 |
1 files changed, 156 insertions, 0 deletions
diff --git a/docs/1-waltham-client-and-receiver.md b/docs/1-waltham-client-and-receiver.md new file mode 100644 index 0000000..2396972 --- /dev/null +++ b/docs/1-waltham-client-and-receiver.md @@ -0,0 +1,156 @@ +**Table of Content** + +1. TOC +{:toc} + +## waltham-client-server-applications + +![image](./images/Waltham_Architecture.jpg) + +### waltham-transmitter-plugin + +*waltham-transmitter-plugin* provides an API to create remote connections and +push surfaces over the network and handles remote input. It is loaded +automatically at start-up, by the compositor. The (remote) outputs are streamed +to the remote side by using the remoting plug-in. + +### waltham-receiver + +waltham-receiver is a sample implementation of the receiver app which should be +running at the remote side. It is developed based on [waltham +server](https://github.com/waltham/waltham/tree/master/tests), using Waltham +protocol to obtain and process remote input -- handled by +waltham-transmitter-plugin, with the remote output handled by the remoting +plugin. This component is designed to be used for evaluating the functionality +of waltham-transmitter plugin. + +## How it works + +### 1. Loading and initialization + +As the compositor is starting up, it loads the waltham-transmitter plugin. + +![image](./images/01_Load_transmitter.jpg) + +### 2. Establishing connection + +At `transmitter_create_remote()`, waltham-transmitter creates +`weston_transmitter_remote` object which expresses the receiver object at +remote side, and it is created for each receiver. + +*waltham-transmitter* uses `wth_display_get_registry()` and `wth_display_sync()` +for the receiver applications in the same manner as the Wayland protocol +would retrieve the interfaces. Then, the receiver applications sends back +the resource list to the waltham-transmitter. + +![image](./images/02_Establish_connection.jpg) + +### 3. Forwarding surface + +A fake mechanism of sending the buffer attachments was previously in place. The +following protocol messages to the receiver app were added to notify about +surface updates. + +`wthp_surface_attach()`, `wthp_surface_damage()` and `wthp_surface_commit()` +correspond to `wl_surface_attach()`, `wl_surface_damage()` and +`wl_surface_commit()` message in wayland protocol. + +- `wthp_surface_attach()` - Send wthp_buffer as a buffer handling. This is not +the actual buffer which contains the data to be rendered but the handle of an +actual buffer. It abstracts the differences of buffer type. +- `wthp_surface_damage()` - Tell the updated region to receiver app. +- `wthp_surface_commit()` - Tell surface gets updated to receiver app. + +These are no longer in place, as the remoting output is being handled entirely +by the remoting plugin. + +![image](./images/03_Forward_surface.jpg) + +### 4. Rendering using GStreamer + +![image](./images/04_Rendering_using_gstreamer.jpg) + +### 5. Input handling + +For handling input events, waltham-transmitter has 2 ways to secure seat. + +1. Use `wl_seat` as weston has. +2. Create a new `wl_seat`. + +Second case is applicable in case the transmitter side does not have an input +device but the receiver at remote side has one. After `wl_seat` is created, +waltham-transmitter sends input events to the client application when +it gets an input event from the receiver via the Waltham protocol. + +The message wthp_send_XXX shows you that input event is forwarded from the +receiver to the transmitter, XXX is filled by the input event name. + +![image](./images/05_Input_handling.jpg) + +### 6. Retry connection + +In case the connection gets disconnected during surface sharing, +waltham-transmitter shall re-establish the connection. The `waltham_display` +objects represents the connection between the transmitter and the receiver, +and contains a flag that can be used to detect a disconnect. + +That happens in `connection_handle_data()` in case a disconnect is detected. +This flag is checked at every call of `transmitter_surface_gather_state`. When +running is in false state, waltham-transmitter starts to retry the handling +sequence. + +It release the waltham protocol objects then it goes to establish a connection +sequence mentioned in 2. Establishing connection. + +![image](./images/06_Retry_connection.jpg) + +## Waltham in practice + +Here is the example how waltham can be used in hypervisor use case of real +project. + +* Weston is used as the wayland compositor. +* waltham-client is implemented for Weston which acts as a Waltham virtual + display. +* Application surface is assigned to Waltham virtual display and it's sent to + the other ECU/OS. Buffers of surface are transferred via GStreamer(UDP), + since transferring raw pixel data via Waltham(TCP) is not fast enough. +* Controlling input events (pointer, keyboard, touch) for the surface is + handled by Waltham. + +![image](./images/Waltham_In_Practice.jpg) + +## How Waltham can be integrated + +Some possible integration examples of waltham follow. + +### As an EGL backend (theoretical possibility) + +Similarly to Wayland back-end for EGL, Waltham client could be a back-end in +the compositor. For better performance, a generic surface sharing mechanism is +needed in a hypervisor environment. Applications need to adapt to Waltham. +As waltham is not designed with this use in mind this usage is just a +theoretical possibility. + +![image](./images/Waltham_Integration_Possibility-01.jpg) + +### As a GStreamer sink (theoretical possibility) + +Similarly to Wayland sink, a Waltham sink GStreamer plugin can be implemented +which sends the buffers to a receiver on another domain/OS. Waltham sink can +utilize frame synchronization and a presentation feedback protocols for video +synchronization. For better performance, a generic surface sharing mechanism +is needed in a hypervisor environment. As Waltham is not designed with this +use in mind this usage is just a theoretical possibility. + +![image](./images/Waltham_Integration_Possibility-02.jpg) + +### As a virtual display in compositor + +Virtual display plugin can be implemented in the compositor. This plugin sends +client buffers to waltham-receiver in another domain. No changes to +applications. For good performance, a generic surface sharing mechanism is +needed in hypervisor environment. This is the intended use in mind during +design. + +![image](./images/Waltham_Integration_Possibility-03.jpg) |