diff options
author | Naoko Tanibata <tnaoko@de.adit-jv.com> | 2020-07-15 13:28:56 +0900 |
---|---|---|
committer | Naoko Tanibata <tnaoko@de.adit-jv.com> | 2020-09-03 15:17:19 +0200 |
commit | e76c57311b7aefbed6e55288108fe197fe12a945 (patch) | |
tree | 40b0b8f2cdf216b501f475a7364aae8dc17808ba /docs/1-waltham-client-and-receiver.md | |
parent | ded331ba2866f6baa32884c5d34c11092c4a1c0f (diff) |
docs: Make paragraph length maximum 80 chars
Task-AGL: SPEC-2757
Update based on review comments.
Signed-off-by: Naoko Tanibata <tnaoko@de.adit-jv.com>
Change-Id: Id8a7ff01e464f77e3eeb111bead872aef36f9707
Diffstat (limited to 'docs/1-waltham-client-and-receiver.md')
-rw-r--r-- | docs/1-waltham-client-and-receiver.md | 108 |
1 files changed, 78 insertions, 30 deletions
diff --git a/docs/1-waltham-client-and-receiver.md b/docs/1-waltham-client-and-receiver.md index 8425556..965b7a0 100644 --- a/docs/1-waltham-client-and-receiver.md +++ b/docs/1-waltham-client-and-receiver.md @@ -8,42 +8,63 @@ ![image](./images/Waltham_Architecture.jpg) ### waltham-client -waltham-client uses Waltham IPC library to connect to remote and transmit client buffers using GStreamer framework. It is designed for AGL upon certain Linux base system with wayland backend. +waltham-client uses Waltham IPC library to connect to remote and transmit client +buffers using GStreamer framework. It is designed for AGL upon certain Linux +base system with wayland backend. waltham-client is divided into two components: * waltham-transmitter plugin: - waltham-transmitter plugin provides API to create remote connections and push surfaces over the network and handles both remote output and remote input. + waltham-transmitter plugin provides API to create remote connections and +push surfaces over the network and handles both remote output and remote input. * waltham-renderer: - waltham-renderer is the implementation for sharing surface. It creates a buffer to be transmitted to other domain. Latest implementation uses GStreamer framework. + waltham-renderer is the implementation for sharing surface. It creates a +buffer to be transmitted to other domain. Latest implementation uses GStreamer +framework. ### waltham-receiver -waltham-receiver is a sample implementation of receiver app which shall be running at remote side. It is developed based on [waltham server](https://github.com/waltham/waltham/tree/master/tests), using Waltham protocol to obtain and process remote output which is sent by waltham-transmitter. -This component is designed to be used for evaluating the functionalities of waltham-transmitter plugin. +waltham-receiver is a sample implementation of receiver app which shall be +running at remote side. It is developed based on [waltham server](https://github.com/ +waltham/waltham/tree/master/tests), using Waltham protocol to obtain and process + remote output which is sent by waltham-transmitter. +This component is designed to be used for evaluating the functionalities of +waltham-transmitter plugin. ## How it works 1. Loading and initialization -The system launches Weston, then Weston loads waltham-transmitter plugin. waltham-client connects to receiver app at remote side during initialization. +The system launches Weston, then Weston loads waltham-transmitter plugin. +waltham-client connects to receiver app at remote side during initialization. ![image](./images/01_Load_transmitter.jpg) -2. Establishing connection +<a id="EstCnnn"></a> +2.</a href="#EstCnnn">Establishing connection</a> -At transmitter_create_remote(), waltham-transmitter creates "struct weston_transmitter_remote" which expresses the receiver object at remote side. Therefore this structure is created for each receiver. -waltham-transmitter sends wth_display_get_registry() and wth_display_sync message() to the receiver app as same manner as Wayland protocol. So that receiver app sends back the resource list to waltham-transmitter. +At transmitter_create_remote(), waltham-transmitter creates +"struct weston_transmitter_remote" which expresses the receiver object at remote +side. Therefore this structure is created for each receiver. +waltham-transmitter sends wth_display_get_registry() and +wth_display_sync_message() to the receiver app as same manner as Wayland protocol. +So that receiver app sends back the resource list to waltham-transmitter. ![image](./images/02_Establish_connection.jpg) 3. Forwarding surface -During Weston redraws surface as waltham-transmitter output, waltham-transmitter sends waltham protocol messages to receiver app to notify surface update. wthp_surface_attach(), wthp_surface_damage() and wthp_surface_commit() which correspond to wl_surface_attach(), wl_surface_damage() and wl_surface_commit() message in wayland protocol. -wthp_surface_attach() - Send wthp_buffer as a buffer handling. This is not the actual buffer which contains the data to be rendered but the handle of actual buffer. It abstracts the differences of buffer type. -wthp_surface_damage() - Tell the updated region to receiver app. -wthp_surface_commit() - Tell surface gets updated to receiver app. +While Weston redraws the surface to waltham-transmitter output, waltham-transmitter + sends waltham protocol messages to receiver app to notify the surface update. +wthp_surface_attach(), wthp_surface_damage() and wthp_surface_commit() which +correspond to wl_surface_attach(), wl_surface_damage() and wl_surface_commit() +message in wayland protocol. +wthp_surface_attach() - Send wthp_buffer to a buffer handling. This is not the a +ctual buffer which contains the data to be rendered but the handle of actual +buffer. It abstracts the differences of buffer type. +wthp_surface_damage() - Tell the updated region to the receiver app. +wthp_surface_commit() - Tell surface gets updated to the receiver app. ![image](./images/03_Forward_surface.jpg) @@ -56,24 +77,42 @@ wthp_surface_commit() - Tell surface gets updated to receiver app. For handling input events, waltham-transmitter has 2 ways to secure seat. 1. Use wl_seat as weston has. 2. Create new wl_seat. -Second case is applicable in case transmitter side does not have input device but receiver at remote side has. After wl_seat is created, waltham-transmitter sends input events to weston client application when it gets input event from receiver via waltham protocol. -The message wthp_send_XXX shows you that input event is forwarded from receiver to transmitter, XXX is filled by input event name. +Second case is applicable in the case transmitter side does not have input device +but the receiver at remote side has. After wl_seat is created, waltham-transmitter +sends input events to weston client application when it gets input event from +receiver via waltham protocol. +The message wthp_send_XXX shows you that input event is forwarded from receiver +to transmitter, XXX is filled by input event name. ![image](./images/05_Input_handling.jpg) 6. Retry connection -In case the connection gets disconnected during surface sharing, waltham-transmitter shall re-establish the connection. The object “struct waltham_display” represents the connection between the transmitter and receiver, which structure has the flag used to detect disconnection named “running”. In function “connection_handle_data()”, running is set to "false" in case disconnection detected. This flag is checked at every “transmitter_surface_gather_state". When running is false state, waltham-transmitter starts retry handling sequence. First of all, It release the waltham protocol objects then goes to Establish connection sequence mentioned in "2. Establishing connection". +In case the connection gets disconnected during surface sharing, +waltham-transmitter shall re-establish the connection. The object +“struct waltham_display” represents the connection between the transmitter and +receiver, which structure has the flag used to detect disconnection named +“running”. In function “connection_handle_data()”, running is set to "false" in +case disconnection detected. This flag is checked at every +“transmitter_surface_gather_state". When running is false state, +waltham-transmitter starts retry handling sequence. First of all, it release the +waltham protocol objects. Second, it establishes connection sequence mentioned +in "[2. Establishing connection](#EstCnnn)". ![image](./images/06_Retry_connection.jpg) ## Waltham in practice -Here is the example how waltham can be used in hypervisor use case of real project . +Here is the example how waltham can be used in hypervisor use case of real +project. * Weston is used as the wayland compositor. -* waltham-client is implemented for Weston which acts as a Waltham virtual display. -* Application surface is assigned to Waltham virtual display and it's sent to the other ECU/OS. - Buffers of surface are transferred via GStreamer(UDP), since transferring raw pixel data via Waltham(TCP) is not enough fast. -* Controlling input events (pointer, keyboard, touch) for the surface is handled by Waltham. +* waltham-client is implemented for Weston which acts as a Waltham virtual + display. +* Application surface is assigned to Waltham virtual display and it's sent to +the other ECU/OS. + Buffers of surface are transferred via GStreamer(UDP), since transferring +raw pixel data via Waltham(TCP) is not enough fast. +* Controlling input events (pointer, keyboard, touch) for the surface is handled + by Waltham. ![image](./images/Waltham_In_Practice.jpg) ## How Waltham can be integrated @@ -81,26 +120,35 @@ Here are possible integration examples of waltham. ### As EGL backend (theoretical possibility) -Similar to Wayland backend for EGL, Waltham client could be backend in compositor. / -For good performance, a generic surface sharing mechanism is needed in hypervisor environment. -Applications need to adapt to Waltham. Waltham is not designed with this use in mind. This usage is just a theoretical possibility. +Similar to Wayland backend for EGL, Waltham client could be backend in +compositor. +For good performance, a generic surface sharing mechanism is needed in hypervisor +environment. +Applications need to adapt to Waltham. Waltham is not designed with this use in +mind. This usage is just a theoretical possibility. ![image](./images/Waltham_Integration_Possibility-01.jpg) ### As GStreamer sink (theoretical possibility) -Similar to Wayland sink, Waltham sink GStreamer plugin can be implemented which sends the buffers to receiver on another domain/OS. -Waltham sink can utilize frame sync and presentation feedback protocols for video synchronization. -For good performance, a generic surface sharing mechanism is needed in hypervisor environment. -Waltham is not designed with this use in mind. This usage is just a theoretical possibility. +Similar to Wayland sink, Waltham sink GStreamer plugin can be implemented which +sends the buffers to receiver on another domain/OS. +Waltham sink can utilize frame sync and presentation feedback protocols for video +synchronization. +For good performance, a generic surface sharing mechanism is needed in +hypervisor environment. +Waltham is not designed with this use in mind. This usage is just a theoretical +possibility. ![image](./images/Waltham_Integration_Possibility-02.jpg) ### As virtual display in compositor -Virtual display plugin can be implemented in compositor. This plugin sends client buffers to waltham-receiver in another domain. +Virtual display plugin can be implemented in compositor. This plugin sends +client buffers to waltham-receiver in another domain. No changes to applications. -For good performance, a generic surface sharing mechanism is needed in hypervisor environment. +For good performance, a generic surface sharing mechanism is needed in +hypervisor environment. This is the intended use in mind during design. ![image](./images/Waltham_Integration_Possibility-03.jpg) |