docs: Add waltham documentationsandbox/mvlad/devel_waltham-transmitter

Bug-AGL: SPEC-2757

Add waltham-book.yml and markdown files into docs.
These are fetched and processed by automotive-grade-linux/docs-webtemplate and
will be visible at docs area on https://docs.automotivelinux.org

Change-Id: I7deb90cdd9c21d71513b9765678505e38681e5f1
Signed-off-by: Naoko Tanibata <tnaoko@de.adit-jv.com>
Signed-off-by: Marius Vlad <marius.vlad@collabora.com>

16 files changed, 503 insertions, 0 deletions

diff --git a/docs/0-waltham-overview.md b/docs/0-waltham-overview.md
new file mode 100644
index 0000000..eecbf60
--- /dev/null
+++ b/docs/0-waltham-overview.md
@@ -0,0 +1,75 @@
+**Table of Content**
+
+1. TOC
+{:toc}
+
+## Context
+
+In today's world, the information for drivers is becoming excessive. For
+example, safety information to let the driver notice the obstacles on the road,
+telematics information about car accident or traffic jam, media information
+from connected phones etc. In the future, it is expected that the
+more displays will be available in the car and show more information.
+
+TFT Cluster will have more information, other than engine speed and a map,
+with a Head-Up-Display projected onto the windshield will bring driver to
+new world.  With the constant bombardment of information drivers will have a
+hard time making sure their eyes on the road, avoiding needless distractions.
+
+As we need more comprehensive Human Machine Interfaces, which displays the
+information that the driver needs on appropriate device and on time with
+a comprehensive user interface, a screen-sharing, remoting mechanism between
+multiple ECUs (Electronic Control Unit) will be necessary. For that, a protocol
+working over the network has been devised that could solve this problem.
+
+## Waltham protocol
+
+[Waltham protocol](https://github.com/waltham/waltham) is a IPC library similar
+to [Wayland](https://wayland.freedesktop.org), developed with networking in
+mind and as such it operates over TCP sockets, while the wayland protocol only
+works locally over a UNIX socket. It retains wayland-esque paradigm, making use
+of XMLs to describe the protocol, and it follow an object-oriented design with
+an asynchronous architecture.
+
+It was developed by Collabora along with ADIT, a joint venture company owned by
+Robert Bosch Car Multimedia GmbH and DENSO corporation. It supports surface
+sharing via network, but sharing function itself is not implemented in Waltham.
+Please refer [Waltham documentation](https://waltham.github.io/waltham/) for
+more details.
+
+### Major differences from Wayland to Waltham
+- Waltham uses TCP sockets for reliable communication
+- Waltham cannot send file descriptors
+- Waltham API is minimal and symmetric between server and client sides
+- Waltham does not provide an event loop implementation
+- The registry implementation is left out of the library, only the interface is
+  provided
+- No multi-threading support for sharing objects between threads
+
+![image](./images/Diffrence_between_Wayland_and_Waltham.jpg)
+
+### Requirements in automotive industry
+In order to use Waltham in automotive industry, the automotive specific
+requirements must be covered.
+
+The below shows very high level requirements. You can find the further
+requirements at [i**Waltham Requirements**](https://confluence.automotivelinux.org/display/UIGRA/Waltham+backend+requirements).
+
+1. Shall be able to support fast/reliable remoting among multiple ECUs
+2. Shall be able to support input handling
+3. Shall be able to share dedicated application
+4. Shall be able to share complete display output depending on additional
+   communication
+
+* Surface sharing is not part of Waltham protocol, Each system needs to
+implement the most efficient way for surface sharing.  On AGL, we implemented
+[Waltham client and Receiver](1-waltham-client-and-receiver.md) to enable
+surface sharing along with GStreamer encoder/decoder. It uses UDP for remoting,
+which is faster than TCP. Input events communicates with Waltham
+protocol.
+
+### Links
+* [Announcement of Waltham](https://lists.freedesktop.org/archives/wayland-devel/2016-October/031413.html)
+* [Bi-weekly meeting minutes](https://wiki.automotivelinux.org/eg-ui-graphics)
+* [UI and Graphics wiki](https://confluence.automotivelinux.org/display/UIGRA/UI+and+Graphics+Home)
+* [Waltham source codes](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/weston-ivi-plugins.git;a=tree;h=refs/heads/master;hb=refs/heads/master)
diff --git a/docs/1-waltham-client-and-receiver.md b/docs/1-waltham-client-and-receiver.md
new file mode 100644
index 0000000..798108b
--- /dev/null
+++ b/docs/1-waltham-client-and-receiver.md
@@ -0,0 +1,156 @@
+**Table of Content**
+
+1. TOC
+{:toc}
+
+## waltham-client-server-applications
+
+![image](./images/Waltham_Architecture.jpg)
+
+### waltham-client
+waltham-client uses Waltham IPC library to connect to remote host and transmit
+client buffers using GStreamer framework.
+
+waltham-client is divided into two components:
+
+* waltham-transmitter plugin: waltham-transmitter plugin provides API to create
+  remote connections and push surfaces over the network and handles both remote
+  output and remote input. Loaded automatically at start-up, by the compositor.
+
+* waltham-renderer: waltham-renderer is the implementation for sharing surface.
+  It creates a buffer to be transmitted to other domain. Latest implementation
+  uses GStreamer framework.
+
+### waltham-receiver
+
+waltham-receiver is a sample implementation of the receiver app which shall be
+running at the remote side. It is developed based on [waltham
+server](https://github.com/waltham/waltham/tree/master/tests), using Waltham
+protocol to obtain and process remote output, which is sent by
+waltham-transmitter.  This component is designed to be used for evaluating the
+functionality of waltham-transmitter plugin.
+
+## How it works
+
+1. Loading and initialization
+
+As the compositor is starting up, it loads the waltham-transmitter plugin.
+waltham-client connects to the receiver application at the remote side during
+initialization.
+
+![image](./images/01_Load_transmitter.jpg)
+
+2. Establishing connection
+
+At transmitter_create_remote(), waltham-transmitter creates
+`weston_transmitter_remote` object which expresses the receiver object at
+remote side, and it is created for each receiver.
+
+*waltham-transmitter* uses wth_display_get_registry() and wth_display_sync()
+for the receiver applications in the same manner as the Wayland protocol
+would retrieve the interfaces. Then, the receiver applications sends back
+the resource list to the waltham-transmitter.
+
+![image](./images/02_Establish_connection.jpg)
+
+3. Forwarding surface
+
+While the compositor redraws the surface, waltham-transmitter sends waltham
+protocol messages to the receiver app to notify about those surface updates.
+`wthp_surface_attach()`, `wthp_surface_damage()` and `wthp_surface_commit()`
+correspond to `wl_surface_attach()`, `wl_surface_damage()` and
+`wl_surface_commit()` message in wayland protocol.
+
+- `wthp_surface_attach()` - Send wthp_buffer as a buffer handling. This is not
+the actual buffer which contains the data to be rendered but the handle of an
+actual buffer. It abstracts the differences of buffer type.
+- `wthp_surface_damage()` - Tell the updated region to receiver app.
+- `wthp_surface_commit()` - Tell surface gets updated to receiver app.
+
+![image](./images/03_Forward_surface.jpg)
+
+4. Rendering using GStreamer
+
+![image](./images/04_Rendering_using_gstreamer.jpg)
+
+5. Input handling
+
+For handling input events, waltham-transmitter has 2 ways to secure seat.
+	1. Use wl_seat as weston has.
+	2. Create a new wl_seat.
+
+Second case is applicable in case the transmitter side does not have an input
+device but the receiver at remote side has one. After wl_seat is created,
+waltham-transmitter sends input events to the client application when
+it gets an input event from the receiver via the Waltham protocol.
+
+The message wthp_send_XXX shows you that input event is forwarded from the
+receiver to the transmitter, XXX is filled by the input event name.
+
+![image](./images/05_Input_handling.jpg)
+
+6. Retry connection
+
+In case the connection gets disconnected during surface sharing,
+waltham-transmitter shall re-establish the connection. The `waltham_display`
+objects represents the connection between the transmitter and the receiver, and
+contains a flag that can be used to detect a disconnect.
+
+That happens in `connection_handle_data()` in case a disconnect
+is detected. This flag is checked at every call of
+`transmitter_surface_gather_state`. When running is in false state,
+waltham-transmitter starts to retry the handling sequence
+
+It release the waltham protocol objects then it goes to establish a connection
+sequence mentioned in 2. Establishing connection.
+
+![image](./images/06_Retry_connection.jpg)
+
+## Waltham in practice
+Here is the example how waltham can be used in hypervisor use case of real
+project.
+
+* Weston is used as the wayland compositor.
+* waltham-client is implemented for Weston which acts as a Waltham  virtual
+  display.
+* Application surface is assigned to Waltham virtual display and it's sent to
+  the other ECU/OS.  Buffers of surface are transferred via GStreamer(UDP),
+  since transferring raw pixel data via Waltham(TCP) is not fast enough.
+* Controlling input events (pointer, keyboard, touch) for the surface is
+  handled by Waltham.
+
+![image](./images/Waltham_In_Practice.jpg)
+## How Waltham can be integrated
+
+Some possible integration examples of waltham follow.
+
+### As an EGL backend (theoretical possibility)
+
+Similarly to Wayland back-end for EGL, Waltham client could be a back-end in
+the compositor.  For better performance, a generic surface sharing mechanism is
+needed in a hypervisor environment.  Applications need to adapt to Waltham.
+As waltham is not designed with this use in mind this usage is just a
+theoretical possibility.
+
+![image](./images/Waltham_Integration_Possibility-01.jpg)
+
+### As a GStreamer sink (theoretical possibility)
+
+Similarly to Wayland sink, a Waltham sink GStreamer plugin can be implemented
+which sends the buffers to a receiver on another domain/OS.  Waltham sink can
+utilize frame synchronization and a presentation feedback protocols for video
+synchronization.  For better performance, a generic surface sharing mechanism
+is needed in a hypervisor environment.  As Waltham is not designed with this
+use in mind this usage is just a theoretical possibility.
+
+![image](./images/Waltham_Integration_Possibility-02.jpg)
+
+### As a virtual display in compositor
+
+Virtual display plugin can be implemented in the compositor. This plugin sends
+client buffers to waltham-receiver in another domain.  No changes to
+applications.  For good performance, a generic surface sharing mechanism is
+needed in hypervisor environment.  This is the intended use in mind during
+design.
+
+![image](./images/Waltham_Integration_Possibility-03.jpg)
diff --git a/docs/2-waltham-how-to-build.md b/docs/2-waltham-how-to-build.md
new file mode 100644
index 0000000..281e852
--- /dev/null
+++ b/docs/2-waltham-how-to-build.md
@@ -0,0 +1,256 @@
+**Table of Content**
+
+1. TOC
+{:toc}
+
+## Important notice
+
+Do note that even though waltham-transmitter can be built, AGL will no longer
+use weston as its default compositor, and although it relies on it (more
+exactly it uses its underlying library -- libweston), the following
+build instructions are most likely just a guide, and not a verified
+walk-through that reflects the current status. They all assume that you'll use
+weston and with it ivi-shell, which is not longer the case.
+
+Integration with the newer compositor is at this point in a WIP state.
+
+## Information
+Please also refer [Readme](https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/weston-ivi-plugins.git;a=tree;h=refs/heads/master;hb=refs/heads/master) in source repository.
+
+## How to build waltham-transmitter
+1. Prerequisite before building
+   AGL Image is already built. Please refer [here](https://docs.automotivelinux.org/docs/en/master/getting_started/reference/getting-started/image-workflow-build.html)
+
+2. Go to AGL build folder and configure your environment.
+```
+       $ cd $AGL_TOP
+       $ source meta-agl/scripts/aglsetup.sh
+```
+3. Build waltham-transmitter by using bitbake.
+
+Since waltham-transmitter is not built by default, this step is mandatory.
+
+```
+       $ bitbake waltham-transmitter
+```
+
+4. You can find the built results under $AGL_TOP/build/tmp/work/<board type>/waltham-transmitter/git-r0/image/
+   - usr/bin/waltham-receiver
+   - usr/lib/transmitter.so
+   - usr/lib/waltham-renderer.so
+
+## How to configure weston.ini and GStreamer pipeline
+
+### weston.ini
+
+In order to load waltham-transmitter plugin to weston, add "transmitter.so" to
+the "modules" key under "[core]" in weston.ini at transmitter side,
+then make sure the "shell" is configured as "ivi-shell.so".
+
+The destination of remoting also needs to be configured in weston.ini.
+Add output name, receiver IP address, port number, output's width and height
+key under "[transmitter-output]". You can speficy multiple [transmitter-output]
+with different output-name.
+
+```
+/* Example_weston.ini - single transmitter-output */
+
+	[core]
+	shell=ivi-shell.so
+	modules=systemd-notify.so,ivi-controller.so,transmitter.so
+
+	[ivi-shell]
+	ivi-module=ivi-controller.so
+	ivi-input-module=ivi-input-controller.so
+
+	[transmitter-output]
+	output-name=transmitter_1
+	server-address=192.168.2.52
+	port=34400
+	width=1920
+	height=1080
+
+```
+
+### GStreamer pipeline
+
+You can use GStreamer pipeline as you want. Please describe pipeline
+configuration in "/etc/xdg/weston/pipeline.cfg".
+Here are some examples.
+
+```
+
+/* General pipeline which does not use any HW encoder */
+	appsrc name=src ! videoconvert ! video/x-raw,format=I420 ! jpegenc ! \
+		rtpjpegpay ! udpsink name=sink host=YOUR_RECIEVER_IP \
+		port=YOUR_RECIEVER_PORT sync=false async=false
+
+/* pipeline to use Intel's HW encoder */
+	appsrc name=src ! videoconvert ! video/x-raw,format=I420 ! \
+			mfxh264enc bitrate=3000000 rate-control=1 ! rtph264pay config-interval=1 ! \
+			udpsink name=sink host=YOUR_RECIEVER_IP port=YOUR_RECIEVER_PORT \
+			sync=false async=false
+
+/* pipeline to use Rcar's HW encoder */
+	appsrc name=src ! videoconvert ! video/x-raw,format=I420 ! \
+				omxh264enc bitrate=3000000 control-rate=2 ! rtph264pay ! \
+				udpsink name=sink host=YOUR_RECIEVER_IP port=YOUR_RECIEVER_PORT \
+				sync=false async=false
+
+```
+
+## Connection Establishment
+
+1. Connect two boards over ethernet.
+2. Assign IP to both boards.
+```Example:
+	transmitter IP: 192.168.2.51
+	waltham-receiver IP: 192.168.2.52
+```
+3. Check if the simple ping works
+```
+        $ ping 192.168.2.52 (you can also ping vice versa)
+```
+
+## Example steps to start remoting
+
+1. Start target boards.
+
+transmitter side must have the above 2 files, the modified weston.ini and
+GStreamer pipeline,cfg.  You can confirm that transmitter is loaded properly
+from weston log as below.
+
+```
+/* Example(/run/platform/display/weston.log) */
+	[12:28:09.127] Loading module '/usr/lib/weston/transmitter.so'
+	[12:28:09.182] Registered plugin API 'transmitter_v1' of size 88
+	[12:28:09.183] Registered plugin API 'transmitter_ivi_v1' of size 16
+	[12:28:09.186] Loading module '/usr/lib/libweston-6/waltham-renderer.so'
+	[12:28:09.255] Transmitter initialized.
+	[12:28:09.255] transmitter_output_attach_head is called
+	[12:28:09.255] Weston head attached successfully to output
+	[12:28:09.255] Output 'transmitter-192.168.2.52:34400-1' enabled with head(s) transmitter-192.168.2.52:34400-1
+	[12:28:09.255] Transmitter weston_seat 0xaaab2209c800
+	[12:28:09.255] Transmitter created pointer=0xaaab220e6b50 for seat 0xaaab2209c800
+	[12:28:09.255] Transmitter created keyboard=0xaaab22038e60 for seat 0xaaab2209c800
+	[12:28:09.255] Transmitter created touch=0xaaab220c7930 for seat 0xaaab2209c800
+```
+
+2. [receiver side] Start receiver application.
+
+The below example shows the case if you use waltham-receiver as receiver application.
+
+1. Add surface to the display where you want to receive transmitted contents.
+You can check which output name is required by using the command
+`LayerManagerControl get scene`.
+
+```
+	$ layer-add-surfaces -s <surface count> -l <layer id> -d <output display name> &
+/* Example */
+	$ layer-add-surfaces -s 1 -l 1 -d HDMI-A-1 &
+```
+2. Start waltham-receiver. You can use debug option with "-v" if you want.
+```
+	$ waltham-receiver -p 34400 -v &
+```
+
+3. Now, receiver side is waiting for the contents coming from transmitter side.
+
+**You must configure and start receiver side first.**
+
+4. [transmitter side] Start an IVI application
+5. Put the surface of IVI application onto transmitter-output by using
+   LayerManagerControl command.
+
+This surface will be transmitted.
+
+```
+	$ layer-add-surfaces -d <transmitter -output name> -s <surface count on receiver> -l <layer id on receiver>
+/* Example */
+	$ layer-add-surfaces -d transmitter-192.168.2.52:34400-1 -s 1 -l 1 &
+```
+
+5. [transmitter side] make sure that weston.log shows remoting has been started.
+
+```
+/* Example(/run/platform/display/weston.log) */
+	[12:29:33.224] surface ID 1
+	[12:29:40.622] gst-setting are :-->
+	[12:29:40.622] ip = 192.168.2.52
+	[12:29:40.622] port = 34400
+	[12:29:40.622] bitrate = 3000000
+	[12:29:40.622] width = 1080
+	[12:29:40.622] height = 1920
+	[12:29:42.177] Parsing GST pipeline:appsrc name=src ! videoconvert ! video/x-raw,format=I420 ! jpegenc ! rtpjjpegpay ! udpsink name=sink host=192.168.2.52 port=3440 sync=false async=false
+```
+
+## Typical issues & Tips
+
+### help functions
+You can find the help information of LayerManagerControl command by using
+
+```
+$ LayerManagerControl help
+```
+
+### waltham-transmitter and waltham-receiver doesn't not communicate
+
+1. Please check ethernet connection. If you assign 192.168.2.51 and
+   192.168.2.52 for waltham-transmitter and waltham-receiver, you shall ping
+   vice versa.
+
+```
+/* At waltham-transmitter side */
+	$ ping 192.168.2.52
+
+/* At waltham-receiver side */
+	$ ping 192.168.2.51
+```
+
+2. Make sure that IP address specified in the weston.ini under
+   [transmitter-output] matches the waltham-receiver IP address.
+
+3. Make sure that IP address in pipeline.cfg on the transmitter side match the
+   waltham-receiver's IP address.
+
+### surface,layer or output information is unknown.
+You can check them by using the below command.
+
+```
+	$ LayerManagerControl get scene
+
+/* Example */
+	screen 0 (0x0)
+	---------------------------------------
+	- connector name:       HDMI-A-1
+	- resolution:           x=1024, y=768
+	- layer render order:   100(0x64),
+
+	    layer 100 (0x64)
+	    ---------------------------------------
+	    - destination region:   x=296, y=0, w=432, h=768
+	    - source region:        x=0, y=0, w=432, h=768
+	    - opacity:              1
+	    - visibility:           1
+	    - surface render order: 1(0x1),
+	    - on screen:            0(0x0)
+
+	        surface 1 (0x1)
+	        ---------------------------------------
+			- created by pid:       3338
+	        - original size:      x=432, y=768
+	        - destination region: x=0, y=0, w=432, h=768
+		- source region:      x=0, y=0, w=432, h=768
+		- opacity:            1
+		- visibility:         1
+		- frame counter:      47
+		- on layer:           100(0x64)
+
+	screen 1 (0x1)
+	---------------------------------------
+	- connector name:       transmitter-192.168.2.52:34400-1
+	- resolution:           x=1920, y=1080
+	- layer render order:
+
+	/* You can know the output name for remoting is "transmitter-192.168.2.52:34400-1" */
+```
diff --git a/docs/images/01_Load_transmitter.jpg b/docs/images/01_Load_transmitter.jpg
new file mode 100755
index 0000000..d9e0527
--- /dev/null
+++ b/docs/images/01_Load_transmitter.jpg
diff --git a/docs/images/02_Establish_connection.jpg b/docs/images/02_Establish_connection.jpg
new file mode 100755
index 0000000..2ad5bc8
--- /dev/null
+++ b/docs/images/02_Establish_connection.jpg
diff --git a/docs/images/03_Forward_surface.jpg b/docs/images/03_Forward_surface.jpg
new file mode 100755
index 0000000..116d36a
--- /dev/null
+++ b/docs/images/03_Forward_surface.jpg
diff --git a/docs/images/04_Rendering_using_gstreamer.jpg b/docs/images/04_Rendering_using_gstreamer.jpg
new file mode 100644
index 0000000..97d7858
--- /dev/null
+++ b/docs/images/04_Rendering_using_gstreamer.jpg
diff --git a/docs/images/05_Input_handling.jpg b/docs/images/05_Input_handling.jpg
new file mode 100644
index 0000000..197bbc0
--- /dev/null
+++ b/docs/images/05_Input_handling.jpg
diff --git a/docs/images/06_Retry_connection.jpg b/docs/images/06_Retry_connection.jpg
new file mode 100644
index 0000000..45cb437
--- /dev/null
+++ b/docs/images/06_Retry_connection.jpg
diff --git a/docs/images/Diffrence_between_Wayland_and_Waltham.jpg b/docs/images/Diffrence_between_Wayland_and_Waltham.jpg
new file mode 100644
index 0000000..73cc5e1
--- /dev/null
+++ b/docs/images/Diffrence_between_Wayland_and_Waltham.jpg
diff --git a/docs/images/Waltham_Architecture.jpg b/docs/images/Waltham_Architecture.jpg
new file mode 100644
index 0000000..f1429f4
--- /dev/null
+++ b/docs/images/Waltham_Architecture.jpg
diff --git a/docs/images/Waltham_In_Practice.jpg b/docs/images/Waltham_In_Practice.jpg
new file mode 100644
index 0000000..f413092
--- /dev/null
+++ b/docs/images/Waltham_In_Practice.jpg
diff --git a/docs/images/Waltham_Integration_Possibility-01.jpg b/docs/images/Waltham_Integration_Possibility-01.jpg
new file mode 100644
index 0000000..4c2d08d
--- /dev/null
+++ b/docs/images/Waltham_Integration_Possibility-01.jpg
diff --git a/docs/images/Waltham_Integration_Possibility-02.jpg b/docs/images/Waltham_Integration_Possibility-02.jpg
new file mode 100644
index 0000000..a45cbe8
--- /dev/null
+++ b/docs/images/Waltham_Integration_Possibility-02.jpg
diff --git a/docs/images/Waltham_Integration_Possibility-03.jpg b/docs/images/Waltham_Integration_Possibility-03.jpg
new file mode 100644
index 0000000..9e98282
--- /dev/null
+++ b/docs/images/Waltham_Integration_Possibility-03.jpg
diff --git a/docs/waltham-book.yml b/docs/waltham-book.yml
new file mode 100644
index 0000000..313ebf7
--- /dev/null
+++ b/docs/waltham-book.yml
@@ -0,0 +1,16 @@
+type: books
+books:
+-
+    id: waltham-docs-source
+    title: "AGL waltham"
+    description: AGL waltham Description
+    keywords:
+    author: "Advanced Driver Information Technology"
+    version: master
+    chapters:
+    - url: 0-waltham-overview.md
+      name: Waltham Overview
+    - url: 1-waltham-client-and-receiver.md
+      name: Waltham Client and Receiver
+    - url: 2-waltham-how-to-build.md
+      name: Waltham How to Build