Docs reorganization and update

Docs reorganization and update using the latest afb-test improvments and
try to make it clearer.

Change-Id: If022cdb46364ef250361bdcd420d45b360f10a2e
Signed-off-by: Romain Forlot <romain.forlot@iot.bzh>

1 files changed, 327 insertions, 0 deletions

diff --git a/docs/3_Launch_the_tests.md b/docs/3_Launch_the_tests.md
new file mode 100644
index 0000000..b577acb
--- /dev/null
+++ b/docs/3_Launch_the_tests.md
@@ -0,0 +1,327 @@
+# How to launch the tests ?
+
+## Natively during the development
+
+It could be convenient to be able to test the software that you are currently
+developing. Then you can ensure that your modifications haven't introduced
+regressions, bugs, etc. This depends upon your tests of course.
+
+As previously saw, you need the `test widget` to be able to launch the tests and
+you need also to have the `afb-test` binding installed and the `application
+framework binder` to be able to execute your tests.
+
+Assuming you already installed those components on your development host, then
+proceed as the following from the project root directory:
+
+```bash
+cd build
+cmake -DBUILD_TEST_WGT=TRUE ..
+make
+make widget
+```
+
+To prepare all files needed for the test launch then use the `afm-test` script:
+
+```bash
+# Usage of afm-test command line utility
+afm-test --help
+Usage: /opt/AGL/bin/afm-test <binding-wgt-rootdir> <test-wgt-rootdir> [-m|--mode <SOLO|SERVICE>] [-t|--timeout <X>] [-l|--lavaoutput]
+binding-wgt-rootdir: path to the test wgt file
+test-wgt-rootdir: path to the test folder file
+-m|--mode: SOLO (1 binder) or SERVICE (2 binders) (Default: SOLO)
+-t|--timeout: timeout in second. (Default 3 seconds)
+-l|--lavaoutput: Flags indicating the binding to add Lava special test markers.
+Error: Test launch failed. Code: 1
+# Launching the test from your build project directory
+afm-test package package-test
+```
+
+### Example with the afb-test selftest suite
+
+Prepare the launch building the `test widget`:
+
+```bash
+$ cd build
+# Cleaning the previous build files
+$ rm -rf *
+# Configuration step
+$ cmake -DBUILD_TEST_WGT=TRUE ..
+-- The C compiler identification is GNU 8.2.1
+-- The CXX compiler identification is GNU 8.2.1
+-- Check for working C compiler: /usr/lib64/ccache/cc
+-- Check for working C compiler: /usr/lib64/ccache/cc -- works
+-- Detecting C compiler ABI info
+-- Detecting C compiler ABI info - done
+-- Detecting C compile features
+-- Detecting C compile features - done
+-- Check for working CXX compiler: /usr/lib64/ccache/c++
+-- Check for working CXX compiler: /usr/lib64/ccache/c++ -- works
+-- Detecting CXX compiler ABI info
+-- Detecting CXX compiler ABI info - done
+-- Detecting CXX compile features
+-- Detecting CXX compile features - done
+Distribution detected (separated by ';' choose one of them) fedora
+Include: /home/claneys/.config/app-templates/cmake.d/00-common-var.cmake
+Include: /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/conf.d/cmake/00-default-osconfig.cmake
+Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/01-build_options.cmake
+-- Found PkgConfig: /usr/bin/pkg-config (found version "1.4.2")
+-- Checking for module 'json-c'
+--   Found json-c, version 0.13.1
+-- Checking for module 'libsystemd>=222'
+--   Found libsystemd, version 238
+-- Checking for module 'afb-daemon>=4.0'
+--   Found afb-daemon, version 6.90.0
+-- Checking for module 'lua>=5.3'
+--   Found lua, version 5.3.4
+Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/02-variables.cmake
+-- Check gcc_minimal_version (found gcc version 8.2.1)  (found g++ version 8.2.1)
+Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/03-macros.cmake
+Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/04-extra_targets.cmake
+-- Overwrite the CMAKE default install prefix with /opt/AGL
+-- Found CURL: /usr/lib64/libcurl.so (found version "7.59.0")
+-- Qt's WebSocket AFB Client: Disabled!
+-- CURL wrapping helpers: Enabled!
+-- Notice: LUA Controler Support Selected
+-- Notice: Using default test widget configuration's file.
+-- If you want to use a customized test-config.xml template then specify TEST_WIDGET_CONFIG_TEMPLATE in your config.cmake file.
+-- Configuring done
+-- Generating done
+-- Build files have been written to: /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build
+# Build the binding
+$ make
+Scanning dependencies of target test-files
+[  3%] Generating test-files
+[  3%] Built target test-files
+Scanning dependencies of target project_populate_test-files
+[  6%] Generating package-test/var/test-files
+[  6%] Built target project_populate_test-files
+Scanning dependencies of target afb-test-config
+[  9%] Generating afb-test-config
+Warning: JSON_CHECKER not found. Not verification made on files !
+[  9%] Built target afb-test-config
+Scanning dependencies of target project_populate_afb-test-config
+[ 12%] Generating package-test/etc/afb-test-config
+[ 12%] Built target project_populate_afb-test-config
+Scanning dependencies of target aftest-config
+[ 16%] Generating aftest-config
+Warning: JSON_CHECKER not found. Not verification made on files !
+[ 16%] Built target aftest-config
+Scanning dependencies of target project_populate_aftest-config
+[ 19%] Generating package/etc/aftest-config
+[ 19%] Built target project_populate_aftest-config
+Scanning dependencies of target fixture-files
+[ 22%] Generating fixture-files
+[ 22%] Built target fixture-files
+Scanning dependencies of target project_populate_fixture-files
+[ 25%] Generating package-test/var/fixture-files
+[ 25%] Built target project_populate_fixture-files
+Scanning dependencies of target afTest-lua
+[ 29%] Generating afTest-lua
+[ 29%] Built target afTest-lua
+Scanning dependencies of target project_populate_afTest-lua
+[ 32%] Generating package/var/afTest-lua
+[ 32%] Built target project_populate_afTest-lua
+Scanning dependencies of target afb-helpers
+[ 35%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/wrap-json.c.o
+[ 38%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/filescan-utils.c.o
+[ 41%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/escape.c.o
+[ 45%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/curl-wrap.c.o
+[ 48%] Linking C static library libafb-helpers.a
+[ 48%] Built target afb-helpers
+Scanning dependencies of target ctl-utilities
+[ 51%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-action.c.o
+[ 54%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-config.c.o
+[ 58%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-onload.c.o
+[ 61%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-plugin.c.o
+[ 64%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-control.c.o
+[ 67%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-event.c.o
+[ 70%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-lua.c.o
+[ 74%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-timer.c.o
+[ 77%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-lua-utils.c.o
+[ 80%] Linking C static library libctl-utilities.a
+[ 80%] Built target ctl-utilities
+Scanning dependencies of target aft
+[ 83%] Building C object src/CMakeFiles/aft.dir/aft.c.o
+[ 87%] Building C object src/CMakeFiles/aft.dir/mapis.c.o
+[ 90%] Linking C shared module aft.so
+[ 90%] Built target aft
+Scanning dependencies of target project_populate_aft
+[ 93%] Generating package/lib/aft.so
+[ 93%] Built target project_populate_aft
+Scanning dependencies of target populate
+[ 96%] Generating package/bin, package/etc, package/lib, package/htdocs, package/var, package-test/bin, package-test/etc, package-test/lib, package-test/htdocs, package-test/var
+[ 96%] Built target populate
+Scanning dependencies of target afTest_build_done
+++ Typical binding launch: afb-daemon --name afTest --port=1234 --workdir=package-test --ldpaths=/opt/AGL/lib64/afb:../package/lib --token=
+[ 96%] Built target afTest_build_done
+Scanning dependencies of target autobuild
+[100%] Built target autobuild
+# Build both widgets classic and test
+$ make widget
+[  2%] Built target test-files
+Scanning dependencies of target packaging_wgt
+[  5%] Generating package/config.xml
+[  8%] Generating package-test/config.xml, package-test/bin/launcher
+[  8%] Built target packaging_wgt
+[ 11%] Generating package-test/var/test-files
+[ 11%] Built target project_populate_test-files
+Warning: JSON_CHECKER not found. Not verification made on files !
+[ 14%] Built target afb-test-config
+[ 17%] Generating package-test/etc/afb-test-config
+[ 17%] Built target project_populate_afb-test-config
+Warning: JSON_CHECKER not found. Not verification made on files !
+[ 20%] Built target aftest-config
+[ 23%] Generating package/etc/aftest-config
+[ 23%] Built target project_populate_aftest-config
+[ 26%] Built target fixture-files
+[ 29%] Generating package-test/var/fixture-files
+[ 29%] Built target project_populate_fixture-files
+[ 32%] Built target afTest-lua
+[ 35%] Generating package/var/afTest-lua
+[ 35%] Built target project_populate_afTest-lua
+[ 50%] Built target afb-helpers
+[ 79%] Built target ctl-utilities
+[ 88%] Built target aft
+[ 91%] Built target project_populate_aft
+[ 94%] Built target populate
+Scanning dependencies of target widget
+[ 97%] Generating aftest.wgt
+NOTICE: -- PACKING widget aftest.wgt from directory /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build/package
+[100%] Generating aftest-test.wgt
+NOTICE: -- PACKING widget aftest-test.wgt from directory /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build/package-test
+++ Install widget file using in the target : afm-util install afTest.wgt
+[100%] Built target widget
+```
+
+Launch the test using the mode SERVICE for the `afb-test` because of a recursion
+problem which loads 2 times the same binding and causes conflict. So it is
+needed to uses 2 binders then each ones loads its binding properly:
+
+```bash
+$ afm-test package package-test/ -m SERVICE
+1..4
+# Started on Tue Oct 30 10:32:46 2018
+# Starting class: TestListverb
+ok     1        TestListverb.testFunction
+# Starting class: TestGetVerb
+ok     2        TestGetVerb.testFunction
+# Starting class: Test_turning_on
+ok     3        Test_turning_on.testFunction
+# Starting class: testLockWait
+ok     4        testLockWait.testFunction
+# Ran 4 tests in 0.001 seconds, 4 successes, 0 failures
+~~~~~~~~~~ BEGIN ALL TESTS ~~~~~~~~~~
+1..59
+# Started on Tue Oct 30 10:32:48 2018
+# Starting class: testAssertEquals
+~~~~~ Begin Test ~~~~~
+~~~~~ Begin Test Assert Equals ~~~~~
+~~~~~ End Test Assert Equals ~~~~~
+~~~~~ End Test ~~~~~
+ok     1        testAssertEquals.testFunction
+# Starting class: testAssertNotEquals
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+ok     2        testAssertNotEquals.testFunction
+# Starting class: testAssertItemsEquals
+~~~~~ Begin Test ~~~~~
+[...]
+~~~~~ End Test ~~~~~
+ok     57       testAssertVerbStatusError.testFunction
+# Starting class: testAssertVerbResponseEqualsError
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+ok     58       testAssertVerbResponseEqualsError.testFunction
+# Starting class: testAssertVerbCbError
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+ok     59       testAssertVerbCbError.testFunction
+# Ran 59 tests in 0.003 seconds, 59 successes, 0 failures
+~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~
+Tests correctly launched.
+```
+
+## Launch test on a target board
+
+If you are trying to launch your test on a target you'll have to use
+a test widget which contains test files, fixture and configuration.
+Then use **afm-test**:
+
+```bash
+# afm-test -h
+Usage: /usr/bin/afm-test [-l|--lava] [-v|--verb <verb>] <path>
+-l|--lavaoutput: flag that enable Lava test marker to the output. (Default: disabled)
+-v|--verb: select a specific verb to launch from the test API. (Default: all)
+path: path to the test wgt file
+```
+
+By default, the test widgets should be located in /usr/AGL/apps/testwgt. This
+will install the widget, launch the tests then display the result on standard
+output. After that it will kill test binding and remove it.
+
+### Example on a target
+
+Here is an example:
+
+```bash
+qemux86-64:~# afm-test /usr/AGL/apps/testwgt/aftest-test.wgt
+afm-test /tmp/aftest-test.wgt
+PASS: aftest-test@6.90 started with pid=1649
+1..59
+~~~~~ Begin Test ~~~~~
+~~~~~ Begin Test Assert Equals ~~~~~
+~~~~~ End Test Assert Equals ~~~~~
+~~~~~ End Test ~~~~~
+PASS: 1 testAssertEquals.testFunction
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+PASS: 2 testAssertNotEquals.testFunction
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+PASS: 3 testAssertItemsEquals.testFunction
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+[...]
+PASS: 58 testAssertVerbResponseEqualsError.testFunction
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+PASS: 59 testAssertVerbCbError.testFunction
+# Ran 59 tests in 0.003 seconds, 59 successes, 0 failures
+~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~
+1..63
+~~~~~ Begin Test ~~~~~
+~~~~~ Begin Test Assert Equals ~~~~~
+~~~~~ End Test Assert Equals ~~~~~
+~~~~~ End Test ~~~~~
+PASS: 1 testAssertEquals.testFunction
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+PASS: 2 testAssertNotEquals.testFunction
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+PASS: 3 testAssertItemsEquals.testFunction
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+[...]
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+PASS: 61 TestGetVerb.testFunction
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+PASS: 62 Test_turning_on.testFunction
+~~~~~ Begin Test ~~~~~
+~~~~~ End Test ~~~~~
+PASS: 63 testLockWait.testFunction
+# Ran 63 tests in 0.003 seconds, 63 successes, 0 failures
+~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~
+PASS: aftest-test@6.90 killed and removed
+```
+
+The command being : ```afm-test /usr/AGL/apps/testwgt/aftest-test.wgt```
+
+You can see here that everything ran as on your pc terminal.
+**Begin Test**  and **End Test** are the
+beforeEach and afterEach functions and
+**END ALL TESTS**  is the after all functions.
+
+ **PASS :** shows the function that is or was running.