# How to launch the tests ? ## Natively during the development It could be convenient to be able to test the software that you are currently developing. Then you can ensure that your modifications haven't introduced regressions, bugs, etc. This depends upon your tests of course. As previously saw, you need the `test widget` to be able to launch the tests and you need also to have the `afb-test` binding installed and the `application framework binder` to be able to execute your tests. Assuming you already installed those components on your development host, then proceed as the following from the project root directory: ```bash cd build cmake -DBUILD_TEST_WGT=TRUE .. make make widget ``` To prepare all files needed for the test launch then use the `afm-test` script: ```bash # Usage of afm-test command line utility afm-test --help Usage: /opt/AGL/bin/afm-test <binding-wgt-rootdir> <test-wgt-rootdir> [-m|--mode <SOLO|SERVICE>] [-t|--timeout <X>] [-l|--lavaoutput] binding-wgt-rootdir: path to the test wgt file test-wgt-rootdir: path to the test folder file -m|--mode: SOLO (1 binder) or SERVICE (2 binders) (Default: SOLO) -t|--timeout: timeout in second. (Default 3 seconds) -l|--lavaoutput: Flags indicating the binding to add Lava special test markers. Error: Test launch failed. Code: 1 # Launching the test from your build project directory afm-test package package-test ``` ### Example with the afb-test selftest suite Prepare the launch building the `test widget`: ```bash $ cd build # Cleaning the previous build files $ rm -rf * # Configuration step $ cmake -DBUILD_TEST_WGT=TRUE .. -- The C compiler identification is GNU 8.2.1 -- The CXX compiler identification is GNU 8.2.1 -- Check for working C compiler: /usr/lib64/ccache/cc -- Check for working C compiler: /usr/lib64/ccache/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/lib64/ccache/c++ -- Check for working CXX compiler: /usr/lib64/ccache/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done Distribution detected (separated by ';' choose one of them) fedora Include: /home/claneys/.config/app-templates/cmake.d/00-common-var.cmake Include: /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/conf.d/cmake/00-default-osconfig.cmake Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/01-build_options.cmake -- Found PkgConfig: /usr/bin/pkg-config (found version "1.4.2") -- Checking for module 'json-c' -- Found json-c, version 0.13.1 -- Checking for module 'libsystemd>=222' -- Found libsystemd, version 238 -- Checking for module 'afb-daemon>=4.0' -- Found afb-daemon, version 6.90.0 -- Checking for module 'lua>=5.3' -- Found lua, version 5.3.4 Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/02-variables.cmake -- Check gcc_minimal_version (found gcc version 8.2.1) (found g++ version 8.2.1) Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/03-macros.cmake Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/04-extra_targets.cmake -- Overwrite the CMAKE default install prefix with /opt/AGL -- Found CURL: /usr/lib64/libcurl.so (found version "7.59.0") -- Qt's WebSocket AFB Client: Disabled! -- CURL wrapping helpers: Enabled! -- Notice: LUA Controler Support Selected -- Notice: Using default test widget configuration's file. -- If you want to use a customized test-config.xml template then specify TEST_WIDGET_CONFIG_TEMPLATE in your config.cmake file. -- Configuring done -- Generating done -- Build files have been written to: /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build # Build the binding $ make Scanning dependencies of target test-files [ 3%] Generating test-files [ 3%] Built target test-files Scanning dependencies of target project_populate_test-files [ 6%] Generating package-test/var/test-files [ 6%] Built target project_populate_test-files Scanning dependencies of target afb-test-config [ 9%] Generating afb-test-config Warning: JSON_CHECKER not found. Not verification made on files ! [ 9%] Built target afb-test-config Scanning dependencies of target project_populate_afb-test-config [ 12%] Generating package-test/etc/afb-test-config [ 12%] Built target project_populate_afb-test-config Scanning dependencies of target aftest-config [ 16%] Generating aftest-config Warning: JSON_CHECKER not found. Not verification made on files ! [ 16%] Built target aftest-config Scanning dependencies of target project_populate_aftest-config [ 19%] Generating package/etc/aftest-config [ 19%] Built target project_populate_aftest-config Scanning dependencies of target fixture-files [ 22%] Generating fixture-files [ 22%] Built target fixture-files Scanning dependencies of target project_populate_fixture-files [ 25%] Generating package-test/var/fixture-files [ 25%] Built target project_populate_fixture-files Scanning dependencies of target afTest-lua [ 29%] Generating afTest-lua [ 29%] Built target afTest-lua Scanning dependencies of target project_populate_afTest-lua [ 32%] Generating package/var/afTest-lua [ 32%] Built target project_populate_afTest-lua Scanning dependencies of target afb-helpers [ 35%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/wrap-json.c.o [ 38%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/filescan-utils.c.o [ 41%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/escape.c.o [ 45%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/curl-wrap.c.o [ 48%] Linking C static library libafb-helpers.a [ 48%] Built target afb-helpers Scanning dependencies of target ctl-utilities [ 51%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-action.c.o [ 54%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-config.c.o [ 58%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-onload.c.o [ 61%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-plugin.c.o [ 64%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-control.c.o [ 67%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-event.c.o [ 70%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-lua.c.o [ 74%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-timer.c.o [ 77%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-lua-utils.c.o [ 80%] Linking C static library libctl-utilities.a [ 80%] Built target ctl-utilities Scanning dependencies of target aft [ 83%] Building C object src/CMakeFiles/aft.dir/aft.c.o [ 87%] Building C object src/CMakeFiles/aft.dir/mapis.c.o [ 90%] Linking C shared module aft.so [ 90%] Built target aft Scanning dependencies of target project_populate_aft [ 93%] Generating package/lib/aft.so [ 93%] Built target project_populate_aft Scanning dependencies of target populate [ 96%] Generating package/bin, package/etc, package/lib, package/htdocs, package/var, package-test/bin, package-test/etc, package-test/lib, package-test/htdocs, package-test/var [ 96%] Built target populate Scanning dependencies of target afTest_build_done ++ Typical binding launch: afb-daemon --name afTest --port=1234 --workdir=package-test --ldpaths=/opt/AGL/lib64/afb:../package/lib --token= [ 96%] Built target afTest_build_done Scanning dependencies of target autobuild [100%] Built target autobuild # Build both widgets classic and test $ make widget [ 2%] Built target test-files Scanning dependencies of target packaging_wgt [ 5%] Generating package/config.xml [ 8%] Generating package-test/config.xml, package-test/bin/launcher [ 8%] Built target packaging_wgt [ 11%] Generating package-test/var/test-files [ 11%] Built target project_populate_test-files Warning: JSON_CHECKER not found. Not verification made on files ! [ 14%] Built target afb-test-config [ 17%] Generating package-test/etc/afb-test-config [ 17%] Built target project_populate_afb-test-config Warning: JSON_CHECKER not found. Not verification made on files ! [ 20%] Built target aftest-config [ 23%] Generating package/etc/aftest-config [ 23%] Built target project_populate_aftest-config [ 26%] Built target fixture-files [ 29%] Generating package-test/var/fixture-files [ 29%] Built target project_populate_fixture-files [ 32%] Built target afTest-lua [ 35%] Generating package/var/afTest-lua [ 35%] Built target project_populate_afTest-lua [ 50%] Built target afb-helpers [ 79%] Built target ctl-utilities [ 88%] Built target aft [ 91%] Built target project_populate_aft [ 94%] Built target populate Scanning dependencies of target widget [ 97%] Generating aftest.wgt NOTICE: -- PACKING widget aftest.wgt from directory /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build/package [100%] Generating aftest-test.wgt NOTICE: -- PACKING widget aftest-test.wgt from directory /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build/package-test ++ Install widget file using in the target : afm-util install afTest.wgt [100%] Built target widget ``` Launch the test using the mode SERVICE for the `afb-test` because of a recursion problem which loads 2 times the same binding and causes conflict. So it is needed to uses 2 binders then each ones loads its binding properly: ```bash $ afm-test package package-test/ -m SERVICE 1..4 # Started on Tue Oct 30 10:32:46 2018 # Starting class: TestListverb ok 1 TestListverb.testFunction # Starting class: TestGetVerb ok 2 TestGetVerb.testFunction # Starting class: Test_turning_on ok 3 Test_turning_on.testFunction # Starting class: testLockWait ok 4 testLockWait.testFunction # Ran 4 tests in 0.001 seconds, 4 successes, 0 failures ~~~~~~~~~~ BEGIN ALL TESTS ~~~~~~~~~~ 1..59 # Started on Tue Oct 30 10:32:48 2018 # Starting class: testAssertEquals ~~~~~ Begin Test ~~~~~ ~~~~~ Begin Test Assert Equals ~~~~~ ~~~~~ End Test Assert Equals ~~~~~ ~~~~~ End Test ~~~~~ ok 1 testAssertEquals.testFunction # Starting class: testAssertNotEquals ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ ok 2 testAssertNotEquals.testFunction # Starting class: testAssertItemsEquals ~~~~~ Begin Test ~~~~~ [...] ~~~~~ End Test ~~~~~ ok 57 testAssertVerbStatusError.testFunction # Starting class: testAssertVerbResponseEqualsError ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ ok 58 testAssertVerbResponseEqualsError.testFunction # Starting class: testAssertVerbCbError ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ ok 59 testAssertVerbCbError.testFunction # Ran 59 tests in 0.003 seconds, 59 successes, 0 failures ~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ Tests correctly launched. ``` ## Launch test on a target board If you are trying to launch your test on a target you'll have to use a test widget which contains test files, fixture and configuration. Then use **afm-test**: ```bash # afm-test -h Usage: /usr/bin/afm-test [-l|--lava] [-v|--verb <verb>] <path> -l|--lavaoutput: flag that enable Lava test marker to the output. (Default: disabled) -v|--verb: select a specific verb to launch from the test API. (Default: all) path: path to the test wgt file ``` By default, the test widgets should be located in /usr/AGL/apps/testwgt. This will install the widget, launch the tests then display the result on standard output. After that it will kill test binding and remove it. ### Example on a target Here is an example: ```bash qemux86-64:~# afm-test /usr/AGL/apps/testwgt/aftest-test.wgt afm-test /tmp/aftest-test.wgt PASS: aftest-test@6.90 started with pid=1649 1..59 ~~~~~ Begin Test ~~~~~ ~~~~~ Begin Test Assert Equals ~~~~~ ~~~~~ End Test Assert Equals ~~~~~ ~~~~~ End Test ~~~~~ PASS: 1 testAssertEquals.testFunction ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ PASS: 2 testAssertNotEquals.testFunction ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ PASS: 3 testAssertItemsEquals.testFunction ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ [...] PASS: 58 testAssertVerbResponseEqualsError.testFunction ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ PASS: 59 testAssertVerbCbError.testFunction # Ran 59 tests in 0.003 seconds, 59 successes, 0 failures ~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ 1..63 ~~~~~ Begin Test ~~~~~ ~~~~~ Begin Test Assert Equals ~~~~~ ~~~~~ End Test Assert Equals ~~~~~ ~~~~~ End Test ~~~~~ PASS: 1 testAssertEquals.testFunction ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ PASS: 2 testAssertNotEquals.testFunction ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ PASS: 3 testAssertItemsEquals.testFunction ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ [...] ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ PASS: 61 TestGetVerb.testFunction ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ PASS: 62 Test_turning_on.testFunction ~~~~~ Begin Test ~~~~~ ~~~~~ End Test ~~~~~ PASS: 63 testLockWait.testFunction # Ran 63 tests in 0.003 seconds, 63 successes, 0 failures ~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ PASS: aftest-test@6.90 killed and removed ``` The command being : ```afm-test /usr/AGL/apps/testwgt/aftest-test.wgt``` You can see here that everything ran as on your pc terminal. **Begin Test** and **End Test** are the beforeEach and afterEach functions and **END ALL TESTS** is the after all functions. **PASS :** shows the function that is or was running.