diff options
-rw-r--r-- | docs/0_Installation.md | 59 | ||||
-rw-r--r-- | docs/1_Write_the_tests.md | 246 | ||||
-rw-r--r-- | docs/2_LaunchTheExample.md | 232 | ||||
-rw-r--r-- | docs/2_The_test_widget.md | 65 | ||||
-rw-r--r-- | docs/3_Launch_the_tests.md | 327 | ||||
-rw-r--r-- | docs/4_Tests_Examples.md (renamed from docs/1_TestExample.md) | 15 | ||||
-rw-r--r-- | docs/Reference/0_BindingTestFunctions.md (renamed from docs/WriteYourTests/Reference/0_BindingTestFunctions.md) | 0 | ||||
-rw-r--r-- | docs/Reference/1_BindingAssertFunctions.md (renamed from docs/WriteYourTests/Reference/1_BindingAssertFunctions.md) | 0 | ||||
-rw-r--r-- | docs/Reference/2_TestFrameworkFunctions.md (renamed from docs/WriteYourTests/Reference/2_TestFrameworkFunctions.md) | 0 | ||||
-rw-r--r-- | docs/Reference/Introduction.md (renamed from docs/WriteYourTests/Reference/Introduction.md) | 0 | ||||
-rw-r--r-- | docs/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md (renamed from docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md) | 0 | ||||
-rw-r--r-- | docs/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md (renamed from docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md) | 0 | ||||
-rw-r--r-- | docs/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md (renamed from docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md) | 0 | ||||
-rw-r--r-- | docs/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md (renamed from docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md) | 0 | ||||
-rw-r--r-- | docs/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md (renamed from docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md) | 0 | ||||
-rw-r--r-- | docs/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md (renamed from docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md) | 0 | ||||
-rw-r--r-- | docs/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md (renamed from docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md) | 0 | ||||
-rw-r--r-- | docs/Reference/LuaUnitAssertionFunctions/Introduction.md (renamed from docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/Introduction.md) | 0 | ||||
-rw-r--r-- | docs/SUMMARY.md | 36 | ||||
-rw-r--r-- | docs/WriteYourTests/0_ProjectTree.md | 172 | ||||
-rw-r--r-- | docs/WriteYourTests/1_BindingConfiguration.md | 112 | ||||
-rw-r--r-- | docs/WriteYourTests/2_LUATestFiles.md | 15 | ||||
-rw-r--r-- | docs/WriteYourTests/Introduction.md | 10 |
23 files changed, 672 insertions, 617 deletions
diff --git a/docs/0_Installation.md b/docs/0_Installation.md index f02fa88..088f4e0 100644 --- a/docs/0_Installation.md +++ b/docs/0_Installation.md @@ -1,65 +1,24 @@ # Installation -## Pre-requisites +[Setup the pre-requisite](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/host-configuration/docs/1_Prerequisites.html). -[Setup the pre-requisite](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/host-configuration/docs/1_Prerequisites.html) then [install the Application Framework](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/host-configuration/docs/2_AGL_Application_Framework.html) on your host. +Use the following command-line to get the `afb-test` binding and all its +dependencies. -You will also need to install lua-devel >= 5.3 to be able to build the project. - -Fedora: - -```bash -dnf install lua-devel -``` - -OpenSuse: +* Fedora (>= 27): ```bash -zypper install lua53-devel +dnf install agl-app-afb-test-devel ``` -Ubuntu (>= Xenial), Debian stable: +* OpenSuse (>= 15.0): ```bash -apt-get install liblua5.3-dev +zypper install agl-app-afb-test-devel ``` -## Grab source and build - -Download the **afb-test** binding source code using git: - -```shell -git clone --recurse-submodules https://gerrit.automotivelinux.org/gerrit/apps/app-afb-test -cd afb-test -mkdir build -cd build -cmake .. && make -``` - -## Test natively on your host - -If you want to use the **afb-test** binding natively on your host, you have to -install it. Then *pkg-config* tool can find the **afb-test.pc** and you can -use **afm-test** launcher: - -```bash -sudo make install -# Eventually set PKG_CONFIG_PATH environment variable if not installed in the -# system directory -export PKG_CONFIG_PATH=<path-to-pkgconfig-dir>:${PKG_CONFIG_PATH} -# The same for the PATH environment variable where afm-test has been installed -export PATH=<path-to-afm-test-dir>:${PATH} -``` - -Then you can test other binding using the **afm-test** launcher. Example here, -with another binding project using **app-templates** submodule or the -**cmake-apps-module** CMake module: - -> **Note** CMake module is the new way to use **app-templates** +* Ubuntu (>= Xenial), Debian stable: ```bash -cd build -cmake .. -make -afm-test package package-test +apt-get install agl-app-afb-test-dev ``` diff --git a/docs/1_Write_the_tests.md b/docs/1_Write_the_tests.md new file mode 100644 index 0000000..500af54 --- /dev/null +++ b/docs/1_Write_the_tests.md @@ -0,0 +1,246 @@ +# Write the tests + +## Create the test tree + +At the root of your project, create a test dedicated directory that will hold +all your tests materials. A classic test tree looks like the following: + +```tree + test + ├── CMakeLists.txt + ├── etc + │ ├── CMakeLists.txt + │ └── aft-middlename.json + ├── fixtures + │ ├── CMakeLists.txt + │ ├── helper.sh + │ ├── data + │ └── plugin.lua + └── tests + ├── CMakeLists.txt + ├── test01.lua + ├── test02.lua + └── test03.lua + ... +``` + +Here is a description of each subdirectory purpose: + +- etc: holds the test binding configuration in a JSON file. +- fixtures: contains all external needed files to run your tests. This is mainly + used to inject data or a plugin with the mock-up APIs code in a LUA or C plugin +- tests: Contains only the tests written in LUA for your binding. + +## Create your configuration file + +The configuration file describes your test api and how it launches the tests. A +test binding doesn't provide verbs and this configuration will describe the api +verb(s), mocked-up apis. Here are the `key` descriptions for this file: + +### `metadata` section + +- `uid`: A simple label mainly useful at debugging purpose +- `version` (optional): the test's version +- `info` (optional): self-explanatory +- `api`: the name that your test binding will take. Used to be the test api's + name prefixed with `aft` (ie: `aft-<tested-api-name>`) +- `require`: the tested api's name. This key ensure that the tested api is + correctly launched and initialized so the test binding could test it. + +### `testVerb` section + +- `uid`: the verb name +- `info` (optional): self-explanatory +- `action`: special string indicating which function to trigger when the verb is + called. It will always be the same about the test binding. +- `args` section: + - `trace`: the name of the tested api that you are going to test. This is + needed to let the test binding tracing the tested api and retrieve through + the binder monitoring feature `calls` and `events`. + - `files`: A string or an array of strings of the LUA files with your tests. + Only provide the file name without the path. + +### `mapis`, stand for Mock-up api, section + +- `uid`: the mocked up api's name +- `info` (optional): self explanatory +- `libs`: LUA script or C plugin file name + +#### `verbs` section + +Describe the implemented mocked up API verbs. Each verb is mapped to a function +name that will be executed at the verb call. + +- `uid`: verb's name +- `info` (optional): self explanatory +- `action`: an URI string that point to a C plugin or LUA script's function that + will be executed at the verb call. The format of the action URI is: + `<lua|plugin|api>`://`<C plugin's name|api's name|lua script name>`#`<function|verb's name>` + +#### `events` section + +This section allows you to trigger a function when a described event is received +. It could be for any event which you need to apply modifications on it. You DO +NOT have to enumerate each possible events that the mocked up api would +generate, you could avoid this section if you do not want to execute a function +at events reception. + +- `uid`: event's name (`<api>/<event-name>`) +- `info` (optional): self explanatory +- `action`: an URI string that point to a C plugin or LUA script's function that + will be executed at the event reception. The format of the action URI is: + `<lua|plugin|api>`://`<C plugin's name|api's name|lua script name>`#`<function|verb's name>` + the action `lua://AFT#_evt_catcher_` is the default `afb-test` events listener. + +### Configuration examples + +Here is a simple example: + +```json +{ + "id": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", + "$schema": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", + "metadata": { + "uid": "Hello_Test", + "version": "1.0", + "api": "aft-example", + "info": "Binding made to test other bindings", + "require": [ + "hello" + ] + }, + "testVerb": { + "uid": "testing-hello", + "info": "Launch the tests against hello api", + "action": "lua://AFT#_launch_test", + "args": { + "trace": "hello", + "files": ["aftTest.lua","helloworld.lua"] + } + } +} +``` + +and another example that mock-up the `low-can` api: + +```json +{ + "id": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", + "$schema": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", + "metadata": { + "uid": "Other_Tests", + "version": "1.0", + "api": "aft-example", + "info": "Binding made to test other bindings", + "require": [ + "tested-api" + ] + }, + "testVerb": { + "uid": "Complete", + "info": "Launch all the tests", + "action": "lua://AFT#_launch_test", + "args": { + "trace": "low-can", + "files": [ "aftTest.lua", "mapis-tests.lua" ] + } + }, + "mapis": [{ + "uid": "low-can", + "info": "Faked low-can API", + "libs": "mapi_low-can.lua", + "verbs": [ + { + "uid": "subscribe", + "info": "Subscribe to CAN signals events", + "action": "lua://low-can#_subscribe" + }, + {...}, + { + "uid": "write", + "info": "Write a CAN messages to the CAN bus.", + "action": "lua://low-can#_write" + } + ], + "events": [{ + "uid": "low-can/diagnostic_messages", + "action": "lua://AFT#_evt_catcher_" + },{ + "uid": "low-can/messages_engine_speed", + "action": "lua://AFT#_evt_catcher_" + },{ + "uid": "low-can/messages_vehicle_speed", + "action": "lua://AFT#_evt_catcher_" + }] + }] +} +``` + +## The LUA test files + +The test framework uses the LUA language to describe the tests. + +You have two different things to have in mind when you write your tests using +this framework: *test* and *assertions* functions. + +- *Assertions* functions are meant to test an atomic operation result. + (ie: `1+1 = 2` is an assertion) +- *Test* functions represent a test (Unbelievable), they represent a set of one + or several *assertions* which are all needed to succeed to valid the test. + +The framework comes with several *test* and *assertion* functions to simply be +able to test verb calls and events receiving. Use the *test* ones and if you +need more use the ones that call a callback. If the test is more complex or +more comprehensive then *describe* your test function using *assert* functions. +See the example below. + +See the test framework functions [References](Reference/Introduction.md) for a +comprehensive list of *tests* and *assertions* functions available. + +### Tests example + +```lua + function _callback(responseJ) + _AFT.assertStrContains(responseJ.response, "Some String") + end + + function _callbackError(responseJ) + _AFT.assertStrContains(responseJ.request.info, "Ping Binder Daemon fails") + end + + function _callbackEvent(eventName, eventData) + _AFT.assertEquals(eventData, {data = { key = 'weird others data', another_key = 123.456 }}) + end + + _AFT.addEventToMonitor("hello/anEvent") + _AFT.addEventToMonitor("hello/anotherEvent", _callbackEvent) + + _AFT.testVerbStatusSuccess('testPingSuccess','hello', 'ping', {}) + _AFT.testVerbResponseEquals('testPingSuccess','hello', 'ping', {}, "Some String") + _AFT.testVerbResponseEquals('testPingSuccess','hello', 'ping', {}, "Unexpected String") + _AFT.testVerbCb('testPingSuccess','hello', 'ping', {}, _callback) + _AFT.testVerbStatusError('testPingError', 'hello', 'pingfail', {}) + _AFT.testVerbResponseEqualsError('testPingError', 'hello', 'pingfail', {}, "Ping Binder Daemon fails") + _AFT.testVerbResponseEqualsError('testPingError', 'hello', 'pingfail', {}, "Ping Binder Daemon succeed") + _AFT.testVerbCbError('testPingError', 'hello', 'pingfail', {}, _callbackError) + + _AFT.testVerbStatusSuccess('testEventAdd', 'hello', 'eventadd', {tag = 'event', name = 'anEvent'}) + _AFT.testVerbStatusSuccess('testEventSub', 'hello', 'eventsub', {tag = 'event'}) + _AFT.testVerbStatusSuccess('testEventPush', 'hello', 'eventpush', {tag = 'event', data = { key = 'some data', another_key = 123}}) + + _AFT.testVerbStatusSuccess('testEventAdd', 'hello', 'eventadd', {tag = 'evt', name = 'anotherEvent'}) + _AFT.testVerbStatusSuccess('testEventSub', 'hello', 'eventsub', {tag = 'evt'}) + _AFT.testVerbStatusSuccess('testEventPush', 'hello', 'eventpush', {tag = 'evt', data = { key = 'weird others data', another_key = 123.456}}) + + _AFT.testVerbStatusSuccess('testGenerateWarning', 'hello', 'verbose', {level = 4, message = 'My Warning message!'}) + + _AFT.testEvtGrpReceived("TestEventGroupReceived",{"hello/anEvent","hello/anotherEvent"},300000) + _AFT.testEvtGrpNotReceived("TestEventGroupNotReceived",{"hello/anEvent","hello/anotherEvent"},300000) + + _AFT.testEvtReceived("testEvent", "hello/anEvent",300000) + _AFT.testEvtReceived("testEventCb", "hello/anotherEvent",300000) + + _AFT.describe("myTestLabel", function() + _AFT.assertEquals(false, false) + end) +``` diff --git a/docs/2_LaunchTheExample.md b/docs/2_LaunchTheExample.md deleted file mode 100644 index 1ef3de7..0000000 --- a/docs/2_LaunchTheExample.md +++ /dev/null @@ -1,232 +0,0 @@ -# Launch The Example - -## From a terminal (on your build host) - -To launch your tests, enter this command. - -```bash -export BUILD_DIR_PATH = "$(pwd)/build" -./afb-test.sh <BUILD_DIR_PATH> -``` - -On afb-daemon startup you should have all the app-framework config displayed: - -```shell ----BEGIN-OF-CONFIG--- --- console: ./AFB-console.out --- rootdir: . --- roothttp: --- rootbase: /opa --- rootapi: /api --- workdir: . --- uploaddir: . --- token: 1 --- name: afbd-test --- aliases: --- dbus_clients: --- dbus_servers: --- ws_clients: --- ws_servers: --- so_bindings: --- ldpaths: /opt/AGL/lib64/afb:lib --- weak_ldpaths: --- calls: --- exec: --- httpdPort: 1234 --- cacheTimeout: 100000 --- apiTimeout: 20 --- cntxTimeout: 32000000 --- nbSessionMax: 10 --- mode: local --- tracereq: common --- traceditf: no --- tracesvc: no --- traceevt: no --- no_ldpaths: no --- noHttpd: no --- background: no --- monitoring: no --- random_token: no ----END-OF-CONFIG--- -INFO: entering foreground mode -INFO: running with pid 20430 -INFO: API monitor added -INFO: binding monitor added to set main -INFO: Scanning dir=[/opt/AGL/lib64/afb] for bindings -INFO: binding [/opt/AGL/lib64/afb/demoContext.so] is a valid AFB binding V1 -INFO: binding [/opt/AGL/lib64/afb/demoContext.so] calling registering function afbBindingV1Register -INFO: API context added -INFO: binding /opt/AGL/lib64/afb/demoContext.so loaded with API prefix context -INFO: binding [/opt/AGL/lib64/afb/helloWorld.so] looks like an AFB binding V2 -INFO: binding hello calling preinit function -NOTICE: [API hello] hello binding comes to live -INFO: API hello added -INFO: binding hello added to set main -INFO: binding [/opt/AGL/lib64/afb/tic-tac-toe.so] looks like an AFB binding V2 -INFO: API tictactoe added -INFO: binding tictactoe added to set main -INFO: binding [/opt/AGL/lib64/afb/demoPost.so] is a valid AFB binding V1 -INFO: binding [/opt/AGL/lib64/afb/demoPost.so] calling registering function afbBindingV1Register -INFO: API post added -INFO: binding /opt/AGL/lib64/afb/demoPost.so loaded with API prefix post -INFO: binding [/opt/AGL/lib64/afb/ave.so] looks like an AFB binding Vdyn -... -... -... -NOTICE: API salut started -INFO: API tictactoe starting... -NOTICE: API tictactoe started -NOTICE: Waiting port=1234 rootdir=. -NOTICE: Browser URL= http://localhost:1234 -``` - -Then in a new terminal launch the client: - -``` bash -afb-client-demo ws://localhost:1234/api?token=1 -afTest launch_all_tests -``` - -You should get something like: - -``` bash -{"response":{"info":"Launching tests"},"jtype":"afb-reply","request":{"status":"success","uuid":"3fa17ce6-0029-4ef9-8e0d-38dba2a9cf38"}} -{"event":"afTest\/results","data":{"info":"Success : 72 Failures : 6"},"jtype":"afb-event"} -``` - -Here you can see that the verb succeeded and that we have 71 Success for 5 failures. - -And on your afb-daemon terminal you have all information about your tests step-by-step (note that it depends on the level of verbosity you gave to the afb-daemon (-vvv option)). - -```shell -DEBUG: received websocket request for afTest/launch_all_tests: null -HOOK: [xreq-000001:afTest/launch_all_tests] BEGIN -HOOK: [xreq-000001:afTest/launch_all_tests] json() -> "null" -HOOK: [xreq-000002:monitor/set] BEGIN -HOOK: [xreq-000002:monitor/set] reply[denied](null, invalid token's identity) -HOOK: [xreq-000002:monitor/set] END -HOOK: [xreq-000003:monitor/trace] BEGIN -HOOK: [xreq-000003:monitor/trace] reply[denied](null, invalid token's identity) -HOOK: [xreq-000003:monitor/trace] END -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -... -... -... -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -~~~~~~~~~~ BEGIN ALL TESTS ~~~~~~~~~~ -HOOK: [xreq-000001:afTest/launch_all_tests] reply[success]({ "info": "Launching tests" }, (null)) -# XML output to var/jUnitResults.xml -# Started on Wed Jul 11 15:42:44 2018 -# Starting class: testPingSuccess -# Starting test: testPingSuccess.testFunction -~~~~~ Begin testPingSuccess ~~~~~ -HOOK: [xreq-000004:hello/ping] BEGIN -HOOK: [xreq-000004:hello/ping] json() -> null -HOOK: [xreq-000004:hello/ping] reply[success]("Some String", Ping Binder Daemon tag=pingSample count=1 query=null) -HOOK: [xreq-000004:hello/ping] END -~~~~~ End testPingSuccess ~~~~~ -# Starting class: testPingSuccessAndResponse -# Starting test: testPingSuccessAndResponse.testFunction -HOOK: [xreq-000005:hello/ping] BEGIN -HOOK: [xreq-000005:hello/ping] json() -> null -HOOK: [xreq-000005:hello/ping] reply[success]("Some String", Ping Binder Daemon tag=pingSample count=2 query=null) -HOOK: [xreq-000005:hello/ping] END -# Starting class: testPingSuccessResponseFail -# Starting test: testPingSuccessResponseFail.testFunction -HOOK: [xreq-000006:hello/ping] BEGIN -HOOK: [xreq-000006:hello/ping] json() -> null -HOOK: [xreq-000006:hello/ping] reply[success]("Some String", Ping Binder Daemon tag=pingSample count=3 query=null) -HOOK: [xreq-000006:hello/ping] END -# Failure: ./var/aft.lua:224: expected: "Unexpected String" -# actual: "Some String" -... -... -... -~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ -HOOK: [xreq-000001:afTest/launch_all_tests] END - -``` - -## On target - -If you are trying to launch your test on a target you'll have to use -a test widget which contains test files, fixture and configuration. -Then use **afm-test**: - -```bash -./afm-test <path> -``` - -By default, the test widgets should be located in /usr/AGL/apps/testwgt. -This it will install the widget, launch the tests then display the result -on standard output. After that it will kill test app and remove it. - -You can produce a widget buy entering ```make widget``` in *build/*. - -Here is an example: - -```bash - -qemux86-64:~# afm-test /usr/AGL/apps/testwgt/aftest-test.wgt -PASS: aftest-test@5.99 started with pid=3857 -null -null -1..62 -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 1 TestListverb.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 2 TestGetVerb.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 3 Test_turning_on.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ Begin Test Assert Equals ~~~~~ -~~~~~ End Test Assert Equals ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 4 testAssertEquals.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 5 testAssertNotEquals.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 6 testAssertItemsEquals.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 7 testAssertAlmostEquals.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 8 testAssertNotAlmostEquals.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 9 testAssertEvalToTrue.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 10 testAssertEvalToFalse.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -[...] -PASS: 60 testAssertVerbStatusError.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 61 testAssertVerbResponseEqualsError.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 62 testAssertVerbCbError.testFunction -# Ran 62 tests in 0.003 seconds, 62 successes, 0 failures -~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ -PASS: aftest-test@5.99 killed and removed - -``` - -The command being : ```afm-test /usr/AGL/apps/testwgt/aftest-test.wgt``` - -You can see here that everything ran as on your pc terminal. -**Begin Test** and **End Test** are the -beforeEach and afterEach functions and -**END ALL TESTS** is the after all functions. - - **PASS :** shows the function that is or was running.
\ No newline at end of file diff --git a/docs/2_The_test_widget.md b/docs/2_The_test_widget.md new file mode 100644 index 0000000..19d77a1 --- /dev/null +++ b/docs/2_The_test_widget.md @@ -0,0 +1,65 @@ +# How to build the test widget using app-templates/cmake-apps-module + +## Defining CMake targets + +Now that the test tree has been created, in each directory you have to create +a `CMakeLists.txt` file to hold the CMake's target definition. For each target +you need to specify a **LABELS** depending on the purpose of the files for each +directory. There are more explanations about using the *cmake-apps-module* (the +former *app-templates* submodule) in the [documentation website](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/sdk-devkit/docs/part-2/2_4-Use-app-templates.html#using-cmake-template-macros). + +Here is a cheat sheet to map the **LABELS** target for each classic test tree +directory: + +* `etc` uses the label **TEST-CONFIG** +* `fixtures` uses the label **TEST-DATA** +* `tests` uses the label **TEST-DATA** + +i.e for the `etc` folder: + +```cmake +PROJECT_TARGET_ADD(afb-test-config) + + file(GLOB CONF_FILES "*.json") + + add_input_files("${CONF_FILES}") + + SET_TARGET_PROPERTIES(${TARGET_NAME} PROPERTIES + LABELS "TEST-CONFIG" + OUTPUT_NAME ${TARGET_NAME} + ) +``` + +> **CAUTION**: make sure that you have CMakeLists files that include your +> subdirectories target (cf: previous chapter `Write the tests`). + +## Build the test widget + +By default, the test widget is not built, you have to specify that you want to +build it or use a special target. + +### Building at the same time than classic widget + +Specify the option `BUILD_TEST_WGT=TRUE` when you configure your build. + +ie: + +```bash +cd build +cmake -DBUILD_TEST_WIDGET=TRUE .. +make +make widget +``` + +### Building separately only the test widget + +Just use the target `test_widget` after a classic build. + +ie: + +```bash +cd build +cmake .. +make +make test_widget +``` diff --git a/docs/3_Launch_the_tests.md b/docs/3_Launch_the_tests.md new file mode 100644 index 0000000..b577acb --- /dev/null +++ b/docs/3_Launch_the_tests.md @@ -0,0 +1,327 @@ +# How to launch the tests ? + +## Natively during the development + +It could be convenient to be able to test the software that you are currently +developing. Then you can ensure that your modifications haven't introduced +regressions, bugs, etc. This depends upon your tests of course. + +As previously saw, you need the `test widget` to be able to launch the tests and +you need also to have the `afb-test` binding installed and the `application +framework binder` to be able to execute your tests. + +Assuming you already installed those components on your development host, then +proceed as the following from the project root directory: + +```bash +cd build +cmake -DBUILD_TEST_WGT=TRUE .. +make +make widget +``` + +To prepare all files needed for the test launch then use the `afm-test` script: + +```bash +# Usage of afm-test command line utility +afm-test --help +Usage: /opt/AGL/bin/afm-test <binding-wgt-rootdir> <test-wgt-rootdir> [-m|--mode <SOLO|SERVICE>] [-t|--timeout <X>] [-l|--lavaoutput] +binding-wgt-rootdir: path to the test wgt file +test-wgt-rootdir: path to the test folder file +-m|--mode: SOLO (1 binder) or SERVICE (2 binders) (Default: SOLO) +-t|--timeout: timeout in second. (Default 3 seconds) +-l|--lavaoutput: Flags indicating the binding to add Lava special test markers. +Error: Test launch failed. Code: 1 +# Launching the test from your build project directory +afm-test package package-test +``` + +### Example with the afb-test selftest suite + +Prepare the launch building the `test widget`: + +```bash +$ cd build +# Cleaning the previous build files +$ rm -rf * +# Configuration step +$ cmake -DBUILD_TEST_WGT=TRUE .. +-- The C compiler identification is GNU 8.2.1 +-- The CXX compiler identification is GNU 8.2.1 +-- Check for working C compiler: /usr/lib64/ccache/cc +-- Check for working C compiler: /usr/lib64/ccache/cc -- works +-- Detecting C compiler ABI info +-- Detecting C compiler ABI info - done +-- Detecting C compile features +-- Detecting C compile features - done +-- Check for working CXX compiler: /usr/lib64/ccache/c++ +-- Check for working CXX compiler: /usr/lib64/ccache/c++ -- works +-- Detecting CXX compiler ABI info +-- Detecting CXX compiler ABI info - done +-- Detecting CXX compile features +-- Detecting CXX compile features - done +Distribution detected (separated by ';' choose one of them) fedora +Include: /home/claneys/.config/app-templates/cmake.d/00-common-var.cmake +Include: /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/conf.d/cmake/00-default-osconfig.cmake +Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/01-build_options.cmake +-- Found PkgConfig: /usr/bin/pkg-config (found version "1.4.2") +-- Checking for module 'json-c' +-- Found json-c, version 0.13.1 +-- Checking for module 'libsystemd>=222' +-- Found libsystemd, version 238 +-- Checking for module 'afb-daemon>=4.0' +-- Found afb-daemon, version 6.90.0 +-- Checking for module 'lua>=5.3' +-- Found lua, version 5.3.4 +Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/02-variables.cmake +-- Check gcc_minimal_version (found gcc version 8.2.1) (found g++ version 8.2.1) +Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/03-macros.cmake +Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/04-extra_targets.cmake +-- Overwrite the CMAKE default install prefix with /opt/AGL +-- Found CURL: /usr/lib64/libcurl.so (found version "7.59.0") +-- Qt's WebSocket AFB Client: Disabled! +-- CURL wrapping helpers: Enabled! +-- Notice: LUA Controler Support Selected +-- Notice: Using default test widget configuration's file. +-- If you want to use a customized test-config.xml template then specify TEST_WIDGET_CONFIG_TEMPLATE in your config.cmake file. +-- Configuring done +-- Generating done +-- Build files have been written to: /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build +# Build the binding +$ make +Scanning dependencies of target test-files +[ 3%] Generating test-files +[ 3%] Built target test-files +Scanning dependencies of target project_populate_test-files +[ 6%] Generating package-test/var/test-files +[ 6%] Built target project_populate_test-files +Scanning dependencies of target afb-test-config +[ 9%] Generating afb-test-config +Warning: JSON_CHECKER not found. Not verification made on files ! +[ 9%] Built target afb-test-config +Scanning dependencies of target project_populate_afb-test-config +[ 12%] Generating package-test/etc/afb-test-config +[ 12%] Built target project_populate_afb-test-config +Scanning dependencies of target aftest-config +[ 16%] Generating aftest-config +Warning: JSON_CHECKER not found. Not verification made on files ! +[ 16%] Built target aftest-config +Scanning dependencies of target project_populate_aftest-config +[ 19%] Generating package/etc/aftest-config +[ 19%] Built target project_populate_aftest-config +Scanning dependencies of target fixture-files +[ 22%] Generating fixture-files +[ 22%] Built target fixture-files +Scanning dependencies of target project_populate_fixture-files +[ 25%] Generating package-test/var/fixture-files +[ 25%] Built target project_populate_fixture-files +Scanning dependencies of target afTest-lua +[ 29%] Generating afTest-lua +[ 29%] Built target afTest-lua +Scanning dependencies of target project_populate_afTest-lua +[ 32%] Generating package/var/afTest-lua +[ 32%] Built target project_populate_afTest-lua +Scanning dependencies of target afb-helpers +[ 35%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/wrap-json.c.o +[ 38%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/filescan-utils.c.o +[ 41%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/escape.c.o +[ 45%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/curl-wrap.c.o +[ 48%] Linking C static library libafb-helpers.a +[ 48%] Built target afb-helpers +Scanning dependencies of target ctl-utilities +[ 51%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-action.c.o +[ 54%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-config.c.o +[ 58%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-onload.c.o +[ 61%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-plugin.c.o +[ 64%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-control.c.o +[ 67%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-event.c.o +[ 70%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-lua.c.o +[ 74%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-timer.c.o +[ 77%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-lua-utils.c.o +[ 80%] Linking C static library libctl-utilities.a +[ 80%] Built target ctl-utilities +Scanning dependencies of target aft +[ 83%] Building C object src/CMakeFiles/aft.dir/aft.c.o +[ 87%] Building C object src/CMakeFiles/aft.dir/mapis.c.o +[ 90%] Linking C shared module aft.so +[ 90%] Built target aft +Scanning dependencies of target project_populate_aft +[ 93%] Generating package/lib/aft.so +[ 93%] Built target project_populate_aft +Scanning dependencies of target populate +[ 96%] Generating package/bin, package/etc, package/lib, package/htdocs, package/var, package-test/bin, package-test/etc, package-test/lib, package-test/htdocs, package-test/var +[ 96%] Built target populate +Scanning dependencies of target afTest_build_done +++ Typical binding launch: afb-daemon --name afTest --port=1234 --workdir=package-test --ldpaths=/opt/AGL/lib64/afb:../package/lib --token= +[ 96%] Built target afTest_build_done +Scanning dependencies of target autobuild +[100%] Built target autobuild +# Build both widgets classic and test +$ make widget +[ 2%] Built target test-files +Scanning dependencies of target packaging_wgt +[ 5%] Generating package/config.xml +[ 8%] Generating package-test/config.xml, package-test/bin/launcher +[ 8%] Built target packaging_wgt +[ 11%] Generating package-test/var/test-files +[ 11%] Built target project_populate_test-files +Warning: JSON_CHECKER not found. Not verification made on files ! +[ 14%] Built target afb-test-config +[ 17%] Generating package-test/etc/afb-test-config +[ 17%] Built target project_populate_afb-test-config +Warning: JSON_CHECKER not found. Not verification made on files ! +[ 20%] Built target aftest-config +[ 23%] Generating package/etc/aftest-config +[ 23%] Built target project_populate_aftest-config +[ 26%] Built target fixture-files +[ 29%] Generating package-test/var/fixture-files +[ 29%] Built target project_populate_fixture-files +[ 32%] Built target afTest-lua +[ 35%] Generating package/var/afTest-lua +[ 35%] Built target project_populate_afTest-lua +[ 50%] Built target afb-helpers +[ 79%] Built target ctl-utilities +[ 88%] Built target aft +[ 91%] Built target project_populate_aft +[ 94%] Built target populate +Scanning dependencies of target widget +[ 97%] Generating aftest.wgt +NOTICE: -- PACKING widget aftest.wgt from directory /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build/package +[100%] Generating aftest-test.wgt +NOTICE: -- PACKING widget aftest-test.wgt from directory /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build/package-test +++ Install widget file using in the target : afm-util install afTest.wgt +[100%] Built target widget +``` + +Launch the test using the mode SERVICE for the `afb-test` because of a recursion +problem which loads 2 times the same binding and causes conflict. So it is +needed to uses 2 binders then each ones loads its binding properly: + +```bash +$ afm-test package package-test/ -m SERVICE +1..4 +# Started on Tue Oct 30 10:32:46 2018 +# Starting class: TestListverb +ok 1 TestListverb.testFunction +# Starting class: TestGetVerb +ok 2 TestGetVerb.testFunction +# Starting class: Test_turning_on +ok 3 Test_turning_on.testFunction +# Starting class: testLockWait +ok 4 testLockWait.testFunction +# Ran 4 tests in 0.001 seconds, 4 successes, 0 failures +~~~~~~~~~~ BEGIN ALL TESTS ~~~~~~~~~~ +1..59 +# Started on Tue Oct 30 10:32:48 2018 +# Starting class: testAssertEquals +~~~~~ Begin Test ~~~~~ +~~~~~ Begin Test Assert Equals ~~~~~ +~~~~~ End Test Assert Equals ~~~~~ +~~~~~ End Test ~~~~~ +ok 1 testAssertEquals.testFunction +# Starting class: testAssertNotEquals +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +ok 2 testAssertNotEquals.testFunction +# Starting class: testAssertItemsEquals +~~~~~ Begin Test ~~~~~ +[...] +~~~~~ End Test ~~~~~ +ok 57 testAssertVerbStatusError.testFunction +# Starting class: testAssertVerbResponseEqualsError +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +ok 58 testAssertVerbResponseEqualsError.testFunction +# Starting class: testAssertVerbCbError +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +ok 59 testAssertVerbCbError.testFunction +# Ran 59 tests in 0.003 seconds, 59 successes, 0 failures +~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ +Tests correctly launched. +``` + +## Launch test on a target board + +If you are trying to launch your test on a target you'll have to use +a test widget which contains test files, fixture and configuration. +Then use **afm-test**: + +```bash +# afm-test -h +Usage: /usr/bin/afm-test [-l|--lava] [-v|--verb <verb>] <path> +-l|--lavaoutput: flag that enable Lava test marker to the output. (Default: disabled) +-v|--verb: select a specific verb to launch from the test API. (Default: all) +path: path to the test wgt file +``` + +By default, the test widgets should be located in /usr/AGL/apps/testwgt. This +will install the widget, launch the tests then display the result on standard +output. After that it will kill test binding and remove it. + +### Example on a target + +Here is an example: + +```bash +qemux86-64:~# afm-test /usr/AGL/apps/testwgt/aftest-test.wgt +afm-test /tmp/aftest-test.wgt +PASS: aftest-test@6.90 started with pid=1649 +1..59 +~~~~~ Begin Test ~~~~~ +~~~~~ Begin Test Assert Equals ~~~~~ +~~~~~ End Test Assert Equals ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 1 testAssertEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 2 testAssertNotEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 3 testAssertItemsEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +[...] +PASS: 58 testAssertVerbResponseEqualsError.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 59 testAssertVerbCbError.testFunction +# Ran 59 tests in 0.003 seconds, 59 successes, 0 failures +~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ +1..63 +~~~~~ Begin Test ~~~~~ +~~~~~ Begin Test Assert Equals ~~~~~ +~~~~~ End Test Assert Equals ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 1 testAssertEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 2 testAssertNotEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 3 testAssertItemsEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +[...] +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 61 TestGetVerb.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 62 Test_turning_on.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 63 testLockWait.testFunction +# Ran 63 tests in 0.003 seconds, 63 successes, 0 failures +~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ +PASS: aftest-test@6.90 killed and removed +``` + +The command being : ```afm-test /usr/AGL/apps/testwgt/aftest-test.wgt``` + +You can see here that everything ran as on your pc terminal. +**Begin Test** and **End Test** are the +beforeEach and afterEach functions and +**END ALL TESTS** is the after all functions. + + **PASS :** shows the function that is or was running. diff --git a/docs/1_TestExample.md b/docs/4_Tests_Examples.md index 6522def..caccea4 100644 --- a/docs/1_TestExample.md +++ b/docs/4_Tests_Examples.md @@ -51,7 +51,7 @@ The example will run some basics tests on API verb calls and events received. _AFT.testEvtReceived("testEvent", "hello/anEvent",300000) _AFT.testEvtReceived("testEventCb", "hello/anotherEvent",300000) - _AFT.testCustom("mytest", function() + _AFT.describe("myTestLabel", function() _AFT.assertEquals(false, false) end) ``` @@ -59,7 +59,6 @@ The example will run some basics tests on API verb calls and events received. ## aftTest.lua ```lua - _AFT.setBeforeEach(function() print("~~~~~ Begin Test ~~~~~") end) _AFT.setAfterEach(function() print("~~~~~ End Test ~~~~~") end) @@ -74,10 +73,10 @@ _AFT.describe("testAssertEquals", function() _AFT.assertEquals(false, false) end function() print("~~~~~ End Test Assert Equals ~~~~~") end) _AFT.describe("testAssertNotEquals", function() _AFT.assertNotEquals(true,false) end) -_AFT.describe("testAssertItemsEquals", function() _AFT.assertItemsEquals({1,2,3},{3,1,2}) end) -_AFT.describe("testAssertAlmostEquals", function() _AFT.assertAlmostEquals(1.25 ,1.5,0.5) end) -_AFT.describe("testAssertNotAlmostEquals", function() _AFT.assertNotAlmostEquals(1.25,1.5,0.125) end) -_AFT.describe("testAssertEvalToTrue", function() _AFT.assertEvalToTrue(true) end) +_AFT.describe("testAssertItemsEquals", function() _AFT.assertItemsEquals({1,2,3},{3,1,2}) end) +_AFT.describe("testAssertAlmostEquals", function() _AFT.assertAlmostEquals(1.25 ,1.5,0.5) end) +_AFT.describe("testAssertNotAlmostEquals", function() _AFT.assertNotAlmostEquals(1.25,1.5,0.125) end) +_AFT.describe("testAssertEvalToTrue", function() _AFT.assertEvalToTrue(true) end) _AFT.describe("testAssertEvalToFalse", function() _AFT.assertEvalToFalse(false) end) _AFT.describe("testAssertStrContains", function() _AFT.assertStrContains("Hello I'm a string","string") end) @@ -90,7 +89,7 @@ _AFT.describe("testAssertNotStrContains", function() _AFT.assertNotStrContains( ... ... ... -_AFT.describe("testAssertNotIsUserdata", function() _AFT.assertNotIsUserdata(2) end) +_AFT.describe("testAssertNotIsUserdata", function() _AFT.assertNotIsUserdata(2) end) function _callback(responseJ) _AFT.assertStrContains(responseJ.response, "Some String") end @@ -105,4 +104,4 @@ _AFT.describe("testAssertVerbCbError",function() _AFT.assertVerbCbError('hello', ``` > **NOTE**: I suggest you to take this lua file example to make your own test -> then read the following the chapter if needed to write more complicated tests.
\ No newline at end of file +> then read the following chapter if needed to write more complex tests. diff --git a/docs/WriteYourTests/Reference/0_BindingTestFunctions.md b/docs/Reference/0_BindingTestFunctions.md index 8da86da..8da86da 100644 --- a/docs/WriteYourTests/Reference/0_BindingTestFunctions.md +++ b/docs/Reference/0_BindingTestFunctions.md diff --git a/docs/WriteYourTests/Reference/1_BindingAssertFunctions.md b/docs/Reference/1_BindingAssertFunctions.md index 918bd04..918bd04 100644 --- a/docs/WriteYourTests/Reference/1_BindingAssertFunctions.md +++ b/docs/Reference/1_BindingAssertFunctions.md diff --git a/docs/WriteYourTests/Reference/2_TestFrameworkFunctions.md b/docs/Reference/2_TestFrameworkFunctions.md index 20184ec..20184ec 100644 --- a/docs/WriteYourTests/Reference/2_TestFrameworkFunctions.md +++ b/docs/Reference/2_TestFrameworkFunctions.md diff --git a/docs/WriteYourTests/Reference/Introduction.md b/docs/Reference/Introduction.md index 8de5a15..8de5a15 100644 --- a/docs/WriteYourTests/Reference/Introduction.md +++ b/docs/Reference/Introduction.md diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md index 4a719b6..4a719b6 100644 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md +++ b/docs/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md index ed927cd..ed927cd 100644 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md +++ b/docs/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md index 9f80a7a..9f80a7a 100644 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md +++ b/docs/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md index a816998..a816998 100644 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md +++ b/docs/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md index 482b98d..482b98d 100644 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md +++ b/docs/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md index 843f651..843f651 100644 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md +++ b/docs/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md index 8d29988..8d29988 100644 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md +++ b/docs/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/Introduction.md b/docs/Reference/LuaUnitAssertionFunctions/Introduction.md index 2bdf05c..2bdf05c 100644 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/Introduction.md +++ b/docs/Reference/LuaUnitAssertionFunctions/Introduction.md diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index cc5e57a..e4a45d5 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -2,23 +2,23 @@ * [Installation](0_Installation.md) -* [Test example](1_TestExample.md) +* [Write the tests](1_Write_the_tests.md) -* [Launch the example](2_LaunchTheExample.md) +* [Launch the example](2_The_test_widget.md) -* [Write your tests](WriteYourTests/Introduction.md) - * [Project Tree](WriteYourTests/0_ProjectTree.md) - * [Binding configuration](WriteYourTests/1_BindingConfiguration.md) - * [LUA test files](WriteYourTests/2_LUATestFiles.md) - * [References](WriteYourTests/Reference/Introduction.md) - * [Binding test functions](WriteYourTests/Reference/0_BindingTestFunctions.md) - * [Binding assert functions](WriteYourTests/Reference/1_BindingAssertFunctions.md) - * [Test framework functions](WriteYourTests/Reference/2_TestFrameworkFunctions.md) - * [LUAUnit assertion functions](WriteYourTests/Reference/LuaUnitAssertionFunctions/Introduction.md) - * [General Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md) - * [Value Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md) - * [Scientific Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md) - * [String Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md) - * [Error Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md) - * [Type Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md) - * [Table Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md)
\ No newline at end of file +* [How to launch the tests ?](3_Launch_the_tests.md) + +* [Tests examples?](4_Tests_Examples.md) + +* [References](Reference/Introduction.md) + * [Binding test functions](Reference/0_BindingTestFunctions.md) + * [Binding assert functions](Reference/1_BindingAssertFunctions.md) + * [Test framework functions](Reference/2_TestFrameworkFunctions.md) + * [LUAUnit assertion functions](Reference/LuaUnitAssertionFunctions/Introduction.md) + * [General Assertions](Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md) + * [Value Assertions](Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md) + * [Scientific Assertions](Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md) + * [String Assertions](Reference/LuaUnitAssertionFunctions/3_StringAssertions.md) + * [Error Assertions](Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md) + * [Type Assertions](Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md) + * [Table Assertions](Reference/LuaUnitAssertionFunctions/6_TableAssertions.md) diff --git a/docs/WriteYourTests/0_ProjectTree.md b/docs/WriteYourTests/0_ProjectTree.md deleted file mode 100644 index 1905d21..0000000 --- a/docs/WriteYourTests/0_ProjectTree.md +++ /dev/null @@ -1,172 +0,0 @@ -# Test architecture - -## Files tree and organization - -```tree - -"test" - +-- etc - | +-- aft-yourbinding.json - | +-- CMakeLists.txt - +-- fixture - | +-- a-script.sh - | +-- any-needed.data - | +-- CMakeLists.txt - | +-- data.json - +-- tests - | +-- CMakeLists.txt - | +-- test01.lua - | +-- test02.lua - | ... - -``` - -To integrate tests in your project, create a **test** subfolder at your project -root directory and fulfill it with appropriate files like shown as above. - -To make it simple you'll have to write your tests using lua language and store it -in the **tests** folder (as shown above) and create a JSON configuration file -*aft-yourbinding.json* to be able to launch your tests. You'll see in the next -section how to write a proper configuration file. - -> **Note** that if you create a new json file, its name has to start with "aft-" -> followed by the binder's name. (e.g. aft-low-can for the low-level-can-service) - -## Integration with CMake using App-templates - -To make the link between your test files, config files, data files -and the test binding, you will have to integrate them with CMake using the -App-templates. - -First you will have to create your CMake target using **PROJECT_TARGET_ADD** -with your target name as parameter, it will include the target to -your project. - -Then add your data files using **add_input_files** with your files in -parameter. - -Use **SET_TARGET_PROPERTIES** to fit the targets properties for macros -usage. Here you have to specify what type of your targets you want to include -in the widget package using the property **LABELS**. It will most likely either -be *TEST-DATA* or *TEST-CONFIG*. - -Here is the LABELS list: - -- **TEST-CONFIG**: JSON configuration files that will be used by the afb-test - binding to know how to execute tests. -- **TEST-DATA**: Resources used to test your binding. It is at least your test - plan and also could be fixtures and any files needed by your tests. These files - will appear in a separate test widget. -- **TEST-PLUGIN**: Shared library meant to be used as a binding - plugin. Binding would load it as a plugin to extend its functionalities. It - should be named with a special extension that you choose with SUFFIX cmake - target property or it'd be **.ctlso** by default. -- **TEST-HTDOCS**: Root directory of a web app. This target has to build its - directory and put its files in the ${CMAKE_CURRENT_BINARY_DIR}/${TARGET_NAME} -- **TEST-EXECUTABLE**: Entry point of your application executed by the AGL - Application Framework -- **TEST-LIBRARY**: An external 3rd party library bundled with the binding for its - own use in case the platform doesn't provide it. - -Here is a mapping between LABELS and directories where files will be placed in -the widget: - -- **EXECUTABLE** : \<wgtrootdir\>/bin -- **BINDING-CONFIG** : \<wgtrootdir\>/etc -- **BINDING** | **BINDINGV2** | **BINDINGV3** | **LIBRARY** : \<wgtrootdir\>/lib -- **PLUGIN** : \<wgtrootdir\>/lib/plugins -- **HTDOCS** : \<wgtrootdir\>/htdocs -- **BINDING-DATA** : \<wgtrootdir\>/var -- **DATA** : \<wgtrootdir\>/var - -And about test dedicated **LABELS**: - -- **TEST-EXECUTABLE** : \<TESTwgtrootdir\>/bin -- **TEST-CONFIG** : \<TESTwgtrootdir\>/etc -- **TEST-PLUGIN** : \<TESTwgtrootdir\>/lib/plugins -- **TEST-HTDOCS** : \<TESTwgtrootdir\>/htdocs -- **TEST-DATA** : \<TESTwgtrootdir\>/var - -> **TIP** you should use the prefix _afb-_ with your **BINDING* targets which -> stand for **Application Framework Binding**. - -You will find in further description about it [here](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/sdk-devkit/docs/part-2/2_4-Use-app-templates.html#targets-properties). - -Here is an example of a proper CMake file to include your LUA test files: - -```CMake -PROJECT_TARGET_ADD(test-files) - - file(GLOB LUA_FILES "*.lua") - add_input_files("${LUA_FILES}") - - SET_TARGET_PROPERTIES(${TARGET_NAME} PROPERTIES - LABELS "TEST-DATA" - OUTPUT_NAME ${TARGET_NAME} - ) -``` - -## Build the test widget - -### Using CMake Apps module or app-templates - -> **Note** the CMake module is the new way to use **app-templates** - -To launch tests on a target board, you need to build a test widget. Using the -SDK, you only have to set the variable *BUILD_TEST_WGT=TRUE* when configuring -the project. - -Example from another project than **afb-test**: - -```bash -mkdir build -cd build -cmake -DBUILD_TEST_WGT=TRUE .. -make -make widget -``` - -### Without using CMake Apps module or app-templates - -Like you'd build a regular widget create a directory where you'll put your tests -materials: LUA tests, configuration and fixture files. - -Then create in that directory a **bin** directory where you'll put this -[script](https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/app-templates.git;a=blob_plain;f=test-widget/launcher.sh.in;h=005c43357db3daa71b66d95d2486cd13f5cee482;hb=refs/heads/master) and name it **launcher**. To finish, you'll also need a -widget configuration file. You can use the example provided [here](https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/app-templates.git;a=blob_plain;f=test-widget/test-config.xml.in;hb=refs/heads/master) and edit it -to fit your needs by replacing the variables surrounded by **@** characters. - -Example from another project than **afb-test**: - -```bash -cd build -BUILDDIR="$(pwd)" -mkdir -p package-test/bin -cd package-test -wget https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/app-templates.git;a=blob_plain;f=test-widget/launcher.sh.in;h=005c43357db3daa71b66d95d2486cd13f5cee482;hb=refs/heads/master -O bin/launcher -wget https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/app-templates.git;a=blob_plain;f=test-widget/test-config.xml.in;hb=refs/heads/master -O config.xml -vim config.xml -``` - -Then once your widget content directory is complete, use **wgtpkg-pack** utility -to create the test widget. - -```bash -cd ${BUILDDIR} -wgtpkg-pack -f -o <project_name>-test.wgt package-test -``` - -## Run the test widget on the target - -Once built you can send the wgt file to your target board and launch the tests -using **afm-test** as follow: - -```bash -afm-test <path-to-your-test.wgt> -``` - -**afm-test** installs the test wgt file, then runs it and uninstalls it. - -> **CAUTION**: Be cautious to not specify a regular widget's app, else it will -> be uninstalled at the end of the operation. So, if it was an important -> service/binding, others bindings or apps relying on it won't work. diff --git a/docs/WriteYourTests/1_BindingConfiguration.md b/docs/WriteYourTests/1_BindingConfiguration.md deleted file mode 100644 index 2bd31c5..0000000 --- a/docs/WriteYourTests/1_BindingConfiguration.md +++ /dev/null @@ -1,112 +0,0 @@ -# Test configuration - -The file `aft-test.json` contains the controller binding configuration. Here, -you have to change or define the *files* key in the *args* object of the -*testVerb* section, *testVerb* is an array of verb definition which are -meant to launch different LUA test files. - -Also you MUST specify which *api* you need to trace to perform your tests. -Specify which api to trace using a pattern. - -Edit the JSON array to point to your tests files. - -Here is an example: - -```json -{ - "id": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", - "$schema": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", - "metadata": { - "uid": "Test", - "version": "1.0", - "api": "afTest", - "info": "Binding made to test other bindings", - "require": [ - "hello" - ] - }, - "testVerb": { - "uid": "launch_all_tests", - "info": "Launch all the tests", - "action": "lua://AFT#_launch_test", - "args": { - "trace": "hello", - "files": ["aftTest.lua","helloworld.lua"] - } - } -} -``` - -and another example which tests the low-can api: - -```json -{ - "id": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", - "$schema": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", - "metadata": { - "uid": "Test", - "version": "1.0", - "api": "aft-aftest", - "info": "Binding made to test other bindings", - "require": [ - "low-can" - ] - }, - "testVerb": { - "uid": "launch_all_tests", - "info": "Launch all the tests", - "action": "lua://AFT#_launch_test", - "args": { - "trace": "low-can", - "files": [ "aftTest.lua", "mapis-tests.lua" ] - } - }, - "mapis": [{ - "uid": "low-can", - "info": "Faked low-can API", - "libs": "mapi_low-can.lua", - "verbs": [ - { - "uid": "subscribe", - "info": "Subscribe to CAN signals events", - "action": "lua://low-can#_subscribe" - }, - { - "uid": "unsubscribe", - "info": "Unsubscribe previously suscribed signals.", - "action": "lua://low-can#_unsubscribe" - }, - { - "uid": "get", - "info": "get a current value of CAN message", - "action": "lua://low-can#_get" - }, - { - "uid": "list", - "info": "get a supported CAN message list", - "action": "lua://low-can#_list" - }, - { - "uid": "auth", - "info": "Authenticate session to be raise Level Of Assurance.", - "action": "lua://low-can#_auth" - }, - { - "uid": "write", - "info": "Write a CAN messages to the CAN bus.", - "action": "lua://low-can#_write" - } - ], - "events": [{ - "uid": "low-can/diagnostic_messages", - "action": "lua://AFT#_evt_catcher_" - },{ - "uid": "low-can/messages_engine_speed", - "action": "lua://AFT#_evt_catcher_" - },{ - "uid": "low-can/messages_vehicle_speed", - "action": "lua://AFT#_evt_catcher_" - }] - }] -} -``` diff --git a/docs/WriteYourTests/2_LUATestFiles.md b/docs/WriteYourTests/2_LUATestFiles.md deleted file mode 100644 index 05752dd..0000000 --- a/docs/WriteYourTests/2_LUATestFiles.md +++ /dev/null @@ -1,15 +0,0 @@ -# LUA Test files - -You have two differents things to take in account when you'll write your tests -using this framework: *test* and *assertions*. - -*Assertions* are functions meant to test an atomic operation result. -(ie: `1+1 = 2` is an assertion) - -*Test* functions represent a test (Unbelievable), they represent a set of one or -several *assertions* which are all needed to succeed to valid the test. - -The framework came with several *test* and *assertion* functions to simply be -able to test verb calls and events receiving. Use the simple one as often as -possible and if you need more use the one that calls a callback. Specifying a -callback let you add assertions and enrich the test. diff --git a/docs/WriteYourTests/Introduction.md b/docs/WriteYourTests/Introduction.md deleted file mode 100644 index a8b457b..0000000 --- a/docs/WriteYourTests/Introduction.md +++ /dev/null @@ -1,10 +0,0 @@ -# Write your test - -Follow this section to know how to write your test. - -Section's Table of content : - -* [Project Tree](0_ProjectTree.html) -* [Binding configuration](1_BindingConfiguration.html) -* [LUA test files](2_LUATestFiles.html) -* [References](Reference/Introduction.html) |