From 16922b73bf17b2f23148438957cd3d18542075f4 Mon Sep 17 00:00:00 2001 From: Romain Forlot Date: Mon, 29 Oct 2018 18:37:26 +0100 Subject: Docs reorganization and update Docs reorganization and update using the latest afb-test improvments and try to make it clearer. Change-Id: If022cdb46364ef250361bdcd420d45b360f10a2e Signed-off-by: Romain Forlot (cherry picked from commit 2f3654275e44f9719818a848da9a0b576d5a1b53) --- docs/0_Installation.md | 59 +--- docs/1_TestExample.md | 108 ------- docs/1_Write_the_tests.md | 246 ++++++++++++++++ docs/2_LaunchTheExample.md | 232 --------------- docs/2_The_test_widget.md | 97 ++++++ docs/3_Launch_the_tests.md | 327 +++++++++++++++++++++ docs/4_Tests_Examples.md | 107 +++++++ docs/Reference/0_BindingTestFunctions.md | 143 +++++++++ docs/Reference/1_BindingAssertFunctions.md | 75 +++++ docs/Reference/2_TestFrameworkFunctions.md | 59 ++++ docs/Reference/Introduction.md | 10 + .../0_GeneralAssertions.md | 63 ++++ .../LuaUnitAssertionFunctions/1_ValueAssertions.md | 79 +++++ .../2_ScientificAssertions.md | 48 +++ .../3_StringAssertions.md | 38 +++ .../LuaUnitAssertionFunctions/4_ErrorAssertions.md | 43 +++ .../LuaUnitAssertionFunctions/5_TypeAssertions.md | 72 +++++ .../LuaUnitAssertionFunctions/6_TableAssertions.md | 19 ++ .../LuaUnitAssertionFunctions/Introduction.md | 13 + docs/SUMMARY.md | 36 +-- docs/WriteYourTests/0_ProjectTree.md | 172 ----------- docs/WriteYourTests/1_BindingConfiguration.md | 112 ------- docs/WriteYourTests/2_LUATestFiles.md | 15 - docs/WriteYourTests/Introduction.md | 10 - .../Reference/0_BindingTestFunctions.md | 100 ------- .../Reference/1_BindingAssertFunctions.md | 64 ---- .../Reference/2_TestFrameworkFunctions.md | 59 ---- docs/WriteYourTests/Reference/Introduction.md | 10 - .../0_GeneralAssertions.md | 32 -- .../LuaUnitAssertionFunctions/1_ValueAssertions.md | 58 ---- .../2_ScientificAssertions.md | 48 --- .../3_StringAssertions.md | 38 --- .../LuaUnitAssertionFunctions/4_ErrorAssertions.md | 30 -- .../LuaUnitAssertionFunctions/5_TypeAssertions.md | 38 --- .../LuaUnitAssertionFunctions/6_TableAssertions.md | 19 -- .../LuaUnitAssertionFunctions/Introduction.md | 13 - 36 files changed, 1466 insertions(+), 1226 deletions(-) delete mode 100644 docs/1_TestExample.md create mode 100644 docs/1_Write_the_tests.md delete mode 100644 docs/2_LaunchTheExample.md create mode 100644 docs/2_The_test_widget.md create mode 100644 docs/3_Launch_the_tests.md create mode 100644 docs/4_Tests_Examples.md create mode 100644 docs/Reference/0_BindingTestFunctions.md create mode 100644 docs/Reference/1_BindingAssertFunctions.md create mode 100644 docs/Reference/2_TestFrameworkFunctions.md create mode 100644 docs/Reference/Introduction.md create mode 100644 docs/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md create mode 100644 docs/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md create mode 100644 docs/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md create mode 100644 docs/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md create mode 100644 docs/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md create mode 100644 docs/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md create mode 100644 docs/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md create mode 100644 docs/Reference/LuaUnitAssertionFunctions/Introduction.md delete mode 100644 docs/WriteYourTests/0_ProjectTree.md delete mode 100644 docs/WriteYourTests/1_BindingConfiguration.md delete mode 100644 docs/WriteYourTests/2_LUATestFiles.md delete mode 100644 docs/WriteYourTests/Introduction.md delete mode 100644 docs/WriteYourTests/Reference/0_BindingTestFunctions.md delete mode 100644 docs/WriteYourTests/Reference/1_BindingAssertFunctions.md delete mode 100644 docs/WriteYourTests/Reference/2_TestFrameworkFunctions.md delete mode 100644 docs/WriteYourTests/Reference/Introduction.md delete mode 100644 docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md delete mode 100644 docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md delete mode 100644 docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md delete mode 100644 docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md delete mode 100644 docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md delete mode 100644 docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md delete mode 100644 docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md delete mode 100644 docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/Introduction.md diff --git a/docs/0_Installation.md b/docs/0_Installation.md index f02fa88..d4a388b 100644 --- a/docs/0_Installation.md +++ b/docs/0_Installation.md @@ -1,65 +1,24 @@ # Installation -## Pre-requisites +[Setup the pre-requisite](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/host-configuration/docs/1_Prerequisites.html). -[Setup the pre-requisite](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/host-configuration/docs/1_Prerequisites.html) then [install the Application Framework](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/host-configuration/docs/2_AGL_Application_Framework.html) on your host. +Use the following command line to get the `afb-test` binding and all its +dependencies. -You will also need to install lua-devel >= 5.3 to be able to build the project. - -Fedora: - -```bash -dnf install lua-devel -``` - -OpenSuse: +* Fedora (>= 27): ```bash -zypper install lua53-devel +dnf install agl-app-afb-test-devel ``` -Ubuntu (>= Xenial), Debian stable: +* OpenSuse (>= 15.0): ```bash -apt-get install liblua5.3-dev +zypper install agl-app-afb-test-devel ``` -## Grab source and build - -Download the **afb-test** binding source code using git: - -```shell -git clone --recurse-submodules https://gerrit.automotivelinux.org/gerrit/apps/app-afb-test -cd afb-test -mkdir build -cd build -cmake .. && make -``` - -## Test natively on your host - -If you want to use the **afb-test** binding natively on your host, you have to -install it. Then *pkg-config* tool can find the **afb-test.pc** and you can -use **afm-test** launcher: - -```bash -sudo make install -# Eventually set PKG_CONFIG_PATH environment variable if not installed in the -# system directory -export PKG_CONFIG_PATH=:${PKG_CONFIG_PATH} -# The same for the PATH environment variable where afm-test has been installed -export PATH=:${PATH} -``` - -Then you can test other binding using the **afm-test** launcher. Example here, -with another binding project using **app-templates** submodule or the -**cmake-apps-module** CMake module: - -> **Note** CMake module is the new way to use **app-templates** +* Ubuntu (>= Xenial), Debian stable: ```bash -cd build -cmake .. -make -afm-test package package-test +apt-get install agl-app-afb-test-dev ``` diff --git a/docs/1_TestExample.md b/docs/1_TestExample.md deleted file mode 100644 index 6522def..0000000 --- a/docs/1_TestExample.md +++ /dev/null @@ -1,108 +0,0 @@ -# Test example - -To launch the binding use the command-line provided at the end of the build, -and the afb-daemon-demo just like in the example below. This will launch the -test of an Helloworld binding example. Tests's codes are available in the LUA -files `conf.d/controller/lua.d/helloworld.lua` and -`conf.d/controller/lua.d/aftTest.lua`. - -The example will run some basics tests on API verb calls and events received. - -## helloworld.lua - -```lua - function _callback(responseJ) - _AFT.assertStrContains(responseJ.response, "Some String") - end - - function _callbackError(responseJ) - _AFT.assertStrContains(responseJ.request.info, "Ping Binder Daemon fails") - end - - function _callbackEvent(eventName, eventData) - _AFT.assertEquals(eventData, {data = { key = 'weird others data', another_key = 123.456 }}) - end - - _AFT.addEventToMonitor("hello/anEvent") - _AFT.addEventToMonitor("hello/anotherEvent", _callbackEvent) - - _AFT.testVerbStatusSuccess('testPingSuccess','hello', 'ping', {}) - _AFT.testVerbResponseEquals('testPingSuccess','hello', 'ping', {}, "Some String") - _AFT.testVerbResponseEquals('testPingSuccess','hello', 'ping', {}, "Unexpected String") - _AFT.testVerbCb('testPingSuccess','hello', 'ping', {}, _callback) - _AFT.testVerbStatusError('testPingError', 'hello', 'pingfail', {}) - _AFT.testVerbResponseEqualsError('testPingError', 'hello', 'pingfail', {}, "Ping Binder Daemon fails") - _AFT.testVerbResponseEqualsError('testPingError', 'hello', 'pingfail', {}, "Ping Binder Daemon succeed") - _AFT.testVerbCbError('testPingError', 'hello', 'pingfail', {}, _callbackError) - - _AFT.testVerbStatusSuccess('testEventAdd', 'hello', 'eventadd', {tag = 'event', name = 'anEvent'}) - _AFT.testVerbStatusSuccess('testEventSub', 'hello', 'eventsub', {tag = 'event'}) - _AFT.testVerbStatusSuccess('testEventPush', 'hello', 'eventpush', {tag = 'event', data = { key = 'some data', another_key = 123}}) - - _AFT.testVerbStatusSuccess('testEventAdd', 'hello', 'eventadd', {tag = 'evt', name = 'anotherEvent'}) - _AFT.testVerbStatusSuccess('testEventSub', 'hello', 'eventsub', {tag = 'evt'}) - _AFT.testVerbStatusSuccess('testEventPush', 'hello', 'eventpush', {tag = 'evt', data = { key = 'weird others data', another_key = 123.456}}) - - _AFT.testVerbStatusSuccess('testGenerateWarning', 'hello', 'verbose', {level = 4, message = 'My Warning message!'}) - - _AFT.testEvtGrpReceived("TestEventGroupReceived",{"hello/anEvent","hello/anotherEvent"},300000) - _AFT.testEvtGrpNotReceived("TestEventGroupNotReceived",{"hello/anEvent","hello/anotherEvent"},300000) - - _AFT.testEvtReceived("testEvent", "hello/anEvent",300000) - _AFT.testEvtReceived("testEventCb", "hello/anotherEvent",300000) - - _AFT.testCustom("mytest", function() - _AFT.assertEquals(false, false) - end) -``` - -## aftTest.lua - -```lua - -_AFT.setBeforeEach(function() print("~~~~~ Begin Test ~~~~~") end) -_AFT.setAfterEach(function() print("~~~~~ End Test ~~~~~") end) - -_AFT.setBeforeAll(function() print("~~~~~~~~~~ BEGIN ALL TESTS ~~~~~~~~~~") return 0 end) -_AFT.setAfterAll(function() print("~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~") return 0 end) - - -local corout = coroutine.create( print ) - -_AFT.describe("testAssertEquals", function() _AFT.assertEquals(false, false) end, - function() print("~~~~~ Begin Test Assert Equals ~~~~~") end, - function() print("~~~~~ End Test Assert Equals ~~~~~") end) - -_AFT.describe("testAssertNotEquals", function() _AFT.assertNotEquals(true,false) end) -_AFT.describe("testAssertItemsEquals", function() _AFT.assertItemsEquals({1,2,3},{3,1,2}) end) -_AFT.describe("testAssertAlmostEquals", function() _AFT.assertAlmostEquals(1.25 ,1.5,0.5) end) -_AFT.describe("testAssertNotAlmostEquals", function() _AFT.assertNotAlmostEquals(1.25,1.5,0.125) end) -_AFT.describe("testAssertEvalToTrue", function() _AFT.assertEvalToTrue(true) end) -_AFT.describe("testAssertEvalToFalse", function() _AFT.assertEvalToFalse(false) end) - -_AFT.describe("testAssertStrContains", function() _AFT.assertStrContains("Hello I'm a string","string") end) -_AFT.describe("testAssertStrContains", function() _AFT.assertStrContains("Hello I'm a second string","second",5) end) - -_AFT.describe("testAssertStrIContains", function() _AFT.assertStrIContains("Hello I'm another string","I'm") end) - -_AFT.describe("testAssertNotStrContains", function() _AFT.assertNotStrContains("Hello it's me again, the other string","banana") end) -_AFT.describe("testAssertNotStrContains", function() _AFT.assertNotStrContains("Hello it's me again, the other string","banana",8) end) -... -... -... -_AFT.describe("testAssertNotIsUserdata", function() _AFT.assertNotIsUserdata(2) end) - - -function _callback(responseJ) _AFT.assertStrContains(responseJ.response, "Some String") end -function _callbackError(responseJ) _AFT.assertStrContains(responseJ.request.info, "Ping Binder Daemon fails") end - -_AFT.describe("testAssertVerbStatusSuccess",function() _AFT.assertVerbStatusSuccess('hello', 'ping', {}) end) -_AFT.describe("testAssertVerbResponseEquals",function() _AFT.assertVerbResponseEquals('hello', 'ping', {},"Some String") end) -_AFT.describe("testAssertVerbCb",function() _AFT.assertVerbCb('hello', 'ping', {},_callback) end) -_AFT.describe("testAssertVerbStatusError",function() _AFT.assertVerbStatusError('hello', 'pingfail', {}) end) -_AFT.describe("testAssertVerbResponseEqualsError",function() _AFT.assertVerbResponseEqualsError('hello', 'pingfail', {},"Ping Binder Daemon fails") end) -_AFT.describe("testAssertVerbCbError",function() _AFT.assertVerbCbError('hello', 'pingfail', {},_callbackError) end) -``` - -> **NOTE**: I suggest you to take this lua file example to make your own test -> then read the following the chapter if needed to write more complicated tests. \ No newline at end of file diff --git a/docs/1_Write_the_tests.md b/docs/1_Write_the_tests.md new file mode 100644 index 0000000..aa55a3c --- /dev/null +++ b/docs/1_Write_the_tests.md @@ -0,0 +1,246 @@ +# Write the tests + +## Create the test tree + +At the root of your project, create a test dedicated directory that will hold +all your tests materials. A classic test tree looks like the following: + +```tree + test + ├── CMakeLists.txt + ├── etc + │ ├── CMakeLists.txt + │ └── aft-middlename.json + ├── fixtures + │ ├── CMakeLists.txt + │ ├── helper.sh + │ ├── data + │ └── plugin.lua + └── tests + ├── CMakeLists.txt + ├── test01.lua + ├── test02.lua + └── test03.lua + ... +``` + +Here is a description of each subdirectory purpose: + +- etc: holds the test binding configuration in a JSON file. +- fixtures: contains all external needed files to run your tests. This is mainly + used to inject data or a plugin with the mock-up APIs code in a LUA or C plugin +- tests: Contains only the tests written in LUA for your binding. + +## test/etc/: Create your configuration file + +The configuration file describes your test api and how it launches the tests. A +test binding doesn't provide verbs and this configuration will describe the api +verb(s), mocked-up apis. Here are the `key` descriptions for this file: + +### `metadata` section + +- `uid`: A simple label mainly useful at debugging purpose +- `version` (optional): the test's version +- `info` (optional): self-explanatory +- `api`: the name that your test binding will take. Used to be the test api's + name prefixed with `aft` (ie: `aft-`) +- `require`: the tested api's name. This key ensure that the tested api is + correctly launched and initialized so the test binding could test it. + +### `testVerb` section + +- `uid`: the verb name +- `info` (optional): self-explanatory +- `action`: special string indicating which function to trigger when the verb is + called. It will always be the same about the test binding. +- `args` section: + - `trace`: the name of the tested api that you are going to test. This is + needed to let the test binding tracing the tested api and retrieve through + the binder monitoring feature `calls` and `events`. + - `files`: A string or an array of strings of the LUA files with your tests. + Only provide the file name without the path. + +### `mapis`, stand for Mock-up api, section + +- `uid`: the mocked up api's name +- `info` (optional): self explanatory +- `libs`: LUA script or C plugin file name + +#### `verbs` section + +Describe the implemented mocked up API verbs. Each verb is mapped to a function +name that will be executed at the verb call. + +- `uid`: verb's name +- `info` (optional): self explanatory +- `action`: an URI string that point to a C plugin or LUA script's function that + will be executed at the verb call. The format of the action URI is: + ``://``#`` + +#### `events` section + +This section allows you to trigger a function when a described event is received +. It could be for any event which you need to apply modifications on it. You DO +NOT have to enumerate each possible events that the mocked up api would +generate, you could avoid this section if you do not want to execute a function +at events reception. + +- `uid`: event's name (`/`) +- `info` (optional): self explanatory +- `action`: an URI string that point to a C plugin or LUA script's function that + will be executed at the event reception. The format of the action URI is: + ``://``#`` + the action `lua://AFT#_evt_catcher_` is the default `afb-test` events listener. + +### Configuration examples + +Here is a simple example: + +```json +{ + "id": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", + "$schema": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", + "metadata": { + "uid": "Hello_Test", + "version": "1.0", + "api": "aft-example", + "info": "Binding made to test other bindings", + "require": [ + "hello" + ] + }, + "testVerb": { + "uid": "testing-hello", + "info": "Launch the tests against hello api", + "action": "lua://AFT#_launch_test", + "args": { + "trace": "hello", + "files": ["aftTest.lua","helloworld.lua"] + } + } +} +``` + +and another example that mock-up the `low-can` api: + +```json +{ + "id": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", + "$schema": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", + "metadata": { + "uid": "Other_Tests", + "version": "1.0", + "api": "aft-example", + "info": "Binding made to test other bindings", + "require": [ + "tested-api" + ] + }, + "testVerb": { + "uid": "Complete", + "info": "Launch all the tests", + "action": "lua://AFT#_launch_test", + "args": { + "trace": "low-can", + "files": [ "aftTest.lua", "mapis-tests.lua" ] + } + }, + "mapis": [{ + "uid": "low-can", + "info": "Faked low-can API", + "libs": "mapi_low-can.lua", + "verbs": [ + { + "uid": "subscribe", + "info": "Subscribe to CAN signals events", + "action": "lua://low-can#_subscribe" + }, + {...}, + { + "uid": "write", + "info": "Write a CAN messages to the CAN bus.", + "action": "lua://low-can#_write" + } + ], + "events": [{ + "uid": "low-can/diagnostic_messages", + "action": "lua://AFT#_evt_catcher_" + },{ + "uid": "low-can/messages_engine_speed", + "action": "lua://AFT#_evt_catcher_" + },{ + "uid": "low-can/messages_vehicle_speed", + "action": "lua://AFT#_evt_catcher_" + }] + }] +} +``` + +## The LUA test files + +The test framework uses the LUA language to describe the tests. + +You have two different things to have in mind when you write your tests using +this framework: *test* and *assertions* functions. + +- *Assertions* functions are meant to test an atomic operation result. + (ie: `1+1 = 2` is an assertion) +- *Test* functions represent a test (Unbelievable), they represent a set of one + or several *assertions* which are all needed to succeed to valid the test. + +The framework comes with several *test* and *assertion* functions to simply be +able to test verb calls and events receiving. Use the *test* ones and if you +need more use the ones that call a callback. If the test is more complex or +more comprehensive then *describe* your test function using *assert* functions. +See the example below. + +See the test framework functions [References](Reference/Introduction.md) for a +comprehensive list of *tests* and *assertions* functions available. + +### Tests example + +```lua + function _callback(responseJ) + _AFT.assertStrContains(responseJ.response, "Some String") + end + + function _callbackError(responseJ) + _AFT.assertStrContains(responseJ.request.info, "Ping Binder Daemon fails") + end + + function _callbackEvent(eventName, eventData) + _AFT.assertEquals(eventData, {data = { key = 'weird others data', another_key = 123.456 }}) + end + + _AFT.addEventToMonitor("hello/anEvent") + _AFT.addEventToMonitor("hello/anotherEvent", _callbackEvent) + + _AFT.testVerbStatusSuccess('testPingSuccess','hello', 'ping', {}) + _AFT.testVerbResponseEquals('testPingSuccess','hello', 'ping', {}, "Some String") + _AFT.testVerbResponseEquals('testPingSuccess','hello', 'ping', {}, "Unexpected String") + _AFT.testVerbCb('testPingSuccess','hello', 'ping', {}, _callback) + _AFT.testVerbStatusError('testPingError', 'hello', 'pingfail', {}) + _AFT.testVerbResponseEqualsError('testPingError', 'hello', 'pingfail', {}, "Ping Binder Daemon fails") + _AFT.testVerbResponseEqualsError('testPingError', 'hello', 'pingfail', {}, "Ping Binder Daemon succeed") + _AFT.testVerbCbError('testPingError', 'hello', 'pingfail', {}, _callbackError) + + _AFT.testVerbStatusSuccess('testEventAdd', 'hello', 'eventadd', {tag = 'event', name = 'anEvent'}) + _AFT.testVerbStatusSuccess('testEventSub', 'hello', 'eventsub', {tag = 'event'}) + _AFT.testVerbStatusSuccess('testEventPush', 'hello', 'eventpush', {tag = 'event', data = { key = 'some data', another_key = 123}}) + + _AFT.testVerbStatusSuccess('testEventAdd', 'hello', 'eventadd', {tag = 'evt', name = 'anotherEvent'}) + _AFT.testVerbStatusSuccess('testEventSub', 'hello', 'eventsub', {tag = 'evt'}) + _AFT.testVerbStatusSuccess('testEventPush', 'hello', 'eventpush', {tag = 'evt', data = { key = 'weird others data', another_key = 123.456}}) + + _AFT.testVerbStatusSuccess('testGenerateWarning', 'hello', 'verbose', {level = 4, message = 'My Warning message!'}) + + _AFT.testEvtGrpReceived("TestEventGroupReceived",{"hello/anEvent","hello/anotherEvent"},300000) + _AFT.testEvtGrpNotReceived("TestEventGroupNotReceived",{"hello/anEvent","hello/anotherEvent"},300000) + + _AFT.testEvtReceived("testEvent", "hello/anEvent",300000) + _AFT.testEvtReceived("testEventCb", "hello/anotherEvent",300000) + + _AFT.describe("myTestLabel", function() + _AFT.assertEquals(false, false) + end) +``` diff --git a/docs/2_LaunchTheExample.md b/docs/2_LaunchTheExample.md deleted file mode 100644 index 1ef3de7..0000000 --- a/docs/2_LaunchTheExample.md +++ /dev/null @@ -1,232 +0,0 @@ -# Launch The Example - -## From a terminal (on your build host) - -To launch your tests, enter this command. - -```bash -export BUILD_DIR_PATH = "$(pwd)/build" -./afb-test.sh -``` - -On afb-daemon startup you should have all the app-framework config displayed: - -```shell ----BEGIN-OF-CONFIG--- --- console: ./AFB-console.out --- rootdir: . --- roothttp: --- rootbase: /opa --- rootapi: /api --- workdir: . --- uploaddir: . --- token: 1 --- name: afbd-test --- aliases: --- dbus_clients: --- dbus_servers: --- ws_clients: --- ws_servers: --- so_bindings: --- ldpaths: /opt/AGL/lib64/afb:lib --- weak_ldpaths: --- calls: --- exec: --- httpdPort: 1234 --- cacheTimeout: 100000 --- apiTimeout: 20 --- cntxTimeout: 32000000 --- nbSessionMax: 10 --- mode: local --- tracereq: common --- traceditf: no --- tracesvc: no --- traceevt: no --- no_ldpaths: no --- noHttpd: no --- background: no --- monitoring: no --- random_token: no ----END-OF-CONFIG--- -INFO: entering foreground mode -INFO: running with pid 20430 -INFO: API monitor added -INFO: binding monitor added to set main -INFO: Scanning dir=[/opt/AGL/lib64/afb] for bindings -INFO: binding [/opt/AGL/lib64/afb/demoContext.so] is a valid AFB binding V1 -INFO: binding [/opt/AGL/lib64/afb/demoContext.so] calling registering function afbBindingV1Register -INFO: API context added -INFO: binding /opt/AGL/lib64/afb/demoContext.so loaded with API prefix context -INFO: binding [/opt/AGL/lib64/afb/helloWorld.so] looks like an AFB binding V2 -INFO: binding hello calling preinit function -NOTICE: [API hello] hello binding comes to live -INFO: API hello added -INFO: binding hello added to set main -INFO: binding [/opt/AGL/lib64/afb/tic-tac-toe.so] looks like an AFB binding V2 -INFO: API tictactoe added -INFO: binding tictactoe added to set main -INFO: binding [/opt/AGL/lib64/afb/demoPost.so] is a valid AFB binding V1 -INFO: binding [/opt/AGL/lib64/afb/demoPost.so] calling registering function afbBindingV1Register -INFO: API post added -INFO: binding /opt/AGL/lib64/afb/demoPost.so loaded with API prefix post -INFO: binding [/opt/AGL/lib64/afb/ave.so] looks like an AFB binding Vdyn -... -... -... -NOTICE: API salut started -INFO: API tictactoe starting... -NOTICE: API tictactoe started -NOTICE: Waiting port=1234 rootdir=. -NOTICE: Browser URL= http://localhost:1234 -``` - -Then in a new terminal launch the client: - -``` bash -afb-client-demo ws://localhost:1234/api?token=1 -afTest launch_all_tests -``` - -You should get something like: - -``` bash -{"response":{"info":"Launching tests"},"jtype":"afb-reply","request":{"status":"success","uuid":"3fa17ce6-0029-4ef9-8e0d-38dba2a9cf38"}} -{"event":"afTest\/results","data":{"info":"Success : 72 Failures : 6"},"jtype":"afb-event"} -``` - -Here you can see that the verb succeeded and that we have 71 Success for 5 failures. - -And on your afb-daemon terminal you have all information about your tests step-by-step (note that it depends on the level of verbosity you gave to the afb-daemon (-vvv option)). - -```shell -DEBUG: received websocket request for afTest/launch_all_tests: null -HOOK: [xreq-000001:afTest/launch_all_tests] BEGIN -HOOK: [xreq-000001:afTest/launch_all_tests] json() -> "null" -HOOK: [xreq-000002:monitor/set] BEGIN -HOOK: [xreq-000002:monitor/set] reply[denied](null, invalid token's identity) -HOOK: [xreq-000002:monitor/set] END -HOOK: [xreq-000003:monitor/trace] BEGIN -HOOK: [xreq-000003:monitor/trace] reply[denied](null, invalid token's identity) -HOOK: [xreq-000003:monitor/trace] END -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -... -... -... -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -~~~~~~~~~~ BEGIN ALL TESTS ~~~~~~~~~~ -HOOK: [xreq-000001:afTest/launch_all_tests] reply[success]({ "info": "Launching tests" }, (null)) -# XML output to var/jUnitResults.xml -# Started on Wed Jul 11 15:42:44 2018 -# Starting class: testPingSuccess -# Starting test: testPingSuccess.testFunction -~~~~~ Begin testPingSuccess ~~~~~ -HOOK: [xreq-000004:hello/ping] BEGIN -HOOK: [xreq-000004:hello/ping] json() -> null -HOOK: [xreq-000004:hello/ping] reply[success]("Some String", Ping Binder Daemon tag=pingSample count=1 query=null) -HOOK: [xreq-000004:hello/ping] END -~~~~~ End testPingSuccess ~~~~~ -# Starting class: testPingSuccessAndResponse -# Starting test: testPingSuccessAndResponse.testFunction -HOOK: [xreq-000005:hello/ping] BEGIN -HOOK: [xreq-000005:hello/ping] json() -> null -HOOK: [xreq-000005:hello/ping] reply[success]("Some String", Ping Binder Daemon tag=pingSample count=2 query=null) -HOOK: [xreq-000005:hello/ping] END -# Starting class: testPingSuccessResponseFail -# Starting test: testPingSuccessResponseFail.testFunction -HOOK: [xreq-000006:hello/ping] BEGIN -HOOK: [xreq-000006:hello/ping] json() -> null -HOOK: [xreq-000006:hello/ping] reply[success]("Some String", Ping Binder Daemon tag=pingSample count=3 query=null) -HOOK: [xreq-000006:hello/ping] END -# Failure: ./var/aft.lua:224: expected: "Unexpected String" -# actual: "Some String" -... -... -... -~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ -HOOK: [xreq-000001:afTest/launch_all_tests] END - -``` - -## On target - -If you are trying to launch your test on a target you'll have to use -a test widget which contains test files, fixture and configuration. -Then use **afm-test**: - -```bash -./afm-test -``` - -By default, the test widgets should be located in /usr/AGL/apps/testwgt. -This it will install the widget, launch the tests then display the result -on standard output. After that it will kill test app and remove it. - -You can produce a widget buy entering ```make widget``` in *build/*. - -Here is an example: - -```bash - -qemux86-64:~# afm-test /usr/AGL/apps/testwgt/aftest-test.wgt -PASS: aftest-test@5.99 started with pid=3857 -null -null -1..62 -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 1 TestListverb.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 2 TestGetVerb.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 3 Test_turning_on.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ Begin Test Assert Equals ~~~~~ -~~~~~ End Test Assert Equals ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 4 testAssertEquals.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 5 testAssertNotEquals.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 6 testAssertItemsEquals.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 7 testAssertAlmostEquals.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 8 testAssertNotAlmostEquals.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 9 testAssertEvalToTrue.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 10 testAssertEvalToFalse.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -[...] -PASS: 60 testAssertVerbStatusError.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 61 testAssertVerbResponseEqualsError.testFunction -~~~~~ Begin Test ~~~~~ -~~~~~ End Test ~~~~~ -PASS: 62 testAssertVerbCbError.testFunction -# Ran 62 tests in 0.003 seconds, 62 successes, 0 failures -~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ -PASS: aftest-test@5.99 killed and removed - -``` - -The command being : ```afm-test /usr/AGL/apps/testwgt/aftest-test.wgt``` - -You can see here that everything ran as on your pc terminal. -**Begin Test** and **End Test** are the -beforeEach and afterEach functions and -**END ALL TESTS** is the after all functions. - - **PASS :** shows the function that is or was running. \ No newline at end of file diff --git a/docs/2_The_test_widget.md b/docs/2_The_test_widget.md new file mode 100644 index 0000000..afd4c6d --- /dev/null +++ b/docs/2_The_test_widget.md @@ -0,0 +1,97 @@ +# How to build the test widget using app-templates/cmake-apps-module + +## Defining CMake targets + +Now that the test tree has been created, in each directory you have to create +a `CMakeLists.txt` file to hold the CMake's target definition. For each target +you need to specify a **LABELS** depending on the purpose of the files for each +directory. There are more explanations about using the *cmake-apps-module* (the +former *app-templates* submodule) in the [documentation website](2_4-Use-app-templates.html#using-cmake-template-macros). + +Here is a cheat sheet to map the **LABELS** target for each classic test tree +directory: + +* `etc` uses the label **TEST-CONFIG** +* `fixtures` uses the label **TEST-DATA** +* `tests` uses the label **TEST-DATA** + +i.e the `etc` folder: + +```cmake +PROJECT_TARGET_ADD(test-config) + + file(GLOB CONF_FILES "*.json") + + add_input_files("${CONF_FILES}") + + SET_TARGET_PROPERTIES(${TARGET_NAME} PROPERTIES + LABELS "TEST-CONFIG" + OUTPUT_NAME ${TARGET_NAME} + ) +``` + +i.e the `fixtures` folder: + +```cmake +PROJECT_TARGET_ADD(test-fixtures) + + # Change the glob pattern to match your file's type + file(GLOB DATA_FILES "*.sh" "*.data") + + add_input_files("${DATA_FILES}") + + SET_TARGET_PROPERTIES(${TARGET_NAME} PROPERTIES + LABELS "TEST-DATA" + OUTPUT_NAME ${TARGET_NAME} + ) +``` + +i.e the `tests` folder: + + +```cmake +PROJECT_TARGET_ADD(tests) + + file(GLOB CONF_FILES "*.lua") + + add_input_files("${CONF_FILES}") + + SET_TARGET_PROPERTIES(${TARGET_NAME} PROPERTIES + LABELS "TEST-DATA" + OUTPUT_NAME ${TARGET_NAME} + ) +``` + +> **CAUTION**: make sure that you have CMakeLists files that include your +> subdirectories target (cf: previous chapter `Write the tests`). + +## Build the test widget + +By default, the test widget is not built, you have to specify that you want to +build it or use a special target. + +### Building at the same time than classic widget + +Specify the option `BUILD_TEST_WGT=TRUE` when you configure your build. + +ie: + +```bash +cd build +cmake -DBUILD_TEST_WIDGET=TRUE .. +make +make widget +``` + +### Building separately only the test widget + +Just use the target `test_widget` after a classic build. + +ie: + +```bash +cd build +cmake .. +make +make test_widget +``` diff --git a/docs/3_Launch_the_tests.md b/docs/3_Launch_the_tests.md new file mode 100644 index 0000000..f3fe18f --- /dev/null +++ b/docs/3_Launch_the_tests.md @@ -0,0 +1,327 @@ +# How to launch the tests ? + +## Natively during the development + +It could be convenient to be able to test the software that you are currently +developing. Then you can ensure that your modifications haven't introduced +regressions, bugs, etc. This depends upon your tests of course. + +As previously explained, you need the `test widget` to be able to launch the tests and +you need also to have the `afb-test` binding installed and the `application +framework binder` to be able to execute your tests. + +Assuming you already installed those components on your development host, then +proceed as the following from the project root directory: + +```bash +cd build +cmake -DBUILD_TEST_WGT=TRUE .. +make +make widget +``` + +To prepare all files needed for the test launch then use the `afm-test` script: + +```bash +# Usage of afm-test command line utility +afm-test --help +Usage: /opt/AGL/bin/afm-test [-m|--mode ] [-t|--timeout ] [-l|--lavaoutput] +binding-wgt-rootdir: path to the test wgt file +test-wgt-rootdir: path to the test folder file +-m|--mode: SOLO (1 binder) or SERVICE (2 binders) (Default: SOLO) +-t|--timeout: timeout in second. (Default 3 seconds) +-l|--lavaoutput: Flags indicating the binding to add Lava special test markers. +Error: Test launch failed. Code: 1 +# Launching the test from your build project directory +afm-test package package-test +``` + +### Example with the afb-test selftest suite + +Prepare the launch building the `test widget`: + +```bash +$ cd build +# Cleaning the previous build files +$ rm -rf * +# Configuration step +$ cmake -DBUILD_TEST_WGT=TRUE .. +-- The C compiler identification is GNU 8.2.1 +-- The CXX compiler identification is GNU 8.2.1 +-- Check for working C compiler: /usr/lib64/ccache/cc +-- Check for working C compiler: /usr/lib64/ccache/cc -- works +-- Detecting C compiler ABI info +-- Detecting C compiler ABI info - done +-- Detecting C compile features +-- Detecting C compile features - done +-- Check for working CXX compiler: /usr/lib64/ccache/c++ +-- Check for working CXX compiler: /usr/lib64/ccache/c++ -- works +-- Detecting CXX compiler ABI info +-- Detecting CXX compiler ABI info - done +-- Detecting CXX compile features +-- Detecting CXX compile features - done +Distribution detected (separated by ';' choose one of them) fedora +Include: /home/claneys/.config/app-templates/cmake.d/00-common-var.cmake +Include: /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/conf.d/cmake/00-default-osconfig.cmake +Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/01-build_options.cmake +-- Found PkgConfig: /usr/bin/pkg-config (found version "1.4.2") +-- Checking for module 'json-c' +-- Found json-c, version 0.13.1 +-- Checking for module 'libsystemd>=222' +-- Found libsystemd, version 238 +-- Checking for module 'afb-daemon>=4.0' +-- Found afb-daemon, version 6.90.0 +-- Checking for module 'lua>=5.3' +-- Found lua, version 5.3.4 +Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/02-variables.cmake +-- Check gcc_minimal_version (found gcc version 8.2.1) (found g++ version 8.2.1) +Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/03-macros.cmake +Include: /usr/share/cmake/Modules/CMakeAfbTemplates/cmake/cmake.d/04-extra_targets.cmake +-- Overwrite the CMAKE default install prefix with /opt/AGL +-- Found CURL: /usr/lib64/libcurl.so (found version "7.59.0") +-- Qt's WebSocket AFB Client: Disabled! +-- CURL wrapping helpers: Enabled! +-- Notice: LUA Controler Support Selected +-- Notice: Using default test widget configuration's file. +-- If you want to use a customized test-config.xml template then specify TEST_WIDGET_CONFIG_TEMPLATE in your config.cmake file. +-- Configuring done +-- Generating done +-- Build files have been written to: /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build +# Build the binding +$ make +Scanning dependencies of target test-files +[ 3%] Generating test-files +[ 3%] Built target test-files +Scanning dependencies of target project_populate_test-files +[ 6%] Generating package-test/var/test-files +[ 6%] Built target project_populate_test-files +Scanning dependencies of target afb-test-config +[ 9%] Generating afb-test-config +Warning: JSON_CHECKER not found. Not verification made on files ! +[ 9%] Built target afb-test-config +Scanning dependencies of target project_populate_afb-test-config +[ 12%] Generating package-test/etc/afb-test-config +[ 12%] Built target project_populate_afb-test-config +Scanning dependencies of target aftest-config +[ 16%] Generating aftest-config +Warning: JSON_CHECKER not found. Not verification made on files ! +[ 16%] Built target aftest-config +Scanning dependencies of target project_populate_aftest-config +[ 19%] Generating package/etc/aftest-config +[ 19%] Built target project_populate_aftest-config +Scanning dependencies of target fixture-files +[ 22%] Generating fixture-files +[ 22%] Built target fixture-files +Scanning dependencies of target project_populate_fixture-files +[ 25%] Generating package-test/var/fixture-files +[ 25%] Built target project_populate_fixture-files +Scanning dependencies of target afTest-lua +[ 29%] Generating afTest-lua +[ 29%] Built target afTest-lua +Scanning dependencies of target project_populate_afTest-lua +[ 32%] Generating package/var/afTest-lua +[ 32%] Built target project_populate_afTest-lua +Scanning dependencies of target afb-helpers +[ 35%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/wrap-json.c.o +[ 38%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/filescan-utils.c.o +[ 41%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/escape.c.o +[ 45%] Building C object afb-helpers/CMakeFiles/afb-helpers.dir/curl-wrap.c.o +[ 48%] Linking C static library libafb-helpers.a +[ 48%] Built target afb-helpers +Scanning dependencies of target ctl-utilities +[ 51%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-action.c.o +[ 54%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-config.c.o +[ 58%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-onload.c.o +[ 61%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-plugin.c.o +[ 64%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-control.c.o +[ 67%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-event.c.o +[ 70%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-lua.c.o +[ 74%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-timer.c.o +[ 77%] Building C object app-controller-submodule/ctl-lib/CMakeFiles/ctl-utilities.dir/ctl-lua-utils.c.o +[ 80%] Linking C static library libctl-utilities.a +[ 80%] Built target ctl-utilities +Scanning dependencies of target aft +[ 83%] Building C object src/CMakeFiles/aft.dir/aft.c.o +[ 87%] Building C object src/CMakeFiles/aft.dir/mapis.c.o +[ 90%] Linking C shared module aft.so +[ 90%] Built target aft +Scanning dependencies of target project_populate_aft +[ 93%] Generating package/lib/aft.so +[ 93%] Built target project_populate_aft +Scanning dependencies of target populate +[ 96%] Generating package/bin, package/etc, package/lib, package/htdocs, package/var, package-test/bin, package-test/etc, package-test/lib, package-test/htdocs, package-test/var +[ 96%] Built target populate +Scanning dependencies of target afTest_build_done +++ Typical binding launch: afb-daemon --name afTest --port=1234 --workdir=package-test --ldpaths=/opt/AGL/lib64/afb:../package/lib --token= +[ 96%] Built target afTest_build_done +Scanning dependencies of target autobuild +[100%] Built target autobuild +# Build both widgets classic and test +$ make widget +[ 2%] Built target test-files +Scanning dependencies of target packaging_wgt +[ 5%] Generating package/config.xml +[ 8%] Generating package-test/config.xml, package-test/bin/launcher +[ 8%] Built target packaging_wgt +[ 11%] Generating package-test/var/test-files +[ 11%] Built target project_populate_test-files +Warning: JSON_CHECKER not found. Not verification made on files ! +[ 14%] Built target afb-test-config +[ 17%] Generating package-test/etc/afb-test-config +[ 17%] Built target project_populate_afb-test-config +Warning: JSON_CHECKER not found. Not verification made on files ! +[ 20%] Built target aftest-config +[ 23%] Generating package/etc/aftest-config +[ 23%] Built target project_populate_aftest-config +[ 26%] Built target fixture-files +[ 29%] Generating package-test/var/fixture-files +[ 29%] Built target project_populate_fixture-files +[ 32%] Built target afTest-lua +[ 35%] Generating package/var/afTest-lua +[ 35%] Built target project_populate_afTest-lua +[ 50%] Built target afb-helpers +[ 79%] Built target ctl-utilities +[ 88%] Built target aft +[ 91%] Built target project_populate_aft +[ 94%] Built target populate +Scanning dependencies of target widget +[ 97%] Generating aftest.wgt +NOTICE: -- PACKING widget aftest.wgt from directory /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build/package +[100%] Generating aftest-test.wgt +NOTICE: -- PACKING widget aftest-test.wgt from directory /home/claneys/Workspace/Sources/IOTbzh/gerrit.automotivelinux.org/apps/app-afb-test/build/package-test +++ Install widget file using in the target : afm-util install afTest.wgt +[100%] Built target widget +``` + +Launch the test using the mode SERVICE for the `afb-test` because of a recursion +problem which loads 2 times the same binding and causes conflict. So it is +needed to uses 2 binders then each ones loads its binding properly: + +```bash +$ afm-test package package-test/ -m SERVICE +1..4 +# Started on Tue Oct 30 10:32:46 2018 +# Starting class: TestListverb +ok 1 TestListverb.testFunction +# Starting class: TestGetVerb +ok 2 TestGetVerb.testFunction +# Starting class: Test_turning_on +ok 3 Test_turning_on.testFunction +# Starting class: testLockWait +ok 4 testLockWait.testFunction +# Ran 4 tests in 0.001 seconds, 4 successes, 0 failures +~~~~~~~~~~ BEGIN ALL TESTS ~~~~~~~~~~ +1..59 +# Started on Tue Oct 30 10:32:48 2018 +# Starting class: testAssertEquals +~~~~~ Begin Test ~~~~~ +~~~~~ Begin Test Assert Equals ~~~~~ +~~~~~ End Test Assert Equals ~~~~~ +~~~~~ End Test ~~~~~ +ok 1 testAssertEquals.testFunction +# Starting class: testAssertNotEquals +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +ok 2 testAssertNotEquals.testFunction +# Starting class: testAssertItemsEquals +~~~~~ Begin Test ~~~~~ +[...] +~~~~~ End Test ~~~~~ +ok 57 testAssertVerbStatusError.testFunction +# Starting class: testAssertVerbResponseEqualsError +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +ok 58 testAssertVerbResponseEqualsError.testFunction +# Starting class: testAssertVerbCbError +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +ok 59 testAssertVerbCbError.testFunction +# Ran 59 tests in 0.003 seconds, 59 successes, 0 failures +~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ +Tests correctly launched. +``` + +## Launch test on a target board + +If you are trying to launch your test on a target you'll have to use +a test widget which contains test files, fixture and configuration. +Then use **afm-test**: + +```bash +# afm-test -h +Usage: /usr/bin/afm-test [-l|--lava] [-v|--verb ] +-l|--lavaoutput: flag that enable Lava test marker to the output. (Default: disabled) +-v|--verb: select a specific verb to launch from the test API. (Default: all) +path: path to the test wgt file +``` + +By default, the test widgets should be located in /usr/AGL/apps/testwgt. This +will install the widget, launch the tests then display the result on standard +output. After that it will kill test binding and remove it. + +### Example on a target + +Here is an example: + +```bash +qemux86-64:~# afm-test /usr/AGL/apps/testwgt/aftest-test.wgt +afm-test /tmp/aftest-test.wgt +PASS: aftest-test@6.90 started with pid=1649 +1..59 +~~~~~ Begin Test ~~~~~ +~~~~~ Begin Test Assert Equals ~~~~~ +~~~~~ End Test Assert Equals ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 1 testAssertEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 2 testAssertNotEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 3 testAssertItemsEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +[...] +PASS: 58 testAssertVerbResponseEqualsError.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 59 testAssertVerbCbError.testFunction +# Ran 59 tests in 0.003 seconds, 59 successes, 0 failures +~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ +1..63 +~~~~~ Begin Test ~~~~~ +~~~~~ Begin Test Assert Equals ~~~~~ +~~~~~ End Test Assert Equals ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 1 testAssertEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 2 testAssertNotEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 3 testAssertItemsEquals.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +[...] +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 61 TestGetVerb.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 62 Test_turning_on.testFunction +~~~~~ Begin Test ~~~~~ +~~~~~ End Test ~~~~~ +PASS: 63 testLockWait.testFunction +# Ran 63 tests in 0.003 seconds, 63 successes, 0 failures +~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~ +PASS: aftest-test@6.90 killed and removed +``` + +The command being : ```afm-test /usr/AGL/apps/testwgt/aftest-test.wgt``` + +You can see here that everything ran as on your pc terminal. +**Begin Test** and **End Test** are the +beforeEach and afterEach functions and +**END ALL TESTS** is the after all functions. + + **PASS :** shows the function that is or was running. diff --git a/docs/4_Tests_Examples.md b/docs/4_Tests_Examples.md new file mode 100644 index 0000000..caccea4 --- /dev/null +++ b/docs/4_Tests_Examples.md @@ -0,0 +1,107 @@ +# Test example + +To launch the binding use the command-line provided at the end of the build, +and the afb-daemon-demo just like in the example below. This will launch the +test of an Helloworld binding example. Tests's codes are available in the LUA +files `conf.d/controller/lua.d/helloworld.lua` and +`conf.d/controller/lua.d/aftTest.lua`. + +The example will run some basics tests on API verb calls and events received. + +## helloworld.lua + +```lua + function _callback(responseJ) + _AFT.assertStrContains(responseJ.response, "Some String") + end + + function _callbackError(responseJ) + _AFT.assertStrContains(responseJ.request.info, "Ping Binder Daemon fails") + end + + function _callbackEvent(eventName, eventData) + _AFT.assertEquals(eventData, {data = { key = 'weird others data', another_key = 123.456 }}) + end + + _AFT.addEventToMonitor("hello/anEvent") + _AFT.addEventToMonitor("hello/anotherEvent", _callbackEvent) + + _AFT.testVerbStatusSuccess('testPingSuccess','hello', 'ping', {}) + _AFT.testVerbResponseEquals('testPingSuccess','hello', 'ping', {}, "Some String") + _AFT.testVerbResponseEquals('testPingSuccess','hello', 'ping', {}, "Unexpected String") + _AFT.testVerbCb('testPingSuccess','hello', 'ping', {}, _callback) + _AFT.testVerbStatusError('testPingError', 'hello', 'pingfail', {}) + _AFT.testVerbResponseEqualsError('testPingError', 'hello', 'pingfail', {}, "Ping Binder Daemon fails") + _AFT.testVerbResponseEqualsError('testPingError', 'hello', 'pingfail', {}, "Ping Binder Daemon succeed") + _AFT.testVerbCbError('testPingError', 'hello', 'pingfail', {}, _callbackError) + + _AFT.testVerbStatusSuccess('testEventAdd', 'hello', 'eventadd', {tag = 'event', name = 'anEvent'}) + _AFT.testVerbStatusSuccess('testEventSub', 'hello', 'eventsub', {tag = 'event'}) + _AFT.testVerbStatusSuccess('testEventPush', 'hello', 'eventpush', {tag = 'event', data = { key = 'some data', another_key = 123}}) + + _AFT.testVerbStatusSuccess('testEventAdd', 'hello', 'eventadd', {tag = 'evt', name = 'anotherEvent'}) + _AFT.testVerbStatusSuccess('testEventSub', 'hello', 'eventsub', {tag = 'evt'}) + _AFT.testVerbStatusSuccess('testEventPush', 'hello', 'eventpush', {tag = 'evt', data = { key = 'weird others data', another_key = 123.456}}) + + _AFT.testVerbStatusSuccess('testGenerateWarning', 'hello', 'verbose', {level = 4, message = 'My Warning message!'}) + + _AFT.testEvtGrpReceived("TestEventGroupReceived",{"hello/anEvent","hello/anotherEvent"},300000) + _AFT.testEvtGrpNotReceived("TestEventGroupNotReceived",{"hello/anEvent","hello/anotherEvent"},300000) + + _AFT.testEvtReceived("testEvent", "hello/anEvent",300000) + _AFT.testEvtReceived("testEventCb", "hello/anotherEvent",300000) + + _AFT.describe("myTestLabel", function() + _AFT.assertEquals(false, false) + end) +``` + +## aftTest.lua + +```lua +_AFT.setBeforeEach(function() print("~~~~~ Begin Test ~~~~~") end) +_AFT.setAfterEach(function() print("~~~~~ End Test ~~~~~") end) + +_AFT.setBeforeAll(function() print("~~~~~~~~~~ BEGIN ALL TESTS ~~~~~~~~~~") return 0 end) +_AFT.setAfterAll(function() print("~~~~~~~~~~ END ALL TESTS ~~~~~~~~~~") return 0 end) + + +local corout = coroutine.create( print ) + +_AFT.describe("testAssertEquals", function() _AFT.assertEquals(false, false) end, + function() print("~~~~~ Begin Test Assert Equals ~~~~~") end, + function() print("~~~~~ End Test Assert Equals ~~~~~") end) + +_AFT.describe("testAssertNotEquals", function() _AFT.assertNotEquals(true,false) end) +_AFT.describe("testAssertItemsEquals", function() _AFT.assertItemsEquals({1,2,3},{3,1,2}) end) +_AFT.describe("testAssertAlmostEquals", function() _AFT.assertAlmostEquals(1.25 ,1.5,0.5) end) +_AFT.describe("testAssertNotAlmostEquals", function() _AFT.assertNotAlmostEquals(1.25,1.5,0.125) end) +_AFT.describe("testAssertEvalToTrue", function() _AFT.assertEvalToTrue(true) end) +_AFT.describe("testAssertEvalToFalse", function() _AFT.assertEvalToFalse(false) end) + +_AFT.describe("testAssertStrContains", function() _AFT.assertStrContains("Hello I'm a string","string") end) +_AFT.describe("testAssertStrContains", function() _AFT.assertStrContains("Hello I'm a second string","second",5) end) + +_AFT.describe("testAssertStrIContains", function() _AFT.assertStrIContains("Hello I'm another string","I'm") end) + +_AFT.describe("testAssertNotStrContains", function() _AFT.assertNotStrContains("Hello it's me again, the other string","banana") end) +_AFT.describe("testAssertNotStrContains", function() _AFT.assertNotStrContains("Hello it's me again, the other string","banana",8) end) +... +... +... +_AFT.describe("testAssertNotIsUserdata", function() _AFT.assertNotIsUserdata(2) end) + + +function _callback(responseJ) _AFT.assertStrContains(responseJ.response, "Some String") end +function _callbackError(responseJ) _AFT.assertStrContains(responseJ.request.info, "Ping Binder Daemon fails") end + +_AFT.describe("testAssertVerbStatusSuccess",function() _AFT.assertVerbStatusSuccess('hello', 'ping', {}) end) +_AFT.describe("testAssertVerbResponseEquals",function() _AFT.assertVerbResponseEquals('hello', 'ping', {},"Some String") end) +_AFT.describe("testAssertVerbCb",function() _AFT.assertVerbCb('hello', 'ping', {},_callback) end) +_AFT.describe("testAssertVerbStatusError",function() _AFT.assertVerbStatusError('hello', 'pingfail', {}) end) +_AFT.describe("testAssertVerbResponseEqualsError",function() _AFT.assertVerbResponseEqualsError('hello', 'pingfail', {},"Ping Binder Daemon fails") end) +_AFT.describe("testAssertVerbCbError",function() _AFT.assertVerbCbError('hello', 'pingfail', {},_callbackError) end) +``` + +> **NOTE**: I suggest you to take this lua file example to make your own test +> then read the following chapter if needed to write more complex tests. diff --git a/docs/Reference/0_BindingTestFunctions.md b/docs/Reference/0_BindingTestFunctions.md new file mode 100644 index 0000000..4ee95e0 --- /dev/null +++ b/docs/Reference/0_BindingTestFunctions.md @@ -0,0 +1,143 @@ +# Binding Test functions + +* **_AFT.testVerbStatusSuccess(testName, api, verb, args, setUp, tearDown)** + + Simply test that the call of a verb successfully returns. + + *setUp* and *tearDown* are functions that can be added to your test context, + it works just like **_AFT.beforeEach()** and **_AFT.afterEach()**, + *setUp* will be ran before your *testFunction* and **_AFT.beforeEach()** + (if set) functions, *tearDown* will be ran after your *testFunction* and + **_AFT.afterEach()** (if set) functions. + + ```lua + _AFT.testVerbStatusSuccess('testPingSuccess','hello', 'ping', {}) + ``` + +* **_AFT.testVerbStatusError(testName, api, verb, args, setUp, tearDown)** + + The inverse of above, test that a verb return does not succeed. + + *setUp* and *tearDown* are functions that can be added to your test context, + it works just like **_AFT.beforeEach()** and **_AFT.afterEach()**, + *setUp* will be ran before your *testFunction* and **_AFT.beforeEach()** + (if set) functions, *tearDown* will be ran after your *testFunction* and + **_AFT.afterEach()** (if set) functions. + + ```lua + _AFT.testVerbStatusSuccess('testPingError', 'hello', 'pingfail', {}) + ``` + +* **_AFT.testVerbResponseEquals(testName, api, verb, args, expectedResponse, setUp, tearDown)** + + Test that the call of a verb successfully returns and that verb's response + is equals to the *expectedResponse*. + + *setUp* and *tearDown* are functions that can be added to your test context, + it works just like **_AFT.beforeEach()** and **_AFT.afterEach()**, *setUp* + will be ran before your *testFunction* and **_AFT.beforeEach()** (if set) + functions, *tearDown* will be ran after your *testFunction* and + **_AFT.afterEach()** (if set) functions. + + ```lua + _AFT.testVerbResponseEquals('testPingSuccess','hello', 'ping', {}, "Some String") + ``` + +* **_AFT.testVerbResponseEqualsError(testName, api, verb, args, expectedResponse, setUp, tearDown)** + + The inverse of above. + + *setUp* and *tearDown* are functions that can be added to your test context, it works + just like **_AFT.beforeEach()** and **_AFT.afterEach()**, *setUp* will be ran + before your *testFunction* and **_AFT.beforeEach()** (if set) functions, + *tearDown* will be ran after your *testFunction* and **_AFT.afterEach()** (if + set) functions. + + ```lua + _AFT.testVerbResponseEqualsError('testPingError', 'hello', 'pingfail', {}, "Ping Binder Daemon fails") + ``` + +* **_AFT.testVerbCb(testName, api, verb, args, expectedResponse, callback, setUp, tearDown)** + + Test the call of a verb with a custom callback. From this callback you + will need to make some assertions on what you need (verb JSON return object + content mainly). + + If you don't need to test the response simply specify an empty LUA table. + + *setUp* and *tearDown* are functions that can be added to your test context, it works + just like **_AFT.beforeEach()** and **_AFT.afterEach()**, *setUp* will be ran + before your *testFunction* and **_AFT.beforeEach()** (if set) functions, + *tearDown* will be ran after your *testFunction* and **_AFT.afterEach()** (if + set) functions. + + ```lua + function _callback(responseJ) + _AFT.assertStrContains(responseJ.response, "Some String") + end + _AFT.testVerbCb('testPingSuccess','hello', 'ping', {}, _callback) + ``` + +* **_AFT.testVerbCbError(testName, api, verb, args, expectedResponse, callback, setUp, tearDown)** + + Should return success on failure. + + *setUp* and *tearDown* are functions that can be added to your test context, it works + just like **_AFT.beforeEach()** and **_AFT.afterEach()**, *setUp* will be ran + before your *testFunction* and **_AFT.beforeEach()** (if set) functions, + *tearDown* will be ran after your *testFunction* and **_AFT.afterEach()** (if + set) functions. + + ```lua + function _callbackError(responseJ) + _AFT.assertStrContains(responseJ.request.info, "Ping Binder Daemon fails") + end + + _AFT.testVerbCbError('testPingError', 'hello', 'pingfail', {}, _callbackError) + ``` + +* **_AFT.testEvtReceived(testName, eventName, timeout, setUp, tearDown)** + + Prior to be able to check that an event has been received, you have to + register the event with the test framework using **_AFT.addEventToMonitor** + function. + + Check if an event has been correctly received in time (timeout in µs). An event + name use the application framework naming scheme: **api/event_name**. + + ```lua + _AFT.testEvtReceived("testEvent", "hello/anEvent",300000) + ``` + +* **_AFT.testEvtNotReceived(testName, eventName, timeout, setUp, tearDown)** + + Prior to be able to check that an event has not been received, you have to + register the event with the test framework using **_AFT.addEventToMonitor** + function. + + Check if an event has not been correctly received in time (timeout in µs). An + event name use the application framework naming scheme: **api/event_name**. + + ```lua + _AFT.testEvtNotReceived("testEventCb", "hello/anotherEvent",300000) + ``` + +* **_AFT.testGrpEvtReceived(testName, eventGrp, timeout, setUp, tearDown)** + + Prior to be able to check that a group of events (a table of events) has been + received, you have to register the events with the test framework using + **_AFT.addEventToMonitor** function. + + The table has to have this format: + ```lua + eventGrp = {["api/event_name_1"]=1, ["api/event_name_2"]=2, ["api/event_name_3"]=5} + ``` + As you can see in the table, event names are table keys and the value stored are + the number of time that the events have to be received. + + Check if events has been correctly received in time (timeout in µs). An + event name use the application framework naming scheme: **api/event_name**. + + ```lua + _AFT.testEvtGrpReceived("TestEventGroupReceived",{["hello/anEvent"]=3,["hello/anotherEvent"]=1}, 300000) + ``` \ No newline at end of file diff --git a/docs/Reference/1_BindingAssertFunctions.md b/docs/Reference/1_BindingAssertFunctions.md new file mode 100644 index 0000000..b05f73b --- /dev/null +++ b/docs/Reference/1_BindingAssertFunctions.md @@ -0,0 +1,75 @@ +# Binding Assert functions + +* **_AFT.assertVerbStatusSuccess(api, verb, args)** + + Simply test that the call of a verb successfully returns. + +* **_AFT.assertVerbStatusError(api, verb, args)** + + The inverse of above. Expects that the verbs fails. + +* **_AFT.assertVerbResponseEquals(api, verb, args, expectedResponse)** + + Test that the call of a verb successfully returns and that verb's response + is equal to the *expectedResponse*. + +* **_AFT.assertVerbResponseEqualsError(api, verb, args, expectedResponse)** + + The inverse of above. Expect an error on the verb call either on the verb or on the expected + response. + +* **_AFT.assertVerbCb(api, verb, args, expectedResponse, callback)** + + Test the call of a verb with a custom callback. From this callback you + will need to make some assertions on what you need (verb JSON return object + content mainly). + + If you don't need to test the response simply specify an empty LUA table. + + ```lua + function _callback(responseJ) + _AFT.assertStrContains(responseJ.response, "Some String") + end + + _AFT.describe("myTestLabel", function() + _AFT.testVerbCb('testPingSuccess','hello', 'ping', {}, _callback) + end) + ``` + +* **_AFT.assertVerbCbError(api, verb, args, expectedResponse, callback)** + + Should return success on failure. + +* **_AFT.assertEvtReceived(eventName, timeout)** + + Prior to be able to check that an event has been received, you have to + register the event with the test framework using **_AFT.addEventToMonitor** + function. + + Check if an event has been correctly received in timeout specified (timeout in µs). + An event name use the application framework naming scheme: **api/event_name**. + +* **_AFT.assertEvtNotReceived(eventName, timeout)** + + Prior to be able to check that an event has been received, you have to + register the event with the test framework using **_AFT.addEventToMonitor** + function. + + Check if an event has been correctly received in time (timeout in µs). + An event name use the application framework naming scheme: **api/event_name**. + +* **_AFT.assertGrpEvtReceived(eventGrp, timeout)** + + Prior to be able to check that a group of events (a table of event) has been + received, you have to register the events with the test framework using + **_AFT.addEventToMonitor** function. + + The table has to have this format: + ```lua + eventGrp = {["api/event_name_1"]=1,["api/event_name_2"]=2,["api/event_name_3"]=5} + ``` + As you can see, in the table, event names are table keys and the value stored are + the number of time that the events have to be received. + + Check if events has been correctly received in timeout specified (timeout in µs). + An event name use the application framework naming scheme: **api/event_name**. \ No newline at end of file diff --git a/docs/Reference/2_TestFrameworkFunctions.md b/docs/Reference/2_TestFrameworkFunctions.md new file mode 100644 index 0000000..c952e3f --- /dev/null +++ b/docs/Reference/2_TestFrameworkFunctions.md @@ -0,0 +1,59 @@ +# Test Framework functions + +* **_AFT.addEventToMonitor(eventName, callback)** + + Add a binding event in the test framework to be able to assert its reception + . You'll need to add as many events as you expect to receive. You could also + specify a callback to test in more details the event received, like the attached + data. The callback will happens after the assertion that it has been received so + you can work on data that the event eventually carry. + +* **_AFT.setJunitFile(filePath)** + + Set the *JUnit* file path. When *JUnit* is set as the output type for the + test framework. + +* **_AFT.setBeforeEach(function)** + + Set the **_AFT.beforeEach()** function which is used to run the specified + *function* before each tests. + +* **_AFT.setAfterEach(function)** + + Set the **_AFT.afterEach()** function which is used to run the specified + *function* after each tests. + +* **_AFT.setBeforeAll(function)** + + Set the **_AFT.beforeAll()** function which is used to run the specified + *function* before all tests. If the given function is successful it has + to return 0 else it will return an error. + +* **_AFT.setAfterAll(function)** + + Set the **_AFT.afterAll()** function which is used to run the specified + *function* after all tests. If the given function is successful it has + to return 0 else it will return an error. + +* **_AFT.describe(testName, testFunction, setUp, tearDown)** + + Describe a manually written test. *testFunction* will be given the name + provided by *testName* and will be tested. + + *setUp* and *tearDown* are functions that can be added to your context, + it works just like **_AFT.beforeEach()** and **_AFT.afterEach()**, + *setUp* will be ran before your *testFunction* and **_AFT.beforeEach()** + (if set) functions, *tearDown* will be ran after your *testFunction* and + **_AFT.afterEach()** (if set) functions. + +* **_AFT.setBefore(testName, beforeTestFunction)** + Set a function to be ran at the beginning of the given *testName* function. + + ```lua + _AFT.testVerbStatusSuccess('testPingSuccess','hello', 'ping', {}) + _AFT.setBefore("testPingSuccess",function() print("~~~~~ Begin testPingSuccess ~~~~~") end) + _AFT.setAfter("testPingSuccess",function() print("~~~~~ End testPingSuccess ~~~~~") end) + ``` + +* **_AFT.setBefore(testName, beforeTestFunction)** + Set a function to be ran at the end of the given *testName* function. \ No newline at end of file diff --git a/docs/Reference/Introduction.md b/docs/Reference/Introduction.md new file mode 100644 index 0000000..8de5a15 --- /dev/null +++ b/docs/Reference/Introduction.md @@ -0,0 +1,10 @@ +# References + +Follow this section to know which functions are available !! + +Section's Table of content : + +* [Binding test functions](0_BindingTestFunctions.html) +* [Binding assert functions](1_BindingAssertFunctions.html) +* [Test framework functions](2_TestFrameworkFunctions.html) +* [LUAUnit assertion functions](LuaUnitAssertionFunctions/Introduction.html) \ No newline at end of file diff --git a/docs/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md new file mode 100644 index 0000000..6282755 --- /dev/null +++ b/docs/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md @@ -0,0 +1,63 @@ +# General Assertions + +* **_AFT.assertEquals(actual, expected)** + + Assert that two values are equal. + + For tables, the comparison is a deep comparison : + + * number of elements must be the same + * tables must contain the same keys + * each key must contain the same values. The values are also compared recursively with deep comparison. + + LuaUnit provides other table-related assertions, see [Table assertions](http://luaunit.readthedocs.io/en/luaunit_v3_2_1/#assert-table) + + ```lua + _AFT.assertEquals("string", "string") + ``` + +* **_AFT.assertNotEquals(actual, expected)** + + Assert that two values are different. The assertion fails if the two values are identical. + + It also uses table deep comparison. + + ```lua + _AFT.assertNotEquals("string", "Another string") + ``` + +* **_AFT.assertAlmostEquals(actual, expected, margin)** + + Assert that two floating point numbers are almost equal. + + When comparing floating point numbers, strict equality does not work. + Computer arithmetic is so that an operation that mathematically yields + 1.00000000 might yield 0.999999999999 in lua . That’s why you need an + almost equals comparison, where you specify the error margin. + + ```lua + -- convert pi/6 radian to 30 degree + pi_div_6_deg_calculated = math.deg(math.pi/6) + pi_div_6_deg_expected = 30 + + -- convert pi/3 radian to 60 degree + pi_div_3_deg_calculated = math.deg(math.pi/3) + pi_div_3_deg_expected = 60 + + -- check absolute error: it is not constant + print( (pi_div_6_deg_expected - pi_div_6_deg_calculated) / lu.EPS ) -- prints: 16 + print( (pi_div_3_deg_expected - pi_div_3_deg_calculated) / lu.EPS ) -- prints: 32 + + -- Better use relative error: + print( ( (pi_div_6_deg_expected - pi_div_6_deg_calculated) / pi_div_6_deg_expected) / lu.EPS ) -- prints: 0.53333 + print( ( (pi_div_3_deg_expected - pi_div_3_deg_calculated) / pi_div_3_deg_expected) / lu.EPS ) -- prints: 0.53333 + + -- relative error is constant. Assertion can take the form of: + assertAlmostEquals( (pi_div_6_deg_expected - pi_div_6_deg_calculated) / pi_div_6_deg_expected, lu.EPS ) + assertAlmostEquals( (pi_div_3_deg_expected - pi_div_3_deg_calculated) / pi_div_3_deg_expected, lu.EPS ) + ``` + +* **_AFT.assertNotAlmostEquals(actual, expected, margin)** + + Assert that two floating point numbers are not almost equal. + \ No newline at end of file diff --git a/docs/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md new file mode 100644 index 0000000..3769b62 --- /dev/null +++ b/docs/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md @@ -0,0 +1,79 @@ +# Value assertions + +* **_AFT.assertEvalToTrue(value)** + + Assert that a given value evals to true. Lua coercion rules are applied so + that values like 0,"",1.17 succeed in this assertion. If provided, extra_msg + is a string which will be printed along with the failure message. + + ```lua + _AFT.assertEvalToTrue(3 == 1+2) + ``` + +* **_AFT.assertEvalToFalse(Value)** + + Assert that a given value evals to *false*. Lua coercion rules are applied so + that *nil* and *false* succeed in this assertion. If provided, extra_msg is a + string which will be printed along with the failure message. + + ```lua + _AFT.assertEvalToFalse(3 == 2) + ``` + +* **_AFT.assertIsTrue(value)** + + Assert that a given value evals to true. Lua coercion rules are applied so + that values like 0, "", 1.17 all compare to true. + + ```lua + _AFT.assertIsTrue(3 == 1+2) + ``` + +* **_AFT.assertIsFalse(value)** + + Assert that a given value evals to false. Lua coercion rules are applied so + that only nil and false all compare to false. + + ```lua + _AFT.assertIsFalse(3 == 2) + ``` + +* **_AFT.assertIsNil(value)** + + Assert that a given value is nil. + + ```lua + var = nil + _AFT.assertIsNil(var) + ``` + +* **_AFT.assertNotIsNil(value)** + + Assert that a given value is not *nil* . Lua coercion rules are applied + so that values like ``0``, ``""``, ``false`` all validate the assertion. + If provided, *extra_msg* is a string which will be printed along with the + failure message. + +* **_AFT.assertIs(actual, expected)** + + Assert that two variables are identical. For string, numbers, boolean and + for nil, this gives the same result as assertEquals() . For the other types, + identity means that the two variables refer to the same object. + + Example : + + ```lua + `s1='toto' + s2='to'..'to' + t1={1,2} + t2={1,2} + _AFT.assertIs(s1,s1) -- ok + _AFT.assertIs(s1,s2) -- ok + _AFT.assertIs(t1,t1) -- ok + _AFT.assertIs(t1,t2) -- fail` + ``` + +* **_AFT.assertNotIs(actual, expected)** + + Assert that two variables are not identical, in the sense that they do not + refer to the same value. See assertIs() for more details. \ No newline at end of file diff --git a/docs/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md new file mode 100644 index 0000000..a9ce270 --- /dev/null +++ b/docs/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md @@ -0,0 +1,48 @@ +# Scientific assertions + +>**Note** +>If you need to deal with value *minus zero* (-0), be very careful because Lua versions +are inconsistent on how they treat the >syntax -0 : it creates either a plus +zero or a minus zero. Multiplying or dividing 0 by -1 also yields inconsistent > +results. The reliable way to create the -0 value is : minusZero = -1 / (1/0). + +* **_AFT.assertIsNaN(value)** + Assert that a given number is a *NaN* (Not a Number), according to the + definition of IEEE-754_ . If provided, *extra_msg* is a string which will + be printed along with the failure message. + +* **_AFT.assertIsPlusInf(value)** + + Assert that a given number is *plus infinity*, according to the definition of + IEEE-754_. If provided, *extra_msg* is a string which will be printed along + with the failure message. + +* **_AFT.assertIsMinusInf(value)** + + Assert that a given number is *minus infinity*, according to the definition of + IEEE-754_. If provided, *extra_msg* is a string which will be printed along + with the failure message. + +* **_AFT.assertIsInf(value)** + + Assert that a given number is *infinity* (either positive or negative), + according to the definition of IEEE-754_. If provided, *extra_msg* is a string + which will be printed along with the failure message. + +* **_AFT.assertIsPlusZero(value)** + + Assert that a given number is *+0*, according to the definition of IEEE-754_. + The verification is done by dividing by the provided number and verifying + that it yields *infinity* . If provided, *extra_msg* is a string which will + be printed along with the failure message. + + Be careful when dealing with *+0* and *-0*, see note above + +* **_AFT.assertIsMinusZero(value)** + + Assert that a given number is *-0*, according to the definition of IEEE-754_. + The verification is done by dividing by the provided number and verifying that + it yields *minus infinity* . If provided, *extra_msg* is a string which will + be printed along with the failure message. + + Be careful when dealing with *+0* and *-0* \ No newline at end of file diff --git a/docs/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md new file mode 100644 index 0000000..929ec14 --- /dev/null +++ b/docs/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md @@ -0,0 +1,38 @@ +# String assertions + +Assertions related to string and patterns. + +* **_AFT.assertStrContains(str, sub[, useRe])** + + Assert that a string contains the given substring or pattern. + + By default, substring is searched in the string. If useRe is provided and is + true, sub is treated as a pattern which is searched inside the string str. + +* **_AFT.assertStrIContains(str, sub)** + + Assert that a string contains the given substring, irrespective of the case. + + Not that unlike assertStrcontains(), you can not search for a pattern. + +* **_AFT.assertNotStrContains(str, sub, useRe)** + + Assert that a string does not contain a given substring or pattern. + + By default, substring is searched in the string. If useRe is provided and is + true, sub is treated as a pattern which is searched inside the string str. + +* **_AFT.assertNotStrIContains(str, sub)** + + Assert that a string does not contain the given substring, irrespective of + the case. + + Note that unlike assertNotStrcontains(), you can not search for a pattern. + +* **_AFT.assertStrMatches(str, pattern[, start[, final]])** + + Assert that a string matches the full pattern pattern. + + If start and final are not provided or are nil, the pattern must match the + full string, from start to end. The functions allows to specify the expected + start and end position of the pattern in the string. \ No newline at end of file diff --git a/docs/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md new file mode 100644 index 0000000..13d381f --- /dev/null +++ b/docs/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md @@ -0,0 +1,43 @@ +# Error assertions + +Error related assertions, to verify error generation and error messages. + +* **_AFT.assertError(func, ...)** + + Assert that calls to the function **func** with the arguments yields an error. If the function does + not yield an error, the assertion fails. + + Note that the error message itself is not checked, which means that this function does not + distinguish between the legitimate error that you expect and another error that might be + triggered by mistake. + + The next functions provide a better approach to error testing, by checking explicitly the error + message content. + +>**Note** +>When testing LuaUnit, switching from assertError() to assertErrorMsgEquals() revealed quite a few bugs! + +* **_AFT.assertErrorMsgEquals(expectedMsg, func, ...)** + + Assert that calls to the function **func** will generate exactly the given error message. If the + function does not yield an error, or if the error message is not identical, the assertion fails. + + Be careful when using this function that error messages usually contain the file name and line + number information of where the error was generated. This is usually inconvenient. To ignore the + filename and line number information, you can either use a pattern with assertErrorMsgMatches() + or simply check if the message contains a string with assertErrorMsgContains() . + +* **_AFT.assertErrorMsgContains(partialMsg, func, ...)** + + Assert that calls to the function **func** will generate an error message containing partialMsg. + If the function does not yield an error, or if the expected message is not contained in the + error message, the assertion fails. + +* **_AFT.assertErrorMsgMatches(expectedPattern, func, ...)** + + Assert that calls to the function **func** will generate an error message matching expectedPattern. + If the function does not yield an error, or if the error message does not match the provided + pattern the assertion fails. + + Note that matching is done from the start to the end of the error message. Be sure to escape + magic all magic characters with % (like -+.?\*) . \ No newline at end of file diff --git a/docs/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md new file mode 100644 index 0000000..d96ef6b --- /dev/null +++ b/docs/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md @@ -0,0 +1,72 @@ +# Type assertions + +The following functions all perform type checking on their argument. If the +received value is not of the right type, the failure message will contain the +expected type, the received type and the received value to help you identify +better the problem. + +* **_AFT.assertIsNumber(value)** + + Assert that the argument is a number (integer or float) + + ```lua + _AFT.assertIsNumber(1) + ``` + +* **_AFT.assertIsString(value)** + + Assert that the argument is a string. + + ```lua + _AFT.assertIsString("string") + ``` + +* **_AFT.assertIsTable(value)** + + Assert that the argument is a table. + + ```lua + _AFT.assertIsTable({key = "value"}) + ``` + +* **_AFT.assertIsBoolean(value)** + + Assert that the argument is a boolean. + + ```lua + _AFT.assertIsBoolean(false) + ``` + +* **_AFT.assertIsFunction(value)** + + Assert that the argument is a function. + + ```lua + _AFT.assertIsFunction(function() print("HELLO WORLD!") end) + ``` + +* **_AFT.assertIsUserdata(value)** + + Assert that the argument is a userdata. + + ```lua + _AFT.assertIsUserdata(someCvariableImportedAsUserDataInLua) + ``` + +* **_AFT.assertIsThread(value)** + + Assert that the argument is a coroutine (an object with type thread ). + + ```lua + local corout = coroutine.create( print ) + + _AFT.assertIsUserdata(corout) + ``` + +* **_AFT.assertNotIsThread(value)** + + Assert that the argument is a not coroutine (an object with type thread ). + + ```lua + _AFT.assertNotIsThread(2) + ``` \ No newline at end of file diff --git a/docs/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md b/docs/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md new file mode 100644 index 0000000..8d29988 --- /dev/null +++ b/docs/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md @@ -0,0 +1,19 @@ +# Table assertions + +* **_AFT.assertItemsEquals(actual, expected)** + + Assert that two tables contain the same items, irrespective of their keys. + + This function is practical for example if you want to compare two lists but + where items are not in the same order: + + ```lua + luaunit.assertItemsEquals( {1,2,3}, {3,2,1} ) -- assertion succeeds + ``` + The comparison is not recursive on the items: if any of the items are tables, + they are compared using table equality (like as in assertEquals() ), where the + key matters. + + ```lua + luaunit.assertItemsEquals( {1,{2,3},4}, {4,{3,2,},1} ) -- assertion fails because {2,3} ~= {3,2} + ``` \ No newline at end of file diff --git a/docs/Reference/LuaUnitAssertionFunctions/Introduction.md b/docs/Reference/LuaUnitAssertionFunctions/Introduction.md new file mode 100644 index 0000000..2bdf05c --- /dev/null +++ b/docs/Reference/LuaUnitAssertionFunctions/Introduction.md @@ -0,0 +1,13 @@ +# LuaUnit Assertion Functions + +Follow this section to know which luaUnit assertion functions are available !! + +Section's Table of content : + +* [General Assertions](0_GeneralAssertions.html) +* [Value Assertions](1_ValueAssertions.html) +* [Scientific Assertions](2_ScientificAssertions.html) +* [String Assertions](3_StringAssertions.html) +* [Error Assertions](4_ErrorAssertions.html) +* [Type Assertions](5_TypeAssertions.html) +* [Table Assertions](6_TableAssertions.html) \ No newline at end of file diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index cc5e57a..e4a45d5 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -2,23 +2,23 @@ * [Installation](0_Installation.md) -* [Test example](1_TestExample.md) +* [Write the tests](1_Write_the_tests.md) -* [Launch the example](2_LaunchTheExample.md) +* [Launch the example](2_The_test_widget.md) -* [Write your tests](WriteYourTests/Introduction.md) - * [Project Tree](WriteYourTests/0_ProjectTree.md) - * [Binding configuration](WriteYourTests/1_BindingConfiguration.md) - * [LUA test files](WriteYourTests/2_LUATestFiles.md) - * [References](WriteYourTests/Reference/Introduction.md) - * [Binding test functions](WriteYourTests/Reference/0_BindingTestFunctions.md) - * [Binding assert functions](WriteYourTests/Reference/1_BindingAssertFunctions.md) - * [Test framework functions](WriteYourTests/Reference/2_TestFrameworkFunctions.md) - * [LUAUnit assertion functions](WriteYourTests/Reference/LuaUnitAssertionFunctions/Introduction.md) - * [General Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md) - * [Value Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md) - * [Scientific Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md) - * [String Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md) - * [Error Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md) - * [Type Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md) - * [Table Assertions](WriteYourTests/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md) \ No newline at end of file +* [How to launch the tests ?](3_Launch_the_tests.md) + +* [Tests examples?](4_Tests_Examples.md) + +* [References](Reference/Introduction.md) + * [Binding test functions](Reference/0_BindingTestFunctions.md) + * [Binding assert functions](Reference/1_BindingAssertFunctions.md) + * [Test framework functions](Reference/2_TestFrameworkFunctions.md) + * [LUAUnit assertion functions](Reference/LuaUnitAssertionFunctions/Introduction.md) + * [General Assertions](Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md) + * [Value Assertions](Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md) + * [Scientific Assertions](Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md) + * [String Assertions](Reference/LuaUnitAssertionFunctions/3_StringAssertions.md) + * [Error Assertions](Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md) + * [Type Assertions](Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md) + * [Table Assertions](Reference/LuaUnitAssertionFunctions/6_TableAssertions.md) diff --git a/docs/WriteYourTests/0_ProjectTree.md b/docs/WriteYourTests/0_ProjectTree.md deleted file mode 100644 index 1905d21..0000000 --- a/docs/WriteYourTests/0_ProjectTree.md +++ /dev/null @@ -1,172 +0,0 @@ -# Test architecture - -## Files tree and organization - -```tree - -"test" - +-- etc - | +-- aft-yourbinding.json - | +-- CMakeLists.txt - +-- fixture - | +-- a-script.sh - | +-- any-needed.data - | +-- CMakeLists.txt - | +-- data.json - +-- tests - | +-- CMakeLists.txt - | +-- test01.lua - | +-- test02.lua - | ... - -``` - -To integrate tests in your project, create a **test** subfolder at your project -root directory and fulfill it with appropriate files like shown as above. - -To make it simple you'll have to write your tests using lua language and store it -in the **tests** folder (as shown above) and create a JSON configuration file -*aft-yourbinding.json* to be able to launch your tests. You'll see in the next -section how to write a proper configuration file. - -> **Note** that if you create a new json file, its name has to start with "aft-" -> followed by the binder's name. (e.g. aft-low-can for the low-level-can-service) - -## Integration with CMake using App-templates - -To make the link between your test files, config files, data files -and the test binding, you will have to integrate them with CMake using the -App-templates. - -First you will have to create your CMake target using **PROJECT_TARGET_ADD** -with your target name as parameter, it will include the target to -your project. - -Then add your data files using **add_input_files** with your files in -parameter. - -Use **SET_TARGET_PROPERTIES** to fit the targets properties for macros -usage. Here you have to specify what type of your targets you want to include -in the widget package using the property **LABELS**. It will most likely either -be *TEST-DATA* or *TEST-CONFIG*. - -Here is the LABELS list: - -- **TEST-CONFIG**: JSON configuration files that will be used by the afb-test - binding to know how to execute tests. -- **TEST-DATA**: Resources used to test your binding. It is at least your test - plan and also could be fixtures and any files needed by your tests. These files - will appear in a separate test widget. -- **TEST-PLUGIN**: Shared library meant to be used as a binding - plugin. Binding would load it as a plugin to extend its functionalities. It - should be named with a special extension that you choose with SUFFIX cmake - target property or it'd be **.ctlso** by default. -- **TEST-HTDOCS**: Root directory of a web app. This target has to build its - directory and put its files in the ${CMAKE_CURRENT_BINARY_DIR}/${TARGET_NAME} -- **TEST-EXECUTABLE**: Entry point of your application executed by the AGL - Application Framework -- **TEST-LIBRARY**: An external 3rd party library bundled with the binding for its - own use in case the platform doesn't provide it. - -Here is a mapping between LABELS and directories where files will be placed in -the widget: - -- **EXECUTABLE** : \/bin -- **BINDING-CONFIG** : \/etc -- **BINDING** | **BINDINGV2** | **BINDINGV3** | **LIBRARY** : \/lib -- **PLUGIN** : \/lib/plugins -- **HTDOCS** : \/htdocs -- **BINDING-DATA** : \/var -- **DATA** : \/var - -And about test dedicated **LABELS**: - -- **TEST-EXECUTABLE** : \/bin -- **TEST-CONFIG** : \/etc -- **TEST-PLUGIN** : \/lib/plugins -- **TEST-HTDOCS** : \/htdocs -- **TEST-DATA** : \/var - -> **TIP** you should use the prefix _afb-_ with your **BINDING* targets which -> stand for **Application Framework Binding**. - -You will find in further description about it [here](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/sdk-devkit/docs/part-2/2_4-Use-app-templates.html#targets-properties). - -Here is an example of a proper CMake file to include your LUA test files: - -```CMake -PROJECT_TARGET_ADD(test-files) - - file(GLOB LUA_FILES "*.lua") - add_input_files("${LUA_FILES}") - - SET_TARGET_PROPERTIES(${TARGET_NAME} PROPERTIES - LABELS "TEST-DATA" - OUTPUT_NAME ${TARGET_NAME} - ) -``` - -## Build the test widget - -### Using CMake Apps module or app-templates - -> **Note** the CMake module is the new way to use **app-templates** - -To launch tests on a target board, you need to build a test widget. Using the -SDK, you only have to set the variable *BUILD_TEST_WGT=TRUE* when configuring -the project. - -Example from another project than **afb-test**: - -```bash -mkdir build -cd build -cmake -DBUILD_TEST_WGT=TRUE .. -make -make widget -``` - -### Without using CMake Apps module or app-templates - -Like you'd build a regular widget create a directory where you'll put your tests -materials: LUA tests, configuration and fixture files. - -Then create in that directory a **bin** directory where you'll put this -[script](https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/app-templates.git;a=blob_plain;f=test-widget/launcher.sh.in;h=005c43357db3daa71b66d95d2486cd13f5cee482;hb=refs/heads/master) and name it **launcher**. To finish, you'll also need a -widget configuration file. You can use the example provided [here](https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/app-templates.git;a=blob_plain;f=test-widget/test-config.xml.in;hb=refs/heads/master) and edit it -to fit your needs by replacing the variables surrounded by **@** characters. - -Example from another project than **afb-test**: - -```bash -cd build -BUILDDIR="$(pwd)" -mkdir -p package-test/bin -cd package-test -wget https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/app-templates.git;a=blob_plain;f=test-widget/launcher.sh.in;h=005c43357db3daa71b66d95d2486cd13f5cee482;hb=refs/heads/master -O bin/launcher -wget https://gerrit.automotivelinux.org/gerrit/gitweb?p=apps/app-templates.git;a=blob_plain;f=test-widget/test-config.xml.in;hb=refs/heads/master -O config.xml -vim config.xml -``` - -Then once your widget content directory is complete, use **wgtpkg-pack** utility -to create the test widget. - -```bash -cd ${BUILDDIR} -wgtpkg-pack -f -o -test.wgt package-test -``` - -## Run the test widget on the target - -Once built you can send the wgt file to your target board and launch the tests -using **afm-test** as follow: - -```bash -afm-test -``` - -**afm-test** installs the test wgt file, then runs it and uninstalls it. - -> **CAUTION**: Be cautious to not specify a regular widget's app, else it will -> be uninstalled at the end of the operation. So, if it was an important -> service/binding, others bindings or apps relying on it won't work. diff --git a/docs/WriteYourTests/1_BindingConfiguration.md b/docs/WriteYourTests/1_BindingConfiguration.md deleted file mode 100644 index 2bd31c5..0000000 --- a/docs/WriteYourTests/1_BindingConfiguration.md +++ /dev/null @@ -1,112 +0,0 @@ -# Test configuration - -The file `aft-test.json` contains the controller binding configuration. Here, -you have to change or define the *files* key in the *args* object of the -*testVerb* section, *testVerb* is an array of verb definition which are -meant to launch different LUA test files. - -Also you MUST specify which *api* you need to trace to perform your tests. -Specify which api to trace using a pattern. - -Edit the JSON array to point to your tests files. - -Here is an example: - -```json -{ - "id": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", - "$schema": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", - "metadata": { - "uid": "Test", - "version": "1.0", - "api": "afTest", - "info": "Binding made to test other bindings", - "require": [ - "hello" - ] - }, - "testVerb": { - "uid": "launch_all_tests", - "info": "Launch all the tests", - "action": "lua://AFT#_launch_test", - "args": { - "trace": "hello", - "files": ["aftTest.lua","helloworld.lua"] - } - } -} -``` - -and another example which tests the low-can api: - -```json -{ - "id": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", - "$schema": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", - "metadata": { - "uid": "Test", - "version": "1.0", - "api": "aft-aftest", - "info": "Binding made to test other bindings", - "require": [ - "low-can" - ] - }, - "testVerb": { - "uid": "launch_all_tests", - "info": "Launch all the tests", - "action": "lua://AFT#_launch_test", - "args": { - "trace": "low-can", - "files": [ "aftTest.lua", "mapis-tests.lua" ] - } - }, - "mapis": [{ - "uid": "low-can", - "info": "Faked low-can API", - "libs": "mapi_low-can.lua", - "verbs": [ - { - "uid": "subscribe", - "info": "Subscribe to CAN signals events", - "action": "lua://low-can#_subscribe" - }, - { - "uid": "unsubscribe", - "info": "Unsubscribe previously suscribed signals.", - "action": "lua://low-can#_unsubscribe" - }, - { - "uid": "get", - "info": "get a current value of CAN message", - "action": "lua://low-can#_get" - }, - { - "uid": "list", - "info": "get a supported CAN message list", - "action": "lua://low-can#_list" - }, - { - "uid": "auth", - "info": "Authenticate session to be raise Level Of Assurance.", - "action": "lua://low-can#_auth" - }, - { - "uid": "write", - "info": "Write a CAN messages to the CAN bus.", - "action": "lua://low-can#_write" - } - ], - "events": [{ - "uid": "low-can/diagnostic_messages", - "action": "lua://AFT#_evt_catcher_" - },{ - "uid": "low-can/messages_engine_speed", - "action": "lua://AFT#_evt_catcher_" - },{ - "uid": "low-can/messages_vehicle_speed", - "action": "lua://AFT#_evt_catcher_" - }] - }] -} -``` diff --git a/docs/WriteYourTests/2_LUATestFiles.md b/docs/WriteYourTests/2_LUATestFiles.md deleted file mode 100644 index 05752dd..0000000 --- a/docs/WriteYourTests/2_LUATestFiles.md +++ /dev/null @@ -1,15 +0,0 @@ -# LUA Test files - -You have two differents things to take in account when you'll write your tests -using this framework: *test* and *assertions*. - -*Assertions* are functions meant to test an atomic operation result. -(ie: `1+1 = 2` is an assertion) - -*Test* functions represent a test (Unbelievable), they represent a set of one or -several *assertions* which are all needed to succeed to valid the test. - -The framework came with several *test* and *assertion* functions to simply be -able to test verb calls and events receiving. Use the simple one as often as -possible and if you need more use the one that calls a callback. Specifying a -callback let you add assertions and enrich the test. diff --git a/docs/WriteYourTests/Introduction.md b/docs/WriteYourTests/Introduction.md deleted file mode 100644 index a8b457b..0000000 --- a/docs/WriteYourTests/Introduction.md +++ /dev/null @@ -1,10 +0,0 @@ -# Write your test - -Follow this section to know how to write your test. - -Section's Table of content : - -* [Project Tree](0_ProjectTree.html) -* [Binding configuration](1_BindingConfiguration.html) -* [LUA test files](2_LUATestFiles.html) -* [References](Reference/Introduction.html) diff --git a/docs/WriteYourTests/Reference/0_BindingTestFunctions.md b/docs/WriteYourTests/Reference/0_BindingTestFunctions.md deleted file mode 100644 index 8da86da..0000000 --- a/docs/WriteYourTests/Reference/0_BindingTestFunctions.md +++ /dev/null @@ -1,100 +0,0 @@ -# Binding Test functions - -* **_AFT.testVerbStatusSuccess(testName, api, verb, args, setUp, tearDown)** - - Simply test that the call of a verb successfully returns. - - *setUp* and *tearDown* are functions that can be added to your context, - it works just like **_AFT.beforeEach()** and **_AFT.afterEach()**, - *setUp* will be ran before your *testFunction* and **_AFT.beforeEach()** - (if set) functions, *tearDown* will be ran after your *testFunction* and - **_AFT.afterEach()** (if set) functions. - -* **_AFT.testVerbStatusError(testName, api, verb, args, setUp, tearDown)** - - The inverse than above. - - *setUp* and *tearDown* are functions that can be added to your context, - it works just like **_AFT.beforeEach()** and **_AFT.afterEach()**, - *setUp* will be ran before your *testFunction* and **_AFT.beforeEach()** - (if set) functions, *tearDown* will be ran after your *testFunction* and - **_AFT.afterEach()** (if set) functions. - -* **_AFT.testVerbResponseEquals(testName, api, verb, args, expectedResponse, setUp, tearDown)** - - Test that the call of a verb successfully returns and that verb's response - is equals to the *expectedResponse*. - - *setUp* and *tearDown* are functions that can be added to your context, - it works just like **_AFT.beforeEach()** and **_AFT.afterEach()**, *setUp* - will be ran before your *testFunction* and **_AFT.beforeEach()** (if set) - functions, *tearDown* will be ran after your *testFunction* and - **_AFT.afterEach()** (if set) functions. - -* **_AFT.testVerbResponseEqualsError(testName, api, verb, args, expectedResponse, setUp, tearDown)** - - The inverse than above. - - *setUp* and *tearDown* are functions that can be added to your context, it works - just like **_AFT.beforeEach()** and **_AFT.afterEach()**, *setUp* will be ran - before your *testFunction* and **_AFT.beforeEach()** (if set) functions, - *tearDown* will be ran after your *testFunction* and **_AFT.afterEach()** (if - set) functions. - -* **_AFT.testVerbCb(testName, api, verb, args, expectedResponse, callback, setUp, tearDown)** - - Test the call of a verb with a custom callback. From this callback you - will need to make some assertions on what you need (verb JSON return object - content mainly). - - If you don't need to test the response simply specify an empty LUA table. - - *setUp* and *tearDown* are functions that can be added to your context, it works - just like **_AFT.beforeEach()** and **_AFT.afterEach()**, *setUp* will be ran - before your *testFunction* and **_AFT.beforeEach()** (if set) functions, - *tearDown* will be ran after your *testFunction* and **_AFT.afterEach()** (if - set) functions. - -* **_AFT.testVerbCbError(testName, api, verb, args, expectedResponse, callback, setUp, tearDown)** - - Should return success on failure. - - *setUp* and *tearDown* are functions that can be added to your context, it works - just like **_AFT.beforeEach()** and **_AFT.afterEach()**, *setUp* will be ran - before your *testFunction* and **_AFT.beforeEach()** (if set) functions, - *tearDown* will be ran after your *testFunction* and **_AFT.afterEach()** (if - set) functions. - -* **_AFT.testEvtReceived(testName, eventName, timeout, setUp, tearDown)** - - Prior to be able to check that an event has been received, you have to - register the event with the test framework using **_AFT.addEventToMonitor** - function. - - Check if an event has been correctly received in time (timeout in µs). An event - name use the application framework naming scheme: **api/event_name**. - -* **_AFT.testEvtNotReceived(testName, eventName, timeout, setUp, tearDown)** - - Prior to be able to check that an event has not been received, you have to - register the event with the test framework using **_AFT.addEventToMonitor** - function. - - Check if an event has not been correctly received in time (timeout in µs). An - event name use the application framework naming scheme: **api/event_name**. - -* **_AFT.testGrpEvtReceived(testName, eventGrp, timeout, setUp, tearDown)** - - Prior to be able to check that a group of event (a table of event) has been - received, you have to register the event with the test framework using - **_AFT.addEventToMonitor** function. - - The table has to have this format: - ```lua - eventGrp = {["api/event_name_1"]=1,["api/event_name_2"]=2,["api/event_name_3"]=5} - ``` - As you can see, in the table, event names are table keys and the value stored are - the number of time that the events have to be received. - - Check if events has been correctly received in time (timeout in µs). An - event name use the application framework naming scheme: **api/event_name**. \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/1_BindingAssertFunctions.md b/docs/WriteYourTests/Reference/1_BindingAssertFunctions.md deleted file mode 100644 index 918bd04..0000000 --- a/docs/WriteYourTests/Reference/1_BindingAssertFunctions.md +++ /dev/null @@ -1,64 +0,0 @@ -# Binding Assert functions - -* **_AFT.assertVerbStatusSuccess(api, verb, args)** - - Simply test that the call of a verb successfully returns. - -* **_AFT.assertVerbStatusError(api, verb, args)** - - The inverse than above. - -* **_AFT.assertVerbResponseEquals(api, verb, args, expectedResponse)** - - Test that the call of a verb successfully returns and that verb's response - is equals to the *expectedResponse*. - -* **_AFT.assertVerbResponseEqualsError(api, verb, args, expectedResponse)** - - The inverse than above. - -* **_AFT.assertVerbCb(api, verb, args, expectedResponse, callback)** - - Test the call of a verb with a custom callback. From this callback you - will need to make some assertions on what you need (verb JSON return object - content mainly). - - If you don't need to test the response simply specify an empty LUA table. - -* **_AFT.assertVerbCbError(api, verb, args, expectedResponse, callback)** - - Should return success on failure. - -* **_AFT.assertEvtReceived(eventName, timeout)** - - Prior to be able to check that an event has been received, you have to - register the event with the test framework using **_AFT.addEventToMonitor** - function. - - Check if an event has been correctly received in time (timeout in µs). - An event name use the application framework naming scheme: **api/event_name**. - -* **_AFT.assertEvtNotReceived(eventName, timeout)** - - Prior to be able to check that an event has been received, you have to - register the event with the test framework using **_AFT.addEventToMonitor** - function. - - Check if an event has been correctly received in time (timeout in µs). - An event name use the application framework naming scheme: **api/event_name**. - -* **_AFT.assertGrpEvtReceived(eventGrp, timeout)** - - Prior to be able to check that a group of event (a table of event) has been - received, you have to register the event with the test framework using - **_AFT.addEventToMonitor** function. - - The table has to have this format: - ```lua - eventGrp = {["api/event_name_1"]=1,["api/event_name_2"]=2,["api/event_name_3"]=5} - ``` - As you can see, in the table, event names are table keys and the value stored are - the number of time that the events have to be received. - - Check if events has been correctly received in time (timeout in µs). - An event name use the application framework naming scheme: **api/event_name**. \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/2_TestFrameworkFunctions.md b/docs/WriteYourTests/Reference/2_TestFrameworkFunctions.md deleted file mode 100644 index 20184ec..0000000 --- a/docs/WriteYourTests/Reference/2_TestFrameworkFunctions.md +++ /dev/null @@ -1,59 +0,0 @@ -# Test Framework functions - -* **_AFT.addEventToMonitor(eventName, callback)** - - Add a binding event in the test framework to be able to assert its reception - . You'll need to add as much as events you expect to receive. You could also - specify a callback to test deeper that the event is as you want to. The - callback will happens after the assertion that it has been received so you - can work on data that the event eventually carry. - -* **_AFT.setJunitFile(filePath)** - - Set the *JUnit* file path. When *JUnit* is set as the output type for the - test framework. - -* **_AFT.setBeforeEach(function)** - - Set the **_AFT.beforeEach()** function which is used to run the *function* - before each tests. - -* **_AFT.setAfterEach(function)** - - Set the **_AFT.afterEach()** function which is used to run the *function* - after each tests. - -* **_AFT.setBeforeAll(function)** - - Set the **_AFT.beforeAll()** function which is used to run the *function* - before all tests. If the given function is successful it has to return 0 - else it will return an error. - -* **_AFT.setAfterAll(function)** - - Set the **_AFT.afterAll()** function which is used to run the *function* - after all tests. If the given function is successful it has to return 0 - else it will return an error. - -* **_AFT.describe(testName, testFunction, setUp, tearDown)** - - Give a context to a custom test. *testFunction* will be given the name - provided by *testName* and will be tested. - - *setUp* and *tearDown* are functions that can be added to your context, - it works just like **_AFT.beforeEach()** and **_AFT.afterEach()**, - *setUp* will be ran before your *testFunction* and **_AFT.beforeEach()** - (if set) functions, *tearDown* will be ran after your *testFunction* and - **_AFT.afterEach()** (if set) functions. - -* **_AFT.setBefore(testName, beforeTestFunction)** - Set a function to be ran at the beginning of the given *testName* function. - - ```lua - _AFT.testVerbStatusSuccess('testPingSuccess','hello', 'ping', {}) - _AFT.setBefore("testPingSuccess",function() print("~~~~~ Begin testPingSuccess ~~~~~") end) - _AFT.setAfter("testPingSuccess",function() print("~~~~~ End testPingSuccess ~~~~~") end) - ``` - -* **_AFT.setBefore(testName, beforeTestFunction)** - Set a function to be ran at the end of the given *testName* function. \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/Introduction.md b/docs/WriteYourTests/Reference/Introduction.md deleted file mode 100644 index 8de5a15..0000000 --- a/docs/WriteYourTests/Reference/Introduction.md +++ /dev/null @@ -1,10 +0,0 @@ -# References - -Follow this section to know which functions are available !! - -Section's Table of content : - -* [Binding test functions](0_BindingTestFunctions.html) -* [Binding assert functions](1_BindingAssertFunctions.html) -* [Test framework functions](2_TestFrameworkFunctions.html) -* [LUAUnit assertion functions](LuaUnitAssertionFunctions/Introduction.html) \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md b/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md deleted file mode 100644 index 4a719b6..0000000 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/0_GeneralAssertions.md +++ /dev/null @@ -1,32 +0,0 @@ -# General Assertions - -* **_AFT.assertEquals(actual, expected)** - - Assert that two values are equal. - - For tables, the comparison is a deep comparison : - - * number of elements must be the same - * tables must contain the same keys - * each key must contain the same values. The values are also compared recursively with deep comparison. - - LuaUnit provides other table-related assertions, see [Table assertions](http://luaunit.readthedocs.io/en/luaunit_v3_2_1/#assert-table) - -* **_AFT.assertNotEquals(actual, expected)** - - Assert that two values are different. The assertion fails if the two values are identical. - - It also uses table deep comparison. - -* **_AFT.assertAlmostEquals(actual, expected, margin)** - - Assert that two floating point numbers are almost equal. - - When comparing floating point numbers, strict equality does not work. - Computer arithmetic is so that an operation that mathematically yields - 1.00000000 might yield 0.999999999999 in lua . That’s why you need an - almost equals comparison, where you specify the error margin. - -* **_AFT.assertNotAlmostEquals(actual, expected, margin)** - - Assert that two floating point numbers are not almost equal. \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md b/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md deleted file mode 100644 index ed927cd..0000000 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/1_ValueAssertions.md +++ /dev/null @@ -1,58 +0,0 @@ -# Value assertions - -* **_AFT.assertEvalToTrue(value)** - - Assert that a given value evals to true. Lua coercion rules are applied so - that values like 0,"",1.17 succeed in this assertion. If provided, extra_msg - is a string which will be printed along with the failure message. - -* **_AFT.assertEvalToFalse(Value)** - - Assert that a given value eval to *false*. Lua coercion rules are applied so - that *nil* and *false* succeed in this assertion. If provided, extra_msg is a - string which will be printed along with the failure message. - -* **_AFT.assertIsTrue(value)** - - Assert that a given value compares to true. Lua coercion rules are applied so - that values like 0, "", 1.17 all compare to true. - -* **_AFT.assertIsFalse(value)** - - Assert that a given value compares to false. Lua coercion rules are applied so - that only nil and false all compare to false. - -* **_AFT.assertIsNil(value)** - - Assert that a given value is nil . - -* **_AFT.assertNotIsNil(value)** - - Assert that a given value is not *nil* . Lua coercion rules are applied - so that values like ``0``, ``""``, ``false`` all validate the assertion. - If provided, *extra_msg* is a string which will be printed along with the - failure message. - -* **_AFT.assertIs(actual, expected)** - - Assert that two variables are identical. For string, numbers, boolean and - for nil, this gives the same result as assertEquals() . For the other types, - identity means that the two variables refer to the same object. - - Example : - - ```lua - `s1='toto' - s2='to'..'to' - t1={1,2} - t2={1,2} - luaunit.assertIs(s1,s1) -- ok - luaunit.assertIs(s1,s2) -- ok - luaunit.assertIs(t1,t1) -- ok - luaunit.assertIs(t1,t2) -- fail` - ``` - -* **_AFT.assertNotIs(actual, expected)** - - Assert that two variables are not identical, in the sense that they do not - refer to the same value. See assertIs() for more details. \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md b/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md deleted file mode 100644 index 9f80a7a..0000000 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/2_ScientificAssertions.md +++ /dev/null @@ -1,48 +0,0 @@ -# Scientific assertions - ->**Note** ->If you need to deal with value minus zero, be very careful because Lua versions -are inconsistent on how they treat the >syntax -0 : it creates either a plus -zero or a minus zero. Multiplying or dividing 0 by -1 also yields inconsistent > -results. The reliable way to create the -0 value is : minusZero = -1 / (1/0). - -* **_AFT.assertIsNaN(value)** - Assert that a given number is a *NaN* (Not a Number), according to the - definition of IEEE-754_ . If provided, *extra_msg* is a string which will - be printed along with the failure message. - -* **_AFT.assertIsPlusInf(value)** - - Assert that a given number is *plus infinity*, according to the definition of - IEEE-754_. If provided, *extra_msg* is a string which will be printed along - with the failure message. - -* **_AFT.assertIsMinusInf(value)** - - Assert that a given number is *minus infinity*, according to the definition of - IEEE-754_. If provided, *extra_msg* is a string which will be printed along - with the failure message. - -* **_AFT.assertIsInf(value)** - - Assert that a given number is *infinity* (either positive or negative), - according to the definition of IEEE-754_. If provided, *extra_msg* is a string - which will be printed along with the failure message. - -* **_AFT.assertIsPlusZero(value)** - - Assert that a given number is *+0*, according to the definition of IEEE-754_. - The verification is done by dividing by the provided number and verifying - that it yields *infinity* . If provided, *extra_msg* is a string which will - be printed along with the failure message. - - Be careful when dealing with *+0* and *-0*, see note above - -* **_AFT.assertIsMinusZero(value)** - - Assert that a given number is *-0*, according to the definition of IEEE-754_. - The verification is done by dividing by the provided number and verifying that - it yields *minus infinity* . If provided, *extra_msg* is a string which will - be printed along with the failure message. - - Be careful when dealing with *+0* and *-0* \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md b/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md deleted file mode 100644 index a816998..0000000 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/3_StringAssertions.md +++ /dev/null @@ -1,38 +0,0 @@ -# String assertions - -Assertions related to string and patterns. - -* **_AFT.assertStrContains(str, sub[, useRe])** - - Assert that a string contains the given substring or pattern. - - By default, substring is searched in the string. If useRe is provided and is - true, sub is treated as a pattern which is searched inside the string str. - -* **_AFT.assertStrIContains(str, sub)** - - Assert that a string contains the given substring, irrespective of the case. - - Not that unlike assertStrcontains(), you can not search for a pattern. - -* **_AFT.assertNotStrContains(str, sub, useRe)** - - Assert that a string does not contain a given substring or pattern. - - By default, substring is searched in the string. If useRe is provided and is - true, sub is treated as a pattern which is searched inside the string str. - -* **_AFT.assertNotStrIContains(str, sub)** - - Assert that a string does not contain the given substring, irrespective of - the case. - - Not that unlike assertNotStrcontains(), you can not search for a pattern. - -* **_AFT.assertStrMatches(str, pattern[, start[, final]])** - - Assert that a string matches the full pattern pattern. - - If start and final are not provided or are nil, the pattern must match the - full string, from start to end. The functions allows to specify the expected - start and end position of the pattern in the string. \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md b/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md deleted file mode 100644 index 482b98d..0000000 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/4_ErrorAssertions.md +++ /dev/null @@ -1,30 +0,0 @@ -# Error assertions - -Error related assertions, to verify error generation and error messages. - -* **_AFT.assertError(func, ...)** - - Assert that calling functions func with the arguments yields an error. If the function does not yield an error, the assertion fails. - - Note that the error message itself is not checked, which means that this function does not distinguish between the legitimate error that you expect and another error that might be triggered by mistake. - - The next functions provide a better approach to error testing, by checking explicitly the error message content. - ->**Note** ->When testing LuaUnit, switching from assertError() to assertErrorMsgEquals() revealed quite a few bugs! - -* **_AFT.assertErrorMsgEquals(expectedMsg, func, ...)** - - Assert that calling function func will generate exactly the given error message. If the function does not yield an error, or if the error message is not identical, the assertion fails. - - Be careful when using this function that error messages usually contain the file name and line number information of where the error was generated. This is usually inconvenient. To ignore the filename and line number information, you can either use a pattern with assertErrorMsgMatches() or simply check if the message contains a string with assertErrorMsgContains() . - -* **_AFT.assertErrorMsgContains(partialMsg, func, ...)** - - Assert that calling function func will generate an error message containing partialMsg . If the function does not yield an error, or if the expected message is not contained in the error message, the assertion fails. - -* **_AFT.assertErrorMsgMatches(expectedPattern, func, ...)** - - Assert that calling function func will generate an error message matching expectedPattern . If the function does not yield an error, or if the error message does not match the provided pattern the assertion fails. - - Note that matching is done from the start to the end of the error message. Be sure to escape magic all magic characters with % (like -+.?\*) . \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md b/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md deleted file mode 100644 index 843f651..0000000 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/5_TypeAssertions.md +++ /dev/null @@ -1,38 +0,0 @@ -# Type assertions - -The following functions all perform type checking on their argument. If the -received value is not of the right type, the failure message will contain the -expected type, the received type and the received value to help you identify -better the problem. - -* **_AFT.assertIsNumber(value)** - - Assert that the argument is a number (integer or float) - -* **_AFT.assertIsString(value)** - - Assert that the argument is a string. - -* **_AFT.assertIsTable(value)** - - Assert that the argument is a table. - -* **_AFT.assertIsBoolean(value)** - - Assert that the argument is a boolean. - -* **_AFT.assertIsFunction(value)** - - Assert that the argument is a function. - -* **_AFT.assertIsUserdata(value)** - - Assert that the argument is a userdata. - -* **_AFT.assertIsThread(value)** - - Assert that the argument is a coroutine (an object with type thread ). - -* **_AFT.assertNotIsThread(value)** - - Assert that the argument is a not coroutine (an object with type thread ). \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md b/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md deleted file mode 100644 index 8d29988..0000000 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/6_TableAssertions.md +++ /dev/null @@ -1,19 +0,0 @@ -# Table assertions - -* **_AFT.assertItemsEquals(actual, expected)** - - Assert that two tables contain the same items, irrespective of their keys. - - This function is practical for example if you want to compare two lists but - where items are not in the same order: - - ```lua - luaunit.assertItemsEquals( {1,2,3}, {3,2,1} ) -- assertion succeeds - ``` - The comparison is not recursive on the items: if any of the items are tables, - they are compared using table equality (like as in assertEquals() ), where the - key matters. - - ```lua - luaunit.assertItemsEquals( {1,{2,3},4}, {4,{3,2,},1} ) -- assertion fails because {2,3} ~= {3,2} - ``` \ No newline at end of file diff --git a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/Introduction.md b/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/Introduction.md deleted file mode 100644 index 2bdf05c..0000000 --- a/docs/WriteYourTests/Reference/LuaUnitAssertionFunctions/Introduction.md +++ /dev/null @@ -1,13 +0,0 @@ -# LuaUnit Assertion Functions - -Follow this section to know which luaUnit assertion functions are available !! - -Section's Table of content : - -* [General Assertions](0_GeneralAssertions.html) -* [Value Assertions](1_ValueAssertions.html) -* [Scientific Assertions](2_ScientificAssertions.html) -* [String Assertions](3_StringAssertions.html) -* [Error Assertions](4_ErrorAssertions.html) -* [Type Assertions](5_TypeAssertions.html) -* [Table Assertions](6_TableAssertions.html) \ No newline at end of file -- cgit 1.2.3-korg