diff options
| -rw-r--r-- | README.md | 402 |
1 files changed, 402 insertions, 0 deletions
diff --git a/README.md b/README.md new file mode 100644 index 0000000..0bd1dde --- /dev/null +++ b/README.md @@ -0,0 +1,402 @@ +# Installation + +## Pre-requisites + +[Setup the pre-requisite](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/host-configuration/docs/1_Prerequisites.html) then [install the Application Framework](http://docs.automotivelinux.org/docs/devguides/en/dev/reference/host-configuration/docs/2_AGL_Application_Framework.html) on your host. + +You will also need to install lua-devel >= 5.3 to be able to build the project. + +Fedora: + +```bash +dnf install lua-devel +``` + +OpenSuse: + +```bash +zypper install lua53-devel +``` + +Ubuntu (>= Xenial), Debian stable: + +```bash +apt-get install liblua5.3-dev +``` + +## Grab source and build + +Download the **afb-test** binding source code using git: + +```bash +git clone --recurse-submodules https://github.com/iotbzh/afb-test +cd afb-test +mkdir build +cd build +cmake .. && make +``` + +## Launch the example + +To launch the binding use the command-line provided at the end of the build. +This will launch the test of an Helloworld binding example. The code of the test +is available from the LUA file `conf.d/project/lua.d/helloworld.lua`. + +The example will run some basics tests on API verb calls and events received. + +> **NOTE**: I suggest you to take this lua file example to make your own test +> then read the following the chapter if needed to write more complicated tests. + +```bash +$ afb-daemon --name afbd-test --port=1234 --workdir=package --ldpaths=/opt/AGL/lib64/afb:lib --token= --tracereq=common --verbose +NOTICE: Can't connect supervision socket to @urn:AGL:afs:supervision:socket: Connection refused +NOTICE: [API hello] hello binding comes to live +NOTICE: [API ave] dynamic binding AVE(ave) comes to live +NOTICE: [API hi] dynamic binding AVE(hi) comes to live +NOTICE: [API salut] dynamic binding AVE(salut) comes to live +NOTICE: [API lib/test-binding.so] Controller in afbBindingVdyn +NOTICE: [API lib/test-binding.so] Controller API='test' info='Binding made to tests other bindings' +NOTICE: API monitor started +HOOK: [xreq-000001:monitor/set] BEGIN +HOOK: [xreq-000001:monitor/set] json() -> { "verbosity": "debug" } +HOOK: [xreq-000001:monitor/set] success(null, (null)) +HOOK: [xreq-000001:monitor/set] END +HOOK: [xreq-000002:monitor/trace] BEGIN +HOOK: [xreq-000002:monitor/trace] json() -> { "add": { "event": "push_after" } } +HOOK: [xreq-000002:monitor/trace] subscribe(monitor/trace:1) -> 0 +HOOK: [xreq-000002:monitor/trace] success(null, (null)) +HOOK: [xreq-000002:monitor/trace] END +HOOK: [xreq-000003:monitor/set] BEGIN +HOOK: [xreq-000003:monitor/set] json() -> { "verbosity": "debug" } +HOOK: [xreq-000003:monitor/set] success(null, (null)) +HOOK: [xreq-000003:monitor/set] END +HOOK: [xreq-000004:monitor/trace] BEGIN +HOOK: [xreq-000004:monitor/trace] json() -> { "add": { "event": "push_after" } } +HOOK: [xreq-000004:monitor/trace] subscribe(monitor/trace:1) -> 0 +HOOK: [xreq-000004:monitor/trace] success(null, (null)) +HOOK: [xreq-000004:monitor/trace] END +# XML output to var/jUnitResults.xml +# Started on Thu Jun 14 18:12:33 2018 +# Starting test: testPingSuccess +INFO: API hello starting... +NOTICE: [API hello] hello binding starting +NOTICE: API hello started +HOOK: [xreq-000005:hello/ping] BEGIN +HOOK: [xreq-000005:hello/ping] json() -> null +HOOK: [xreq-000005:hello/ping] success("Some String", Ping Binder Daemon tag=pingSample count=1 query=null) +HOOK: [xreq-000005:hello/ping] END +# Starting test: testPingSuccess +HOOK: [xreq-000006:hello/ping] BEGIN +HOOK: [xreq-000006:hello/ping] json() -> null +HOOK: [xreq-000006:hello/ping] success("Some String", Ping Binder Daemon tag=pingSample count=2 query=null) +HOOK: [xreq-000006:hello/ping] END +# Starting test: testPingSuccess +HOOK: [xreq-000007:hello/ping] BEGIN +HOOK: [xreq-000007:hello/ping] json() -> null +HOOK: [xreq-000007:hello/ping] success("Some String", Ping Binder Daemon tag=pingSample count=3 query=null) +HOOK: [xreq-000007:hello/ping] END +# Failure: var//aft.lua:127: expected: "Unexpected String" +# actual: "Some String" +# Starting test: testPingSuccess +HOOK: [xreq-000008:hello/ping] BEGIN +HOOK: [xreq-000008:hello/ping] json() -> null +HOOK: [xreq-000008:hello/ping] success("Some String", Ping Binder Daemon tag=pingSample count=4 query=null) +HOOK: [xreq-000008:hello/ping] END +# Starting test: testPingError +HOOK: [xreq-000009:hello/pingfail] BEGIN +HOOK: [xreq-000009:hello/pingfail] fail(failed, Ping Binder Daemon fails) +HOOK: [xreq-000009:hello/pingfail] END +# Starting test: testPingError +HOOK: [xreq-000010:hello/pingfail] BEGIN +HOOK: [xreq-000010:hello/pingfail] fail(failed, Ping Binder Daemon fails) +HOOK: [xreq-000010:hello/pingfail] END +# Failure: var//aft.lua:145: Received the not expected value: "Ping Binder Daemon fails" +# Starting test: testPingError +HOOK: [xreq-000011:hello/pingfail] BEGIN +HOOK: [xreq-000011:hello/pingfail] fail(failed, Ping Binder Daemon fails) +HOOK: [xreq-000011:hello/pingfail] END +# Starting test: testPingError +HOOK: [xreq-000012:hello/pingfail] BEGIN +HOOK: [xreq-000012:hello/pingfail] fail(failed, Ping Binder Daemon fails) +HOOK: [xreq-000012:hello/pingfail] END +# Starting test: testEventAdd +HOOK: [xreq-000013:hello/eventadd] BEGIN +HOOK: [xreq-000013:hello/eventadd] get(tag) -> { name: tag, value: event, path: (null) } +HOOK: [xreq-000013:hello/eventadd] get(name) -> { name: name, value: anEvent, path: (null) } +HOOK: [xreq-000013:hello/eventadd] success(null, (null)) +HOOK: [xreq-000013:hello/eventadd] END +# Starting test: testEventSub +HOOK: [xreq-000014:hello/eventsub] BEGIN +HOOK: [xreq-000014:hello/eventsub] get(tag) -> { name: tag, value: event, path: (null) } +HOOK: [xreq-000014:hello/eventsub] subscribe(hello/anEvent:2) -> 0 +HOOK: [xreq-000014:hello/eventsub] success(null, (null)) +HOOK: [xreq-000014:hello/eventsub] END +# Starting test: testEventPush +HOOK: [xreq-000015:hello/eventpush] BEGIN +HOOK: [xreq-000015:hello/eventpush] get(tag) -> { name: tag, value: event, path: (null) } +HOOK: [xreq-000015:hello/eventpush] get(data) -> { name: data, value: { "another_key": 123, "key": "some data" }, path: (null) } +DEBUG: [API test] Received event=hello/anEvent, query={ "another_key": 123, "key": "some data" } +WARNING: [API test] CtlDispatchEvent: fail to find uid=hello/anEvent in action event section [/home/claneys/Workspace/Sources/IOTbzh/afb-test/app-controller-submodule/ctl-lib/ctl-event.c:46,CtrlDispatchApiEvent] +DEBUG: [API test] Received event=monitor/trace, query={ "time": "10944.010648", "tag": "trace", "type": "event", "id": 106, "event": { "id": 2, "name": "hello\/anEvent", "action": "push_after" }, "data": { "data": { "another_key": 123, "key": "some data" }, "result": 1 } } +DEBUG: [API test] Received event=monitor/trace, query={ "time": "10944.010648", "tag": "trace", "type": "event", "id": 106, "event": { "id": 2, "name": "hello\/anEvent", "action": "push_after" }, "data": { "data": { "another_key": 123, "key": "some data" }, "result": 1 } } +HOOK: [xreq-000015:hello/eventpush] success(null, (null)) +HOOK: [xreq-000015:hello/eventpush] END +# Starting test: testEventAdd +HOOK: [xreq-000016:hello/eventadd] BEGIN +HOOK: [xreq-000016:hello/eventadd] get(tag) -> { name: tag, value: evt, path: (null) } +HOOK: [xreq-000016:hello/eventadd] get(name) -> { name: name, value: anotherEvent, path: (null) } +HOOK: [xreq-000016:hello/eventadd] success(null, (null)) +HOOK: [xreq-000016:hello/eventadd] END +# Starting test: testEventSub +HOOK: [xreq-000017:hello/eventsub] BEGIN +HOOK: [xreq-000017:hello/eventsub] get(tag) -> { name: tag, value: evt, path: (null) } +HOOK: [xreq-000017:hello/eventsub] subscribe(hello/anotherEvent:3) -> 0 +HOOK: [xreq-000017:hello/eventsub] success(null, (null)) +HOOK: [xreq-000017:hello/eventsub] END +# Starting test: testEventPush +HOOK: [xreq-000018:hello/eventpush] BEGIN +HOOK: [xreq-000018:hello/eventpush] get(tag) -> { name: tag, value: evt, path: (null) } +HOOK: [xreq-000018:hello/eventpush] get(data) -> { name: data, value: { "another_key": 123.456, "key": "weird others data" }, path: (null) } +DEBUG: [API test] Received event=hello/anotherEvent, query={ "another_key": 123.456, "key": "weird others data" } +WARNING: [API test] CtlDispatchEvent: fail to find uid=hello/anotherEvent in action event section [/home/claneys/Workspace/Sources/IOTbzh/afb-test/app-controller-submodule/ctl-lib/ctl-event.c:46,CtrlDispatchApiEvent] +DEBUG: [API test] Received event=monitor/trace, query={ "time": "10944.011045", "tag": "trace", "type": "event", "id": 128, "event": { "id": 3, "name": "hello\/anotherEvent", "action": "push_after" }, "data": { "data": { "another_key": 123.456, "key": "weird others data" }, "result": 1 } } +DEBUG: [API test] Received event=monitor/trace, query={ "time": "10944.011045", "tag": "trace", "type": "event", "id": 128, "event": { "id": 3, "name": "hello\/anotherEvent", "action": "push_after" }, "data": { "data": { "another_key": 123.456, "key": "weird others data" }, "result": 1 } } +HOOK: [xreq-000018:hello/eventpush] success(null, (null)) +HOOK: [xreq-000018:hello/eventpush] END +# Starting test: testEvent +# Starting test: testEventCb +# Ran 16 tests in 0.003 seconds, 14 successes, 2 failures +DEBUG: Init config done +INFO: API auth starting... +NOTICE: API auth started +INFO: API ave starting... +NOTICE: [API ave] dynamic binding AVE(ave) starting +NOTICE: API ave started +INFO: API context starting... +NOTICE: API context started +INFO: API dbus starting... +NOTICE: API dbus started +INFO: API hi starting... +NOTICE: [API hi] dynamic binding AVE(hi) starting +NOTICE: API hi started +INFO: API post starting... +NOTICE: API post started +INFO: API salut starting... +NOTICE: [API salut] dynamic binding AVE(salut) starting +NOTICE: API salut started +INFO: API test starting... +NOTICE: API test started +INFO: API tictactoe starting... +NOTICE: API tictactoe started +NOTICE: Waiting port=1234 rootdir=. +NOTICE: Browser URL= http://localhost:1234 +``` + +## Write your own tests + +### Binding configuration + +In the package directory you have a file name `test-config.json` that contains +the controller binding configuration. Here, you have to change or define the +*files* key in the *args* object of the *onload* section. + +Also you MUST specify which *api* you need to trace to perform your tests. +Specify which api to trace using a pattern. + +Edit the JSON array to point to your tests files. + +Here is an example: + +```json +{ + "id": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", + "$schema": "http://iot.bzh/download/public/schema/json/ctl-schema.json#", + "metadata": { + "uid": "Test", + "version": "1.0", + "api": "test", + "info": "Binding made to tests other bindings", + "require": [ + "hello" + ] + }, + "onload": { + "uid": "launch_all_tests", + "info": "Launch all the tests", + "action": "lua://AFT#_launch_test", + "args": { + "trace": "hello", + "files": ["helloworld.lua"] + } + } +} +``` + +> **CAUTION** : It must stay a JSON array, don't change the type + +### LUA Test files + +First, ensure that you put your LUA tests files in the `var` directory from the +binding root directory. + +You have two differents things to take in account when you'll write your tests +using this framework: *test* and *assertions*. + +*Assertions* are functions mean to test an atomic operation result. +(ie: `1+1 = 2` is an assertion) + +*Test* functions represent a test (Unbelievable), they represent a set of one or +several *assertions* which are all needed to succeed to valid the test. + +The framework came with several *test* and *assertion* functions to simply be +able to test verb calls and events receiving. Use the simple one as often as +possible and if you need more use the one that call a callback. Specifying a +callback let you add assertions and enrich the test. + +### Reference + +#### Binding Test functions + +* **_AFT.testVerbStatusSuccess(testName, api, verb, args)** + + Simply test that the call of a verb successfully returns. + +* **_AFT.testVerbStatusError(testName, api, verb, args)** + + The inverse than above + +* **_AFT.testVerbResponseEquals(testName, api, verb, args, expectedResponse)** + + Test that the call of a verb successfully returns and that verb's response + is equals to the *expectedResponse*. + +* **_AFT.testVerbResponseEqualsError(testName, api, verb, args, expectedResponse)** + + The inverse than above + +* **_AFT.testVerbCb(testName, api, verb, args, expectedResponse, callback)** + + Test that the call of a verb with a custom callback. From this callback you + will need to make some assertions on what you need (verb JSON return object + content mainly). + + If you doesn't need to test the response simply specify an empty LUA table. + +* **_AFT.testVerbCbError(testName, api, verb, args, expectedResponse, callback)** + + Should return success on failure. + +* **_AFT.testEvtReceived(testName, eventName, timeout)** + + Prior to be able to check that an event has been received, you have to + register the event with the test framework using **_AFT.addEventToMonitor** + function. + + Check if an event has been correctly received. An event name use the + application framework naming scheme: **api/event_name**. + + The timeout argument is in seconds and could be omitted if you doesn't need + to wait to receive the event. + +> **NOTE**: about now this timeout arguments simply wait a number of seconds +> instead of interrupt as soon as the event has been received. + +* **_AFT.testLogReceived(testName, logMsg, timeout)** + + Prior to be able to check that an event has been received, you have to + register the event with the test framework using **_AFT.addLogToMonitor** + function. + + Check if a log message sent by the daemon has been correctly received. A log + message is caracterized by its *api* and the message level *error*, + *warning*, *notice*, *info* or *debug*. + + The timeout argument is in seconds and could be omitted if you doesn't need + to wait to receive the event. + +> **NOTE**: about now this timeout arguments simply wait a number of seconds +> instead of interrupt as soon as the event has been received. + +#### Binding Assert functions + +* **_AFT.assertVerbStatusSuccess(api, verb, args)** + + Simply test that the call of a verb successfully returns. + +* **_AFT.assertVerbStatusError(testName, api, verb, args)** + + The inverse than above + +* **_AFT.assertVerbResponseEquals(api, verb, args, expectedResponse)** + + Test that the call of a verb successfully returns and that verb's response + is equals to the *expectedResponse*. + +* **_AFT.assertVerbResponseEqualsError(api, verb, args, expectedResponse)** + + The inverse than above + +* **_AFT.assertVerbCb(api, verb, args, expectedResponse, callback)** + + Test that the call of a verb with a custom callback. From this callback you + will need to make some assertions on what you need (verb JSON return object + content mainly). + + If you doesn't need to test the response simply specify an empty LUA table. + +* **_AFT.assertVerbCbError(api, verb, args, expectedResponse, callback)** + + Should return success on failure. + +* **_AFT.assertEvtReceived(eventName)** + + Prior to be able to check that an event has been received, you have to + register the event with the test framework using **_AFT.addEventToMonitor** + function. + + Check if an event has been correctly received. An event name use the + application framework naming scheme: **api/event_name**. + + The timeout argument is in seconds and could be omitted if you doesn't need + to wait to receive the event. + +> **NOTE**: about now this timeout arguments simply wait a number of seconds +> instead of interrupt as soon as the event has been received. + +* **_AFT.assertLogReceived(testName, logMsg, timeout)** + + Prior to be able to check that an event has been received, you have to + register the event with the test framework using **_AFT.addLogToMonitor** + function. + + Check if a log message sent by the daemon has been correctly received. A log + message is caracterized by its *api* and the message level *error*, + *warning*, *notice*, *info* or *debug*. + + The timeout argument is in seconds and could be omitted if you doesn't need + to wait to receive the event. + +> **NOTE**: about now this timeout arguments simply wait a number of seconds +> instead of interrupt as soon as the event has been received. + +#### Test Framework functions + +* **_AFT.addEventToMonitor(eventName, callback)** + + Add a binding event in the test framework to be able to assert its reception + . You'll need to add as much as events you expect to receive. You could also + specify a callback to test deeper that the event is as you want to. The + callback will happens after the assertion that it has been received so you + can work on data that the event eventually carry. + +* **_AFT.addLogToMonitor(api, type, message, callback)** + + Add a message log in the test framework to be able to assert its reception. + The behavior is the same than for the api *events* except that in addition + it also check that the message is coming from a specified *api* and has the + specified type *error*, *warning*, *notice*, *info* or *debug*. + +* **_AFT.setJunitFile(filePath)** + + Set the *JUnit* file path. When *JUnit* is set as the output type for the + test framework. |