From eca44ae331bfe8bc4021541d5bf00c0fd8525f08 Mon Sep 17 00:00:00 2001 From: Romain Forlot Date: Fri, 23 Jun 2017 16:35:29 +0200 Subject: Update signaling chapter Change-Id: Icbaa18b740403a1e43ef6b61fbdc45ba3fad1437 Signed-off-by: Romain Forlot --- signaling/architecture.md | 558 +++++++++++++--------- signaling/high-level-viwi-service.md | 236 ++++++++++ signaling/images/CAN_bindings_communication.png | Bin 0 -> 43504 bytes signaling/images/CAN_level_mapping.png | Bin 0 -> 56087 bytes signaling/images/OpenXC_to_AGL.png | Bin 0 -> 84031 bytes signaling/images/can-generator.svg | 244 ++++++++++ signaling/images/distributed-arch.png | Bin 0 -> 73736 bytes signaling/images/high-level-arch.png | Bin 0 -> 30810 bytes signaling/images/iotbzh_logo_small.png | Bin 0 -> 6989 bytes signaling/images/logo_iot_bzh.svg | 142 ++++++ signaling/images/signal-agent-arch.svg | 439 ------------------ signaling/images/signal-service-arch.svg | 296 ++++++++++++ signaling/index.md | 26 +- signaling/low-level-can-service-guide.md | 584 ++++++++++++++++++++++++ 14 files changed, 1858 insertions(+), 667 deletions(-) create mode 100644 signaling/high-level-viwi-service.md create mode 100644 signaling/images/CAN_bindings_communication.png create mode 100644 signaling/images/CAN_level_mapping.png create mode 100644 signaling/images/OpenXC_to_AGL.png create mode 100644 signaling/images/can-generator.svg create mode 100644 signaling/images/distributed-arch.png create mode 100755 signaling/images/high-level-arch.png create mode 100644 signaling/images/iotbzh_logo_small.png create mode 100644 signaling/images/logo_iot_bzh.svg delete mode 100644 signaling/images/signal-agent-arch.svg create mode 100644 signaling/images/signal-service-arch.svg create mode 100644 signaling/low-level-can-service-guide.md (limited to 'signaling') diff --git a/signaling/architecture.md b/signaling/architecture.md index 47d11bc..8c6064d 100644 --- a/signaling/architecture.md +++ b/signaling/architecture.md @@ -14,10 +14,9 @@ layout: techdoc 1. TOC {:toc} - ## Context -Automotive applications need to understand in real time the context in which +Automotive applications need to understand in real time the context in which vehicles operate. In order to do so, it is critical for automotive application to rely on a simple, fast and secure method to access data generated by the multiple sensors/ECU embedded in modern cars. @@ -25,55 +24,82 @@ multiple sensors/ECU embedded in modern cars. This signaling problem is neither new, neither unique to the automotive and multiple solutions often described as Message Broker or Signaling Gateway have been around for a while. -In the context of Linux for Automotive, the "Automotive Message Broker Daemon -" (aka AMBd) has been around for few years and is pretty well known within the -Linux Automotive community. -The present discussion is a proposal to enhance existing signaling/message -models. Our proposal relies on [[APbinder]] binder/bindings model to minimize -complexity while keeping the system fast and secure. -We propose a model with multiple transport options and a full set of security -feature to protect the agent generating the signal as well as consuming them. +The present document is the now implemented since AGL Daring Dab version, to +handle existing signaling/message in a car. It relies on [[APbinder]] +binder/bindings model to minimize complexity while keeping the system fast +around secure. We propose a model with multiple transport options and a full set +of security feature to protect the service generating the signal as well as +consuming them. ## Objectives -Our objectives are to solve following 3 key issues: - -1. reduce as much as possible the amount of exchanged data to the meaningful subset really used by applications -2. offer a high level API that obfuscates low level and proprietary interface to improve stability in time of the code -3. hide specificities of low level implementation as well as the chosen deployment distribution model. +Our objectives are solving following 3 key issues: -To reach first objective, events emission frequency should be controlled at the lowest level it possibly can. -Aggregation, composition, treatment, filtering of signals should be supported at software level when -not supported by the hardware. +1. reduce as much as possible the amount of exchanged data to the meaningful + subset really used by applications +2. offer a high level API that obfuscates low level and proprietary interface to + improve stability in time of the code +3. hide specificities of low level implementation as well as the chosen + deployment distribution model. -Second objectives of offering long term stable hight level API while allowing flexibility in changing low level implementation -may look somehow conflicting. Nevertheless by isolating low level interface from high level and allowing dynamic composition -it is possible to mitigate both objectives. +To reach first objective, events emission frequency should be controlled at the +lowest level it possibly can. Aggregation, composition, treatment, filtering of +signals should be supported at software level when not supported by the hardware. +Second objectives of offering long term stable hight level API while allowing +flexibility in changing low level implementation may look somehow conflicting. +Nevertheless by isolating low level interface from high level and allowing +dynamic composition it is possible to mitigate both objectives. ## Architecture -Good practice is often based on modularity with clearly separated components assembled within a common framework. -Such modularity ensures separation of duties, robustness, resilience and achievable long term maintenance. - -This document uses the term "AGENT" to define a specific instance of this proposed common framework used to host a group -of dedicated separated components that handle targeted signals/events. Each agent exposes to services/applications -the signals/events it is responsible for. - -As an example, a CAN-bus agent may want to mix non-public proprietary API with CANopen compatible devices while hiding this -complexity to applications. The goal is on one hand to isolate proprietary piece of code in such a way that it is -as transparent as possible for the remaining part of the architecture. On a second hand isolation of code related -to a specific device provides a better separation of responsibilities, keeping all specificity related to a given -component clearly isolated and much easier to test or maintain. Last but not least if needed this model may also -help to provide some proprietary code directly as binary and not as source code. - -![image](./images/signal-agent-arch.svg "Signal Agent Architecture") - -While in some cases it may be chosen to implement a single agent responsible for everything, other scenarii may chose -to split responsibility between multiple agents. Those multiple agents may run on a single ECU or on multiple ECUs. -Chosen deployment distribution strategy should not impact the development of components responsible for signals/events -capture. As well as it should have a loose impact on applications/services consuming those events. +Good practice is often based on modularity with clearly separated components +assembled within a common framework. Such modularity ensures separation of +duties, robustness, resilience and achievable long term maintenance. + +This document uses the term "**Service**" to define a specific instance of this +proposed common framework used to host a group of dedicated separated components +that handle targeted signals/events. Each service exposes to services/applications +the signals/events it is responsible for. + +As an example, a CAN service may want to mix non-public proprietary API with +CANopen compatible devices while hiding this complexity to applications. The +goal is on one hand to isolate proprietary piece of code in such a way that it +is as transparent as possible for the remaining part of the architecture. On a +second hand isolation of code related to a specific device provides a better +separation of responsibilities, keeping all specificity related to a given +component clearly isolated and much easier to test or maintain. Last but not +least if needed this model may also help to provide some proprietary code +directly as binary and not as source code. + +Communicating between the car and regular apps should be done using a 2 levels +AGL services which have two distincts roles: + +- low level should handle communication with CAN bus device (read, decoding, + basic and efficient filtering, caching, ...) +- high level should handle more complex tasks (signals compositions, complex + algorythms like Kalman filter, business logic...) + +![image](./images/signal-service-arch.svg "Signal Agent Architecture") + +To do so, the choice has been to use a similar architecture than [[OpenXC]], a +Ford project. Principle is simple, from a JSON file that describes all CAN +signals wanted to be handled, in general a conversion from a **dbc** file, AGL +generator convert it to a C++ source code file. This file which in turn is used +as part of the low level CAN service which can now be compiled. This service +reads, decodes and serves this CAN signals to a high level CAN service that +holds business logic and high level features like described is the above +chapter. + +![image](./images/can-generator.svg "AGL CAN generator") + +While in some cases it may be chosen to implement a single service responsible +for everything, other scenarii may chose to split responsibility between +multiple services. Those multiple services may run on a single ECU or on +multiple ECUs. Chosen deployment distribution strategy should not impact the +development of components responsible for signals/events capture. As well as it +should have a loose impact on applications/services consuming those events. A distributed capable architecture may provide multiple advantages: @@ -82,16 +108,24 @@ A distributed capable architecture may provide multiple advantages: - it simplifies security by enabling isolation and sandboxing - it clearly separates responsibilities and simplifies resolution of conflicts -![image](./images/distributed-arch.svg "Distributed Architecture") +Distributed architecture has to be discussed and about now is not fully +implemented. Low level CAN service isn't fully functional nor tested to assume +this feature but its architecture let the possibility open and will be +implemented later. + +![image](./images/distributed-arch.png "Distributed Architecture") Performance matters. There is a trade-off between modularity and efficiency. -This is specially critical for signals where propagation time from one module to the other -should remain as short as possible and furthermore should consume as little computing resources as possible. +This is specially critical for signals where propagation time from one module to +the other should remain as short as possible and furthermore should consume as +little computing resources as possible. -A flexible solution should provide enough versatility to either compose modules in separate processes; -either chose a model where everything is hosted within a single process. Chosen deployment model should have minor -or no impact on development/integration processes. Deployment model should be something easy to change, it should remain -a tactical decision and never become a structuring decision. +A flexible solution should provide enough versatility to either compose modules +in separate processes; either chose a model where everything is hosted within a +single process. Chosen deployment model should have minor or no impact on +development/integration processes. Deployment model should be something easy to +change, it should remain a tactical decision and never become a structuring +decision. Nevertheless while grouping modules may improve performance and reduce resource consumption, on the other hand, it has a clear impact on security. No one should forget that some signals have very different level of security from other ones. @@ -107,21 +141,28 @@ Providing such flexibility constrains the communication model used by modules: implementation shouldn't impact how it is used - The cost of the abstraction for modules grouped in a same process must be as little as possible -- The cost of separating modules with the maximum of security must remain as minimal as possible - -Another point impacting performance relates to a smart limitation on the number of emitted signals. -Improving the cost of sending a signal is one thing, reducing the number of signals is an other one. -No one should forget that the faster you ignore a useless signal the better it is. -The best way to achieve this is by doing the filtering of useless signal as close as possible -of the component generating the signal and when possible directly at the hardware level. - -To enable the right component to filter useless signals, consumer clients must describe -precisely the data they need. Then services and agent must organise their computing to provide -only expected data at the expected rate. Also, any data not required by any client should at -the minimum never be transmitted. Furthermore when possible then should even not be computed at all. - -Describing expected data in a precise but nevertheless simple manner remains a challenge. -It implies to manage: +- The cost of separating modules with the maximum of security must remain as + minimal as possible + +Another point impacting performance relates to a smart limitation on the number +of emitted signals. Improving the cost of sending a signal is one thing, +reducing the number of signals is an other one. No one should forget that the +faster you ignore a useless signal the better it is. The best way to achieve +this is by doing the filtering of useless signal as close as possible of the +component generating the signal and when possible directly at the hardware level. + +To enable the right component to filter useless signals, consumer clients must +describe precisely the data they need. A filter on frequency is provided since +Daring Dab version, as well as minimum and maximum limits. These filters can be +specified at subscription time. Also, any data not required by any client should +at the minimum never be transmitted. So only changed data is transmitted and if +another service needs to receive at a regular time, it has to assume that if no +events are received then it is that the value hasn't change. Furthermore when +possible then should even not be computed at all, a CAN signal received on +socket is purely ignored if no one asks for it. + +Describing expected data in a precise but nevertheless simple manner remains a +challenge. It implies to manage: - requested frequency of expected data - accuracy of data to avoid detection of inaccurate changes @@ -132,225 +173,289 @@ It implies to manage: subtracting, running logical operators like AND/OR/XOR, getting the mean, ...) - etc... -It is critical to enable multiple features in signal queries to enable modules -to implement the best computing method. The best computing method may have an impact on -which device to query as well as on which filters should be applied. Furthermore -filtering should happen as soon as possible and obviously when possible directly at hardware level. - -## Leveraging Existing solutions - -Unfortunately, there is no existing solution ready to be integrated. Obviously some parts of required technologies to -implement such solution exist, but a significant part of the work remains to be done. If we look at the main building -blocks used in the proposed architecture. +It is critical to enable multiple features in signal queries to enable modules +to implement the best computing method. The best computing method may have an +impact on which device to query as well as on which filters should be applied. +Furthermore filtering should happen as soon as possible and obviously when +possible directly at hardware level. ### Transport Solutions -D-Bus is the standard choice for Linux, nevertheless it has some serious performance limitation due to -internal verbosity. Nevertheless because it is available and pre-integrated with almost every Linux component, D-Bus may -still remains an acceptable choice for signal with low rate of emission (i.e. HMI). +D-Bus is the standard choice for Linux, nevertheless it has some serious +performance limitation due to internal verbosity. Nevertheless because it is +available and pre-integrated with almost every Linux component, D-Bus may still +remains an acceptable choice for signal with low rate of emission (i.e. HMI). -For a faster communication, Jaguar-Land-Rover proposes a memory shared signal infrastructure. -Unfortunately this solution is far from solving all issues and has some drawbacks. -Let check the open issues it has: +For a faster communication, Jaguar-Land-Rover proposes a memory shared signal +infrastructure. Unfortunately this solution is far from solving all issues and +has some drawbacks. Let check the open issues it has: - there is no management of what requested data are. This -translate in computing data even when not needed. -- on top of shared memory, an extra side channel is required for processes to communicate with the -daemon. + translate in computing data even when not needed. +- on top of shared memory, an extra side channel is required for processes + to communicate with the daemon. - a single shared memory implies a lot of concurrency handling. This might -introduce drawbacks that otherwise would be solved through communication buffering. - -ZeroMQ, NanoMSG and equivalent libraries focused on fast communication. Some (e.g. ZeroMQ) come with -a commercial licensing model when others (e.g. NanoMSG) use an open source licensing. -Those solutions are well suited for both communicating inside a unique ECU or across several ECUs. -However, most of them are using Unix domain sockets and TCP sockets and typically do not use shared memory for inter-process communication. - -Last but not least Android binder, Kdbus and other leverage shared memory, zero copy and sit directly within Linux kernel. -While this may boost information passing between local processes, it also has some limitations. The first one is -the non support of a multi-ECU or vehicle to cloud distribution. The second one is that none of them is -approved upstream in kernel tree. This last point may create some extra burden each time a new version -of Linux kernel is needed or when porting toward a new hardware is required. + introduce drawbacks that otherwise would be solved through communication + buffering. + +ZeroMQ, NanoMSG and equivalent libraries focused on fast communication. Some +(e.g. ZeroMQ) come with a commercial licensing model when others (e.g. NanoMSG) +use an open source licensing. Those solutions are well suited for both +communicating inside a unique ECU or across several ECUs. However, most of them +are using Unix domain sockets and TCP sockets and typically do not use shared +memory for inter-process communication. + +Last but not least Android binder, Kdbus and other leverage shared memory, zero +copy and sit directly within Linux kernel. While this may boost information +passing between local processes, it also has some limitations. The first one is +the non support of a multi-ECU or vehicle to cloud distribution. The second one +is that none of them is approved upstream in kernel tree. This last point may +create some extra burden each time a new version of Linux kernel is needed or +when porting toward a new hardware is required. ### Query and Filtering Language -Description language for filtering of expected data remains an almost green field where nothing really fit signal agent requirements. -Languages like Simulink or signal processing graphical languages are valuable modelling tools. Unfortunately they -cannot be inserted in the car. Furthermore those languages have many features that are not useful in proposed signal agent -context and cost of integrating such complex languages might not be justified for something as simple as a signal agent. -The same remarks apply for automation languages. - -Further investigations would be needed. Even if there is no obvious solutions fit or need. Existing solution use a different context or industry may exist or nearby solutions -might worse the value be adapted to our requirements. +Description language for filtering of expected data remains an almost green +field where nothing really fit signal service requirements. Languages like +Simulink or signal processing graphical languages are valuable modelling tools. +Unfortunately they cannot be inserted in the car. Furthermore those languages +have many features that are not useful in proposed signal service context and +cost of integrating such complex languages might not be justified for something +as simple as a signal service. The same remarks apply for automation languages. + +Further investigations leads to some specifications already presents like the +one from Jaguar Land Rover [[VISS]], for **Vehicule Information Service +Specification** and another from Volkwagen AG named [[ViWi]], stand for +**Volkwagen Infotainment Web Interface**. Each ones has their pro and cons. +Let's see a little comparison: + +TODO: List of features not pro cons +| VISS | ViWi | +|-------------------------------------------------------|-------------------------------------------------------------------| +| + Filtering on node (not possible on several nodes or branches) | | + Describe a protocol | +| + Access restrictions to signals | + Ability to specify custom signals | +| +/- Use high level development languages | + RESTful HTTP calls | +| - One big Server that handle requests | + Stateless | +| - Filtering | + Filtering, sorting | +| - Static signals tree not extensible [[VSS]] | -/+ Use JSON objects to communicate | +| - Use of AMB ? | - Identification of resources may be a bit heavy going using UUID | +| - Use of Websocket | | + +About **[[VISS]]** specification, the major problem comes from the fact that +signals are specified under the [[VSS]], **Vehicle Signal Specification**. So, +problem is that it is difficult, if not impossible, to make a full inventory +of all signals existing for each car. More important, each evolution in signals +must be reported in the specification and it is without seeing the fact that +car makers have their names and set of signals that would mostly don't +comply with the [[VSS]]. VISS doesn't seems to be an valuable way to handle +car's signals, a big component that responds requests, use of **Automotive +Message Broker** that use DBus is a performance problem. Fujitsu Ten recent +study[[1]] highlights that processor can't handle an heavy load on CAN bus and +that Low level binding adopted for AGL is about 10 times[[2]] less impact on +performance. ## Describing Signal Subscriptions using JSON -JSON is a rich structured representation of data. For requested data, it allows the expression of multiple features and constraints. -JSON is both very flexible and efficient. -There are significant advantages in describing requested data at subscription time using a language like JSON. -Another advantage of JSON is that no parser is required to analyse the request. +JSON is a rich structured representation of data. For requested data, it allows +the expression of multiple features and constraints. JSON is both very flexible +and efficient. There are significant advantages in describing requested data at +subscription time using a language like JSON. Another advantage of JSON is that +no parser is required to analyse the request. -Here is an example of what could be a subscription specification: +Existing works exists to describe a signals that comes first from Vector with +its proprietary database (`DBC`) which widely used in industry. Make a +description based on this format appears to be a good solution and Open Source +community already has existing tools that let you convert proprietary file +format to an open one. So, a JSON description based on work from [[OpenXC]] is +specified [here](https://github.com/openxc/vi-firmware/blob/master/docs/config/reference.rst) +which in turn is used in Low level CAN service in AGL: ```json -{ - "define": { - "speed": { - "signal": "average-wheel-speed", - "unit": "km/h", - "accuracy": 0.5, - "filtered": "yes" - }, - "expect": { - "value": "speed", - "when": { - "cross": { "speed": [ 50, 90, 110, 130 ] } - }, - "max-rate": 1 - } +{ "name": "example", + "extra_sources": [], + "initializers": [], + "loopers": [], + "buses": {}, + "commands": [], + "0x3D9": { + "bus": "hs", + "signals": { + "PT_FuelLevelPct": { + "generic_name": "fuel.level", + "bit_position": 8, + "bit_size": 8, + "factor": 0.392157, + "offset": 0 + }, + "PT_EngineSpeed": { + "generic_name": "engine.speed", + "bit_position": 16, + "bit_size": 16, + "factor": 0.25, + "offset": 0 + }, + "PT_FuelLevelLow": { + "generic_name": "fuel.level.low", + "bit_position": 55, + "bit_size": 1, + "factor": 1, + "offset": 0, + "decoder": "decoder_t::booleanDecoder" + } + } + } } ``` -A generic library to handle such request definition should be provided. -This library should allow the agent to process the request, search if its business logic or any of present low level modules support requested signal. Finally it should notify the concerned lower module to optimise signal generation. - -While such an approach might not be the most common approach, organisation like [Signal-K] already uses JSON to transmit NMEA-2000 signals on boats. Others might already used it as well and some further investigations on existing solutions would be needed before moving in such a direction. +From a description like the above one, low level CAN generator will output +a C++ source file which let low level CAN service that uses it to handle such +signal definition. ## Naming Signal -Naming and defining signals is something very complex. -For example just ***speed***, as a signal, is difficult to define. +Naming and defining signals is something very complex. For example just +***speed***, as a signal, is difficult to define. What unit is used (km/h, M/h, m/s, ...)? From which source (wheels, GPS, AccelMeter)? How was it captured (period of measure, instantaneous, mean, filtered)? -In order to simplify application development we should nevertheless agree on some naming convention for key signals. -Those names might be relatively complex and featured. They may include a unit, a rate, a precision, etc. +In order to simplify application development we should nevertheless agree on +some naming convention for key signals. Those names might be relatively complex +and featured. They may include a unit, a rate, a precision, etc. + +How these names should be registered, documented and managed is out of scope of +this document but extremely important and at some point in time should be +addressed. Nevertheless this issue should not prevent from moving forward +developing a modern architecture. Developers should be warned that naming is a +complex task, and that in the future naming scheme should be redefined, and +potential adjustments would be required. + +About Low level CAN signals naming a doted notation, like the one used by Jaguar +Landrover, is a good compromise as it describe a path to an car element. It +separates and organize names into hierarchy. From the left to right, you +describe your names using the more common ancestor at the left then more you go +to the right the more it will be accurate. Using this notation let you subscribe +or unsubscribe several signals at once using a globbing expression. + +Example using OBD2 standard PID: + +```path +engine.load +engine.coolant.temperature +fuel.pressure +intake.manifold.pressure +engine.speed +vehicle.speed +intake.air.temperature +mass.airflow +throttle.position +running.time +EGR.error +fuel.level +barometric.pressure +commanded.throttle.position +ethanol.fuel.percentage +accelerator.pedal.position +hybrid.battery-pack.remaining.life +engine.oil.temperature +engine.torque +``` + +Here you can chose to subscribe to all engine component using an expression +like : `engine.*` + +## Reusing existing/legacy code + +About now provided services use: + +- **Low Level** [[OpenXC]] project provides logic and some useful libraries to + access a CAN bus. It is the choice for AGL. -How these names should be registered, documented and managed is out of scope of this document but -extremely important and at some point in time should be addressed. Nevertheless this issue should not prevent -from moving forward developing a modern architecture. -Developers should be warned that naming is a complex -task, and that in the future naming scheme should be redefined, and potential adjustments would be required. +- **High Level** In many cases accessing to low level signal is not enough. + Low level information might need to be composed (i.e. GPS+Gyro+Accel). + Writing this composition logic might be quite complex and reusing existing + libraries like: LibEkNav for Kalman filtering [[9]] or Vrgimbal for 3 axes + control[[10]] may help saving a lot of time. AGL apps should access CAN + signals through High Level service. High level can lean on as many low level + service as needed to compute its **Virtual signals** coming from differents + sources. Viwi protocol seems to be a good solution. ## Leveraging AGL binder -Such a model is loosely coupled with AGL binder. Low level signal modules as well as virtual signal -components may potentially run within any hosting environment that would provide the right API with corresponding required facilities. -Nevertheless leveraging [[APbinder]] has multiple advantages. It already implements event notification to support -a messaging/signaling model for distributed agents. It enables a subscribe model responding to the requirement and finally it uses JSON -natively. +Such a model is loosely coupled with AGL binder. Low level CAN service as well +as virtual signal components may potentially run within any hosting environment +that would provide the right API with corresponding required facilities. +Nevertheless leveraging [[APbinder]] has multiple advantages. It already +implements event notification to support a messaging/signaling model for +distributed services. It enables a subscribe model responding to the +requirement and finally it uses JSON natively. -This messaging/signalling model already enforces the notion of subscription for receiving data. -It implies that unexpected data are not sent and merely not computed. -When expected data is available, it is pushed to all waiting subscriber only one time. +This messaging/signalling model already enforces the notion of subscription for +receiving data. It implies that unexpected data are not sent and merely not +computed. When expected data is available, it is pushed to all waiting +subscriber only one time. The [[APbinder]] provides transparency of communication. It currently implements the transparency over D-Bus/Kdbus and WebSocket. Its transparency mechanism of communication is easy to extend to other technologies: pools of shared memory or any proprietary transport model. -When modules are loaded by the same service agent, the [[APbinder]] provides transparently "in-memory" communication. -This in-memory communication is really efficient: on one hand, the exchanged -JSON objects are not serialized (because not streamed), on the other hand, -those JSON objects provide a high level of abstraction able to transfer any -data. +When bindings/services are loaded by the same binder, it provides transparently +`in-memory` communication. This in-memory communication is really efficient: on +one hand, the exchanged JSON objects are not serialized (because not streamed), +on the other hand, those JSON objects provide a high level of abstraction able +to transfer any data. -Technically an agent is a standard [[APbinder]] service. +Technically a service is a standard [[APbinder]] binding which is also handled +by the system and launched as a daemon by systemD. Therefore Signal/Agent inherits of security protection through SMACK, access control through Cynara, transparency of API to transport layer, life cycle -management, ... Like any other [[APbinder]] service Signal/Agent is composed of a set of bindings. In signal agent specific case, those bindings are in fact the "signal modules". +management, ... Like any other [[APbinder]] process is composed of a set of +bindings. In signal service specific case, those bindings are in fact the +`signal modules`. -The proposed model allows to implement low level dependencies as independent signal modules. -Those modules when developed are somehow like "Lego Bricks". -They can be spread or grouped within one or multiple agents depending on -deployment constraints (performance, multi-ECU, security & isolation constraints, -...). +The proposed model allows to implement low level dependencies as independent +signal modules. Those modules when developed are somehow like "Lego Bricks". +They can be spread or grouped within one or multiple services depending on +deployment constraints (performance, multi-ECU, security & isolation +constraints,...). + +On top of that low level signal modules, you should use a high level service. +A first implementation of [[ViWi]] is available [here](https://github.com/iotbzh/high-level-viwi-service) +and can be use to integrate business logic and high level features. The model naturally uses JSON to represent data. -**Multi-ECU and Vehicule to Cloud interactions:** while this might not be a -show stopper for current projects, it is obvious that in the near future -Signal/Agent should support a fully distributed architectures. -Some event may come from the cloud (i.e. request to start monitoring a given -feature), some may come from SmartCity and nearby vehicles, and last but -not least some may come from another ECU within the same vehicle or from a -virtualized OS within the same ECU (e.g. cluster & IVI). -In order to do so, Signal modules should enable composition within one or more -[[APbinder]] inside the same ECU. -Furthermore they should also support chaining with the outside world. +## Multi-ECU and Vehicule to Cloud interactions + +While this might not be a show stopper for current projects, it is obvious that +in the near future Signal/Agent should support a fully distributed +architectures. Some event may come from the cloud (i.e. request to start +monitoring a given feature), some may come from SmartCity and nearby vehicles, +and last but not least some may come from another ECU within the same vehicle or +from a virtualized OS within the same ECU (e.g. cluster & IVI). In order to do +so, Signal modules should enable composition within one or more [[APbinder]] +inside the same ECU. Furthermore they should also support chaining with the +outside world. -![image](./images/cloud-arch.svg "CLoud & Multi-ECU Architecture") +![image](./images/cloud-arch.svg "Cloud & Multi-ECU Architecture") 1. Application requests Virtual Signal exactly like if it was a low level signal 2. Agent Signal has direct relation to low level signal -3. Agent needs to proxy to an other agent inside the same ECU to access the +3. Agent needs to proxy to an other service inside the same ECU to access the signal 4. Signal is not present on current ECU. Request has to be proxied to the outside world -## Work to be Done - -As stated before, the foundation for event signaling is already in place in -development branch of [[APbinder]]. -Nevertheless in order to provide a real added value to developers missing -components should be developed. - -- **Agent Core Components** - - Create a simple configuration model to move from low level signal - description to something more generic that could help to abstract the - hardware. - - Search for an already existing library that provide high level signal - subscription and adapt it. - If no such library exist, create it. - - Implement some form of introspection to allow application to discover - the list of signal supported by a given agent. -- **Few Common Agents** - - Provide at least a reference implementation for a generic CAN agent - - Agent simulation, most developers may not have access to physical - devices during development phase, some form of simulator would be more - than welcome. - - etc ... - -## Reusing existing/legacy code - -Whatever the final architecture will be, it is important not to forget how -to leverage existing/legacy code within the new model. -There are multiple sources for reusable code that can be classified in two -main categories: - -- **Low Level** Typically those existing code interface with CAN-BUS or any - other low level device. - Some AMBd[[6]] plugins could be a good source of input for low level - implementation. - But also more community oriented projects like: ArduPilot/Rover[[7]] - Barometer or Camera interfaces[[8]]. - -- **Virtual Signal** In many cases accessing to low level signal is not enough. - Low level information might need to be composed (i.e. GPS+Gyro+Accel). - Raw value might need to be processed (i.e.: change unit, average on the - last xx seconds, ...). - Writing this composition logic might be quite complex and reusing existing - libraries like: LibEkNav for Kalman filtering [[9]] or Vrgimbal for 3 axes - control[[10]] may help saving a lot of time. - -## Proposal - -In order to start moving forward one option would be to take a concrete example -with some existing can devices (i.e. the one from MicroChips use within ALS demo). - -An other option would be to leverage Porter+Cogent-Extension[[11]] that supports -both a GPS and a 9 axis sensor to implement a smart geoposition agent. -We could reuse some existing code like the one from Rover[[7]] to provide a -fine grain positioning service and then interface this new GeoPositioning -agent with an application like GpsNavi[[12]] to get a end-to-end demonstrator. - [AppFw]: http://iot.bzh/download/public/2016/appfw/01_Introduction-to-AppFW-for-AGL-1.0.pdf "Application Framework" [APcore]: http://iot.bzh/download/public/2016/appfw/03_Documentation-AppFW-Core-1.0.pdf "AppFw Core" [APmain]: https://gerrit.automotivelinux.org/gerrit/#/q/project:src/app-framework-main "AppFw Main" [APbinder]: https://gerrit.automotivelinux.org/gerrit/#/q/project:src/app-framework-binder "AppFw Binder" [APsamples]: https://gerrit.automotivelinux.org/gerrit/gitweb?p=src/app-framework-binder.git;a=tree;f=bindings/samples "AppFw Samples" [Signal-K]: http://signalk.org/overview.html +[1]: http://schd.ws/hosted_files/aglmmwinter2017/37/20170201_AGL-AMM_F10_kusakabe.pdf +[2]: https://wiki.automotivelinux.org/_media/agl-distro/20170402_ften_can_kusakabe_v2.pdf [6]: https://github.com/otcshare/automotive-message-broker [7]: http://ardupilot.org/rover/index.html [8]: https://github.com/ArduPilot/ardupilot/tree/master/libraries @@ -358,4 +463,9 @@ agent with an application like GpsNavi[[12]] to get a end-to-end demonstrator. [10]: http://ardupilot.org/rover/docs/common-vrgimbal.html [11]: http://elinux.org/R-Car/Boards/Porter:PEXT01 [12]: https://github.com/gpsnavi/gpsnavi - +[VISS]: http://rawgit.com/w3c/automotive/gh-pages/vehicle_data/vehicle_information_service.html +[VSS]: https://github.com/GENIVI/vehicle_signal_specification +[ViWi]: https://www.w3.org/Submission/2016/SUBM-viwi-protocol-20161213/ +[OpenXC]: http://openxcplatform.com/ +[low level CAN service]: https://gerrit.automotivelinux.org/gerrit/#/admin/projects/src/low-level-can-generator +[high level ViWi]: https://github.com/iotbzh/high-level-viwi-service \ No newline at end of file diff --git a/signaling/high-level-viwi-service.md b/signaling/high-level-viwi-service.md new file mode 100644 index 0000000..6b512b9 --- /dev/null +++ b/signaling/high-level-viwi-service.md @@ -0,0 +1,236 @@ +# AGL VIWI HIGH-CAN binding architecture + +This binding is intended to act between low-level binding(s) and clients. It +builds ViWi resources as defined in a json configuration file. It implements +subscribe/unsubscribe/get verbs for the clients accordingly with protocol +specification. + +Each ViWi resource can be composed of several elements, for which subscriptions +will be made to the low-level binding with configurable frequencies or filters. + +![ViWi High Level binding architecture](images/high-level-arch.png) + +# BRIEF VIWI DESCRIPTION + +ViWi (Volkswagen Infotainment Web Interface) protocol defines a serie of +objects, which can be queried or updated via JSon messages. + +Each object is assigned with a unique URI. + +The depth of the URI tree is limited to 3, i.e. +_/service/resource>/element/_, for instance +**/car/doors/3901a278-ba17-44d6-9aef-f7ca67c04840**. + +To retrieve the list of elements for a given resource, one can use the get +command, for instance **get /car/doors/**. + +It is also possible to subscribe to elements or group of elements, for +instance **subscribe /car/doors/3901a278-ba17-44d6-9aef-f7ca67c04840**. +Requests can also have various filters, or specify a frequency. + +More details in the [ViWi general documentation](https://www.w3.org/Submission/viwi-protocol/) +and in the [ViWi.service.car documentation](https://www.w3.org/Submission/viwi-service-car/) + +# Installation + +## Prerequisites + +Low level CAN service (>=4.0) must be installed. Prerequisites are the same. +Follow instructions from **Low level CAN service Guide**. + +```bash +git clone --recursive https://gerrit.automotivelinux.org/gerrit/apps/low-level-can-service +``` + +## Clone + +```bash +export WD=$(pwd) +git clone --recusive https://github.com/iotbzh/high-level-viwi-service.git +``` + +## Build + +Use the classic build flow: + +```bash +cd $WD/high-level-viwi-service +mkdir build +cd build +cmake .. +make +``` + +# Usage + +## JSON configuration file + +This file must be named *high.json*, and must accessible from afb-daemon. + +The json configuration file consists in 2 sections: + +### Definitions section + +This section describes each resources defined in the high-level binding. Each +resource is composed with different properties having a name, a type and a +description. + +Type can be boolean, double, string, or int. Properties "id", "uri" and "name" +are compulsory. + +For instance: + +```json +{ + "name": "/car/demoboard/", + "properties": { + "id": { + "type": "string", + "description": "identifier" + }, + "uri": { + "type": "string", + "description": "object uri" + }, + "name": { + "type": "string", + "description": "name" + }, + "unit": { + "type": "string", + "description": "units" + }, + "speed": { + "type": "double", + "description": "vehicle centerpoint speed as shown by the instrument cluster" + }, + "rpm": { + "type": "double", + "description": "engine rotations per minute" + }, + "level": { + "type": "double", + "description": "level of tankage" + }, + "load": { + "type": "double", + "description": "engine load" + } + } +} +``` + +### Resources section + +This section defines which values should be assigned to resource's properties +as defined in the definitions section. The link to the definitions section is +made through the name of the resource. + +Some values are static, some are linked to low-level requests. + +In case a value is linked to a low-level request, the value will start with +"${" and end with "}". In that case the value will consist in the name of the +low-level signal, followed with the frequency of the signal in ms. -1 in the +frequency means that high level binding should subscribe to low level binding +for all changes, without specifying a frequency. + +For instance: + +```json +{ + "name": "/car/demoboard/", + "values": [{ + "name": "vehicleSpeed", + "unit": "km/h", + "speed": "${diagnostic_messages.vehicle.speed,1000}" + }, { + "name": "engineSpeed", + "unit": "rpm", + "rpm": "${diagnostic_messages.engine.speed,1000}" + }, { + "name": "fuelLevel", + "unit": "litre", + "level": "${diagnostic_messages.fuel.level,1000}" + }, { + "name": "engineLoad", + "unit": "Nm", + "load": "${diagnostic_messages.engine.load,1000}" + }] +} +``` + +## Running and testing + +### Launch the binder together with the two bindings + +The Json high level configuration file *high.json* must be placed in the +directory where you launch afb-daemon. + +```bash +cp $WD/high-level-viwi-service/high.json $WD + cd $WD +``` + +Then you can natively under linux you can launch afb-daemon with the low-level +and high-level bindings with a command like: + +```bash +cd $WD +afb-daemon --rootdir=$WD/low-level-can-service/CAN-binder/build/package --binding=$WD/low-level-can-service/CAN-binder/build/package/lib/afb-low-can.so --binding=$WD/high-level-viwi-service/build/package/lib/afb-high-can.so --port=1234 --tracereq=common --token=1 --verbose +``` + +### Use afb-client-demo to test high level binding + +On another terminal, connect to the binding using previously installed +_**AFB Websocket CLI**_ tool: + +```bash +$ afb-client-demo ws://localhost:1234/api?token=1 +``` + +You will be on an interactive session where you can communicate directly with +the binding API. + +The binding provides at this moment 3 verbs, _get_, _subscribe_ and +_unsubscribe_, which can take a JSON object as an argument. + +To use the _**AFB Websocket CLI**_ tool, a command line will be like the +following: + +``` + +``` + +Where: + +* API : _**high-can**_. +* Verb : _**get**_, _**subscribe**_ or _**unsubscribe**_ +* Arguments : _**{ "name": "/car/doors/" }**_ + +You can therefore use commands such as: + +``` +high-can subscribe {"name":"/car/doors/","interval":10000} +high-can unsubscribe {"name":"/car/doors/","interval":10000} +high-can get {"name":"/car/demoboard/"} +high-can get {"name":"/car/demoboard/","fields":["fuelLevel","engineLoad"]} +``` + +For instance the output of the third command should be: + +``` +high-can get {"name":"/car/demoboard/"} +ON-REPLY 1:high-can/get: {"response":{"\/car\/demoboard\/2159e2-5b638a-39e242-7a2f5":{"id":"2159e2-5b638a-39e242-7a2f5","name":"vehicleSpeed","speed":0.000000,"unit":"km\/h","uri":"\/car\/demoboard\/2159e2-5b638a-39e242-7a2f5"},"\/car\/demoboard\/22ad2c-5a3c2b-50fabb-324c82":{"id":"22ad2c-5a3c2b-50fabb-324c82","level":0.000000,"name":"fuelLevel","unit":"litre","uri":"\/car\/demoboard\/22ad2c-5a3c2b-50fabb-324c82"},"\/car\/demoboard\/3a3ab9-2bd52c-11d30-689acf":{"id":"3a3ab9-2bd52c-11d30-689acf","name":"engineSpeed","rpm":0.000000,"unit":"rpm","uri":"\/car\/demoboard\/3a3ab9-2bd52c-11d30-689acf"},"\/car\/demoboard\/5ae808-8093cb-99716-30a605":{"id":"5ae808-8093cb-99716-30a605","load":0.000000,"name":"engineLoad","unit":"Nm","uri":"\/car\/demoboard\/5ae808-8093cb-99716-30a605"}},"jtype":"afb-reply","request":{"status":"success","uuid":"44ce03f9-a7ca-49e1-a62a-40c74db0caa0"}} +``` + +As you can see for the moment all values are 0, because we didn't inject any +CAN data in the binder. To do this, you can use **canplayer** to feed the +bindings with some data. + +You can find an example of data in high level binding, "samples" directory. + +For instance, on a third terminal: + +```bash +$ canplayer -I candata +``` \ No newline at end of file diff --git a/signaling/images/CAN_bindings_communication.png b/signaling/images/CAN_bindings_communication.png new file mode 100644 index 0000000..426e2f7 Binary files /dev/null and b/signaling/images/CAN_bindings_communication.png differ diff --git a/signaling/images/CAN_level_mapping.png b/signaling/images/CAN_level_mapping.png new file mode 100644 index 0000000..1f1d906 Binary files /dev/null and b/signaling/images/CAN_level_mapping.png differ diff --git a/signaling/images/OpenXC_to_AGL.png b/signaling/images/OpenXC_to_AGL.png new file mode 100644 index 0000000..6e40336 Binary files /dev/null and b/signaling/images/OpenXC_to_AGL.png differ diff --git a/signaling/images/can-generator.svg b/signaling/images/can-generator.svg new file mode 100644 index 0000000..f5a567c --- /dev/null +++ b/signaling/images/can-generator.svg @@ -0,0 +1,244 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + CAN Low Level Bindng(Shared Library) + + + + + + + + + + + + + + OPENXC SignalDescription(JSON) + + + + + + + + + + + + + + + + + + + + + Low LevelBindingStatic Code(AGL) + + + + + + + + + + + + + + CANDecoding/EncodingC++ Code(vendor) + + + + + + + + + + + + + + OptionalMessageHandlers(vendor) + + + + + + + + + + + + + + + Call for decode/encode + + + + + + + + + + CANConfigGenerator + + + + + + + + + + C/C++Compiler + + + + + + + + + + + + + + OPENXC SignalDescription(JSON) + + + + + + + + + + + + + + OPENXC SignalDescription(JSON) + + + + + + + + \ No newline at end of file diff --git a/signaling/images/distributed-arch.png b/signaling/images/distributed-arch.png new file mode 100644 index 0000000..3c4f4a0 Binary files /dev/null and b/signaling/images/distributed-arch.png differ diff --git a/signaling/images/high-level-arch.png b/signaling/images/high-level-arch.png new file mode 100755 index 0000000..b4a1e8c Binary files /dev/null and b/signaling/images/high-level-arch.png differ diff --git a/signaling/images/iotbzh_logo_small.png b/signaling/images/iotbzh_logo_small.png new file mode 100644 index 0000000..6a98c60 Binary files /dev/null and b/signaling/images/iotbzh_logo_small.png differ diff --git a/signaling/images/logo_iot_bzh.svg b/signaling/images/logo_iot_bzh.svg new file mode 100644 index 0000000..aa23e84 --- /dev/null +++ b/signaling/images/logo_iot_bzh.svg @@ -0,0 +1,142 @@ + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + IOT + + BZH + + + + diff --git a/signaling/images/signal-agent-arch.svg b/signaling/images/signal-agent-arch.svg deleted file mode 100644 index 1a8677e..0000000 --- a/signaling/images/signal-agent-arch.svg +++ /dev/null @@ -1,439 +0,0 @@ - -image/svg+xmlGeneric CAN-Bus Signal Agent -Can Virtual Signal(Business Logic) -Transport + Access Control -SignalsSubscription -Module-A: Open-APIs CAN-bus Dev-ABC -isDoorOpendimInternalLightOutetc... -GetPositionGetSpeedIsMoving... -Common Signal Agent Framework -Module-B: Privat-APIs CAN-bus Dev-XX -Module-C: Privat-APIs CAN-bus Dev-YY - \ No newline at end of file diff --git a/signaling/images/signal-service-arch.svg b/signaling/images/signal-service-arch.svg new file mode 100644 index 0000000..3dee802 --- /dev/null +++ b/signaling/images/signal-service-arch.svg @@ -0,0 +1,296 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Binder + + + + + + + + CAN Low Level Binding(s)Decoding / EncodingAuthentication / Crypto / FirewallingTransaction (set… ack ...)Stats & MathsCaching (low freq. Signals, get() call)Debug + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + CAN High Level Binding(s)LogicAggregation (« vehicle.doors.any.open »)Advanced Ops + + + + + + + + + + + + + + + + + + + + + + + + + CAN BUS + + + + + + + + + + + + + + + + CAN frames - 011010010 + + + + + + + + + Signals - « vehicle.doors.left.open »(Binder Events) + + + + + + + + UI + + + + + + + + Publish Subscribe + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/signaling/index.md b/signaling/index.md index d3ebcd0..cb466ae 100644 --- a/signaling/index.md +++ b/signaling/index.md @@ -1,8 +1,8 @@ -# AGL Message Signaling (proposal) +# AGL Message Signaling -## Architecture Proposal +## Architecture -This [document](./architecture.md) presents a proposal of architecture for message signaling in AGL. +This [document](./architecture.md) presents an architecture for message signaling in AGL. Also available as a [PDF Document](http://iot.bzh/download/public/2016/signaling/AGL-Message-Signaling-Architecture.pdf) @@ -18,7 +18,19 @@ A reference HTML5 application has been developed: see [github:iotbzh/txc-demo](h This application uses a [OpenXC trace file](http://openxcplatform.com/resources/traces.html) to display 4 different panels representing live vehicle data. -It's available as an [AGL Application paclage](http://iot.bzh/download/public/2016/afb-demos/txc-demo_0.1.wgt) installable through AGL Application Framework. +It's available as an [AGL Application package](http://iot.bzh/download/public/2016/afb-demos/txc-demo_0.1.wgt) installable through AGL Application Framework. + +## Low level CAN service + +A project to access and decode CAN bus has been developed and part of AGL since Daring Dab version: [Low level CAN service](https://gerrit.automotivelinux.org/gerrit/#/admin/projects/apps/low-level-can-service) + +This rewrite of OpenXC to adapt the project to AGL. + +Must be used in conjunction with the [low level CAN generator](https://gerrit.automotivelinux.org/gerrit/#/admin/projects/src/low-level-can-generator) to custom your service. + +## High Level ViWi service + +An implementation of [ViWi](https://www.w3.org/Submission/2016/SUBM-viwi-protocol-20161213/) protocol has been made and available : [github.com:iotbzh/high-level-viwi-service](https://github.com/iotbzh/high-level-viwi-service) ## Benchmarks @@ -28,5 +40,11 @@ Some tests to evaluate the performances of the framework have been done by simul [Jose's presentation at AGL AMM Munich'16](http://iot.bzh/download/public/2016/build-agl-application-AMM-Munich-2016/) +## AMM Tokyo'17 Presentation + +[Kusakabe-san from Fujitsu-Ten presentation at AGL AMM Tokyo'17](http://schd.ws/hosted_files/aglmmwinter2017/37/20170201_AGL-AMM_F10_kusakabe.pdf) +## F2F Karslruhe'17 Presentation +[Romain's presentation at AGL F2F Karslruhe'17](http://iot.bzh/download/public/2017/F2F-Karslruhe/AGL-Signaling.pdf) +[Kusakabe-san from Fujitsu-Ten presentation at AGL F2F Karslruhe'17](https://wiki.automotivelinux.org/_media/agl-distro/20170402_ften_can_kusakabe_v2.pdf) diff --git a/signaling/low-level-can-service-guide.md b/signaling/low-level-can-service-guide.md new file mode 100644 index 0000000..6b3eb47 --- /dev/null +++ b/signaling/low-level-can-service-guide.md @@ -0,0 +1,584 @@ +# AGL CAN binding architecture + +It's meant to generate, from a JSON file describing CAN messages and diagnostic +message \(OBD2 for now\), a cpp file to integrate with the project. + +Once generated binding is built with it and result will be a widget file to +install on an AGL target system. + +![From OpenXC firmware to AGL binding](images/OpenXC_to_AGL.png) + +Bringing CAN management into the AGL project is more than allowing decode and +print CAN messages, lot of tools can do that (Wireshark, CAN-utils, ...). + +The goal is to provide a common API and abstraction to the CAN bus then you can +bring some more high level functionalities to the system. + +CAN binding will be separated in two parts: + +![CAN low and high level bindings mapping](images/CAN_level_mapping.png) + +- High level: Binding from which others applications will connect to. + It provides valuable access to the CAN bus by aggregate signals or providing + new signals from several originals. For example, a signal exposing whether or + not a door is open, no matter which one it is. Also, we can imagine an + application which supervise if there is no one in the car but moving (1m, 2m ?) + to alert the owner of an unexpected behavior. The high level binding will sends + a single event representing that behavior to the application which in turn will + send a phone message to. + +- Low level: Decode messages that transit and send event through **Application + Framework** to the subscribers with human readable message. It provides some + basic access to the bus + some basic mathematical, statistical features + (last_values, min, max, timestamps, averaging) as well as basic filter to get + discerning signal only (This part are not implemented yet in the low level). + +![Communication between CAN bindings and third applications](images/CAN_bindings_communication.png) + +Last but not least, the low level binding can be shipped as binary only using +OpenXC inspired [AGL low level CAN binding Generator](http://github.com/iotbzh/can-config-generator). + +# Prerequisites + +- An AGL system installed with latest Daring Dab version with latest Application + framework version >= 0.6. +- Make sure you built the AGL generator else you will not be able to generate + custom low-level CAN binding. +It will produce a _application-generated.cpp_ file to paste in the source, + _CAN-binder/low-can-binding/binding/_, directory. +- Make sure you already set up the AGL SDK using the following + [SDK Quick Setup Guide](http://docs.iot.bzh/docs/getting_started/en/dev/reference/setup-sdk-environment.html). + Alternatively, please refer to official guides available on [AGL Developer Site](http://docs.automotivelinux.org/docs/devguides/en/dev/#guides). + +If you need to have the graphic stack inside your SDK, you have to prepare your + environment with the **iotbzh**, or **Daring Dab** flavor using _prepare_meta_ + tool. To do so, run the following command in your docker image in the step 4 + in place of `... [ prepare build environment ] ...`: + +> **NOTE** These commands assume that proprietary graphic drivers for Renesas +Porter board are located in _/home/devel/share/proprietary-renesas-rcar_ directory. + +```bash +prepare_meta -f iotbzh -o /xdt -l /home/devel/mirror -p /home/devel/share/proprietary-renesas-rcar/ -t m3ulcb -e wipeconfig -e rm_work -e cleartemp +/xdt/build/m3ulcb/agl-init-build-env +``` + +- (Optionnal) An [USB CAN adapter](http://shop.8devices.com/usb2can) connected to + connector through the [right cable](http://www.mouser.fr/ProductDetail/EasySync/OBD-M-DB9-F-ES/)) +if you want to connect to a real car through the OBD2 connector. + +# Getting started + +## CAN config generator usage + +### Build requirements + +- CMake version 3.3 or later +- G++, Clang++ or any C++11 compliant compiler. + +### Compile + +```bash +source /xdt/sdk/environment-setup-aarch64-agl-linux +export PATH=$PATH:/xdt/sdk/sysroots/x86_64-aglsdk-linux/usr/bin +export WD=$(pwd) +git clone --recursive https://gerrit.automotivelinux.org/gerrit/apps/low-level-can-service -b Renesas_delivery_Q2 +git clone --recursive https://gerrit.automotivelinux.org/gerrit/apps/low-level-can-generator +cd ${WD}/low-level-can-generator +mkdir -p build +cd build +cmake -G "Unix Makefiles" .. +make +``` + +### Naming convention + +We chose a doted naming convention because it's a well know schema. + +It separates and organize names into hierarchy. From the left to right, you +describe your names using the more common ancestor at the left then more you go +to the right the more it will be accurate. + +Let's take an example, here is an example about standard PID name following this +convention: + +``` +engine.load +engine.coolant.temperature +fuel.pressure +intake.manifold.pressure +engine.speed +vehicle.speed +intake.air.temperature +mass.airflow +throttle.position +running.time +EGR.error +fuel.level +barometric.pressure +commanded.throttle.position +ethanol.fuel.percentage +accelerator.pedal.position +hybrid.battery-pack.remaining.life +engine.oil.temperature +engine.torque +``` + +> **NOTE** It's recommended that you follow this naming convention to named +> your CAN signals. +> +> There is only character `*` that is forbidden in names because it's used as +> wildcard for subscription and unsubscription. +> +> This described in the below chapter. + +### Available decoder + +You can use some basic decoder provided by default by the binding which are: + +- ***decoder_t::noopDecoder*** : Default decoder if not specified, return raw + value from signal's bitfield. +- ***decoder_t::booleanDecoder*** : Coerces a numerical value to a boolean. +- ***decoder_t::stateDecoder***s : Find and return the corresponding string + state for a CAN signal's raw integer value. + +### Generating JSON from Vector CANoe Database + +> **CAUTION** This chapter has not been tested since it haven't necessary +> automotive tools for that. + +If you use CANoe to store your `gold standard` CAN signal definitions, you may +be able to use the OpenXC `xml_to_json.py` script to make your JSON for you. +First, export the Canoe .dbc file as XML - you can do this with Vector CANdb++. +Next, create a JSON file according to the format defined above, but only define: + +- CAN messages. +- Name of CAN signals within messages and their generic_name. +- Optionnaly name of diagnostic messages and their name. + +To install the OpenXC utilities and runs `xml_to_json.py` script: + +```bash +sudo pip install openxc +cd /usr/local/lib/python2.7/dist-packages/openxc/generator +``` + +Assuming the data exported from Vector is in `signals.xml` and your minimal mapping +file is `mapping.json`, run the script: + +```bash +python -m openxc.utils ./xml_to_json.py signals.xml mapping.json signals.json +``` + +The script scans `mapping.json` to identify the CAN messages and signals that +you want to use from the XML file. It pulls the neccessary details of the +messages (bit position, bit size, offset, etc) and outputs the resulting subset +as JSON into the output file, `signals.json`. + +The resulting file together with `mapping.json` will work as input to the code +generation script. + +### Generate your config file + +To generate your config file you just have to run the generator using the `-m` +option to specify your JSON file. + +```bash +./can-config-generator -m ../tests/basic.json -o application-generated.cpp +``` + +If you omit the `-o` option, then code is generated on the stdout. +You also can specify a header and a footer file. +These files must be valid C++ fragment as long as they will be inserted as is. +Use the `-h` option to display help. + +> **CAUTION:** Each `diagnostic_message` must define the same `bus` as the +> binding will use only one bus. + +### Supported OpenXC items + +About now, compliance with OpenXC reference is in progress, can-config-generator +and CAN\_signaling will implement them soon. `initializers`, `loopers`, +`commands` and `handlers` nodes are ignored for now. + +This generator will follow OpenXC support status of the low level CAN signaling +binding. + +> **NOTE**: The `buses` item will not be supported by this generator because +> the binding use another way to declare and configure buses. Please refer to +> the binding's documentation. + +## Compile and install the binding + +### Build requirements + +- Kernel >= 4.8 +- CMake version 3.3 or later +- G++, Clang++ or any C++11 compliant compiler. + +### Compile + +Clone the binding repository, copy the generated file and updated the git +submodules. + +Execute the following commands from this repository: + +```bash +cd ${WD}/low-level-can-service +cp ${WD}/low-level-can-generator/build/application-generated.cpp ../low-can-binding/binding +``` + +### Installation + +```bash +cd ${WD}/low-level-can-service +mkdir build +cd build +cmake .. +make +make widget +``` + +To install it manually, you need to copy the _low-can-service.wgt_ file on your +target, then from it execute the following commands : + +On your host, to copy over the network : + +```bash +scp low-can-service.wgt root@:~ +``` + +On the target, assuming _**wgt**_ file is in the root home directory: + +```bash +afm-util install low-can-service.wgt +{ "added": "low-can-service@4.0" } +``` +# Configure the AGL system + +## Virtual CAN device + +Connected to the target, here is how to load the virtual CAN device driver and +set up a new vcan device : + +```bash +modprobe vcan +ip link add vcan0 type vcan +ip link set vcan0 up +``` + +You also can named your linux CAN device like you want and if you need name it +`can0` : + +```bash +modprobe vcan +ip link add can0 type vcan +ip link set can0 up +``` + +## CAN device using the USB CAN adapter + +Using real connection to CAN bus of your car using the USB CAN adapter +connected to the OBD2 connector. + +Once connected, launch `dmesg` command and search which device to use: + +```bash +dmesg +[...] +[ 131.871441] usb 1-3: new full-speed USB device number 4 using ohci-pci +[ 161.860504] can: controller area network core (rev 20120528 abi 9) +[ 161.860522] NET: Registered protocol family 29 +[ 177.561620] usb 1-3: USB disconnect, device number 4 +[ 191.061423] usb 1-2: USB disconnect, device number 3 +[ 196.095325] usb 1-2: new full-speed USB device number 5 using ohci-pci +[ 327.568882] usb 1-2: USB disconnect, device number 5 +[ 428.594177] CAN device driver interface +[ 1872.551543] usb 1-2: new full-speed USB device number 6 using ohci-pci +[ 1872.809302] usb_8dev 1-2:1.0 can0: firmware: 1.7, hardware: 1.0 +[ 1872.809356] usbcore: registered new interface driver usb_8dev +``` + +Here device is named `can0`. + +This instruction assuming a speed of 500000kbps for your CAN bus, you can try +others supported bitrate like 125000, 250000 if 500000 doesn't work: + +```bash +ip link set can0 type can bitrate 500000 +ip link set can0 up +ip link show can0 + can0: mtu 16 qdisc pfifo_fast state UNKNOWN qlen 10 + link/can + can state ERROR-ACTIVE (berr-counter tx 0 rx 0) restart-ms 0 + bitrate 500000 sample-point 0.875 + tq 125 prop-seg 6 phase-seg1 7 phase-seg2 2 sjw 1 + sja1000: tseg1 1..16 tseg2 1..8 sjw 1..4 brp 1..64 brp-inc 1 + clock 16000000 +``` + +On a Rcar Gen3 board, you'll have your CAN device as `can1` because `can0` +already exists as an embedded device. + +The instructions will be the same: + +```bash +ip link set can1 type can bitrate 500000 +ip link set can1 up +ip link show can1 + can0: mtu 16 qdisc pfifo_fast state UNKNOWN qlen 10 + link/can + can state ERROR-ACTIVE (berr-counter tx 0 rx 0) restart-ms 0 + bitrate 500000 sample-point 0.875 + tq 125 prop-seg 6 phase-seg1 7 phase-seg2 2 sjw 1 + sja1000: tseg1 1..16 tseg2 1..8 sjw 1..4 brp 1..64 brp-inc 1 + clock 16000000 +``` + +## Rename an existing CAN device + +You can rename an existing CAN device using following command and doing so move +an existing `can0` device to anything else and then use another device as +`can0`. For a Rcar Gen3 board do the following by example: + +```bash +sudo ip link set can0 down +sudo ip link set can0 name bsp-can0 +sudo ip link set bsp-can0 up +``` + +Then connect your USB CAN device that will be named `can0` by default. + +# Configure the binding + +The binding reads system configuration file _/etc/dev-mapping.conf_ at start to +map logical name from signals described in JSON file to linux devices name +initialized by the system. Edit file _/etc/dev-mappping.conf_ and add mapping +in section `CANbus-mapping`. + +Default binding configuration use a CAN bus named `hs` so you need to map it to +the real one, here are some examples: + +- Using virtual CAN device as described in the previous chapter: + +```ini +[CANbus-mapping] +hs="vcan0" +ls="vcan1" +``` + +- Using real CAN device, this example assume CAN bus traffic will be on can0. + +```ini +[CANbus-mapping] +hs="can0" +ls="can1" +``` + +- On a Rcar Gen3 board there is an embedded CAN device so `can0` already + exists. So you might want to use your USB CAN adapter plugged to the OBD2 + connector, in this case use `can1`: + +```ini +[CANbus-mapping] +hs="can1" +``` + +> **CAUTION VERY IMPORTANT:** Make sure the CAN bus\(es\) you specify in your +> configuration file match those specified in your generated source file with +> the `CAN-config-generator`. + +# Run it, test it, use it ! + +You can run the binding using **afm-util** tool, here is the classic way to go: + +```bash +afm-util run low-can-service@4.0 +1 +``` + +You can find instructions to use afm-util tool [here](http://docs.iot.bzh/docs/apis_services/en/dev/reference/af-main/afm-daemons.html#using-afm-util), +as well as documentation about Application Framework. + +But you can't control nor interact with it because you don't know security +token that **Application Framework** gaves it at launch. + +So, to test it, it is better to launch the binding manually. In the following +example, it will use port **1234** and left empty security token for testing +purpose: + +```bash +afb-daemon --binding=/var/lib/afm/applications/low-can-service/4.0/lib/afb-low-can.so --rootdir=/var/lib/afm/applications/low-can-service/4.0/ --port=1234 --token=1 +NOTICE: binding [/usr/lib/afb/afb-dbus-binding.so] calling registering function afbBindingV1Register +NOTICE: binding /usr/lib/afb/afb-dbus-binding.so loaded with API prefix dbus +NOTICE: binding [/usr/lib/afb/authLogin.so] calling registering function afbBindingV1Register +NOTICE: binding /usr/lib/afb/authLogin.so loaded with API prefix auth +NOTICE: binding [/var/lib/afm/applications/low-can-service/4.0/libs//low-can-binding.so] calling registering function afbBindingV1Register +NOTICE: binding /var/lib/afm/applications/low-can-service/4.0/libs//low-can-binding.so loaded with API prefix low-can +NOTICE: Waiting port=1234 rootdir=/var/lib/afm/applications/low-can-service/4.0/ +NOTICE: Browser URL= http:/*localhost:1234 +``` + +On another terminal, connect to the binding using previously installed +_**AFB Websocket CLI**_ tool: + +```bash +afb-client-demo ws://localhost:1234/api?token=1 +``` + +You will be on an interactive session where you can communicate directly with +the binding API. + +The binding provides at this moment 2 verbs, _subscribe_ and _unsubscribe_, +which can take argument by a JSON **event** object. + +The argument value is the CAN message **generic\_name** as described in the +JSON file used to generate cpp file for the binding. + +To use the _**AFB Websocket CLI**_ tool, a command line will be like the +following : + +``` + +``` + +Where: + +* API : _**low-can**_. +* Verb : _**subscribe**_ or _**unsubscribe**_ +* Arguments : _**{ "event": "driver.doors.open" }**_ + +## Subscription and unsubscription + +You can ask to subscribe to chosen CAN event with a call to _subscribe_ API +verb with the CAN messages name as JSON argument. + +> **NOTE:** If no argument is provided, then you'll subscribe to all signals at +> once. + +For example from a websocket session: + +```json +low-can subscribe { "event": "doors.driver.open" } +ON-REPLY 1:low-can/subscribe: {"jtype":"afb-reply","request":{"status":"success","uuid":"a18fd375-b6fa-4c0e-a1d4-9d3955975ae8"}} +``` + +Subscription and unsubscription can take wildcard in their _event_ value. + +To receive all doors events : + +```json +low-can subscribe { "event" : "doors*" } +ON-REPLY 1:low-can/subscribe: {"jtype":"afb-reply","request":{"status":"success","uuid":"511c872e-d7f3-4f3b-89c2-aa9a3e9fbbdb"}} +``` + +Then you will receive an event each time a CAN message is decoded for the event +named _doors.driver.open_ + +```json +ON-EVENT low-can/messages.doors.driver.open({"event":"low-can\/messages.doors.driver.open","data":{"name":"messages.doors.driver.open","value":true},"jtype":"afb-event"}) +``` + +Notice that event shows you that the CAN event is named +_messages.doors.driver.open_ but you ask for event about _doors.driver.open_. + +This is because all CAN messages or diagnostic messages are prefixed by the +JSON parent node name, **messages** for CAN messages and +**diagnostic\_messages** for diagnostic messages like OBD2. + +This will let you subscribe or unsubcribe to all signals at once, not +recommended, and better make filter on subscribe operation based upon their +type. Examples: + +```json +low-can subscribe { "event" : "*speed*" } --> will subscribe to all messages with speed in their name. Search will be make without prefix for it. +low-can subscribe { "event" : "speed*" } --> will subscribe to all messages begin by speed in their name. Search will be make without prefix for it. +low-can subscribe { "event" : "messages*speed*" } --> will subscribe to all CAN messages with speed in their name. Search will be on prefixed messages here. +low-can subscribe { "event" : "messages*speed" } --> will subscribe to all CAN messages ending with speed in their name. Search will be on prefixed messages here. +low-can subscribe { "event" : "diagnostic*speed*" } --> will subscribe to all diagnostic messages with speed in their name. Search will be on prefixed messages here. +low-can subscribe { "event" : "diagnostic*speed" } --> will subscribe to all diagnostic messages ending with speed in their name. Search will be on prefixed messages here. +``` + +You can stop receiving event from it by unsubscribe the signal the same way you +did for subscribe + +```json +low-can unsubscribe { "event": "doors.driver.open" } +ON-REPLY 2:low-can/unsubscribe: {"jtype":"afb-reply","request":{"status":"success"}} +low-can unsubscribe { "event" : "doors*" } +ON-REPLY 3:low-can/unsubscribe: {"jtype":"afb-reply","request":{"status":"success"}} +``` + +### Filtering capabilities + +It is possible to limits received event notifications into minimum and maximum +boundaries as well as doing frequency thinning. This is possible using the +argument filter with one or more of the filters available : + +- frequency: specify in Hertz the frequency which will be used to getting + notified of new CAN events for the designated signal. If, during the + blocked time, further changed CAN messages are received, the last valid one + will be transferred after the lockout with a RX_CHANGED. +- min: Minimum value that the decoded value needs to be above to get pushed + to the subscribed client(s). +- max: Maximum value that the decoded value needs to be below to get pushed to + the subscribed client(s) + +Order doesn't matter neither the number of filters chosen, you can use one, two +or all of them at once. + +Usage examples : + +```json +low-can subscribe {"event": "messages.engine.speed", "filter": { "frequency": 3, "min": 1250, "max": 3500}} +low-can subscribe {"event": "messages.engine.load", "filter": { "min": 30, "max": 100}} +low-can subscribe {"event": "messages.vehicle.speed", "filter": { "frequency": 2}} +``` + +## Using CAN utils to monitor CAN activity + +You can watch CAN traffic and send custom CAN messages using can-utils +preinstalled on AGL target. + +To watch watch going on a CAN bus use: + +```bash +candump can0 +``` + +Or for an USB CAN adapter connected to porter board: + +```bash +candump can1 +``` + +Send a custom message: + +```bash +cansend can0 ID#DDDDAAAATTTTAAAA +``` + +You can also replay a previously dumped CAN logfiles. These logfiles can be +found in _can_samples_ directory under Git repository. Following examples use a +real trip from an Auris Toyota car. + +Trace has been recorded from a CAN device `can0` so you have to map it to the +correct one you use for your tests. + +Replay on a virtual CAN device `vcan0`: + +```bash +canplayer -I trip_test_with_obd2_vehicle_speed_requests vcan0=can0 +``` + +Replay on a CAN device `can0`: + +```bash +canplayer -I trip_test_with_obd2_vehicle_speed_requests can0 +``` + +Replay on a CAN device `can1` (porter by example): + +```bash +canplayer -I trip_test_with_obd2_vehicle_speed_requests can1=can0 +``` -- cgit 1.2.3-korg