diff options
Diffstat (limited to 'docs/06_Component_Documentation')
14 files changed, 469 insertions, 267 deletions
diff --git a/docs/06_Component_Documentation/01_AGL_components.md b/docs/06_Component_Documentation/01_AGL_components.md index 3508096..4aa9a17 100644 --- a/docs/06_Component_Documentation/01_AGL_components.md +++ b/docs/06_Component_Documentation/01_AGL_components.md @@ -24,3 +24,8 @@ title: AGL Components ### Application Lifecycle and Services - [Application Framework](Application_Framework/01_Introduction.md) + +### Application for Demo + +- [Momi Navi](Demo_Application/01_Momi_Navi.md) + diff --git a/docs/06_Component_Documentation/09_AGL_Demo_Control_Panel.md b/docs/06_Component_Documentation/09_AGL_Demo_Control_Panel.md index af42523..2b99242 100644 --- a/docs/06_Component_Documentation/09_AGL_Demo_Control_Panel.md +++ b/docs/06_Component_Documentation/09_AGL_Demo_Control_Panel.md @@ -5,7 +5,10 @@ title: AGL Demo Control Panel ## Introduction -This document describes the design and usage of the **AGL Demo Control Panel**, a **Qt5-based** tool that allows you to control and interact with various **Automotive Grade Linux (AGL)** demo applications. The tool uses **Kuksa.val** and **CAN frame messages** to communicate with the target machine that runs the AGL image(s). You can use the tool to perform tasks such as starting and stopping scripts, changing the vehicle speed and engine RPM, adjusting the HVAC settings, and providing Steering Inputs. The tool is designed to **demonstrate** the capabilities and features of AGL in a **user-friendly** and **interactive** way. +This document describes the design and usage of the **AGL Demo Control Panel**, a **Qt6-based** tool that allows you to control and interact with various **Automotive Grade Linux (AGL)** demo applications. The tool uses **Kuksa.val** and **CAN frame messages** to communicate with the target machine that runs the AGL image(s). You can use the tool to perform tasks such as starting and stopping scripts, changing the vehicle speed and engine RPM, adjusting the HVAC settings, and providing Steering Inputs. The tool is designed to **demonstrate** the capabilities and features of AGL in a **user-friendly** and **interactive** way. + +It also supports file playback from CARLA based CAN messages. Refer to the [CARLA with AGL](13_CARLA_with_AGL.md) for more information. + ### Application Overview To use the control panel, you need to connect the main machine that runs the control panel to the target machine that runs the AGL image(s) using a **LAN/ethernet cable**. You also need to configure the IP address of the Kuksa server and set your preferences in the tool’s settings menu. @@ -13,16 +16,6 @@ To use the control panel, you need to connect the main machine that runs the con  ## Installation - - -- _Note_: - If errors occur in Debian based/Rasbian OS during installation, follow the steps mentioned below and skip to step 2: -```bash -$ nano requirements.txt -# -> Comment pyqt5 dependency using "#" -$ sudo apt install python3-pyqt5 python3-qtpy pyqt5-dev-tools python3-pyqt5.qtsvg -y -``` - - Step 1 ```bash $ python3 -m venv control-panel @@ -32,7 +25,7 @@ $ source control-panel/bin/activate - Step 2 ```bash $ pip3 install -r requirements.txt -$ pyrcc5 assets/res.qrc -o res_rc.py +$ pyside6-rcc assets/res.qrc -o res_rc.py ``` ## Setup @@ -40,8 +33,9 @@ Before using the `AGL Demo Control Panel`, we need to make sure to run the Kuks ### 1. Connect the Machines -First, we need to connect the machines, i.e. the host machine (Running the control panel) and the target machine (running the AGL image) via LAN or a bridged network (QEMU or VM) -### 2. CAN interface (WIP) +First, we need to connect the machines, i.e. the host machine (Running the control panel) and the target machine (running the AGL image) via LAN or a bridged network (QEMU or VM). + +### 2. CAN interface To set up the CAN interface between the Host system and the target machine(s) we use [cannelloni](https://github.com/mguentner/cannelloni), diff --git a/docs/06_Component_Documentation/10_agl_voice_agent_assistant.md b/docs/06_Component_Documentation/10_agl_voice_agent_assistant.md index 97bdd0d..e75382e 100644 --- a/docs/06_Component_Documentation/10_agl_voice_agent_assistant.md +++ b/docs/06_Component_Documentation/10_agl_voice_agent_assistant.md @@ -5,10 +5,15 @@ title: AGL Voice Agent / Assistant # AGL Voice Agent / Assistant # Introduction -A gRPC-based voice agent designed for Automotive Grade Linux (AGL). This service leverages GStreamer, Vosk, Snips, and RASA to seamlessly process user voice commands. It converts spoken words into text, extracts intents from these commands, and performs actions through the Kuksa interface. The voice agent is designed to be modular and extensible, allowing for the addition of new speech recognition and intent extraction models. +A gRPC-based voice agent designed for Automotive Grade Linux (AGL). This service leverages GStreamer, Vosk, Whisper AI, and Snips to process user voice commands seamlessly. It converts spoken words into text, extracts intents from these commands, and performs actions through the Kuksa interface. The voice agent is modular and extensible, allowing for the addition of new speech recognition and intent extraction models. Whisper AI is available for both online and offline modes, offering more accurate transcription compared to Vosk, though it may run slower in offline mode. + +Note: RASA NLU is currently not available as it is not supported by Python 3.12, the version used in AGL. + # Installation and Usage -Before we dive into the detailed components documentation, let's first take a look at how to install and use the voice agent service. All of the features of the voice agent service are encapsulated in the `meta-offline-voice-agent` sub-layer which can be found under `meta-agl-devel` layer. These features are currently part of the `master` branch only. This sub-layer can be built into the final image by using following commands: +Before diving into the detailed components documentation, let's first look at how to install and use the voice agent service. All of the features of the voice agent service are encapsulated in the `meta-offline-voice-agent` sub-layer, which can be found under the `meta-agl-devel layer`. These features are currently available in the `master` branch only. You can build this sub-layer into the final image using the following commands: + +### Building for QEMU x86-64 ```shell $ source master/meta-agl/scripts/aglsetup.sh -m qemux86-64 -b build-master agl-demo agl-devel agl-offline-voice-agent @@ -16,36 +21,73 @@ $ source agl-init-build-env $ bitbake agl-ivi-demo-flutter ``` -After the build is complete, you can run the final image using QEMU. Once the image is running, you can start the voice agent service by running the following command: +### Building for Raspberry Pi 5 + +Whisper AI does not work on the emulator; therefore, building and running on a physical device like the Raspberry Pi 5 is recommended. You can set up the build environment specifically for Raspberry Pi 5 using: + +```shell +$ source master/meta-agl/scripts/aglsetup.sh -m raspberrypi5 -b build-master agl-demo agl-devel agl-offline-voice-agent +$ source agl-init-build-env +$ bitbake agl-ivi-demo-flutter +``` + +<br> +**Note**: The voice assistant client is already integrated into the `flutter-ics-homescreen`, which is the default homescreen app for `agl-ivi-demo-flutter`. However, if you wish to build any other image, you can use the `flutter-voiceassistant` app, which is a standalone flutter-based client for the voice assistant. You can add `flutter-voiceassistant` to your build by adding the following line to the `conf/local.conf` file: + ```shell -$ voiceagent-service run-server --default +FEATURE_PACKAGES_agl-offline-voice-agent:append = " \ + flutter-voiceassistant \ +" ``` -The `--default` flag loads the voice agent service with default configuration. The default configuration file looks like this: +The voice agent service will automatically start on startup with the default configuration located at +`/etc/default/voice-agent-config.ini.` + + +The default configuration file looks like this: ```ini [General] base_audio_dir = /usr/share/nlu/commands/ -stt_model_path = /usr/share/vosk/vosk-model-small-en-us-0.15/ +vosk_model_path = /usr/share/vosk/vosk-model-small-en-us-0.15/ +whisper_model_path = /usr/share/whisper/tiny.pt +whisper_cpp_path = /usr/bin/whisper-cpp +whisper_cpp_model_path = /usr/share/whisper-cpp/models/tiny.en.bin wake_word_model_path = /usr/share/vosk/vosk-model-small-en-us-0.15/ snips_model_path = /usr/share/nlu/snips/model/ channels = 1 sample_rate = 16000 bits_per_sample = 16 -wake_word = hello auto +wake_word = hey automotive server_port = 51053 server_address = 127.0.0.1 rasa_model_path = /usr/share/nlu/rasa/models/ rasa_server_port = 51054 -rasa_detached_mode = 0 +rasa_detached_mode = 1 base_log_dir = /usr/share/nlu/logs/ store_voice_commands = 0 +online_mode = 1 +online_mode_address = online-whisper-asr-service-address +online_mode_port = online-whisper-asr-service-port +online_mode_timeout = 15 +mpd_ip = 127.0.0.1 +mpd_port = 6600 [Kuksa] ip = 127.0.0.1 -port = 8090 -protocol = ws -insecure = True -token = /usr/lib/python3.10/site-packages/kuksa_certificates/jwt/super-admin.json.token +port = 55555 +protocol = grpc +insecure = 0 +token = /usr/lib/python3.12/site-packages/kuksa_certificates/jwt/super-admin.json.token +tls_server_name = Server + +[VSS] +hostname = localhost +port = 55555 +protocol = grpc +insecure = 0 +token_filename = /etc/xdg/AGL/agl-vss-helper/agl-vss-helper.token +ca_cert_filename = /etc/kuksa-val/CA.pem +tls_server_name = Server [Mapper] intents_vss_map = /usr/share/nlu/mappings/intents_vss_map.json @@ -56,8 +98,6 @@ Most of the above configuration variable are self explanatory, however, I'll div - **`store_voice_commands`**: This variable is used to enable/disable the storage of voice commands. If this variable is set to `1`, then the voice commands will be stored in the `base_audio_dir` directory. The voice commands are stored in the following format: `base_audio_dir/<timestamp>.wav`. The `timestamp` is the time at which the voice command was received by the voice agent service. -- **`rasa_detached_mode`**: This variable is used to enable/disable the detached mode for the RASA NLU engine. If this variable is set to `1`, then the RASA NLU engine will be run in detached mode, i.e. the voice agent service won't run it and will assume that RASA is already running. This is useful when you want to run the RASA NLU engine on a separate machine. If this variable is set to `0`, then the RASA NLU engine will be run as a sub process of the voice agent service. - - **`intents_vss_map`**: This is the path to the file that actually maps the intent output from our intent engine to the VSS signal specification. This file is in JSON format and contains the mapping for all the intents that we want to support. The default file looks like this: ```json @@ -70,18 +110,18 @@ Most of the above configuration variable are self explanatory, however, I'll div }, "HVACFanSpeed": { "signals": [ - "Vehicle.Cabin.HVAC.Station.Row1.Left.FanSpeed", - "Vehicle.Cabin.HVAC.Station.Row1.Right.FanSpeed", - "Vehicle.Cabin.HVAC.Station.Row2.Left.FanSpeed", - "Vehicle.Cabin.HVAC.Station.Row2.Right.FanSpeed" + "Vehicle.Cabin.HVAC.Station.Row1.Driver.FanSpeed", + "Vehicle.Cabin.HVAC.Station.Row1.Passenger.FanSpeed", + "Vehicle.Cabin.HVAC.Station.Row2.Driver.FanSpeed", + "Vehicle.Cabin.HVAC.Station.Row2.Passenger.FanSpeed" ] }, "HVACTemperature": { "signals": [ - "Vehicle.Cabin.HVAC.Station.Row1.Left.Temperature", - "Vehicle.Cabin.HVAC.Station.Row1.Right.Temperature", - "Vehicle.Cabin.HVAC.Station.Row2.Left.Temperature", - "Vehicle.Cabin.HVAC.Station.Row2.Right.Temperature" + "Vehicle.Cabin.HVAC.Station.Row1.Driver.Temperature", + "Vehicle.Cabin.HVAC.Station.Row1.Passenger.Temperature", + "Vehicle.Cabin.HVAC.Station.Row2.Driver.Temperature", + "Vehicle.Cabin.HVAC.Station.Row2.Passenger.Temperature" ] } } @@ -142,21 +182,27 @@ $ voiceagent-service run-server --config path/to/config.ini One thing to note here is that all the directory paths in the configuration file should be absolute and always end with a `/`. # High Level Architecture -.png) + # Components - Voice Agent Service + - Whisper AI - Vosk Kaldi - Snips - RASA - Voice Assistant App # Voice Agent Service -The voice agent service is a gRPC-based service that is responsible for converting spoken words into text, extracting intents from these commands, and performing actions through the Kuksa interface. The service is composed of three main components: Vosk Kaldi, RASA, and Snips. +The voice agent service is a gRPC-based service that is responsible for converting spoken words into text, extracting intents from these commands, and performing actions through the Kuksa interface. The service is composed of three main components: Whisper AI, Vosk Kaldi, RASA, and Snips. + +## Whisper AI +Whisper AI is a versatile speech recognition model developed by OpenAI. It is designed to convert spoken words into text with high accuracy and supports multiple languages out of the box. Whisper AI offers various pre-trained models, each optimized for different hardware capabilities and performance requirements. The current voice agent service uses Whisper AI for **speech recognition**. In offline mode, Whisper AI processes speech locally on the device, providing an accurate transcription of user commands. While Whisper AI may be slightly slower than other models like Vosk, it offers higher accuracy, especially for complex and noisy inputs. It does not currently support wake word detection, so it is used alongside other tools like Vosk for a comprehensive voice assistant solution. ## Vosk Kaldi Vosk Kaldi is a speech recognition toolkit that is based on Kaldi and Vosk. It is used to convert spoken words into text. It provides us with some official pre-trained models for various popular languages. We can also train our own models using the Vosk Kaldi toolkit. The current voice agent service requires two different models to run, one for **wake-word detection** and one for **speech recognition**. The wake word detection model is used to detect when the user says the wake word, which is "Hey Automotive" by default, we can easily change the default wake word by modifying the config file. The speech recognition model is used to convert the user's spoken words into text. +**Note:** The current **wake word** is set to **"Hey Automotive,"** but you can change it to any wake word of your choice. However, if you plan to use a more specific wake word, you may need to train the Vosk wake word model yourself, as the pre-trained models may not be optimized for your specific wake word. + ## Snips Snips NLU (Natural Language Understanding) is a Python based Intent Engine that allows to extract structured information from sentences written in natural language. The NLU engine first detects what the intention of the user is (a.k.a. intent), then extracts the parameters (called slots) of the query. The developer can then use this to determine the appropriate action or response. Our voice agent service uses either Snips or RASA to extract intents from the user's spoken commands. @@ -203,6 +249,9 @@ To set up and run the Snips NLU Intent Engine, follow these steps: - The Snips NLU engine is not very accurate as compared to RASA, however, its extremely lightweight and really fast. ## RASA + +**Note:** : RASA is currently not included into the build as it is not supported by Python 3.12, the version used in AGL. + RASA is an open-source machine learning framework for building contextual AI assistants and chatbots. It is based on Python and TensorFlow. It is used to extract intents from the user's spoken commands. The RASA NLU engine is trained on a dataset that contains intents, entities, and sample utterances. The RASA NLU engine is used to parse user queries and extract the intent and relevant entities for further processing. It is recommended to take a brief look at [RASA Official Documentation](https://rasa.com/docs/rasa/) to get a better understanding of how RASA works. @@ -279,8 +328,19 @@ To set up and run the RASA NLU Intent Engine, follow these steps: - The underlying AI arhictecture of the RASA NLU is extensible and changeable thanks to the TensorFlow backend. - The RASA NLU engine is very accurate as compared to Snips, however, its heavy and slightly slow. -# Voice Assistant App -The voice assistant app is a flutter based application made for Automotive Grade Linux (AGL). It is responsible for interacting with the voice agent service for user voice command recognition, intent extraction, and command execution. It also receives the response from the voice agent service and displays it on the screen. Some app UI screenshots are attached below. +# Voice Assistant Client + +## flutter-ics-homescreen Integration +The voice assistant client is integrated into the `flutter-ics-homescreen` app, which can interact with the voice agent service to process user voice commands. + +|  |  | +|:--:|:--:| + +|  |  | +|:--:|:--:| + +## Voice Assistant App +The voice assistant app is a standalone, Flutter-based client designed for Automotive Grade Linux (AGL). It is responsible for interacting with the voice agent service for user voice command recognition, intent extraction, and command execution. It also receives the response from the voice agent service and displays it on the screen. Some app UI screenshots are attached below. - - +|  |  | +|:--:|:--:|
\ No newline at end of file diff --git a/docs/06_Component_Documentation/12_AGL_Persistent_Storage_API.md b/docs/06_Component_Documentation/12_AGL_Persistent_Storage_API.md index 97ec0ac..04ee9ee 100644..100755 --- a/docs/06_Component_Documentation/12_AGL_Persistent_Storage_API.md +++ b/docs/06_Component_Documentation/12_AGL_Persistent_Storage_API.md @@ -1,225 +1,229 @@ -# Persistent Storage API for the Automotive Grade Linux demo - -The [AGL Persistent Storage API](https://github.com/LSchwiedrzik/agl-persistent-storage-api) -is a grpc API for [AGL](https://www.automotivelinux.org/) -that serves as persistent storage API for the demo. The API is written -in Rust and makes use of [tonic](https://crates.io/crates/tonic-build) for grpc -functionality as well as [RocksDB](https://rocksdb.org/) as a database backend, -using [rust-rocksdb](https://crates.io/crates/rust-rocksdb). Use cases include -retaining settings over a system shutdown (e.g. audio, HVAC, profile data, Wifi -settings, radio presets, metric vs imperial units). - -The most important hardware consideration for this project is that the AGL demo -runs on embedded hardware with flash storage, so we want to minimize number of -write operations. This impacts the choice of database; we have chosen to work -with RocksDB as it is well-suited for embedded computing and tunable with -respect to write amplification. In principle the API is flexible with -respect to database used (pluggable backends), but only RocksDB is implemented. -This API is part of the AGL demo as of release 'Royal Ricefish'. - -The AGL Persistent Storage API is constructed using a layered architecture: - -- Controller layer: translates proto calls to service calls. -- Service layer: communicates with the controller and facade layers, implements - the business logic -- Facade layer: implements RocksDB. - -## API Specification - -**Namespaces** -The rpcs described below interact with keys belonging to specific namespaces. This feature enables applications to maintain private namespaces within the same database. Not specifying a namespace when calling the API will result in the default namespace "" being used. Alternatively, a specific namespace (e.g. "AppName") can be chosen. With the exception of DestroyDB, which acts on the entire database, all rpcs can only interact with one namespace at a time. - -- `DestroyDB() -> StandardResponse(success: boolean, message: string)` - - - Consumer wants to destroy the entire database. - - ```text - DestroyDB() -> //destroys entire database. - ``` - -- `Write(key: string, value: string, namespace: string) -> StandardResponse(success: boolean, message: string)` - - - Consumer wants to save *key* + *value* to a given *namespace* (default is ""), (e.g. - 'Vehicle.Infotainment.Radio.CurrentStation':'hr5'). - - This overwrites existing *value* under *key*. - - An empty string cannot be used as a *key*. - - ```text - Write('Vehicle.Infotainment.Radio.CurrentStation':'wdr 4') -> Response - - Write('Vehicle.Infotainment':'yes') -> Response - - Write('test':'1') -> Response - - Write('':'test') -> Error - - Write(key: 'Private.Info', value: 'test', namespace: 'AppName') -> Response - ``` - -- `Read(key: string, namespace: string) -> ReadResponse(success: boolean, message: string, value: string)` - - - Consumer wants to read *value* of existing *key* in a given *namespace* (default is ""), e.g. - 'Vehicle.Infotainment.Radio.CurrentStation': - - ```text - Read('Vehicle.Infotainment.Radio.CurrentStation') -> 'wdr 4' - - Read('Vehicle.doesNotExist') -> ERROR - - Read(key: 'Private.Info', namespace: 'AppName') -> 'test' - ``` - -- `Delete(key: string, namespace: string) -> StandardResponse(success: boolean, message: string)` - - - Consumer wants to delete an existing *key* + *value* from a given *namespace* (default is ""), e.g. - 'Vehicle.Infotainment.Radio.CurrentStation': - - ```text - Delete('Vehicle.Infotainment.Radio.CurrentStation') -> Response - - Delete('Vehicle.doesNotExist') -> ERROR - - Delete(key: 'Private.Info', namespace: 'AppName') -> Response - ``` - -- `Search(key: string, namespace: string) -> ListResponse(success: boolean, message: string, keys: repeated string)` - - - Consumer wants to list all keys that contain *key* in a given *namespace* (default is ""), e.g. 'Radio' - - ```text - Search('Radio') -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Communication.Radio.Volume') - - Search('Info') -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature') - - Search('nt.Rad') -> ('Vehicle.Infotainment.Radio.CurrentStation') - - Search('') -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature', 'Vehicle.Communication.Radio.Volume') - - Search(key: '', namespace: 'AppName') -> ('Private.Info') - ``` - -- `DeleteNodes(key: string, namespace: string) -> StandardResponse(success: boolean, message: string)` - - - Consumer wants to delete all keys located in the subtree with root *key*, within the given *namespace* (default is ""), e.g. - 'Vehicle.Infotainment' - - `key = ''` returns `ERROR` - - This rpc assumes that keys follow a VSS-like tree structure. - - ```text - DeleteNodes('Vehicle.Infotainment') -> Response //deletes ('Vehicle.Infotainment', 'Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature') - - DeleteNodes('Vehicle') -> Response //deletes ('Vehicle.Infotainment', 'Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature', 'Vehicle.Communication.Radio.Volume') - - DeleteNodes('') -> ERROR - - DeleteNodes('DoesNotExist') -> ERROR - - DeleteNodes('Vehic') -> ERROR - - DeleteNodes(key: 'Private', namespace: 'AppName') -> Response //deletes ('Private.Info') - ``` - -- `ListNodes(node: string, layers: optional int, namespace: string) -> ListResponse(boolean, message, repeated string keys)` - - - Consumer wants to list all nodes located in the subtree with root *node* exactly *layers* - layers deep, within the given *namespace* (default is "") , e.g. 'Vehicle.Infotainment' - - - `layers = 0` lists all keys that start in *node* any number of *layers* deep - - `layers` default value is 1 - - `node = ''` returns top-level root node(s) - - This rpc assumes that keys follow a VSS-like tree structure. - - ```text - ListNodes('Vehicle.Infotainment', 1) -> ('Vehicle.Infotainment.Radio', 'Vehicle.Infotainment.HVAC') - - ListNodes('Vehicle.Infotainment', 2) -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature') - - ListNodes('Vehicle', 0) -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature', 'Vehicle.Communication.Radio.Volume', 'Vehicle.Infotainment') - - ListNodes('', 0) -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature', 'Vehicle.Communication.Radio.Volume', 'Vehicle.Infotainment', 'test') - - ListNodes('Vehicle.Infotainment') -> ('Vehicle.Infotainment.Radio', 'Vehicle.Infotainment.HVAC') - - ListNodes('', 1) -> ('Vehicle', 'test') - - ListNodes('Vehicle.Infotainment.Radio.Volume', 1) -> () - - ListNodes('Vehicle', -1) -> ERROR - - ListNodes('Vehicle.DoesNotExist', 1) -> ERROR - - ListNodes(key: 'Private', namespace: 'AppName') -> ('Private.Info') - - For empty data base: - ListNodes('', 1) -> () - ``` - -## Example Tree - -Note: nodes marked by \* are keys (and therefore have a value) - -**Namespace: ""** -- Vehicle - - Infotainment \* - - Radio - - CurrentStation \* - - Volume \* - - HVAC - - OutdoorTemperature \* - - Communication - - Radio - - Volume \* -- test \* - -**Namespace: "AppName"** -- Private - - Info \* - -## Setup instructions - -1. Install rust - -2. Download or install protobuf (e.g. from - [here](https://github.com/protocolbuffers/protobuf/releases)) and set the - `PROTOC` environment variable: - `echo -n "export PROTOC=/path/to/protoc.exe" >> ~/.bashrc` - -3. Build application - - ```bash - cargo build - ``` - -4. Run tests - - ```bash - cargo test - ``` - -5. Start server - - ```bash - cargo run --release --bin server - ``` - -## rpc Usage - -Insomnia usage for manual testing is describd in -https://konghq.com/blog/engineering/building-grpc-apis-with-rust - -```text -DestroyDB: {} - -Write: { "key": "foo", "value": "foobar", "namespace": "bar" } - -Read: { "key": "foo", "namespace": "bar" } - -Delete: { "key": "foo", "namespace": "bar" } - -Search: { "key": "foo", "namespace": "bar" } - -DeleteNodes: { "key": "foo", "namespace": "bar" } - -ListNodes: { "key": "foo", "layers": 1, "namespace": "bar" } - -``` +# Persistent Storage API for the Automotive Grade Linux demo
+
+The AGL Persistent Storage API
+is a grpc API for [AGL](https://www.automotivelinux.org/)
+that serves as persistent storage API for the demo. The API is written
+in Rust and makes use of [tonic](https://crates.io/crates/tonic-build) for grpc
+functionality as well as [RocksDB](https://rocksdb.org/) as a database backend,
+using [rust-rocksdb](https://crates.io/crates/rust-rocksdb). Use cases include
+retaining settings over a system shutdown (e.g. audio, HVAC, profile data, Wifi
+settings, radio presets, metric vs imperial units).
+
+The most important hardware consideration for this project is that the AGL demo
+runs on embedded hardware with flash storage, so we want to minimize number of
+write operations. This impacts the choice of database; we have chosen to work
+with RocksDB as it is well-suited for embedded computing and tunable with
+respect to write amplification. In principle the API is flexible with
+respect to database used (pluggable backends), but only RocksDB is implemented.
+This API is part of the AGL demo as of release 'Royal Ricefish'.
+
+The AGL Persistent Storage API is constructed using a layered architecture:
+
+- Controller layer: translates proto calls to service calls.
+- Service layer: communicates with the controller and facade layers, implements
+ the business logic
+- Facade layer: implements RocksDB.
+
+By default, the API can be accessed through port 50054. This can be changed in main.rs.
+The RocksDB database files are stored in directory "AGLPersistentStorageAPI",
+located in the home directory of your system. This can be changed in service.rs.
+
+## API Specification
+
+### Namespaces
+
+The rpcs described below interact with keys belonging to specific namespaces. This feature enables applications to maintain private namespaces within the same database. Not specifying a namespace when calling the API will result in the default namespace "" being used. Alternatively, a specific namespace (e.g. "AppName") can be chosen. With the exception of DestroyDB, which acts on the entire database, all rpcs can only interact with one namespace at a time.
+
+### Remote procedure calls
+
+- `DestroyDB() -> StandardResponse(success: boolean, message: string)`
+
+ - Consumer wants to destroy the entire database.
+
+ ```text
+ DestroyDB() -> //destroys entire database.
+ ```
+
+- `Write(key: string, value: string, namespace: string) -> StandardResponse(success: boolean, message: string)`
+
+ - Consumer wants to save *key* + *value* to a given *namespace* (default is ""), (e.g.
+ 'Vehicle.Infotainment.Radio.CurrentStation':'hr5').
+ - This overwrites existing *value* under *key*.
+ - An empty string cannot be used as a *key*.
+
+ ```text
+ Write('Vehicle.Infotainment.Radio.CurrentStation':'wdr 4') -> Response
+
+ Write('Vehicle.Infotainment':'yes') -> Response
+
+ Write('test':'1') -> Response
+
+ Write('':'test') -> Error
+
+ Write(key: 'Private.Info', value: 'test', namespace: 'AppName') -> Response
+ ```
+
+- `Read(key: string, namespace: string) -> ReadResponse(success: boolean, message: string, value: string)`
+
+ - Consumer wants to read *value* of existing *key* in a given *namespace* (default is ""), e.g.
+ 'Vehicle.Infotainment.Radio.CurrentStation':
+
+ ```text
+ Read('Vehicle.Infotainment.Radio.CurrentStation') -> 'wdr 4'
+
+ Read('Vehicle.doesNotExist') -> ERROR
+
+ Read(key: 'Private.Info', namespace: 'AppName') -> 'test'
+ ```
+
+- `Delete(key: string, namespace: string) -> StandardResponse(success: boolean, message: string)`
+
+ - Consumer wants to delete an existing *key* + *value* from a given *namespace* (default is ""), e.g.
+ 'Vehicle.Infotainment.Radio.CurrentStation':
+
+ ```text
+ Delete('Vehicle.Infotainment.Radio.CurrentStation') -> Response
+
+ Delete('Vehicle.doesNotExist') -> ERROR
+
+ Delete(key: 'Private.Info', namespace: 'AppName') -> Response
+ ```
+
+- `Search(key: string, namespace: string) -> ListResponse(success: boolean, message: string, keys: repeated string)`
+
+ - Consumer wants to list all keys that contain *key* in a given *namespace* (default is ""), e.g. 'Radio'
+
+ ```text
+ Search('Radio') -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Communication.Radio.Volume')
+
+ Search('Info') -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature')
+
+ Search('nt.Rad') -> ('Vehicle.Infotainment.Radio.CurrentStation')
+
+ Search('') -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature', 'Vehicle.Communication.Radio.Volume')
+
+ Search(key: '', namespace: 'AppName') -> ('Private.Info')
+ ```
+
+- `DeleteNodes(key: string, namespace: string) -> StandardResponse(success: boolean, message: string)`
+
+ - Consumer wants to delete all keys located in the subtree with root *key*, within the given *namespace* (default is ""), e.g.
+ 'Vehicle.Infotainment'
+ - `key = ''` returns `ERROR`
+ - This rpc assumes that keys follow a VSS-like tree structure. *key* must be the full name of an existing node.
+
+ ```text
+ DeleteNodes('Vehicle.Infotainment') -> Response //deletes ('Vehicle.Infotainment', 'Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature')
+
+ DeleteNodes('Vehicle') -> Response //deletes ('Vehicle.Infotainment', 'Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature', 'Vehicle.Communication.Radio.Volume')
+
+ DeleteNodes('') -> ERROR
+
+ DeleteNodes('DoesNotExist') -> ERROR
+
+ DeleteNodes('Vehic') -> ERROR
+
+ DeleteNodes(key: 'Private', namespace: 'AppName') -> Response //deletes ('Private.Info')
+ ```
+
+- `ListNodes(node: string, layers: optional int, namespace: string) -> ListResponse(boolean, message, repeated string keys)`
+
+ - Consumer wants to list all nodes located in the subtree with root *node* exactly *layers*
+ layers deep, within the given *namespace* (default is "") , e.g. 'Vehicle.Infotainment'
+
+ - `layers = 0` lists all keys that start in *node* any number of *layers* deep
+ - `layers` default value is 1
+ - `node = ''` returns top-level root node(s)
+ - This rpc assumes that keys follow a VSS-like tree structure. *node* must be the full name of an existing node.
+
+ ```text
+ ListNodes('Vehicle.Infotainment', 1) -> ('Vehicle.Infotainment.Radio', 'Vehicle.Infotainment.HVAC')
+
+ ListNodes('Vehicle.Infotainment', 2) -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature')
+
+ ListNodes('Vehicle', 0) -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature', 'Vehicle.Communication.Radio.Volume', 'Vehicle.Infotainment')
+
+ ListNodes('', 0) -> ('Vehicle.Infotainment.Radio.CurrentStation', 'Vehicle.Infotainment.Radio.Volume', 'Vehicle.Infotainment.HVAC.OutdoorTemperature', 'Vehicle.Communication.Radio.Volume', 'Vehicle.Infotainment', 'test')
+
+ ListNodes('Vehicle.Infotainment') -> ('Vehicle.Infotainment.Radio', 'Vehicle.Infotainment.HVAC')
+
+ ListNodes('', 1) -> ('Vehicle', 'test')
+
+ ListNodes('Vehicle.Infotainment.Radio.Volume', 1) -> ()
+
+ ListNodes('Vehicle', -1) -> ERROR
+
+ ListNodes('Vehicle.DoesNotExist', 1) -> ERROR
+
+ ListNodes(key: 'Private', namespace: 'AppName') -> ('Private.Info')
+
+ For empty data base:
+ ListNodes('', 1) -> ()
+ ```
+
+## Example Tree
+
+Note: nodes marked by \* are keys (and therefore have a value)
+
+**Namespace: ""**
+- Vehicle
+ - Infotainment \*
+ - Radio
+ - CurrentStation \*
+ - Volume \*
+ - HVAC
+ - OutdoorTemperature \*
+ - Communication
+ - Radio
+ - Volume \*
+- test \*
+
+**Namespace: "AppName"**
+- Private
+ - Info \*
+
+## Setup instructions
+
+1. Install [rust](https://rustup.rs/).
+
+2. Install the Protobuf Compiler, e.g. by downloading the latest pre-built binary for your system [here](https://github.com/protocolbuffers/protobuf/releases) and following the installation instructions included in the readme. Be sure to add your Protobuf installation to your PATH. See also the general [Protobuf installation instructions](https://github.com/protocolbuffers/protobuf?tab=readme-ov-file#protobuf-compiler-installation).
+
+3. Install a clang compiler, e.g. by downloading the latest pre-built LLVM binary for your system [here](https://github.com/llvm/llvm-project/releases) and adding the LIBCLANG_PATH variable to your environment.
+
+4. Build application.
+
+ ```bash
+ cargo build
+ ```
+
+5. Run tests.
+
+ ```bash
+ cargo test
+ ```
+
+6. Start server.
+
+ ```bash
+ cargo run --release --bin server
+ ```
+
+## Remote Procedure Call Usage
+
+To ensure your API is working as expected, you can use [Insomnia](https://insomnia.rest/) to manually send remote procedure calls to the API, following the instructions provided in the [Insomnia documentation](https://docs.insomnia.rest/insomnia/requests#send-a-grpc-request). For each procedure call, an example is given below:
+
+```text
+DestroyDB: {}
+
+Write: { "key": "foo", "value": "foobar", "namespace": "bar" }
+
+Read: { "key": "foo", "namespace": "bar" }
+
+Delete: { "key": "foo", "namespace": "bar" }
+
+Search: { "key": "foo", "namespace": "bar" }
+
+DeleteNodes: { "key": "foo", "namespace": "bar" }
+
+ListNodes: { "node": "foo", "layers": 1, "namespace": "bar" }
+```
diff --git a/docs/06_Component_Documentation/13_CARLA_with_AGL.md b/docs/06_Component_Documentation/13_CARLA_with_AGL.md new file mode 100644 index 0000000..075da80 --- /dev/null +++ b/docs/06_Component_Documentation/13_CARLA_with_AGL.md @@ -0,0 +1,104 @@ +--- +title: CARLA with AGL +--- +# CARLA with AGL (WIP) + +## Setting up CARLA + +You can follow the steps provided in the [CARLA documentation](https://carla.readthedocs.io/en/latest/start_quickstart/#carla-installation) for installing CARLA. + +We recommend using the [latest release](https://github.com/carla-simulator/carla/releases/), and using the supported Python version to run the `carla_to_CAN.py` Script. + +1. Running the CARLA Server + + ```bash + # Move to the installation directory + $ cd /path/to/CARLA_<version> + + # Start the CARLA Server + $ ./CarlaUE4.sh + + # To run using minimum resources + $ ./CarlaUE4.sh -quality-level=Low -prefernvidia + ``` + + You may also add the `-RenderOffScreen` flag to start CARLA in off-screen mode. Refer to the various [rendering options](https://carla.readthedocs.io/en/latest/adv_rendering_options/#no-rendering-mode) for more details. + + Another way of running the CARLA server without a display is by using [CARLA in Docker](https://carla.readthedocs.io/en/latest/build_docker/). + +2. Starting a Manual Simulation + + ```bash + # Navigate to directory containing the demo python scripts + # + $ cd /path/to/CARLA_<version>/PythonAPI/examples + ``` + + Create a Python virtual environment and resolve dependencies + ```bash + $ python3 -m venv carlavenv + $ source carlavenv/bin/activate + $ pip3 install -r requirements.txt + + # Start the manual_control.py script + $ python3 manual_control.py + ``` + +## Converting CARLA data into CAN + +The `carla_to_CAN.py` script can be run run alongside an existing CARLA simulation to fetch data and convert it into CAN messages based on the [agl-vcar.dbc](https://git.automotivelinux.org/src/agl-dbc/plain/agl-vcar.dbc) file. + +While the `record_playback.py` script is responsible for recording amd playing back the CAN data for later sessions. + +_NOTE_: This does **not** require the CARLA server to be running. + +To access these scripts, clone the [AGL Demo Control Panel](https://gerrit.automotivelinux.org/gerrit/admin/repos/src/agl-demo-control-panel,general) project. + +```bash +# Move to the Scripts directory +$ cd /path/to//agl-demo-control-panel/Scripts + +# Fetch the agl-vcar.dbc file +$ wget -nd -c "https://git.automotivelinux.org/src/agl-dbc/plain/agl-vcar.dbc" +``` + +Create a Python virtual environment and resolve dependencies +```bash +$ python3 -m venv carlavenv +$ source carlavenv/bin/activate +$ pip3 install -r requirements.txt + +# Optionally, set up the vcan0 interface +$ ./vcan.sh +``` + +1. Converting CARLA Data into CAN + + ```bash + $ python -u carla_to_CAN.py + # OR + $ python -u carla_to_CAN.py --host <carla_server_ip> --port <carla_server_port> + ``` + +2. Recording and Playback of CAN messages + + ```bash + $ python -u record_playback.py + # OR + $ python -u record_playback.py --interface (or) -i can0 # default vcan0 + ``` + + CLI Options: + + - 1: Captures CAN messages and writes them into 'can_messages.txt' + - 2: Replays captured CAN messages + - 3: Exit + + +## CAN interface to AGL Demo Platform + +To use the **`carla_to_CAN.py`** and **`record_playback.py`** scripts to send messages on the CAN interface, one can use the CAN bus or use CAN over Ethernet using **cannelloni**. + +**cannelloni** is available in AGL, just add `IMAGE_INSTALL:append = " cannelloni"` to your `conf/local.conf`. + +To set up the CAN interface between the Host system and the target machine(s) refer to the [cannelloni docs](https://github.com/mguentner/cannelloni). diff --git a/docs/06_Component_Documentation/Demo_Application/01_Momi_Navi.md b/docs/06_Component_Documentation/Demo_Application/01_Momi_Navi.md new file mode 100644 index 0000000..ac0f886 --- /dev/null +++ b/docs/06_Component_Documentation/Demo_Application/01_Momi_Navi.md @@ -0,0 +1,35 @@ +--- +title: Momi Navi +--- + +# What is Momi Navi + +Momi Navi is a example application for navigation. It run on simple ivi demo (that name is momi IVI) for the instrument cluster container integration. + + + +## How to use + +When you build the instrument cluster integration without momi navi specific setting, it will not show map. You need momi navi specific setting. + +The momi navi uses mapbox gl plugin fo Qt location. It requires own access token to download map. You can create own mapbox access token following [this page](https://docs.mapbox.com/help/getting-started/access-tokens/). + +After creation, please set environment variable to local.conf (or site.conf). + +``` +MOMIMAP_MAPBOX_ACCESS_TOKEN = "Your access token." +``` + +## Extra configuration + +Momi navi supports configuration for customize. + +When you set environment variable to local.conf (or site.conf), you can customize Momi navi. + +| Environment variable | Customizing parametor | +|:---|:---| +| MOMIMAP_MAPBOX_STYLE | Map style | +| MOMIMAP_INITIAL_LATITUDE | Default location | +| MOMIMAP_INITIAL_LONGITUDE | Default location | + +How to create map style, please check [this page](https://docs.mapbox.com/studio-manual/guides/map-styling/). diff --git a/docs/06_Component_Documentation/Demo_Application/images/mominavi.jpg b/docs/06_Component_Documentation/Demo_Application/images/mominavi.jpg Binary files differnew file mode 100644 index 0000000..8e30c05 --- /dev/null +++ b/docs/06_Component_Documentation/Demo_Application/images/mominavi.jpg diff --git a/docs/06_Component_Documentation/images/agl-voice-agent/AGL_VoiceAgent_updated_architecture.png b/docs/06_Component_Documentation/images/agl-voice-agent/AGL_VoiceAgent_updated_architecture.png Binary files differnew file mode 100644 index 0000000..b302a45 --- /dev/null +++ b/docs/06_Component_Documentation/images/agl-voice-agent/AGL_VoiceAgent_updated_architecture.png diff --git a/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-1.png b/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-1.png Binary files differnew file mode 100644 index 0000000..ecd5c04 --- /dev/null +++ b/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-1.png diff --git a/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-2.png b/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-2.png Binary files differnew file mode 100644 index 0000000..7a12045 --- /dev/null +++ b/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-2.png diff --git a/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-3.png b/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-3.png Binary files differnew file mode 100644 index 0000000..2e2b5d1 --- /dev/null +++ b/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-3.png diff --git a/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-settings.png b/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-settings.png Binary files differnew file mode 100644 index 0000000..ebc5007 --- /dev/null +++ b/docs/06_Component_Documentation/images/agl-voice-agent/flutter-ics-homescreen-voice-assistant-settings.png diff --git a/docs/06_Component_Documentation/images/agl-voice-agent/voice-assistant-updated-1.png b/docs/06_Component_Documentation/images/agl-voice-agent/voice-assistant-updated-1.png Binary files differnew file mode 100644 index 0000000..c32b395 --- /dev/null +++ b/docs/06_Component_Documentation/images/agl-voice-agent/voice-assistant-updated-1.png diff --git a/docs/06_Component_Documentation/images/agl-voice-agent/voice-assistant-updated-2.png b/docs/06_Component_Documentation/images/agl-voice-agent/voice-assistant-updated-2.png Binary files differnew file mode 100644 index 0000000..1d70e6c --- /dev/null +++ b/docs/06_Component_Documentation/images/agl-voice-agent/voice-assistant-updated-2.png |
