Add meta-agl-kuksa-val

Add new meta-agl-kuksa-val layer, and populate it with the base
KUKSA.val databroker and VSS schema recipes from meta-agl, as well
as the recipes for the Python modules they depend on.

Additional changes:
- Installation of AGL-specific configuration has been removed from
  the kuksa-can-provider recipe.  It now packages the upstream
  example/test configuration in a new kuksa-can-provider-conf-example
  package that can be replaced via the RPROVIDES scheme.  Some minor
  tweaks for paths are made to the upstream .ini file to match
  FHS expectations.

Bug-AGL: SPEC-5136

Change-Id: I3128a223f4fcf16e496cb27e26afcaee0a28979a
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Reviewed-on: https://gerrit.automotivelinux.org/gerrit/c/AGL/meta-agl/+/30040
Tested-by: Jenkins Job builder account
ci-image-build: Jenkins Job builder account
ci-image-boot-test: Jenkins Job builder account
Reviewed-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>

6 files changed, 378 insertions, 0 deletions

diff --git a/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0001-dbc2val-add-installation-mechanism.patch b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0001-dbc2val-add-installation-mechanism.patch
new file mode 100644
index 000000000..6a9c1ba14
--- /dev/null
+++ b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0001-dbc2val-add-installation-mechanism.patch
@@ -0,0 +1,77 @@
+From 73dd680486b72d15d6f4c7aa129219ecdbcbd7b2 Mon Sep 17 00:00:00 2001
+From: Scott Murray <scott.murray@konsulko.com>
+Date: Wed, 19 Apr 2023 15:55:01 -0400
+Subject: [PATCH 1/4] dbc2val: add installation mechanism
+
+Add setup.py and setup.cfg to allow installing the dbcfeederlib
+module and dbcfeeder.py in a way suitable for packaging.
+
+Upstream-Status: pending
+
+Signed-off-by: Scott Murray <scott.murray@konsulko.com>
+---
+ setup.cfg | 30 ++++++++++++++++++++++++++++++
+ setup.py  | 14 ++++++++++++++
+ 2 files changed, 44 insertions(+)
+ create mode 100644 setup.cfg
+ create mode 100644 setup.py
+
+diff --git a/setup.cfg b/setup.cfg
+new file mode 100644
+index 0000000..cb64407
+--- /dev/null
++++ b/setup.cfg
+@@ -0,0 +1,30 @@
++[metadata]
++name = dbcfeeder
++author = Sebastian Schildt, Naresh Nayak, Wenwen Chen
++author_email = sebastian.schildt@de.bosch.com, naresh.nayak@de.bosch.com, wenwen.chen@de.bosch.com
++description = KUKSA.val CAN provider
++long_description = file:README.md
++long_description_content_type = text/markdown
++url=https://github.com/eclipse-kuksa/kuksa-can-provider
++project_urls=
++  Source=https://github.com/eclipse-kuksa/kuksa-can-provider
++  Bug Tracker=https://github.com/eclipse-kuksa/kuksa-can-provider/issues
++classifiers =
++    Intended Audience :: Developers
++    Development Status :: 3 - Alpha
++    Environment :: Console
++    Programming Language :: Python :: 3
++    License :: OSI Approved :: Apache Software License
++    Operating System :: OS Independent
++    Topic :: Software Development
++
++license_file = LICENSE
++
++[options]
++python_requires = >=3.6
++install_requires=
++    pyserial
++    pyyaml
++    kuksa-client
++packages=dbcfeederlib
++scripts=dbcfeeder.py
+diff --git a/setup.py b/setup.py
+new file mode 100644
+index 0000000..c5fb2b7
+--- /dev/null
++++ b/setup.py
+@@ -0,0 +1,14 @@
++from setuptools import setup
++
++setup(
++    version_config={
++        "template": "{tag}",
++        "dev_template": "{tag}-{ccount}",
++        "dirty_template": "{tag}+{ccount}-dirty",
++        "starting_version": "0.1.0",
++        "version_callback": None,
++        "version_file": None,
++        "count_commits_from_version_file": False
++    },
++    setup_requires=['setuptools-git-versioning'],
++)
+-- 
+2.44.0
+
diff --git a/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0002-dbc2val-usability-improvements.patch b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0002-dbc2val-usability-improvements.patch
new file mode 100644
index 000000000..91a309240
--- /dev/null
+++ b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0002-dbc2val-usability-improvements.patch
@@ -0,0 +1,80 @@
+From 2e4e1f9147f1ebe5b545ae0cab41341e3abb00ae Mon Sep 17 00:00:00 2001
+From: Scott Murray <scott.murray@konsulko.com>
+Date: Sat, 15 Jun 2024 13:13:17 -0400
+Subject: [PATCH 2/4] dbc2val: usability improvements
+
+Changes:
+- Tweaked default configuration file search path to better match
+  Linux FHS.  First look for a config.ini or dbc_feeder.ini in
+  /etc/kuksa-can-provider, then fall back to /etc/dbc_feeder.ini
+  before using trying other possible paths.
+- Add catching of exceptions around CAN device opening so that the
+  script can exit cleanly with an error message if the device is
+  not available.
+- Tweaked DBC default value file command line argument parsing so
+  that it does not attempt to fallback to "dbc_default_values.json"
+  in the current working directory.  That likely works for upstream
+  test scenarios, but prevents running on a target.
+
+Upstream-Status: pending
+
+Signed-off-by: Scott Murray <scott.murray@konsulko.com>
+---
+ dbcfeeder.py | 19 +++++++++++++++----
+ 1 file changed, 15 insertions(+), 4 deletions(-)
+
+diff --git a/dbcfeeder.py b/dbcfeeder.py
+index a1ef174..c252503 100755
+--- a/dbcfeeder.py
++++ b/dbcfeeder.py
+@@ -144,7 +144,11 @@ class Feeder:
+                     whitelisted_frame_ids.append(filter.can_id)  # type: ignore
+                 elm2canbridge.elm2canbridge(canport, self._elmcan_config, whitelisted_frame_ids)
+ 
+-            self._reader.start()
++            try:
++                self._reader.start()
++            except:
++                log.error("Could not open %s, exiting", canport)
++                sys.exit(-1)
+ 
+             receiver = threading.Thread(target=self._run_receiver)
+             receiver.start()
+@@ -165,7 +169,12 @@ class Feeder:
+             # For now creating another bus
+             # Maybe support different buses for downstream/upstream in the future
+ 
+-            self._canclient = CANClient(interface="socketcan", channel=canport, can_fd=can_fd)
++            self._canclient = None
++            try:
++                self._canclient = CANClient(interface="socketcan", channel=canport, can_fd=can_fd)
++            except:
++                log.error("Could not open %s, exiting", canport)
++                sys.exit(-1)
+ 
+             transmitter = threading.Thread(target=self._run_transmitter)
+             transmitter.start()
+@@ -335,8 +344,10 @@ def _parse_config(filename: str) -> configparser.ConfigParser:
+         configfile = filename
+     else:
+         config_candidates = [
+-            "/config/dbc_feeder.ini",
++            "/etc/kuksa-can-provider/config.ini",
++            "/etc/kuksa-can-provider/dbc_feeder.ini",
+             "/etc/dbc_feeder.ini",
++            "/config/dbc_feeder.ini",
+             "config/dbc_feeder.ini",
+         ]
+         for candidate in config_candidates:
+@@ -534,7 +545,7 @@ def main(argv):
+     elif os.environ.get("DBC_DEFAULT_FILE"):
+         dbc_default = os.environ.get("DBC_DEFAULT_FILE")
+     else:
+-        dbc_default = config.get(CONFIG_SECTION_CAN, CONFIG_OPTION_DBC_DEFAULT_FILE, fallback="dbc_default_values.json")
++        dbc_default = config.get(CONFIG_SECTION_CAN, CONFIG_OPTION_DBC_DEFAULT_FILE, fallback=None)
+ 
+     if args.mapping:
+         mappingfile = args.mapping
+-- 
+2.44.0
+
diff --git a/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0003-dbc2val-fix-token-file-configuration-option.patch b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0003-dbc2val-fix-token-file-configuration-option.patch
new file mode 100644
index 000000000..faf94e0af
--- /dev/null
+++ b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0003-dbc2val-fix-token-file-configuration-option.patch
@@ -0,0 +1,32 @@
+From 0b5822c6a8f8fa489bc7ae67f91284ac150f6518 Mon Sep 17 00:00:00 2001
+From: Scott Murray <scott.murray@konsulko.com>
+Date: Sat, 15 Jun 2024 13:18:08 -0400
+Subject: [PATCH 3/4] dbc2val: fix token file configuration option
+
+The client library changed the token option name to
+'token_or_tokenfile', update things to match so that token
+location can be configured again for dbcfeeder.py.
+
+Upstream-Status: pending
+
+Signed-off-by: Scott Murray <scott.murray@konsulko.com>
+---
+ dbcfeederlib/serverclientwrapper.py | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/dbcfeederlib/serverclientwrapper.py b/dbcfeederlib/serverclientwrapper.py
+index 498e1b6..fa43d28 100644
+--- a/dbcfeederlib/serverclientwrapper.py
++++ b/dbcfeederlib/serverclientwrapper.py
+@@ -56,7 +56,7 @@ class ServerClientWrapper(clientwrapper.ClientWrapper):
+         self._client_config["insecure"] = not self._tls
+         # Do not set token if it is empty to allow default client lib info to be used
+         if self._token_path != "":
+-            self._client_config["token"] = self._token_path
++            self._client_config["token_or_tokenfile"] = self._token_path
+ 
+         if self._root_ca_path:
+             self._client_config['cacertificate'] = self._root_ca_path
+-- 
+2.44.0
+
diff --git a/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0004-Enable-val2dbc-for-sensor-values.patch b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0004-Enable-val2dbc-for-sensor-values.patch
new file mode 100644
index 000000000..e45fb4cf5
--- /dev/null
+++ b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/0004-Enable-val2dbc-for-sensor-values.patch
@@ -0,0 +1,174 @@
+From 937218a357ac1914fe410cf3ad31a67d54a63270 Mon Sep 17 00:00:00 2001
+From: Scott Murray <scott.murray@konsulko.com>
+Date: Mon, 17 Jun 2024 17:07:44 -0400
+Subject: [PATCH 4/4] Enable val2dbc for sensor values
+
+Rework to allow val2dbc mode to write out sensor values in
+addition to actuator target values.
+
+Upstream-Status: pending
+
+Signed-off-by: Scott Murray <scott.murray@konsulko.com>
+---
+ dbcfeeder.py                            |  8 ++++++--
+ dbcfeederlib/databrokerclientwrapper.py | 18 ++++++++++++------
+ dbcfeederlib/dbc2vssmapper.py           | 21 ++++++++++-----------
+ dbcfeederlib/serverclientwrapper.py     |  2 +-
+ mapping/README.md                       |  2 --
+ 5 files changed, 29 insertions(+), 22 deletions(-)
+
+diff --git a/dbcfeeder.py b/dbcfeeder.py
+index c252503..c1e20c4 100755
+--- a/dbcfeeder.py
++++ b/dbcfeeder.py
+@@ -289,19 +289,23 @@ class Feeder:
+             log.debug("Processing %d VSS Data Entry updates", len(updates))
+             dbc_signal_names: Set[str] = set()
+             for update in updates:
++                value = None
+                 if update.entry.value is not None:
+-                    # This should never happen as we do not subscribe to current value
+                     log.warning(
+                         "Current value for %s is now: %s of type %s",
+                         update.entry.path, update.entry.value.value, type(update.entry.value.value)
+                     )
++                    value = update.entry.value.value
+ 
+                 if update.entry.actuator_target is not None:
+                     log.debug(
+                         "Target value for %s is now: %s of type %s",
+                         update.entry.path, update.entry.actuator_target, type(update.entry.actuator_target.value)
+                     )
+-                    affected_signals = self._mapper.handle_update(update.entry.path, update.entry.actuator_target.value)
++                    value = update.entry.actuator_target.value
++
++                if value != None:
++                    affected_signals = self._mapper.handle_update(update.entry.path, value)
+                     dbc_signal_names.update(affected_signals)
+ 
+             messages_to_send: Set[Message] = set()
+diff --git a/dbcfeederlib/databrokerclientwrapper.py b/dbcfeederlib/databrokerclientwrapper.py
+index 716ee6d..db2b80a 100644
+--- a/dbcfeederlib/databrokerclientwrapper.py
++++ b/dbcfeederlib/databrokerclientwrapper.py
+@@ -199,14 +199,20 @@ class DatabrokerClientWrapper(clientwrapper.ClientWrapper):
+     def supports_subscription(self) -> bool:
+         return True
+ 
+-    async def subscribe(self, vss_names: List[str], callback):
++    async def subscribe(self, vss_entries: dict[str, str], callback):
+         """Create a subscription and invoke the callback when data received."""
+         entries: List[SubscribeEntry] = []
+-        for name in vss_names:
+-            # Always subscribe to target
+-            subscribe_entry = SubscribeEntry(name, View.FIELDS, [Field.ACTUATOR_TARGET])
+-            log.info("Subscribe entry: %s", subscribe_entry)
+-            entries.append(subscribe_entry)
++        for name, signal_type in vss_entries.items():
++            if signal_type == "actuator":
++                subscribe_entry = SubscribeEntry(name, View.FIELDS, [Field.ACTUATOR_TARGET])
++                log.info("Subscribe entry: %s", subscribe_entry)
++                entries.append(subscribe_entry)
++            if signal_type == "sensor":
++                subscribe_entry = SubscribeEntry(name, View.FIELDS, [Field.VALUE])
++                log.info("Subscribe entry: %s", subscribe_entry)
++                entries.append(subscribe_entry)
++        if not entries:
++            return
+ 
+         # If there is a path VSSClient will request a secure connection
+         if self._tls and self._root_ca_path:
+diff --git a/dbcfeederlib/dbc2vssmapper.py b/dbcfeederlib/dbc2vssmapper.py
+index 218f693..2be5e98 100644
+--- a/dbcfeederlib/dbc2vssmapper.py
++++ b/dbcfeederlib/dbc2vssmapper.py
+@@ -69,12 +69,13 @@ class VSSMapping:
+     parser: Parser = Parser()
+ 
+     def __init__(self, vss_name: str, dbc_name: str, transform: dict, interval_ms: int,
+-                 on_change: bool, datatype: str, description: str):
++                 on_change: bool, signal_type: str, datatype: str, description: str):
+         self.vss_name = vss_name
+         self.dbc_name = dbc_name
+         self.transform = transform
+         self.interval_ms = interval_ms
+         self.on_change = on_change
++        self.signal_type = signal_type
+         self.datatype = datatype
+         self.description = description
+         # For time comparison (interval_ms) we store last value used for comparison. Unit seconds.
+@@ -347,7 +348,7 @@ class Mapper(DBCParser):
+         if can_signal_name not in self._dbc2vss_mapping:
+             self._dbc2vss_mapping[can_signal_name] = []
+         mapping_entry = VSSMapping(expanded_name, can_signal_name, transformation_definition, interval, on_change,
+-                                   node["datatype"], node["description"])
++                                   node["type"], node["datatype"], node["description"])
+         self._dbc2vss_mapping[can_signal_name].append(mapping_entry)
+ 
+         for msg_def in self.get_messages_for_signal(can_signal_name):
+@@ -398,7 +399,7 @@ class Mapper(DBCParser):
+             log.warning("Ignoring \"interval_ms\" property of mapping definition for %s", expanded_name)
+ 
+         mapping_entry = VSSMapping(expanded_name, can_signal_name, transform, interval, on_change,
+-                                   node["datatype"], node["description"])
++                                   node["type"], node["datatype"], node["description"])
+         if can_signal_name not in self._vss2dbc_mapping:
+             self._vss2dbc_mapping[expanded_name] = []
+         self._vss2dbc_mapping[expanded_name].append(mapping_entry)
+@@ -426,12 +427,7 @@ class Mapper(DBCParser):
+         if dbc2vss_def is not None:
+             self._analyze_dbc2vss(expanded_name, node, dbc2vss_def)
+         if "vss2dbc" in node:
+-            if node["type"] == "actuator":
+-                self._analyze_vss2dbc(expanded_name, node, node["vss2dbc"])
+-            else:
+-                # vss2dbc is handled by subscription to target value, so only makes sense for actuators
+-                log.error("vss2dbc only allowed for actuators, VSS signal %s is not an actuator!", expanded_name)
+-                sys.exit(-1)
++            self._analyze_vss2dbc(expanded_name, node, node["vss2dbc"])
+ 
+     def _traverse_vss_node(self, name, node, prefix=""):
+         """
+@@ -474,9 +470,12 @@ class Mapper(DBCParser):
+         """Get all CAN signal names for which a mapping to a VSS Data Entry exists."""
+         return self._dbc2vss_mapping.keys()
+ 
+-    def get_vss2dbc_entries(self) -> KeysView[str]:
++    def get_vss2dbc_entries(self) -> Dict[str, str]:
+         """Get all VSS Data Entry paths for which a mapping to a CAN signal name exists."""
+-        return self._vss2dbc_mapping.keys()
++        entries: Dict[str, str] = {}
++        for name, mappings in self._vss2dbc_mapping.items():
++            entries[name] = mappings[0].signal_type
++        return entries
+ 
+     def get_vss_names(self) -> Set[str]:
+         """Get all VSS names used in mappings, both vss2dbc and dbc2vss"""
+diff --git a/dbcfeederlib/serverclientwrapper.py b/dbcfeederlib/serverclientwrapper.py
+index fa43d28..86b2ceb 100644
+--- a/dbcfeederlib/serverclientwrapper.py
++++ b/dbcfeederlib/serverclientwrapper.py
+@@ -122,6 +122,6 @@ class ServerClientWrapper(clientwrapper.ClientWrapper):
+         log.info("Feature not implemented")
+         return False
+ 
+-    async def subscribe(self, vss_names: List[str], callback):
++    async def subscribe(self, vss_entries: dict[str, str], callback):
+         log.error("Feature not implemented")
+         return
+diff --git a/mapping/README.md b/mapping/README.md
+index 2155f28..ea6de07 100644
+--- a/mapping/README.md
++++ b/mapping/README.md
+@@ -44,8 +44,6 @@ This is built on the assumption that the DBC provider always send target values
+ Having separate configurations (`dbc2vss` and `vss2dbc`) is needed as wanted value and actual value never are sent
+ by the same DBC signal, they are not even part of the same CAN-frame.
+ 
+-*This means that `vss2dbc` only can be used for actuators, as only actuators have target values!*
+-
+ ## Example mapping files
+ 
+ Example mapping files for various VSS versions can be found in this folder.
+-- 
+2.44.0
+
diff --git a/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/kuksa-can-provider.default b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/kuksa-can-provider.default
new file mode 100644
index 000000000..aad0bc419
--- /dev/null
+++ b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/kuksa-can-provider.default
@@ -0,0 +1,3 @@
+# For output only mode:
+#EXTRA_ARGS="--val2dbc --no-dbc2val --dbc-default /etc/kuksa-can-provider/dbc_default_values.json"
+#LOG_LEVEL=debug
diff --git a/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/kuksa-can-provider.service b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/kuksa-can-provider.service
new file mode 100644
index 000000000..e43dd47fd
--- /dev/null
+++ b/meta-agl-kuksa-val/recipes-connectivity/kuksa-val/kuksa-can-provider/kuksa-can-provider.service
@@ -0,0 +1,12 @@
+[Unit]
+Description=Eclipse KUKSA.val DBC feeder
+Requires=kuksa-databroker.service
+After=kuksa-databroker.service
+
+[Service]
+EnvironmentFile=-/etc/default/kuksa-can-provider
+ExecStart=/usr/bin/dbcfeeder.py $EXTRA_ARGS
+Restart=on-failure
+
+[Install]
+WantedBy=multi-user.target