kuksa-dbc-feeder: add recipe and dependencies

Add a kuksa-dbc-feeder recipe to build the sample CAN feeder for the
KUKSA.val Vehicle Information Service (VIS) server and add it to the
agl-demo-platform image by adding it to packagegroup-agl-ivi-services.
Local patches are applied to enable building with OpenEmbedded, make
installation into standard Linux FHS locations feasible, and improve
usability on target with respect to logging and error handling.
These will be discussed with upstream to hopefully get them
integrated.

Additional changes:
- Add a recipe for the required kuksa-viss-client Python module
  included with the KUKSA.val source tree.  The module is also
  available via pypi.org, but keeping all the pieces in lockstep for
  now seems better, so it is built out of a clone of the kuksa.val
  repo using the same SRCREV as the server and kuksa-dbc-feeder.
- Add backports of recipes for required Python modules:
    python3-bitstruct, python3-can, python3-cantools, python3-cmd2,
    python3-diskcache, python3-importlib-metadata,
    python3-setuptools-scm, python3-textparser, python3-toml
  This is required to both add recipes missing from dunfell as well
  as to meet version requirements for kuksa-viss-client and
  kuksa-dbc-feeder.  For the most part the recipes were picked from
  the poky and meta-openembedded kirkstone branches before the
  significant Python module build rework done for the kirkstone M4
  milestone.  There is some potential for updating them to newer
  backports, but this will be a low priority unless there are
  explicit requests.
- Add bbappend for python3-cantools to add some missing runtime
  dependencies discovered during testing.  This will be addressed
  upstream in meta-python as time permits.
- Add new recipes for required Python modules:
    python3-argparse-addons, python3-can-j1939,
    python3-py-expression-eval, python3-setuptools-git-versioning
  These will be upstreamed to meta-python as time permits.
- Add minimal DBC file and mapping configuration.  At present the
  agl-vcar.dbc file only contains the minimum message definitions
  known to be required for the AGL demo platform, namely vehicle
  and engine speeds and the steering wheel events.
- Add can-dev-helper recipe to install a systemd unit and script to
  ensure a CAN interface is available for testing.  This is a tweaked
  version of what was previously used with agl-service-can-low-level.

Bug-AGL: SPEC-4405

Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Change-Id: I4f9fe9c34165da30005d39689739c73dfdbbc615

7 files changed, 356 insertions, 0 deletions

diff --git a/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/0001-dbc2val-add-installation-mechanism.patch b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/0001-dbc2val-add-installation-mechanism.patch
new file mode 100644
index 000000000..42ab7886d
--- /dev/null
+++ b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/0001-dbc2val-add-installation-mechanism.patch
@@ -0,0 +1,166 @@
+From 6e7176cf6cbf7587c7e7999da56b59a32cf81f82 Mon Sep 17 00:00:00 2001
+From: Scott Murray <scott.murray@konsulko.com>
+Date: Wed, 11 May 2022 15:31:25 -0400
+Subject: [PATCH] dbc2val: add installation mechanism
+
+Add setup.py and setup.cfg to allow installing the dbc2val module
+and dbcfeeder.py in a way suitable for packaging.  Some of the
+imports in the scripts have been tweaked to enable running against
+an installed copy of dbc2val.
+
+Signed-off-by: Scott Murray <scott.murray@konsulko.com>
+---
+ kuksa_feeders/dbc2val/__init__.py      |  0
+ kuksa_feeders/dbc2val/dbc2vssmapper.py | 12 ++++-----
+ kuksa_feeders/dbc2val/dbcfeeder.py     |  6 ++---
+ kuksa_feeders/setup.cfg                | 31 ++++++++++++++++++++++
+ kuksa_feeders/setup.py                 | 36 ++++++++++++++++++++++++++
+ 5 files changed, 75 insertions(+), 10 deletions(-)
+ create mode 100644 kuksa_feeders/dbc2val/__init__.py
+ create mode 100644 kuksa_feeders/setup.cfg
+ create mode 100644 kuksa_feeders/setup.py
+
+diff --git a/kuksa_feeders/dbc2val/__init__.py b/kuksa_feeders/dbc2val/__init__.py
+new file mode 100644
+index 0000000..e69de29
+diff --git a/kuksa_feeders/dbc2val/dbc2vssmapper.py b/kuksa_feeders/dbc2val/dbc2vssmapper.py
+index a43d1bd..1718154 100644
+--- a/kuksa_feeders/dbc2val/dbc2vssmapper.py
++++ b/kuksa_feeders/dbc2val/dbc2vssmapper.py
+@@ -11,8 +11,8 @@
+ ########################################################################
+ 
+ import yaml 
+-import transforms.mapping
+-import transforms.math
++from dbc2val.transforms import mapping
++from dbc2val.transforms import math
+ 
+ 
+ class mapper:
+@@ -22,9 +22,9 @@ class mapper:
+             self.mapping = yaml.full_load(file)
+ 
+         self.transforms={}
+-        self.transforms['fullmapping']=transforms.mapping.mapping(discard_non_matching_items=True)
+-        self.transforms['partialmapping']=transforms.mapping.mapping(discard_non_matching_items=False)
+-        self.transforms['math']=transforms.math.math()
++        self.transforms['fullmapping']=mapping.mapping(discard_non_matching_items=True)
++        self.transforms['partialmapping']=mapping.mapping(discard_non_matching_items=False)
++        self.transforms['math']=math.math()
+ 
+ 
+ 
+@@ -62,4 +62,4 @@ class mapper:
+ 
+     def __getitem__(self, item):
+         return self.mapping[item]
+-    
+\ No newline at end of file
++    
+diff --git a/kuksa_feeders/dbc2val/dbcfeeder.py b/kuksa_feeders/dbc2val/dbcfeeder.py
+index 26a0c23..a7cec86 100755
+--- a/kuksa_feeders/dbc2val/dbcfeeder.py
++++ b/kuksa_feeders/dbc2val/dbcfeeder.py
+@@ -16,10 +16,7 @@ import configparser
+ import queue
+ import json
+ 
+-import dbc2vssmapper
+-import dbcreader
+-import j1939reader
+-import elm2canbridge
++from dbc2val import dbc2vssmapper, dbcreader, j1939reader, elm2canbridge
+ 
+ scriptDir= os.path.dirname(os.path.realpath(__file__))
+ sys.path.append(os.path.join(scriptDir, "../../"))
+@@ -27,6 +24,7 @@ from kuksa_viss_client import KuksaClientThread
+ 
+ print("kuksa.val DBC example feeder")
+ config_candidates=['/config/dbc_feeder.ini', '/etc/dbc_feeder.ini', os.path.join(scriptDir, 'config/dbc_feeder.ini')]
++configfile = None
+ for candidate in config_candidates:
+     if os.path.isfile(candidate):
+         configfile=candidate
+diff --git a/kuksa_feeders/setup.cfg b/kuksa_feeders/setup.cfg
+new file mode 100644
+index 0000000..ca6ad0b
+--- /dev/null
++++ b/kuksa_feeders/setup.cfg
+@@ -0,0 +1,31 @@
++[metadata]
++name = dbc2val
++author = Sebastian Schildt, Naresh Nayak, Wenwen Chen
++author_email = sebastian.schildt@de.bosch.com, naresh.nayak@de.bosch.com, wenwen.chen@de.bosch.com
++description = kuksa.val DBC feeder
++long_description = file:README.md
++long_description_content_type = text/markdown
++url=https://github.com/eclipse/kuksa.val
++project_urls=
++  Source=https://github.com/eclipse/kuksa.val/tree/master/kuksa_feeders/dbc2val
++  Bug Tracker=https://github.com/eclipse/kuksa.val/issues
++classifiers = 
++    Intended Audience :: Developers
++    Development Status :: 3 - Alpha
++    Environment :: Console
++    Programming Language :: Python :: 3
++    License :: OSI Approved :: Eclipse Public License 2.0 (EPL-2.0)
++    Operating System :: OS Independent
++    Topic :: Software Development
++    
++license_file = LICENSE
++
++[options]
++python_requires = >=3.6
++install_requires=
++    pyserial
++    pyyaml
++    kuksa-viss-client
++packages=find:
++include_package_data = True
++scripts=dbc2val/dbcfeeder.py
+diff --git a/kuksa_feeders/setup.py b/kuksa_feeders/setup.py
+new file mode 100644
+index 0000000..ad08d17
+--- /dev/null
++++ b/kuksa_feeders/setup.py
+@@ -0,0 +1,36 @@
++# To avoid shipping dbcfeeder.py in the module itself, use the
++# technique outlined at:
++#
++# https://stackoverflow.com/a/50592100
++#
++# This can be removed if the directory structure is ever reorganized
++# more along the lines of upstream recommendations.
++#
++
++import fnmatch
++from setuptools import find_packages, setup
++from setuptools.command.build_py import build_py as build_py_orig
++
++exclude = ['dbc2val.dbcfeeder']
++
++class build_py(build_py_orig):
++    def find_package_modules(self, package, package_dir):
++        modules = super().find_package_modules(package, package_dir)
++        print("modules = %s" % modules)
++        return [(pkg, mod, file, ) for (pkg, mod, file, ) in modules
++                if not any(fnmatch.fnmatchcase(pkg + '.' + mod, pat=pattern)
++                           for pattern in exclude)]
++
++setup(
++    version_config={
++        "template": "{tag}",
++        "dev_template": "{tag}-{ccount}",
++        "dirty_template": "{tag}-{ccount}-dirty",
++        "starting_version": "0.1.11",
++        "version_callback": None,
++        "version_file": None,
++        "count_commits_from_version_file": False
++    },
++    setup_requires=['setuptools-git-versioning'],
++    cmdclass={'build_py': build_py},
++)
+-- 
+2.35.1
+
diff --git a/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/0002-dbc2val-usability-improvements.patch b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/0002-dbc2val-usability-improvements.patch
new file mode 100644
index 000000000..12e366aa8
--- /dev/null
+++ b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/0002-dbc2val-usability-improvements.patch
@@ -0,0 +1,120 @@
+From fe10a3645e77cd8122d3d312d317bedcb88bc683 Mon Sep 17 00:00:00 2001
+From: Scott Murray <scott.murray@konsulko.com>
+Date: Thu, 12 May 2022 17:39:56 +0200
+Subject: [PATCH] dbc2val: usability improvements
+
+Changes:
+- Tweaked default configuration file search path to better match
+  Linux FHS and kuksa-val-server.  First look for a config.ini in
+  /etc/kuksa-dbc-feeder, then /etc/dbc_feeder.ini.
+- Added a command-line option to specify configuration file, this
+  should allow running two instances against different interfaces.
+- Added verbosity command-line option and made several messages
+  verbose mode only to avoid log spamming.
+- Added '-u' option to python invocation to disable output buffering.
+  The intent is to make logging immediate, otherwise errors may not
+  get logged for some time (or at all).
+- Add catching of exceptions around CAN device opening so that the
+  script can exit cleanly with an error message if the device is
+  not available.
+
+Upstream-Status: pending
+Signed-off-by: Scott Murray <scott.murray@konsulko.com>
+---
+ kuksa_feeders/dbc2val/dbcfeeder.py | 40 ++++++++++++++++++++++--------
+ 1 file changed, 29 insertions(+), 11 deletions(-)
+
+diff --git a/kuksa_feeders/dbc2val/dbcfeeder.py b/kuksa_feeders/dbc2val/dbcfeeder.py
+index 56c316a..d2d70b9 100755
+--- a/kuksa_feeders/dbc2val/dbcfeeder.py
++++ b/kuksa_feeders/dbc2val/dbcfeeder.py
+@@ -1,4 +1,4 @@
+-#!/usr/bin/env python
++#!/usr/bin/env -S python -u
+ 
+ ########################################################################
+ # Copyright (c) 2020 Robert Bosch GmbH
+@@ -15,15 +15,21 @@ import os, sys, signal
+ import configparser
+ import queue
+ import json
++import argparse
+ 
+ from dbc2val import dbc2vssmapper, dbcreader, j1939reader, elm2canbridge
+-
+-scriptDir= os.path.dirname(os.path.realpath(__file__))
+-sys.path.append(os.path.join(scriptDir, "../../"))
+ from kuksa_viss_client import KuksaClientThread
+ 
+-print("kuksa.val DBC example feeder")
+-config_candidates=['/config/dbc_feeder.ini', '/etc/dbc_feeder.ini', os.path.join(scriptDir, 'config/dbc_feeder.ini')]
++parser = argparse.ArgumentParser("kuksa.val DBC example feeder")
++parser.add_argument("-c", "--config", dest="userconfig")
++parser.add_argument("-v", "--verbose", action="store_true")
++args = parser.parse_args()
++
++if args.verbose:
++    print("kuksa.val DBC example feeder")
++config_candidates=['/etc/kuksa-dbc-feeder/config.ini', '/etc/dbc_feeder.ini']
++if args.userconfig is not None:
++    config_candidates.insert(0, args.userconfig)
+ configfile = None
+ for candidate in config_candidates:
+     if os.path.isfile(candidate):
+@@ -54,10 +60,12 @@ cancfg = config['can']
+ canport = cancfg['port']
+ 
+ if config["can"].getboolean("j1939", False):
+-    print("Use j1939 reader")
++    if args.verbose:
++        print("Use j1939 reader")
+     reader = j1939reader.J1939Reader(cancfg,canQueue,mapping)
+ else:
+-    print("Use dbc reader")
++    if args.verbose:
++        print("Use dbc reader")
+     reader = dbcreader.DBCReader(cancfg, canQueue,mapping)
+ 
+ if canport == 'elmcan':
+@@ -65,10 +73,18 @@ if canport == 'elmcan':
+         print("section {} missing from configuration, exiting".format(canport))
+         sys.exit(-1)
+ 
+-    print("Using elmcan. Trying to set up elm2can bridge")
++    if args.verbose:
++        print("Using elmcan. Trying to set up elm2can bridge")
+     elmbr=elm2canbridge.elm2canbridge(canport, config[canport], reader.canidwl)
+ 
+-reader.start_listening()
++try:
++    reader.start_listening()
++except:
++    print("Could not open {}, exiting".format(canport))
++    kuksa.stop()
++    reader.stop()
++    sys.exit(-1)
++
+ running = True
+ 
+ def terminationSignalreceived(signalNumber, frame):
+@@ -77,6 +93,7 @@ def terminationSignalreceived(signalNumber, frame):
+     kuksa.stop()
+     reader.stop()
+     print("Received termination signal. Shutting down")
++    sys.exit(0)
+ 
+ signal.signal(signal.SIGINT, terminationSignalreceived)
+ signal.signal(signal.SIGQUIT, terminationSignalreceived)
+@@ -88,7 +105,8 @@ while running:
+         for target in mapping[signal]['targets']:
+             tv=mapping.transform(signal,target,value)
+             if tv is not None: #none indicates the transform decided to not set the value
+-                print("Update VSS path {} to {} based on signal {}".format(target, tv, signal))
++                if args.verbose:
++                    print("Update VSS path {} to {} based on signal {}".format(target, tv, signal))
+                 resp=json.loads(kuksa.setValue(target, str(tv)))
+                 if "error" in resp:
+                     if "message" in resp["error"]: 
+-- 
+2.35.1
+
diff --git a/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/agl-vcar.dbc b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/agl-vcar.dbc
new file mode 100644
index 000000000..c7efcf6d3
--- /dev/null
+++ b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/agl-vcar.dbc
@@ -0,0 +1,31 @@
+VERSION "AGL Virtual Car 1.0"
+
+BS_:
+
+BO_ 1001 Vehicle_Status_1: 8 Vector_XXX
+  SG_ PT_VehicleAvgSpeed : 7|15@0+ (0.015625,0) [0|0] "" Vector_XXX
+
+BO_ 985 Vehicle_Status_2: 8 Vector_XXX
+  SG_ PT_FuelLevelPct : 8|8@1+ (0.392157,0) [0|0] "" Vector_XXX
+  SG_ PT_EngineSpeed : 23|16@0+ (0.25,0) [0|0] "" Vector_XXX
+  SG_ PT_FuelLevelLow : 55|1@1+ (1,0) [0|1] "" Vector_XXX
+
+BO_ 33 Steering_Wheel: 8 Vector_XXX
+  SG_ SW_Previous : 32|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_VolumeUp : 33|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_Mode : 34|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_VolumeDown : 35|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_Next : 36|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_Info : 38|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_VolumeMute : 39|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_Voice : 45|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_PhoneHangup : 46|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_PhoneCall : 47|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_CruiseEnable : 48|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_CruiseResume : 49|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_CruiseSet : 51|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_CruiseCancel : 52|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_CruiseLimit : 54|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_CruiseDistance : 55|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_Horn : 56|1@1+ (1,0) [0|1] "" Vector_XXX
+  SG_ SW_LaneDepartureWarning : 63|1@1+ (1,0) [0|1] "" Vector_XXX
diff --git a/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/config.ini b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/config.ini
new file mode 100644
index 000000000..25c9fd273
--- /dev/null
+++ b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/config.ini
@@ -0,0 +1,15 @@
+[kuksa_val]
+# kuksa VSS server address
+server=wss://localhost:8090
+# JWT security token file
+token=/etc/kuksa-dbc-feeder/dbc_feeder.json.token
+# VSS mapping file
+mapping=/etc/kuksa-dbc-feeder/mapping.yml
+
+[can]
+# CAN port
+port=can0
+#Enable SAE-J1939 Mode. False: ignore
+j1939=False
+# DBC file used to parse CAN messages
+dbcfile=/etc/kuksa-dbc-feeder/agl-vcar.dbc
diff --git a/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/dbc_feeder.json.token b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/dbc_feeder.json.token
new file mode 100644
index 000000000..b8df66663
--- /dev/null
+++ b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/dbc_feeder.json.token
@@ -0,0 +1 @@
+eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJrdWtzYS52YWwiLCJpc3MiOiJFY2xpcHNlIEtVS1NBIERldiIsImFkbWluIjp0cnVlLCJtb2RpZnlUcmVlIjp0cnVlLCJpYXQiOjE1MTYyMzkwMjIsImV4cCI6MTc2NzIyNTU5OSwia3Vrc2EtdnNzIjp7IioiOiJydyJ9fQ.p2cnFGH16QoQ14l6ljPVKggFXZKmD-vrw8G6Vs6DvAokjsUG8FHh-F53cMsE-GDjyZH_1_CrlDCnbGlqjsFbgAylqA7IAJWp9_N6dL5p8DHZTwlZ4IV8L1CtCALs7XVqvcQKHCCzB63Y8PgVDCAqpQSRb79JPVD4pZwkBKpOknfEY5y9wfbswZiRKdgz7o61_oFnd-yywpse-23HD6v0htThVF1SuGL1PuvGJ8p334nt9bpkZO3gaTh1xVD_uJMwHzbuBCF33_f-I5QMZO6bVooXqGfe1zvl3nDrPEjq1aPulvtP8RgREYEqE6b2hB8jouTiC_WpE3qrdMw9sfWGFbm04qC-2Zjoa1yYSXoxmYd0SnliSYHAad9aXoEmFENezQV-of7sc-NX1-2nAXRAEhaqh0IRuJwB4_sG7SvQmnanwkz-sBYxKqkoFpOsZ6hblgPDOPYY2NAsZlYkjvAL2mpiInrsmY_GzGsfwPeAx31iozImX75rao8rm-XucAmCIkRlpBz6MYKCjQgyRz3UtZCJ2DYF4lKqTjphEAgclbYZ7KiCuTn9HualwtEmVzHHFneHMKl7KnRQk-9wjgiyQ5nlsVpCCblg6JKr9of4utuPO3cBvbjhB4_ueQ40cpWVOICcOLS7_w0i3pCq1ZKDEMrYDJfz87r2sU9kw1zeFQk
\ No newline at end of file
diff --git a/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/kuksa-dbc-feeder.service b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/kuksa-dbc-feeder.service
new file mode 100644
index 000000000..dfcbb4df8
--- /dev/null
+++ b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/kuksa-dbc-feeder.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=Eclipse KUKSA.val DBC feeder
+Requires=kuksa-val.service can-dev-helper.service
+After=kuksa-val.service can-dev-helper.service
+
+[Service]
+ExecStart=/usr/bin/dbcfeeder.py
+Restart=on-failure
+
+[Install]
+WantedBy=multi-user.target
diff --git a/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/mapping.yml b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/mapping.yml
new file mode 100644
index 000000000..a493168d4
--- /dev/null
+++ b/recipes-connectivity/kuksa-val/kuksa-dbc-feeder/mapping.yml
@@ -0,0 +1,12 @@
+# Mapping Speed
+PT_VehicleAvgSpeed:
+  minupdatedelay: 100
+  targets: 
+    Vehicle.Speed: {}
+
+PT_EngineSpeed:
+  minupdatedelay: 100
+  targets: 
+    Vehicle.Powertrain.CombustionEngine.Engine.Speed:
+      transform:
+        math: "floor(x+0.5)"