summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2021-01-13Support specifying CAN interface for testskoi_10.93.0koi_10.92.0koi/10.93.0koi/10.92.010.93.010.92.0Scott Murray2-0/+6
Add AGL_CAN_INTERFACE environment variable hook to allow specifying the CAN interface to be used in the CAN tests. This is required when using an interface other than the default of "can0", since the specific interface is needed for playback of CAN data to test binding behavior. The switch in AGL CI to using vcan0 on all test boards will require this hook. Bug-AGL: SPEC-3756 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: If84731ae39fd65f8fa88d24ddc0370da63ca5489
2021-01-11Mark CAN J1939 tests as xfail for nowScott Murray1-0/+17
Since CI builds of the CAN binding widget are done standalone with the SDK, J1939 support ends up not being enabled, and the associated tests fail when run by CIAT for builds of agl-service-can-low-level repo changes. For now, mark the tests as xfail to avoid blocking CIAT. This should be reverted once the issue is resolved, e.g. by enabling J1939 support by default (see SPEC-3766). Bug-AGL: SPEC-3765 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I74eca93cf746aba2dddfa87bd2b8546714d5e0c0
2020-12-03Add HVAC bindings and testskoi_10.91.0koi/10.91.010.91.0Edi Feschiyan2-0/+176
The current implementation of agl-service-hvac does not have test widgets implemented as reference point so these tests are new. Writing to verbs related to changing LED brightness results in I2C errors and therefore the tests are marked as hwrequired and xfail, as no hardware demo units are available to test with. The markers will stay until it is decided whether it is enough for testing or mock tests should be implemented. Bug-AGL: SPEC-3660 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: Ibc051c6b4a06e0d126be9f8a0e0efe5876244671
2020-11-11Add marks for Internet requiring testsScott Murray3-0/+7
Add "internet" mark to geoclue and weather binding tests to allow skipping them with the pytest keyword option. Bug-AGL: SPEC-3685 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I330666ff57f044f04f42d1e89d0768e1a61406ae
2020-11-11Add simple wrapper scriptScott Murray1-0/+3
Add simple "pyagl" script that wraps running pytest. Bug-AGL: SPEC-3684 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: Ibdd71359fbdd82f10a277903a86bca5f92d8d9be
2020-11-10Add CAN value testsEdi Feschiyan1-0/+89
Porting CAN value tests from lua test widgets to complete the CAN testing suite. Bug-AGL: SPEC-3660 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: I1f977e00fd69c3dac5a34746e514c262647461f5
2020-10-19Allow pyagl CAN test test_write_wo_auth to xfailJan-Simon Moeller1-1/+1
Until the root cause can be traced down, xfail the write_wo_auth test. Reason: we do execute pyagl multiple times and this will always fail on the 2nd invocation. This is a bug in the appfw and is tracked in Bug-AGL: SPEC-3648 Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I9d5e6058c8ab9dd2efec461c551c884230d1bb4b
2020-10-06Add radio binding testsScott Murray3-2/+741
Initial implementation of a radio binding wrapper and associated tests. There are some known test limitations with respect to scanning and RDS, and while assumptions about hardware behavior have been avoided so that the tests should work on both the null and hardware backend implementations, it is possible that something has been missed, as testing has been with the null implementation. See the comment header in test_radio.py for more information. Additionally, the trimming of pytest parameters from test names for LAVA's TEST_CASE_ID value has been removed, as the new radio tests do use parameterization to run multiple instances of the same test. It is currently believed that LAVA will be okay with the brace characters in the parameterized names, this will need to be verified. Note that the invalid frequency offset test passing is dependent on a fix to the radio binding behavior with respect to error replies that will be submitted separately. Bug-AGL: SPEC-3585 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I001b046c705b9e18cd317b5ac26ef2b7d2cb722e
2020-10-01Fix CAN test data installationScott Murray1-1/+2
Update MANIFEST.in so the new CAN test data does actually get installed, not quite sure how it looked like it was working in my testing. Bug-AGL: SPEC-3585 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I6ae35e5b5d57b6324bff0a9c5952556b7ffc9a94
2020-09-30Skip signal composer tests by defaultScott Murray2-3/+13
Fix signal composer test markings (there were accidentally marked as can_low_level), and add logic to skip them by default for now due to SPEC-3615. They can be enabled by passing --run-signal-composer to pytest. Once SPEC-3615 is resolved, this can be removed or the default changed to not skipping. Bug-AGL: SPEC-3585 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: Ie64d2868d90bdd242b762c7ad35cac1be11677e6
2020-09-30CAN test rework to improve robustnessScott Murray14-106/+1471
Changes: - Rework CAN test CAN data playing with canplayer to fix racing against listening for the binding events the data will result in. The previous use of a fixture to drive canplayer has been replaced with a small wrapper class that is inline in the test code to ensure the correct order of operations. - Copy required CAN test CAN data files used in the afb-test widget into pyagl/tests/data/can to remove dependency on test widget presence and contents. The previous prepare_replay_files fixture has been changed into a per-test fixture that scp's the file into /tmp on the target if running remotely. When running on target, they are used directly out of the pyagl install. - Improve test_Filter_Test_01_Step_2 using the updated canreplay file copied from agl-service-can-low-level, there are now several messages that test the filter range. - Add a tweak to cut the fixture parameter(s) off the test names when generating LAVA output. - Reordered test_auth to be after the write without authorization test to avoid spurious failures. Also removed the duplicate test in the J1939 tests and replaced it with an appropriate dependency. - Replace xfail marks on J1939 tests with logic to detect J1939 kernel support and mark them dynamically. - Add missing can_j1939 marking to the test test_low_can_subscribe_j1939_event. - Use known_hosts=None in asyncssh.connect calls to ensure remote testing is not affected by local SSH known_hosts configuration. - Tweak - Update some comment formatting to improve readability. - Simplify logic a bit in AGLBaseService.listener function. Bug-AGL: SPEC-3585 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I4ef660c175468ef57873c03a33cb5ec7e6f16568
2020-09-30Remove "regular" test markingScott Murray15-108/+13
Remove the "regular" test marking to simplify things. At the moment it provides no real separation from the set of tests proscribed by "not hwrequired", and things like J1939 need to be dynamically skipped based on platform support instead of requiring manual mark specification. Bug-AGL: SPEC-3585 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: Ic21cb84c499af9b63c83d145f3ffefaae4fda357
2020-09-23Add CANService testsEdi Feschiyan4-9/+491
Porting most of the lua tests for agl-service-can-low to pyagl Handle None in AGLBaseService.afbresponse() and AFBResponse class for timeouts Add new can_j1939 marker for J1939 CAN tests, also xfail marked until a way to determine if J1939 kernel support is available signal-composer verb queries tend to crash/choke after service startup right after boot, but when the service is restarted, tests run fine - so adding a timeout handling on verb queries in tests Bug-AGL: SPEC-3585 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: Ib1fc259f2b28fb11c5f89b61df0b066d2c02045e
2020-09-23Add timeout ability to AGLBaseService.listener()Edi Feschiyan1-6/+17
When doing subscription and event testing, there are cases which need to test lack of emitted events, changing the .listener() async generator to be able to return None. Set to None by default which would use self.timeout Adding extra data on ERROR response from afb if it is available Changing AFBResponse to raise TypeError instead return Bug-AGL: SPEC-3585 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: I09a62c559a1e8ca4bebe6f696167ca61ffb82b59
2020-09-17Fix signalcomposer testEdi Feschiyan1-1/+15
Add a test to test all available signals reported by list verb Bug-AGL: SPEC-3585 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: I73f4cb75ef24b38948756b2c81bc4e3552d7eb3c
2020-09-17Override un/subscribe methods in CANServiceEdi Feschiyan1-0/+5
All agl-services use 'value' keyword in JSON request for subscription which is set as the default behavior for base.py, however agl-service-can-low uses 'event'. Bug-AGL: SPEC-3585 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: Id93cf1bfdbb5c60614e3e5eca3814d5d16b907a9
2020-09-17Fix AGLBaseService.afbresponse() for EVENT dataEdi Feschiyan1-1/+1
afbresponse() in base.py does wrong assignment at the end which effectively makes it work only for responses and discards EVENT data because there is no 'response' key in the json emitted for the event Bug-AGL: SPEC-3585 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: I69b15127ca386148687d83b3a231c3aa53e789b7
2020-09-16Update tox.ini and requirements.txtEdi Feschiyan2-2/+3
Update requirements.txt to match AsyncSSH with newer 2.3.0 version Update tox.ini with proper path to run in CI and set AGL_AVAILABLE_INTERFACES to "ethernet" by default Bug-AGL: SPEC-3583 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: If5f000b2dd17354cb97aeddd882bf7978b3ddb97
2020-09-10Remove hwrequired marking of audiomixer testsjellyfish_9.99.4jellyfish/9.99.49.99.4Scott Murray1-6/+0
Since CI runs qemu with a soundcard enabled, and it is easy to do so with runqemu with the "audio" option, remove the "hwrequired" markers on the audiomixer tests that have them. Bug-AGL: SPEC-3572 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I290845e802168116f4769dd011c84875d2e4b5ac
2020-09-10Improve LAVA outputScott Murray2-14/+110
Changes: - Disable progress and verbose output when in LAVA output mode to clean up output. - Disable color output in LAVA output mode. - Removed STARTTC and ENDTC lines for passed/skipped tests to reduce output volume, they are still printed around the output of failed tests. - Tweaked error output in LAVA output mode so it will be usefully parsed out by LAVA and show up in results. - Fixed skipped test output in LAVA output mode, the logic needed tweaking to get a TESTCASE line output for them. Also, tweaked the LAVA result to 'skip' instead of 'pass' for skipped tests, as LAVA understands 'skip'. - By request, add 0.25 second delay after each test in LAVA output mode to slow down serial output. - Updated documentation to add some invocation examples. Bug-AGL: SPEC-3572 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I2e2b213dba2a9751210512d698afb67f7543cbe1
2020-09-02Add .gitreviewScott Murray1-0/+5
Add .gitreview file for AGL Gerrit usage. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-09-02Merge branch 'smurray/fixes' into 'master'Edi Feschiyan13-22/+29
Updates to prepare for AGL CI use See merge request konsulko/internal/pyagl!2
2020-09-01Add timeout for binding responsesScott Murray2-1/+8
To ensure automated use of the tests cannot get hung, add a timeout on waiting for binding responses. The default value is 5 seconds, it may be over-riden by setting the AGL_TEST_TIMEOUT environment variable. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-09-01audiomixer and mediascanner bindings do not need to be startedScott Murray4-5/+6
The changes in AGL for SPECs 3557 and 3558 mean that the audiomixer and mediascanner bindings do not need to be started before running tests, disable doing so. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-09-01Rework per-binding test markingScott Murray6-9/+10
Rework the test marking to allow deriving the test mark name for a given binding from the binding name, i.e. agl-service-X. Due to dashes not being valid in pytest marks, underscores have been used instead, i.e. agl-service-can-low-level -> can_low_level. The intent is to allow CI tooling to easily work up a pytest -k option to run the tests for a specific binding. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-09-01Assume network connectivityScott Murray2-7/+5
Revert the xfail changes to the geoclue and weather tests, as it's been clarified that network connectivity is assumed present in AGL CI. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-09-01Adding author to license heading in filesEdi Feschiyan33-30/+103
2020-08-31adding an option during service init to run the service it is missingEdi Feschiyan5-88/+104
adding a fix where if multiple instances of the same service are available fixing geoclue to xfail due to lack of internet connection and removing subscription wait moving scannable interfaces to a fixture adding dependency and moving network tests order
2020-08-31mediascanner service is not running in the system on boot, so adding ↵Edi Feschiyan3-20/+78
runafmservice and startsystemdservice methods and adjusting the portfinder() to run the missing service.
2020-08-31Merge branch 'master' of gitlab.com:konsulko/internal/pyagl into masterEdi Feschiyan32-30/+552
 Conflicts:  pyagl/tests/test_audiomixer.py  pyagl/tests/test_bluetooth.py
2020-08-31Adding can-low basic verb tests and cli invocationsEdi Feschiyan12-116/+116
Decorator cleanups
2020-08-29Merge branch 'smurray/fixes' into 'master'Edi Feschiyan32-30/+546
Assorted fixes See merge request konsulko/internal/pyagl!1
2020-08-28Rename AGL_AVAILABLE_TECHSScott Murray2-19/+31
Rename AGL_AVAILABLE_TECHS to AGL_AVAILABLE_INTERFACES for clarity, and update README.md to document it and improve the test running section. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-08-28Mark some more tests as hwrequiredScott Murray2-7/+7
Mark more tests that seem to require hardware in the audiomixer and bluetooth tests as hwrequired, with an eye towards having no failures when running with "not hwrequired" on qemu. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-08-28Add license file and copyrightsScott Murray31-4/+508
Add Apache 2.0 license text as LICENSE file, and copyright headers to all source files. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-08-19test_signal_composer.py - adding tests forEdi Feschiyan2-3/+51
2020-08-17bluetooth.py and test_bluetooth.py - adding missing avrcp_controls support ↵Edi Feschiyan12-17/+229
and test mediascanner.py - adding missing agl service mediascanner support and tests test_network.py - adding missing scan_services test adding base service and tests files for signal_composer and can_low
2020-08-07Changing portfinder method to run locally without sshEdi Feschiyan1-35/+64
2020-08-07Adding changes to setup.py to include ini files from MANIFEST.inEdi Feschiyan2-1/+7
2020-08-07moving conftest to pyagl subdir and creating a manifestEdi Feschiyan2-0/+1
2020-08-07moving conftest conftest under a packageEdi Feschiyan1-0/+0
2020-08-07updating .gitignore for toxEdi Feschiyan1-0/+4
2020-08-07Registering "regular" and "hwrequired" test markers, adding them to testsEdi Feschiyan14-27/+270
Enabling custom pytest --lava option for LAVA CI integration Enabling tox for CI bluetooth::default_adapter should support setting a default one according to documentation in README.md
2020-07-23removing unused pytest-reverseEdi Feschiyan2-2/+1
2020-07-23Fixed catching wrong TimeoutError exception thrown by the task in ↵Edi Feschiyan1-1/+1
test_location_events
2020-07-23changes to README.mdEdi Feschiyan1-1/+12
2020-07-17fix xfail message for bluetooth fixtureEdi Feschiyan4-7/+15
fix test_entry_phonebook to xfail instead fail when no records are returned but query is successful replace 'is not' with !=
2020-07-14When using asyncio.timeout on a response() task it will return NoneEdi Feschiyan1-3/+9
If not caught when that happens, it will raise TypeErrors and random unawaited coros/tasks
2020-07-13converting test_subscribe_location into task and waiting for it more ↵Edi Feschiyan2-4/+8
intelligently
2020-07-13when pytests are run on Python 3.8 the test gathering and execution order is ↵Edi Feschiyan2-1/+2
reversed and may break tests which require state to be carried over adding pytest-reverse to requirements/setup.py to be readily available if needed