summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2020-11-18Add marks for Internet requiring testsScott Murray3-0/+7
Add "internet" mark to geoclue and weather binding tests to allow skipping them with the pytest keyword option. Bug-AGL: SPEC-3685 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I330666ff57f044f04f42d1e89d0768e1a61406ae
2020-11-18Add simple wrapper scriptScott Murray1-0/+3
Add simple "pyagl" script that wraps running pytest. Bug-AGL: SPEC-3684 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: Ibdd71359fbdd82f10a277903a86bca5f92d8d9be
2020-11-18Add CAN value testsEdi Feschiyan1-0/+89
Porting CAN value tests from lua test widgets to complete the CAN testing suite. Bug-AGL: SPEC-3660 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: I1f977e00fd69c3dac5a34746e514c262647461f5
2020-11-18Allow pyagl CAN test test_write_wo_auth to xfailJan-Simon Moeller1-1/+1
Until the root cause can be traced down, xfail the write_wo_auth test. Reason: we do execute pyagl multiple times and this will always fail on the 2nd invocation. This is a bug in the appfw and is tracked in Bug-AGL: SPEC-3648 Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I9d5e6058c8ab9dd2efec461c551c884230d1bb4b
2020-10-07Add radio binding testsScott Murray3-2/+741
Initial implementation of a radio binding wrapper and associated tests. There are some known test limitations with respect to scanning and RDS, and while assumptions about hardware behavior have been avoided so that the tests should work on both the null and hardware backend implementations, it is possible that something has been missed, as testing has been with the null implementation. See the comment header in test_radio.py for more information. Additionally, the trimming of pytest parameters from test names for LAVA's TEST_CASE_ID value has been removed, as the new radio tests do use parameterization to run multiple instances of the same test. It is currently believed that LAVA will be okay with the brace characters in the parameterized names, this will need to be verified. Note that the invalid frequency offset test passing is dependent on a fix to the radio binding behavior with respect to error replies that will be submitted separately. Bug-AGL: SPEC-3585 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I001b046c705b9e18cd317b5ac26ef2b7d2cb722e (cherry picked from commit 8e3e0588037f9601dbb527c4302f908380c20e5f)
2020-10-06Fix CAN test data installationScott Murray1-1/+2
Update MANIFEST.in so the new CAN test data does actually get installed, not quite sure how it looked like it was working in my testing. Bug-AGL: SPEC-3585 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I6ae35e5b5d57b6324bff0a9c5952556b7ffc9a94 (cherry picked from commit 333ca4bc131ab6ba41f10d8d447b6130ca777dd7)
2020-10-01Skip signal composer tests by defaultScott Murray2-3/+13
Fix signal composer test markings (there were accidentally marked as can_low_level), and add logic to skip them by default for now due to SPEC-3615. They can be enabled by passing --run-signal-composer to pytest. Once SPEC-3615 is resolved, this can be removed or the default changed to not skipping. Bug-AGL: SPEC-3585 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I66a644eecf90308b7873f605685137cf403adee5
2020-10-01CAN test rework to improve robustnessScott Murray14-106/+1471
Changes: - Rework CAN test CAN data playing with canplayer to fix racing against listening for the binding events the data will result in. The previous use of a fixture to drive canplayer has been replaced with a small wrapper class that is inline in the test code to ensure the correct order of operations. - Copy required CAN test CAN data files used in the afb-test widget into pyagl/tests/data/can to remove dependency on test widget presence and contents. The previous prepare_replay_files fixture has been changed into a per-test fixture that scp's the file into /tmp on the target if running remotely. When running on target, they are used directly out of the pyagl install. - Improve test_Filter_Test_01_Step_2 using the updated canreplay file copied from agl-service-can-low-level, there are now several messages that test the filter range. - Add a tweak to cut the fixture parameter(s) off the test names when generating LAVA output. - Reordered test_auth to be after the write without authorization test to avoid spurious failures. Also removed the duplicate test in the J1939 tests and replaced it with an appropriate dependency. - Replace xfail marks on J1939 tests with logic to detect J1939 kernel support and mark them dynamically. - Add missing can_j1939 marking to the test test_low_can_subscribe_j1939_event. - Use known_hosts=None in asyncssh.connect calls to ensure remote testing is not affected by local SSH known_hosts configuration. - Tweak - Update some comment formatting to improve readability. - Simplify logic a bit in AGLBaseService.listener function. Bug-AGL: SPEC-3585 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I24d0e2f9ac75d1d2aabd5ff2197e7997dae59273
2020-10-01Remove "regular" test markingScott Murray15-108/+13
Remove the "regular" test marking to simplify things. At the moment it provides no real separation from the set of tests proscribed by "not hwrequired", and things like J1939 need to be dynamically skipped based on platform support instead of requiring manual mark specification. Bug-AGL: SPEC-3585 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: Ia1908058db59b52ced39a08acb6efb374c9a097a
2020-10-01Add CANService testsEdi Feschiyan4-9/+491
Porting most of the lua tests for agl-service-can-low to pyagl Handle None in AGLBaseService.afbresponse() and AFBResponse class for timeouts Add new can_j1939 marker for J1939 CAN tests, also xfail marked until a way to determine if J1939 kernel support is available signal-composer verb queries tend to crash/choke after service startup right after boot, but when the service is restarted, tests run fine - so adding a timeout handling on verb queries in tests Bug-AGL: SPEC-3585 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: I6e66a8aab82e404f6a416960592d18995a04c7ba
2020-10-01Add timeout ability to AGLBaseService.listener()Edi Feschiyan1-6/+17
When doing subscription and event testing, there are cases which need to test lack of emitted events, changing the .listener() async generator to be able to return None. Set to None by default which would use self.timeout Adding extra data on ERROR response from afb if it is available Changing AFBResponse to raise TypeError instead return Bug-AGL: SPEC-3585 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: I86c3e9d6242fcbe2ea0c402bcb211e2a89eb9d76
2020-10-01Fix signalcomposer testEdi Feschiyan1-1/+15
Add a test to test all available signals reported by list verb Bug-AGL: SPEC-3585 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: I125d23e5adc975b05606a886132e1faa18889cb1
2020-10-01Override un/subscribe methods in CANServiceEdi Feschiyan1-0/+5
All agl-services use 'value' keyword in JSON request for subscription which is set as the default behavior for base.py, however agl-service-can-low uses 'event'. Bug-AGL: SPEC-3585 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: I3bd264cf459b49aefc645814fa9a3a1040aa83ca
2020-10-01Fix AGLBaseService.afbresponse() for EVENT dataEdi Feschiyan1-1/+1
afbresponse() in base.py does wrong assignment at the end which effectively makes it work only for responses and discards EVENT data because there is no 'response' key in the json emitted for the event Bug-AGL: SPEC-3585 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: I198d7be7ea68d056b8269e20d9ce067016d7975b
2020-10-01Update tox.ini and requirements.txtEdi Feschiyan2-2/+3
Update requirements.txt to match AsyncSSH with newer 2.3.0 version Update tox.ini with proper path to run in CI and set AGL_AVAILABLE_INTERFACES to "ethernet" by default Bug-AGL: SPEC-3583 Signed-off-by: Edi Feschiyan <edi.feschiyan@konsulko.com> Change-Id: Id1e880d80d6c6bedb6650deeb4fe6f3befdcf50b
2020-09-15Add gitreview file for jellyfish branchjellyfish_10.0.0jellyfish/10.0.010.0.0Jan-Simon Möller1-1/+1
Update the .gitreview file. Bug-AGL: SPEC-3581 Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
2020-09-10Remove hwrequired marking of audiomixer testsjellyfish_9.99.4jellyfish/9.99.49.99.4Scott Murray1-6/+0
Since CI runs qemu with a soundcard enabled, and it is easy to do so with runqemu with the "audio" option, remove the "hwrequired" markers on the audiomixer tests that have them. Bug-AGL: SPEC-3572 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I290845e802168116f4769dd011c84875d2e4b5ac
2020-09-10Improve LAVA outputScott Murray2-14/+110
Changes: - Disable progress and verbose output when in LAVA output mode to clean up output. - Disable color output in LAVA output mode. - Removed STARTTC and ENDTC lines for passed/skipped tests to reduce output volume, they are still printed around the output of failed tests. - Tweaked error output in LAVA output mode so it will be usefully parsed out by LAVA and show up in results. - Fixed skipped test output in LAVA output mode, the logic needed tweaking to get a TESTCASE line output for them. Also, tweaked the LAVA result to 'skip' instead of 'pass' for skipped tests, as LAVA understands 'skip'. - By request, add 0.25 second delay after each test in LAVA output mode to slow down serial output. - Updated documentation to add some invocation examples. Bug-AGL: SPEC-3572 Signed-off-by: Scott Murray <scott.murray@konsulko.com> Change-Id: I2e2b213dba2a9751210512d698afb67f7543cbe1
2020-09-02Add .gitreviewScott Murray1-0/+5
Add .gitreview file for AGL Gerrit usage. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-09-02Merge branch 'smurray/fixes' into 'master'Edi Feschiyan13-22/+29
Updates to prepare for AGL CI use See merge request konsulko/internal/pyagl!2
2020-09-01Add timeout for binding responsesScott Murray2-1/+8
To ensure automated use of the tests cannot get hung, add a timeout on waiting for binding responses. The default value is 5 seconds, it may be over-riden by setting the AGL_TEST_TIMEOUT environment variable. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-09-01audiomixer and mediascanner bindings do not need to be startedScott Murray4-5/+6
The changes in AGL for SPECs 3557 and 3558 mean that the audiomixer and mediascanner bindings do not need to be started before running tests, disable doing so. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-09-01Rework per-binding test markingScott Murray6-9/+10
Rework the test marking to allow deriving the test mark name for a given binding from the binding name, i.e. agl-service-X. Due to dashes not being valid in pytest marks, underscores have been used instead, i.e. agl-service-can-low-level -> can_low_level. The intent is to allow CI tooling to easily work up a pytest -k option to run the tests for a specific binding. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-09-01Assume network connectivityScott Murray2-7/+5
Revert the xfail changes to the geoclue and weather tests, as it's been clarified that network connectivity is assumed present in AGL CI. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-09-01Adding author to license heading in filesEdi Feschiyan33-30/+103
2020-08-31adding an option during service init to run the service it is missingEdi Feschiyan5-88/+104
adding a fix where if multiple instances of the same service are available fixing geoclue to xfail due to lack of internet connection and removing subscription wait moving scannable interfaces to a fixture adding dependency and moving network tests order
2020-08-31mediascanner service is not running in the system on boot, so adding ↵Edi Feschiyan3-20/+78
runafmservice and startsystemdservice methods and adjusting the portfinder() to run the missing service.
2020-08-31Merge branch 'master' of gitlab.com:konsulko/internal/pyagl into masterEdi Feschiyan32-30/+552
 Conflicts:  pyagl/tests/test_audiomixer.py  pyagl/tests/test_bluetooth.py
2020-08-31Adding can-low basic verb tests and cli invocationsEdi Feschiyan12-116/+116
Decorator cleanups
2020-08-29Merge branch 'smurray/fixes' into 'master'Edi Feschiyan32-30/+546
Assorted fixes See merge request konsulko/internal/pyagl!1
2020-08-28Rename AGL_AVAILABLE_TECHSScott Murray2-19/+31
Rename AGL_AVAILABLE_TECHS to AGL_AVAILABLE_INTERFACES for clarity, and update README.md to document it and improve the test running section. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-08-28Mark some more tests as hwrequiredScott Murray2-7/+7
Mark more tests that seem to require hardware in the audiomixer and bluetooth tests as hwrequired, with an eye towards having no failures when running with "not hwrequired" on qemu. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-08-28Add license file and copyrightsScott Murray31-4/+508
Add Apache 2.0 license text as LICENSE file, and copyright headers to all source files. Signed-off-by: Scott Murray <scott.murray@konsulko.com>
2020-08-19test_signal_composer.py - adding tests forEdi Feschiyan2-3/+51
2020-08-17bluetooth.py and test_bluetooth.py - adding missing avrcp_controls support ↵Edi Feschiyan12-17/+229
and test mediascanner.py - adding missing agl service mediascanner support and tests test_network.py - adding missing scan_services test adding base service and tests files for signal_composer and can_low
2020-08-07Changing portfinder method to run locally without sshEdi Feschiyan1-35/+64
2020-08-07Adding changes to setup.py to include ini files from MANIFEST.inEdi Feschiyan2-1/+7
2020-08-07moving conftest to pyagl subdir and creating a manifestEdi Feschiyan2-0/+1
2020-08-07moving conftest conftest under a packageEdi Feschiyan1-0/+0
2020-08-07updating .gitignore for toxEdi Feschiyan1-0/+4
2020-08-07Registering "regular" and "hwrequired" test markers, adding them to testsEdi Feschiyan14-27/+270
Enabling custom pytest --lava option for LAVA CI integration Enabling tox for CI bluetooth::default_adapter should support setting a default one according to documentation in README.md
2020-07-23removing unused pytest-reverseEdi Feschiyan2-2/+1
2020-07-23Fixed catching wrong TimeoutError exception thrown by the task in ↵Edi Feschiyan1-1/+1
test_location_events
2020-07-23changes to README.mdEdi Feschiyan1-1/+12
2020-07-17fix xfail message for bluetooth fixtureEdi Feschiyan4-7/+15
fix test_entry_phonebook to xfail instead fail when no records are returned but query is successful replace 'is not' with !=
2020-07-14When using asyncio.timeout on a response() task it will return NoneEdi Feschiyan1-3/+9
If not caught when that happens, it will raise TypeErrors and random unawaited coros/tasks
2020-07-13converting test_subscribe_location into task and waiting for it more ↵Edi Feschiyan2-4/+8
intelligently
2020-07-13when pytests are run on Python 3.8 the test gathering and execution order is ↵Edi Feschiyan2-1/+2
reversed and may break tests which require state to be carried over adding pytest-reverse to requirements/setup.py to be readily available if needed
2020-07-13catching immediate emit from geoclue subscription, otherwise breaks ↵Edi Feschiyan2-4/+7
unsubscribe test below commenting out homescreen tap test because the api currently does not respond to it
2020-07-13weather tests adding dependenciesEdi Feschiyan1-1/+2