Improve LAVA output

Changes:
- Disable progress and verbose output when in LAVA output mode to
  clean up output.
- Disable color output in LAVA output mode.
- Removed STARTTC and ENDTC lines for passed/skipped tests to
  reduce output volume, they are still printed around the output
  of failed tests.
- Tweaked error output in LAVA output mode so it will be usefully
  parsed out by LAVA and show up in results.
- Fixed skipped test output in LAVA output mode, the logic needed
  tweaking to get a TESTCASE line output for them.  Also, tweaked
  the LAVA result to 'skip' instead of 'pass' for skipped tests,
  as LAVA understands 'skip'.
- By request, add 0.25 second delay after each test in LAVA output
  mode to slow down serial output.
- Updated documentation to add some invocation examples.

Bug-AGL: SPEC-3572

Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Change-Id: I2e2b213dba2a9751210512d698afb67f7543cbe1

1 files changed, 31 insertions, 2 deletions

diff --git a/README.md b/README.md
index 6f690e5..be8c283 100644
--- a/README.md
+++ b/README.md
@@ -46,11 +46,9 @@ Until the package is uploaded onto PyPI, either:
 * `pip install .`
 
 or
-
 * `pip install` the generated zip from the repository(when public)
 
 or
-
 * git clone \<repo\>
 * mkdir packages && cd packages
 * pip wheel ../\<repo\>
@@ -84,6 +82,21 @@ Note that the tests have been labelled with `pytest` markers to allow selecting
 * regular - all regular verb tests with expected values
 * hwrequired - verb tests requiring available physical hardware
 
+#### Examples
+Running just the tests for a single binding (audiomixer):
+```
+pytest -k "audiomixer" /usr/lib/python3.8/site-packages/pyagl/tests
+```
+Note that the per-binding markers cannot use dashes ('-') in their names, so generating the marker for a specific binding can be done with something like:
+```
+echo agl-service-can-low-level | cut -d- -f3- | tr - _
+```
+Running tests with LAVA compatible output:
+```
+pytest --lava "audiomixer" /usr/lib/python3.8/site-packages/pyagl/tests
+```
+Note that `--lava` and `-L` are equivalent, either will activate LAVA compatible output.  As well, note that when using LAVA output, the `pytest` `-v` verbose option is ignored.
+
 ### Test Configuration
  Running the tests remotely involves the export of the following environment variables:
 * AGL_TGT_IP - required - point at an IP Address with AGL instance
@@ -101,3 +114,19 @@ Some specific tests are dependent on additional configuration via the following
 * AGL_BT_TEST_ADDR - optional, for the Bluetooth tests pair/connect/disconnect with an actual Bluetooth device(phone)'s address .  The address should have the DBus address style as "dev_00_01_02_03_04_05" instead of a regular colon-separated MAC address.
 
 Should some of the optional variables be omitted, the fixtures and their dependent tests should fail gracefully with a "XFAIL" (expected fail) or "SKIP" result.  There are tests that are dependent on other tests, therefore if the dependent tests fail, the whole test chain will be skipped.
+
+### Running Inside QEMU
+To run `QEMU` from inside the AGL build environment to be able to run tests, the `runqemu` command can be used.  For example:
+```
+$ runqemu kvm publicvnc serial slirp audio
+```
+A note on some of the `runqemu` options:
+ - serial - enables a serial console
+ - slirp - enables user networking (no root login / `sudo` required)
+ - audio - enables audio
+
+The tests can then be invoked after logging in:
+```
+ AGL_AVAILABLE_INTERFACES=ethernet pytest -k "not hwrequired" /usr/lib/python3.8/site-packages/pyagl/tests
+```
+If running `QEMU` outside of the AGL build environment, note that you will likely want to pass the `-soundhw hda` option to enable audio support if running the audiomixer tests, and an appropriate `-netdev` option to enable some form of networking for the network binding tests.