summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2020-10-19Extend skiplists and fix typo in exportJan-Simon Möller7-2/+8
We the can test 1_pyagl-tests.can_write_wo_auth can easily fail on the 2nd invocation. Work this around until fixed. Bug-AGL: SPEC-3648 Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org> Change-Id: I25af2c86604672f2e439fef19b400c83fff358d9
2020-10-13Add AGL_AVAILABLE_INTERFACES to application testJan-Simon Möller1-0/+3
The standalone run did have it defined, but the application test did mit the setting of AGL_AVAILABLE_INTERFACES=ethernet. If necessary it has to be redefined in the test for bluetooth and wifi per board. Bug-AGL: SPEC-3597 Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org> Change-Id: I0772f9a6cc449406785fb167f575d6ca65e6267c
2020-10-07Disable afm-test executionJan-Simon Moeller1-21/+24
agl-service-homescreen crashes when a testwgt is executed. Until resolved, disable the test wgt execution. Bug-AGL: SPEC-3608 Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I415b2fe80338f4090ef037c77d61af8205fbb77e
2020-10-06SPEC-3484: Use the new gcovr-wrapperCorentin LABBE1-14/+11
For using gcov, a new gcovr-wrapper was created. Let's use it, capture its output ang generate some metrics. Bug-AGL: SPEC-3484 Change-Id: Idbd9aa34d37d31414d18341b035440e28df5dc83 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-09-14Merge "SPEC-3484: install and basicly test coverage package"Jan-Simon Moeller1-1/+14
2020-09-14Merge "Change regular pyagl invocation to nohw"Jan-Simon Moeller4-21/+21
2020-09-14Change regular pyagl invocation to nohwJan-Simon Moeller4-21/+21
In CI we will execute the 'not hwrequired' flag. Make this the default case in the test-suites. Bug-AGL: SPEC-3572 Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I370bb3c12953d7455a1a4c06c573cbc10ee17bda
2020-09-14SPEC-3484: install and basicly test coverage packageCorentin LABBE1-1/+14
Install and test coverage the same way than normal package. Handling coverage generated files will be handled later. Bug-AGL: SPEC-3484 Change-Id: I02d4ea1aa77954a7c85317accb6b565ece8676c0 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-09-14SPEC-3550: fix parsing of idCorentin LABBE1-1/+1
The id could be not the first element, and so the parsing is wrong. Change-Id: I325aac6de4733a166c7be8e3515b85e9edff4365 Bug-AGL: SPEC-3550 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-09-10SPEC-3550: name could be in uppercasejellyfish_9.99.4jellyfish/9.99.49.99.4Corentin LABBE1-1/+1
In config.xml, name could be in uppercase, so let's use id which is lowercase. Change-Id: Ibc3e9e145692c7773f70a1308c7ff0de27e9f0bd Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-09-09Add not hwrequired to regular script caseJan-Simon Moeller1-1/+1
Qemu by default has no hardware attached. Limit the run to not execut hardware-specific tests. Bug-AGL: SPEC-3572 Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I3305d0c34737cf026989e2bde197d8ce3d96db19
2020-09-09Add test definitions for pyaglJan-Simon Moeller3-0/+26
We can now call pytest using the pyagl/tests module. These expose -k regular and -k hwrequired, thus create two caller scripts. Also create a sample testsuite entry using the regular variant with limited AGL_AVAILABLE_INTERFACES. Bug-AGL: SPEC-3572 Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: Ic6b977da959f946ca88348715d5249748c70f736
2020-08-28SPEC-3550: handle package with service name different than their filenameCorentin LABBE1-5/+15
Some package has their service name different than their filename. This patch handle this situation Bug-AGL: SPEC-3550 Change-Id: I99c102f411ce5a0636fa14528cd2890694d986c6 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-08-13SPEC-3484: create new application tests using new wgt typesjellyfish_9.99.3jellyfish/9.99.39.99.3Corentin LABBE1-32/+122
The wgt-test and wgt-coverage needs to be handled differently. This patchs splits the application tests to handle all files encountered. First we download all files, then we use them in a specific order. Change-Id: Idc71ecfe36496db314b288f2b8ca7c2d61887de7 Bug-AGL: SPEC-3484 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-08-12SPEC-3482: handle new package nameCorentin LABBE1-1/+1
Now we build all package type, we need to handle coverage/test packages. Bug-AGL: SPEC-3482 Change-Id: Idb6dc2d453f26183094d6679dcc5c1f553780a16 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-08-10Print the syslog when the application does not startJan-Simon Moeller1-0/+2
Dump the journal when the app does not start. Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: Ia886e87599721deca57dd8664c6637c3cd1f9b3f
2020-08-10SPEC-3533: unzip wgt files in a dedicated directoryCorentin LABBE1-1/+7
For finding wgtname, we download an index.html and the WGT archive could contains a file with the same name and unzip complain about replacing a file. The solution is to unzip in an empty dedicated directory. Change-Id: I9680eb8909256e749a2b5177ed1d942add841af5 BUG-AGL: SPEC-3533 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-07-16SPEC-3495: fix greping of systemd unitsjellyfish_9.99.2jellyfish/9.99.29.99.2Corentin LABBE1-4/+4
The grep pattern for systemd units is not enough restrictive. So let's made grep a little more restrictive. Change-Id: If0f75919a89c39da63e5375e3b3cdd020c6ea324 Bug-AGL: SPEC-3495 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-07-13SPEC-3495: handle better all package nameCorentin LABBE1-3/+7
Package could have -debug in their filename, but their service name is without it. Some package could exists with a basename-extension, so we need to grep better to not catch them all. Example: agl-service-bluetooth catch agl-service-bluetooth-map. Change-Id: Ibc750e7893a8cbe3fc22a0d8f13a32c28a898bd1 Bug-AGL: SPEC-3495 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-07-03SPEC-3427: check if RID is nullCorentin LABBE1-1/+33
RID could be null if process failed to start. In that case, we will retry to start it. On a second fail, we will not continue the test suite and fail. In th same time add a second status check, but after a delay to be sure the service is still working. Bug-AGL: SPEC-3427 Change-Id: I566f1aefa06cbb791674373340ce87be220b18a2 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2020-06-01Make the script runnable locally and adapt ps calljellyfish_9.99.1jellyfish/9.99.19.99.1Jan-Simon Moeller1-16/+23
The script can be run for testing locally now and afm-util ps needs to start as the target user. Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: Ief0f785a46439531d4257c2eef9ced15be1b20fb
2020-05-28Remove versioning of wgts from testJan-Simon Möller1-2/+2
This was removed thus there is no '@?.?' . Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
2020-03-24Update skiplists.Jan-Simon Moeller4-0/+4
lava-read-feedback fail on qemu due to a sequence issues. This is merely a timing issue on the lava side that we mask here. Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I19070931704e7b9bf0a27e9370fdffd5adb899ba
2019-12-21Disable the can module unload/load testicefish_8.99.5icefish/8.99.58.99.5Jan-Simon Möller2-4/+4
And also fix a logic bug in the test_can_modules. The test can stay disabled. There is no need to unload the modules. Bug-AGL: SPEC-3070 . Change-Id: Ic94d5bb9cad3d6cdcbc6534cfc4003213ad82aaa Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
2019-12-20SPEC-3064: handle removal of can_bcmicefish_8.99.4icefish/8.99.48.99.4Corentin LABBE1-0/+12
The test removing can module fail on DRA7 due to can_bcm using it. This patchs handle this case. Bug-AGL: SPEC-3064 Change-Id: I272e0ea62786fb43c4ea68601f2fb12864a279b6 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-12-10Merge "Skip the can test"icefish_8.99.3icefish/8.99.38.99.3Jan-Simon Moeller1-1/+1
2019-12-10SPEC-2970: Do not fail ofone and bluetooth services if bluetooth tag is non ↵Corentin LABBE2-0/+20
present When no bluetooth dongle is present, there is a timing issue which let the service fail. So if no bluetooth tag is present, skip the service test instead of failling. Bug-AGL: SPEC-2907 Change-Id: I10755ec698ab632ffcf0eec18ebe6bc41b1b5ec1 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-12-09Skip the can testKhouloud Touil1-1/+1
As the behaviour of the test not determined, to avoid the fail of the test, we will skip it if the value of no_transmission not found. AGL-bug: SPEC-3025 Change-Id: If24e512f2cc83ee7018a4309cb64434cf3a2b938 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>
2019-12-04SPEC-2970: Add bluetooth and ofono on qemu ARM/ARM64 skiplistsicefish_8.99.2icefish/8.99.28.99.2Corentin LABBE2-0/+4
Since bluetooth is still failling on qemu ARM and ARM64 (and bring down ofono by dependency), let's add them to the skiplist. Change-Id: I1ac3485f339763f16d49e7527faa74f6efe71b30 Bug-AGL: SPEC-2970 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-11-28Also call kill in the right contextJan-Simon Moeller1-1/+1
depending on the app. Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I7208497e81fb3fcb2636bf7559b27ff2b5ce7676
2019-11-28Call info ps and status in the context of the app userJan-Simon Moeller1-4/+4
We need to call the info and status call under the context of user under which the app runs. Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I25abff89db22c5effb6addfbc59ec30575aae2ed
2019-11-28Merge "SPEC-2996: handle rcar_can"Jan-Simon Moeller1-1/+13
2019-11-28SPEC-2996: handle rcar_canCorentin LABBE1-1/+13
The test remove can_dv module fail on H3 due to rcar_can using it. This patchs handle this case. Change-Id: I452a61813567ad09f3f819ce73c1c11753326514 Bug-AGL: SPEC-2996 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-11-28SPEC-2996: Fix ip checkCorentin LABBE2-4/+5
BusyBox ip print its version stderr, so we need to redirect it to stdin. Furthermore, using set -x reset the error code, we need to store it before. Bug-AGL: SPEC-2996 Change-Id: I4b80475b0f9291fc55e661362a981ac14dbdb9b0 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-11-27SPEC-2996: rework the CAN modules testsCorentin LABBE1-89/+55
The CAN modules tests have lots of redundant code, let's add some function for probing/removing modules. Furthermore, some tests cannot work with busybox ip, I adds some tests for it. When reworking the tests, I have removed the "lsmod |grep can" tests which was of no value. I have also removed all sleep which was undocumented(probably for some log flushing), if something need them I will re-add it with comment. Bug-AGL: SPEC-2996 Change-Id: I2e8a109d799c58726d848814fd4c21a9fbd2eef5 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-11-26Verbose outputJan-Simon Möller1-2/+3
Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
2019-11-26Merge "test_can_loopback_mode: Add ip util version check"Jan-Simon Moeller1-1/+13
2019-11-26test_can_loopback_mode: Add ip util version checkCorentin LABBE1-1/+13
CAN tests fail on M3 and H3 because their build includes busybox version of ip (and not iproute one). This patchs adds a test for detecting if the right ip util is installed. Change-Id: I5909bc0af23271626dec9c1ed7003bdfc9129041 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-11-26SPEC-2970: Improove logging sizeCorentin LABBE3-0/+3
When TERM is dump, the width of logging is cropped at 80columns which made debugging harder. This patchs increase columns size to 1000. Bug-AGL: SPEC-2970 Change-Id: Ia70136b6aa6242362b1286631ae4b00ab567ae2e Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-11-07Cynagora: Replace cynara with cynagoraicefish_8.99.1icefish/8.99.18.99.1Jose Bollo3-5/+5
Currently we disable ptests/socket tests to be enabled in SPEC-2940 and SPEC-2941. Shift from the permission database cynara to cynagora permission database with a compatibility library. Bug-AGL: SPEC-2844 Bug-AGL: SPEC-2941 Signed-off-by: Jose Bollo <jose.bollo@iot.bzh> Change-Id: I673e47ae237a83926f15d003dd2d2623a08eefb5
2019-10-26Mask the bluetooth test until issues are resolvedJan-Simon Moeller1-0/+1
Currently the bluetooth service fails exclusively on m3ulcb-nogfx. We need to adapt the CI infra (lava labs) either with additional hw or adapt the dtb used conditionally. This needs further work. Meanwhile mask the bluetooth service test on the m3ulcb-nogfx target to avoid false-positives being reported. Bug-AGL: SPEC-2921 . Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I592ea7f8a6b8ddc4ff10eac8428162c44a1d3ac4
2019-10-23remove debuging information of do_afm_utilCorentin LABBE1-2/+0
This printing breaks 'rid' parsing, remove it. Bug-AGL: SPEC-2903 Bug-AGL: SPEC-2925 Change-Id: I505ab9d2994f983de53849fce25d87c8af44362d Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-10-22SPEC-2903: fix the grep test methodCorentin LABBE1-2/+6
While inspecting if application test is working with our latest SPEC-2903 changes I see in job 8845 the following errors: + grep urn:AGL:widget:provided-api config.xml + '<feature' 'name=\"urn:AGL:widget:provided-api\">' ./common/scripts/application-lifecycle.sh: line 74: <feature: command not found So let's change the grep method. Furthermore, add a debug for the do_afm_util for being sure it work as needed. Bug-AGL: SPEC-2903 Change-Id: I5a8a6d98cd9dc6c1b2d1626fc6841eb2c2b16d53 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-10-21SPEC-2903: Use a wrapper instead of constructing a commandCorentin LABBE1-23/+19
This patch change the way we ran afm-util commands by using a simple wrapper. Furthermore, since we are in a loop, let's reset all variable (SERVICE_PLATFORM/etc..) to 0 at the beginning of the loop. Extra-furthermore: restore the unzip error code test, but in non-fatal way, since busybox unzip seems to dont like our CRC. Bug-AGL: SPEC-2903 Change-Id: I5d7357fd74307ed69a6675beac57d8601fdcdb52 Signed-off-by: Corentin LABBE <clabbe@baylibre.com>
2019-10-19Improve lifecycle testJan-Simon Moeller1-7/+9
Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I9076278096e15829b4bca52f6b5ef9e9a3b39e9e
2019-10-17Merge "Differ platform and user scope in service and application"Jan-Simon Moeller1-2/+38
2019-10-17Differ platform and user scope in service and applicationJan-Simon Moeller1-2/+38
When executing the lifecycle test, we need to take into account if the service or app runs in platform or user scope. Bug-AGL: SPEC-2903 . Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Change-Id: I49a53a5a2b14666757b65b0f8203eb73ed2c7b45
2019-10-16Merge "Inverse the fail/pass logic"Jan-Simon Moeller1-2/+2
2019-10-16Merge "qa-testdefinitions: Split can tests to avoid failures"Jan-Simon Moeller3-28/+21
2019-10-15Inverse the fail/pass logicKhouloud Touil1-2/+2
This test should be failing, so it should be set to pass when it fails, and should be set to fail when it passes. AGL-bug: SPEC-2878 Change-Id: I43921717eea77d654921eca4691c936bb2901e54 Signed-off-by: Khouloud Touil <ktouil@baylibre.com>