aboutsummaryrefslogtreecommitdiffstats
path: root/tests
AgeCommit message (Collapse)AuthorFilesLines
2023-04-21agl-test-framework: Optimize common function log_processQiu Tingting18-268/+20
Add new commone function log_process_gnu for the following tests: acl attr bash bc bluez5 busybox cpio libpam libxml2 zlib SPEC-4758 Signed-off-by: Qiu Tingting <qiutt@fujitsu.com> Change-Id: I92de3b781badac9cb6d55acd5cc0c601dc4a7c65
2023-04-18ltp/cve : set environment variablesduerpei1-0/+15
If the kernel config file cannot be found, set the environment variable "KCONFIG_SKIP_CHECK" = "True" to avoid some case errors in ltp/cve. The format of the case error report is as follows: <<<test_output>>> incrementing stop tst_kconfig.c:65: TINFO: Couldn't locate kernel config! tst_kconfig.c:201: TBROK: Cannot parse kernel .config Bug-AGL: SPEC-4755 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: Idbe419ee2938436196893389c69a706b0083431e
2023-04-14agl-test-framework: add ptest bluez5Qiu Tingting3-0/+47
Add ptest bluez5 in agl-test-framework. Testing costs 36s. Bug-AGL: SPEC-4753 Signed-off-by: Qiu Tingting <qiutt@fujitsu.com> Change-Id: I9a6c548f28db761883845f40e3fcd612574946c5
2023-04-11agl-test-framework: add ptest attrQiu Tingting2-0/+38
Add ptest attr in agl-test-framework. Testing costs 7s. Bug-AGL: SPEC-4753 Signed-off-by: Qiu Tingting <qiutt@fujitsu.com> Change-Id: I9f4b74331dac5db4ba64ce56d268fd856d7ad8b9
2023-04-11agl-test-framework: add ptest aclQiu Tingting2-0/+38
Add ptest acl in agl-test-framework. Testing costs 1m46s. Bug-AGL: SPEC-4753 Signed-off-by: Qiu Tingting <qiutt@fujitsu.com> Change-Id: Id43d68b85f560625407971f01322a45aef9a53cf
2023-04-03agl-test-framework: add ptest json-glibYan2-0/+30
add json-glib ptest into agl-test-framework. Bug-AGL: SPEC-4733 Signed-off-by: Yan <yanxk.fnst@fujitsu.com> Change-Id: I03ef4122fbef0c6cee3eec5e5d23915698b35e51
2023-03-08agl-test-framework: solve bugs in the test frameworkduerpei6-5/+44
Fixed problems: 1.When all the executed test suites are skipped, no new summary- report is generated. 2.When there is test suite successfully executed, the generated summary-report will contain the test suites which have logs in the "/tmp-log" directory, rather than just the test suites which has just executed. Correction method: 1.Move the function in "conftest.py" to "agl_test_report.py" and call it in the "agl-test" script to ensure that the function will be executed. 2.Set the timestamp to record the information of each executed test suite or skipped test suite to the file of "test_list_timestamp". And generate a summar-report according to the above file. Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: I47bfd09706e37ce6bdc13f3f9f266bc62f74f777
2023-07-22agl-test-framework: add "cpio"yanxk3-0/+66
it is new ptest, newly supported in YP master-next by us, now we add it into our test framework. sample output: tests/cpio/run_tests.py::test_cpio[cpio version ] PASSED tests/cpio/run_tests.py::test_cpio[basic functionality: copyin/copyout ] PASSED tests/cpio/run_tests.py::test_cpio[symlink ] PASSED Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I08d8c18c1f1f4ec7df5f5447e6177490958fad37
2023-07-17agl-test-framework: add memory test "linus_stress"yanxk2-0/+55
this test was part of core test, testing memory. it is now part of agl-test-framework. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I04caa643c440de18707ea112c655f694fc62f257
2023-07-12agl-test-framework: replace "-" with "_"duerpei6-0/+0
Replace "-" with "_" in the name of the directory to solve the following errors when running agl-test: import file mismatch: imported module 'run_tests' has this __file__ attribute: /usr/AGL/agl-test/tests/aio-stress/run_tests.py which is not the same as the test file we want to collect: /usr/AGL/agl-test/tests/gdk-pixbuf/run_tests.py HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: Id8028dfbda618a9b7858602d3832bf2eb7f7ef5e
2021-03-30agl-test-framework: add LTP syscalls tests into frameworkyanxk5-1/+91
syscalls in LTP version 20220121 contains some bugs, those have been adjusted in order to provide trustable test reports. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I41aba8e64e9b916066f9a60ffac863ffdbabbf47
2023-01-23agl-test-framework: fix bug caused by prefixyanxk1-2/+2
prefix should be set to 'LTP/', otherwise the get_workdir() method in class AGLBaseTest would not work. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I8a9eeaa78acde2fc6a8f085c0bcdef3b5b800b45
2023-01-23agl-test-framework: fix pytest warningyanxk1-1/+1
fix a pytest warning caused by missing escape character. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: Iec8a4faf09322b6518409cb575d0149cad6d34a9
2023-01-20agl-test-framework: enable ptest of 'busybox'duerpei4-0/+78
Enable the test suite of 'busybox' from ptest Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: I960f013d47f893f65766628da443ad6dce6f1009
2023-01-18agl-test-framework: add posix conformance testsyanxk2-0/+109
conformance tests from Open Posix Test Suite is added into agl-test now. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I4da7dbc2cd423c0de01f9ee53651f5590181e68b
2023-01-17agl-test-framework: add new functions for LTP classduerpei1-9/+14
Add new function of "get_log_path" and "get_log_file" for LTP class. Change the judgment rule in "run_test_fun" to make it more reasonable. Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: Id2e805046fe48551f910f04ad1dcfca5c76b8019
2023-01-13agl-test-framework: add script for ltp/cveduerpei2-0/+24
Add run_test.py script for the test suite of ltp/cve, the script used the class of LTPBase. Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: I4eb27c18f1b49f51308f531b687c46f736461f5e
2023-01-13agl-test-framework: make sure "crashme" would have return valueyanxk1-0/+30
a fact is "crashme" would crash qemux86 under mid or high level, but low level looks fine, we provide kill -15 solution to solve "no return" hang up. so there will always be a return value. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I3d9ebb64b8b57a64d0ca002f20cf908141099ad6
2022-10-12agl-test-framework: add LTPBase classduerpei4-0/+154
Add LTPBase class for test suites from LTP(linux-test-project). The LTPBase inherits from AGLBaseTest class. It contains some common functions, which are used to run the test suites from LTP and process the log. It makes it easy to support test suites from LTP. And add run_test scripts for the test suite of ltp/math, the scripts used the class of LTPBase. Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: I683aaca37ddfb84a12e570a8918934ea8391ded6
2022-10-07agl-test-framework: fix bug for ptest testsuitesQiu Tingting9-9/+0
Fix bug for ptest testsuites which introduced by the following commit: agl-test-framework: add "XPASS" and "XFAIL" states Bug-AGL: SPEC-4345 Signed-off-by: Qiu Tingting <qiutt@fujitsu.com> Change-Id: I77309e05468045bf1deba1ce94aed81f9492d5e9
2023-01-03agl-test-framework: enable ptest of gdk-pixbufQiu Tingting2-0/+29
Add test scripts for enabling ptest of gdk-pixbuf. This test will cost 12 seconds time in raspberrypi4. Bug-AGL: SPEC-4345 Signed-off-by: Qiu Tingting <qiutt@fujitsu.com> Change-Id: I67fefce8552fe9033734cf340758fe92efc5a58d
2023-01-03agl-test-framework: enable ptest of python3Qiu Tingting3-0/+134
Add test scripts for enabling ptest of python3. This test will cost 1h50min time in raspberrypi4. Bug-AGL: SPEC-4345 Signed-off-by: Qiu Tingting <qiutt@fujitsu.com> Change-Id: I840f4fa3e2d457f87260ec79fc72a06327157f22
2022-09-23agl-test-framework: add test "stress-ng"duerpei2-14/+12
Replace "stress" with "stress-ng", the stress-ng package which is already in OE-core. Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: Ia79453f2fc82d4aed0a7a405df93d2560da253de
2022-12-08agl-test-framework: add test 'bash'duerpei3-0/+68
Add test script for 'bash' Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: Id50dcc3d1a80b8d088b8af897aee82f6492e1a96
2022-12-02agl-test-framework: add test 'crashme'yanxk3-0/+115
'agl-test' modified, now, usr would not run dangerous tests like this 'crashme' unless they do self selection by '-m' pytest option. Set default to 'not dangerous'. a new marker 'dangerous' has been add to the pytest.ini. add new method into the agl_test_base.py, to help get path of the 'spec.json', this file help stress tests like 'crashme' to run under different stress levels. for run_tests.py, modifications focus on how to run 'crashme' under that different levels. new marker 'order' would always ensure this 'crashme' test to be placed as the so called 'last' test of all. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: Icb36d7c5c52430d89787f4373a2852b1b03ec8a6
2022-08-27agl-test-framework: add test 'stress'duerpei2-0/+57
Add test script for 'stress' Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: I1c0c048dc907e5cff5d50396ab28b7550371b32e
2022-11-11agl-test-framework: enable ptest of 'bc'yanxk3-0/+65
we have already submitted patches that enabled ptest of 'bc' to the OpenEmbedded community and the patches got merged. but it is not yet in the YP 4.0.5, so it would skip now. it can be seen in YP 4.1. we now add it into the test framework, for it would be available in the next YP release, maybe, or several versions later. Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: Iffdef472c9449c9e3179c3b3bd1273fb2ec0e032
2022-08-14agl-test-framework: deploy parametric implementation for old ‘glib-2.0’ testyanxk1-34/+6
Single definitions are no longer used in this test framework, they are now replaced by another parametric implementation. The benefit of this is that it minimizes the likelihood of requiring manual maintenance due to the small but frequent updates from the upstream. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I86a0edabb27c7117d8279e1478be94f456894533
2022-08-14agl-test-framework: enable ptest of 'openssl'yanxk3-0/+72
Base on the agl-test-framework, the ptest results of 'openssl' can now be collected, analyzed and then reported. Here are 3 samples that the output result might look like: tests/openssl/run_tests.py::test_openssl[30-test_evp_fetch_prov] PASSED tests/openssl/run_tests.py::test_openssl[30-test_evp_kdf] PASSED tests/openssl/run_tests.py::test_openssl[30-test_evp_libctx] PASSED Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: Id28a36ce081e8cfb49dcb7af8db369244f1fd258
2022-11-10agl-test-framework: deploy parametric implementation for old ‘expat’ testyanxk1-3416/+6
Single definitions are no longer used in this test framework, they are now replaced by another parametric implementation. The benefit of this is that it minimizes the likelihood of requiring manual maintenance due to the small but frequent updates from the upstream. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I326c429b0750e7d7e623f45d36d4e53e5c5b97c4
2022-11-10agl-test-framework: deploy parametric implementation for old ‘zlib’ testyanxk1-24/+6
Single definitions are no longer used in this test framework, they are now replaced by another parametric implementation. The benefit of this is that it minimizes the likelihood of requiring manual maintenance due to the small but frequent updates from the upstream. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I18ee28b6ebce12c61a55e87fef9f1b96f6db958c
2022-11-10agl-test-framework: deploy parametric implementation for old ‘bzip2’ testyanxk1-2015/+6
Single definitions are no longer used in this test framework, they are now replaced by another parametric implementation. The benefit of this is that it minimizes the likelihood of requiring manual maintenance due to the small but frequent updates from the upstream. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I609a4c54edcc2b68a43d759fa60e9eee64d9c2cf
2022-11-10agl-test-framework: enable ptest of 'libxml2'yanxk3-0/+69
Base on the agl-test-framework, the ptest results of 'libxml2' can now be collected, analyzed and then reported. Here are 3 samples that the output result might look like: tests/libxml2/run_tests.py::test_libxml2[Reader regression tests] PASSED tests/libxml2/run_tests.py::test_libxml2[Reader entities substitution regression tests] PASSED tests/libxml2/run_tests.py::test_libxml2[Reader on memory regression tests] PASSED Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I8361fe54177d951c36732ebfbe8dd8915975d7a9
2022-11-08agl-test-framework: enable ptest of 'libpam'yanxk3-0/+68
Base on the agl-test-framework, the ptest results of 'libpam' can now be collected, analyzed and then reported. Here are 3 samples that the output result might look like: tests/libpam/run_tests.py::test_libpam[tst-pam_acct_mgmt] PASSED tests/libpam/run_tests.py::test_libpam[tst-pam_authenticate] PASSED tests/libpam/run_tests.py::test_libpam[tst-pam_chauthtok] PASSED Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I11c73689b5073ca4bf24b34d128287e029afdb3a
2022-07-24Add precheck() funQiu Tingting5-1/+41
Checking env before run testsuite is necessary. 1. Add common check in agl_test_base.py. It is currently an empty function. If necessary, common check sould be added. 2. Add ptest common check in agl_test_ptest_base.py. For ptest testsuite, check whether ptest-runner and run-ptest script exist. 3. Add special check in run_tests.py. For aio-stress testsuite, check whether compiled aio-stress script exists For bzip2 testsuite, check whether bzip2 cmd exists. Bug-AGL: SPEC-4345 Signed-off-by: Qiu Tingting <qiutt@fujitsu.com> Change-Id: Id7841c0337465266dd607403ecb3e1c4377c6198
2022-07-23agl-test-framework: enable ptest of 'zlib'yanxk3-0/+79
Base on the agl-test-framework, the test results of OSS "zlib" can now be collected, analyzed and then reported. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I023c080cc365facb7d647f4c1085b7c9739f0b0e
2022-07-21tests: solve the problem of duplicate nameduerpei3-2/+2
To avoid error caused by the same name between the test set and pyhton module, add an empty '__ init__. py' file in '/tests' Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: I7748e6ae90b2eb12b59a39b3659b5910fc487810
2022-07-11agl-test-framework: enable Ptest of "bzip2"yanxk3-0/+2082
Base on the agl-test-framework, the test results of OSS "bzip2" can now be collected, analyzed and then reported. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: I28df7079cfc0bc4b95b3ca195119f435b9ba8b6b
2022-09-30agl-test-framework: add test set of aio-stressduerpei2-0/+50
Add the script file that calls the test set 'aiostress'. Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: Ib3906fed1f85031d3554342a5555aa3a046b8c09
2022-07-02agl-test-framework: enable Ptest of "Expat"yanxk3-0/+3494
Base on the agl-test-framework, the test results of OSS "Expat" can now be collected, analyzed and then reported. Bug-AGL: SPEC-4345 Signed-off-by: yanxk <yanxk.fnst@fujitsu.com> Change-Id: Ibd1cccc6f78f4ee6a640b47eb74d2b35f40b6d31
2022-06-16agl-test-framework: use PTESTBaseTest classduerpei2-0/+52
Move the "run_tests" script of glib2 under "qa-test-misc" to "agl-test-framework", this is more reasonable. And modify the script file run_tests under glib2, make it use PTESTBaseTest class. Later, I will delete the "run_tests" scripts under "qa-test-misc" Bug-AGL: SPEC-4345 Signed-off-by: duerpei <duep.fnst@fujitsu.com> Change-Id: Id055b829954bf8bdf75a90ff5b417baae0c4d5fb