diff options
author | 2023-10-10 14:33:42 +0000 | |
---|---|---|
committer | 2023-10-10 14:33:42 +0000 | |
commit | af1a266670d040d2f4083ff309d732d648afba2a (patch) | |
tree | 2fc46203448ddcc6f81546d379abfaeb323575e9 /roms/u-boot/doc/develop/tests_sandbox.rst | |
parent | e02cda008591317b1625707ff8e115a4841aa889 (diff) |
Change-Id: Iaf8d18082d3991dec7c0ebbea540f092188eb4ec
Diffstat (limited to 'roms/u-boot/doc/develop/tests_sandbox.rst')
-rw-r--r-- | roms/u-boot/doc/develop/tests_sandbox.rst | 209 |
1 files changed, 209 insertions, 0 deletions
diff --git a/roms/u-boot/doc/develop/tests_sandbox.rst b/roms/u-boot/doc/develop/tests_sandbox.rst new file mode 100644 index 000000000..84608dcb8 --- /dev/null +++ b/roms/u-boot/doc/develop/tests_sandbox.rst @@ -0,0 +1,209 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +Sandbox tests +============= + +Test Design +----------- + +Most uclasses and many functions of U-Boot have sandbox tests. This allows much +of the code to be checked in an developer-friendly environment. + +Sandbox provides a way to write and run unit tests. The traditional approach to +unit tests is to build lots of little executables, one for each test or +category of tests. With sandbox, so far as possible, all the tests share a +small number of executables (e.g. 'u-boot' for sandbox, 'u-boot-spl' and +'u-boot' for sandbox_spl) and can be run very quickly. The vast majority of +tests can run on the 'sandbox' build, + +Available tests +--------------- + +Some of the available tests are: + + - command_ut: Unit tests for command parsing and handling + - compression: Unit tests for U-Boot's compression algorithms, useful for + security checking. It supports gzip, bzip2, lzma and lzo. + - image: Unit tests for images: + + - test/image/test-imagetools.sh - multi-file images + - test/py/tests/test-fit.py - FIT images + - tracing: test/trace/test-trace.sh tests the tracing system (see + README.trace) + - verified boot: test/py/tests/test_vboot.py + +If you change or enhance any U-Boot subsystem, you should write or expand a +test and include it with your patch series submission. Test coverage in some +older areas of U-Boot is still somewhat limited and we need to work to improve +it. + +Note that many of these tests are implemented as commands which you can +run natively on your board if desired (and enabled). + +To run all tests, use 'make check'. + + +Running sandbox tests directly +------------------------------ + +Typically tests are run using the pytest suite. Running pytests on sandbox is +easy and always gets things right. For example some tests require files to be +set up before they can work. + +But it is also possible to run some sandbox tests directly. For example, this +runs the dm_test_gpio() test which you can find in test/dm/gpio.c:: + + $ ./u-boot -T -c "ut dm gpio" + + + U-Boot 2021.01 + + Model: sandbox + DRAM: 128 MiB + WDT: Started with servicing (60s timeout) + MMC: mmc2: 2 (SD), mmc1: 1 (SD), mmc0: 0 (SD) + In: serial + Out: vidconsole + Err: vidconsole + Model: sandbox + SCSI: + Net: eth0: eth@10002000, eth5: eth@10003000, eth3: sbe5, eth6: eth@10004000 + Test: dm_test_gpio: gpio.c + Test: dm_test_gpio: gpio.c (flat tree) + Failures: 0 + +The -T option tells the U-Boot sandbox to run with the 'test' devicetree +(test.dts) instead of -D which selects the normal sandbox.dts - this is +necessary because many tests rely on nodes or properties in the test devicetree. +If you try running tests without -T then you may see failures, like:: + + $ ./u-boot -c "ut dm gpio" + + + U-Boot 2021.01 + + DRAM: 128 MiB + WDT: Not found! + MMC: + In: serial + Out: serial + Err: serial + SCSI: + Net: No ethernet found. + Please run with test device tree: + ./u-boot -d arch/sandbox/dts/test.dtb + Test: dm_test_gpio: gpio.c + test/dm/gpio.c:37, dm_test_gpio(): 0 == gpio_lookup_name("b4", &dev, &offset, &gpio): Expected 0x0 (0), got 0xffffffea (-22) + Test: dm_test_gpio: gpio.c (flat tree) + test/dm/gpio.c:37, dm_test_gpio(): 0 == gpio_lookup_name("b4", &dev, &offset, &gpio): Expected 0x0 (0), got 0xffffffea (-22) + Failures: 2 + +The message above should provide a hint if you forget to use the -T flag. Even +running with -D will produce different results. + +You can easily use gdb on these tests, without needing --gdbserver:: + + $ gdb u-boot --args -T -c "ut dm gpio" + ... + (gdb) break dm_test_gpio + Breakpoint 1 at 0x1415bd: file test/dm/gpio.c, line 37. + (gdb) run -T -c "ut dm gpio" + Starting program: u-boot -T -c "ut dm gpio" + Test: dm_test_gpio: gpio.c + + Breakpoint 1, dm_test_gpio (uts=0x5555558029a0 <global_dm_test_state>) + at files/test/dm/gpio.c:37 + 37 ut_assertok(gpio_lookup_name("b4", &dev, &offset, &gpio)); + (gdb) + +You can then single-step and look at variables as needed. + + +Running sandbox_spl tests directly +---------------------------------- + +SPL is the phase before U-Boot proper. It is present in the sandbox_spl build, +so you can run SPL like this:: + + ./spl/u-boot-spl + +SPL tests are special in that they run (only in the SPL phase, of course) if the +-u flag is given:: + + ./spl/u-boot-spl -u + + U-Boot SPL 2021.01-00723-g43c77b51be5-dirty (Jan 24 2021 - 16:38:24 -0700) + Running 5 driver model tests + Test: dm_test_of_plat_base: of_platdata.c (flat tree) + Test: dm_test_of_plat_dev: of_platdata.c (flat tree) + Test: dm_test_of_plat_parent: of_platdata.c (flat tree) + Test: dm_test_of_plat_phandle: of_platdata.c (flat tree) + Test: dm_test_of_plat_props: of_platdata.c (flat tree) + Failures: 0 + + + U-Boot 2021.01-00723-g43c77b51be5-dirty (Jan 24 2021 - 16:38:24 -0700) + + DRAM: 128 MiB + ... + +It is not possible to run SPL tests in U-Boot proper, firstly because they are +not built into U-Boot proper and secondly because the environment is very +different, e.g. some SPL tests rely on of-platdata which is only available in +SPL. + +Note that after running, SPL continues to boot into U-Boot proper. You can add +'-c exit' to make U-Boot quit without doing anything further. It is not +currently possible to run SPL tests and then stop, since the pytests require +that U-Boot produces the expected banner. + +You can use the -k flag to select which tests run:: + + ./spl/u-boot-spl -u -k dm_test_of_plat_parent + +Of course you can use gdb with sandbox_spl, just as with sandbox. + + +Running all tests directly +-------------------------- + +A fast way to run all sandbox tests is:: + + ./u-boot -T -c "ut all" + +It typically runs single-thread in 6 seconds on 2021 hardware, with 2s of that +to the delays in the time test. + +This should not be considered a substitute for 'make check', but can be helpful +for git bisect, etc. + + +What tests are built in? +------------------------ + +Whatever sandbox build is used, which tests are present is determined by which +source files are built. For sandbox_spl, the of_platdata tests are built +because of the build rule in test/dm/Makefile:: + + ifeq ($(CONFIG_SPL_BUILD),y) + obj-$(CONFIG_SPL_OF_PLATDATA) += of_platdata.o + else + ...other tests for non-spl + endif + +You can get a list of tests in a U-Boot ELF file by looking for the +linker_list:: + + $ nm /tmp/b/sandbox_spl/spl/u-boot-spl |grep 2_dm_test + 000000000001f200 D _u_boot_list_2_dm_test_2_dm_test_of_plat_base + 000000000001f220 D _u_boot_list_2_dm_test_2_dm_test_of_plat_dev + 000000000001f240 D _u_boot_list_2_dm_test_2_dm_test_of_plat_parent + 000000000001f260 D _u_boot_list_2_dm_test_2_dm_test_of_plat_phandle + 000000000001f280 D _u_boot_list_2_dm_test_2_dm_test_of_plat_props + + +Writing tests +------------- + +See :doc:`tests_writing` for how to write new tests. + |