aboutsummaryrefslogtreecommitdiffstats
path: root/roms/u-boot/doc/develop/tests_sandbox.rst
diff options
context:
space:
mode:
authorAngelos Mouzakitis <a.mouzakitis@virtualopensystems.com>2023-10-10 14:33:42 +0000
committerAngelos Mouzakitis <a.mouzakitis@virtualopensystems.com>2023-10-10 14:33:42 +0000
commitaf1a266670d040d2f4083ff309d732d648afba2a (patch)
tree2fc46203448ddcc6f81546d379abfaeb323575e9 /roms/u-boot/doc/develop/tests_sandbox.rst
parente02cda008591317b1625707ff8e115a4841aa889 (diff)
Add submodule dependency filesHEADmaster
Change-Id: Iaf8d18082d3991dec7c0ebbea540f092188eb4ec
Diffstat (limited to 'roms/u-boot/doc/develop/tests_sandbox.rst')
-rw-r--r--roms/u-boot/doc/develop/tests_sandbox.rst209
1 files changed, 209 insertions, 0 deletions
diff --git a/roms/u-boot/doc/develop/tests_sandbox.rst b/roms/u-boot/doc/develop/tests_sandbox.rst
new file mode 100644
index 000000000..84608dcb8
--- /dev/null
+++ b/roms/u-boot/doc/develop/tests_sandbox.rst
@@ -0,0 +1,209 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+Sandbox tests
+=============
+
+Test Design
+-----------
+
+Most uclasses and many functions of U-Boot have sandbox tests. This allows much
+of the code to be checked in an developer-friendly environment.
+
+Sandbox provides a way to write and run unit tests. The traditional approach to
+unit tests is to build lots of little executables, one for each test or
+category of tests. With sandbox, so far as possible, all the tests share a
+small number of executables (e.g. 'u-boot' for sandbox, 'u-boot-spl' and
+'u-boot' for sandbox_spl) and can be run very quickly. The vast majority of
+tests can run on the 'sandbox' build,
+
+Available tests
+---------------
+
+Some of the available tests are:
+
+ - command_ut: Unit tests for command parsing and handling
+ - compression: Unit tests for U-Boot's compression algorithms, useful for
+ security checking. It supports gzip, bzip2, lzma and lzo.
+ - image: Unit tests for images:
+
+ - test/image/test-imagetools.sh - multi-file images
+ - test/py/tests/test-fit.py - FIT images
+ - tracing: test/trace/test-trace.sh tests the tracing system (see
+ README.trace)
+ - verified boot: test/py/tests/test_vboot.py
+
+If you change or enhance any U-Boot subsystem, you should write or expand a
+test and include it with your patch series submission. Test coverage in some
+older areas of U-Boot is still somewhat limited and we need to work to improve
+it.
+
+Note that many of these tests are implemented as commands which you can
+run natively on your board if desired (and enabled).
+
+To run all tests, use 'make check'.
+
+
+Running sandbox tests directly
+------------------------------
+
+Typically tests are run using the pytest suite. Running pytests on sandbox is
+easy and always gets things right. For example some tests require files to be
+set up before they can work.
+
+But it is also possible to run some sandbox tests directly. For example, this
+runs the dm_test_gpio() test which you can find in test/dm/gpio.c::
+
+ $ ./u-boot -T -c "ut dm gpio"
+
+
+ U-Boot 2021.01
+
+ Model: sandbox
+ DRAM: 128 MiB
+ WDT: Started with servicing (60s timeout)
+ MMC: mmc2: 2 (SD), mmc1: 1 (SD), mmc0: 0 (SD)
+ In: serial
+ Out: vidconsole
+ Err: vidconsole
+ Model: sandbox
+ SCSI:
+ Net: eth0: eth@10002000, eth5: eth@10003000, eth3: sbe5, eth6: eth@10004000
+ Test: dm_test_gpio: gpio.c
+ Test: dm_test_gpio: gpio.c (flat tree)
+ Failures: 0
+
+The -T option tells the U-Boot sandbox to run with the 'test' devicetree
+(test.dts) instead of -D which selects the normal sandbox.dts - this is
+necessary because many tests rely on nodes or properties in the test devicetree.
+If you try running tests without -T then you may see failures, like::
+
+ $ ./u-boot -c "ut dm gpio"
+
+
+ U-Boot 2021.01
+
+ DRAM: 128 MiB
+ WDT: Not found!
+ MMC:
+ In: serial
+ Out: serial
+ Err: serial
+ SCSI:
+ Net: No ethernet found.
+ Please run with test device tree:
+ ./u-boot -d arch/sandbox/dts/test.dtb
+ Test: dm_test_gpio: gpio.c
+ test/dm/gpio.c:37, dm_test_gpio(): 0 == gpio_lookup_name("b4", &dev, &offset, &gpio): Expected 0x0 (0), got 0xffffffea (-22)
+ Test: dm_test_gpio: gpio.c (flat tree)
+ test/dm/gpio.c:37, dm_test_gpio(): 0 == gpio_lookup_name("b4", &dev, &offset, &gpio): Expected 0x0 (0), got 0xffffffea (-22)
+ Failures: 2
+
+The message above should provide a hint if you forget to use the -T flag. Even
+running with -D will produce different results.
+
+You can easily use gdb on these tests, without needing --gdbserver::
+
+ $ gdb u-boot --args -T -c "ut dm gpio"
+ ...
+ (gdb) break dm_test_gpio
+ Breakpoint 1 at 0x1415bd: file test/dm/gpio.c, line 37.
+ (gdb) run -T -c "ut dm gpio"
+ Starting program: u-boot -T -c "ut dm gpio"
+ Test: dm_test_gpio: gpio.c
+
+ Breakpoint 1, dm_test_gpio (uts=0x5555558029a0 <global_dm_test_state>)
+ at files/test/dm/gpio.c:37
+ 37 ut_assertok(gpio_lookup_name("b4", &dev, &offset, &gpio));
+ (gdb)
+
+You can then single-step and look at variables as needed.
+
+
+Running sandbox_spl tests directly
+----------------------------------
+
+SPL is the phase before U-Boot proper. It is present in the sandbox_spl build,
+so you can run SPL like this::
+
+ ./spl/u-boot-spl
+
+SPL tests are special in that they run (only in the SPL phase, of course) if the
+-u flag is given::
+
+ ./spl/u-boot-spl -u
+
+ U-Boot SPL 2021.01-00723-g43c77b51be5-dirty (Jan 24 2021 - 16:38:24 -0700)
+ Running 5 driver model tests
+ Test: dm_test_of_plat_base: of_platdata.c (flat tree)
+ Test: dm_test_of_plat_dev: of_platdata.c (flat tree)
+ Test: dm_test_of_plat_parent: of_platdata.c (flat tree)
+ Test: dm_test_of_plat_phandle: of_platdata.c (flat tree)
+ Test: dm_test_of_plat_props: of_platdata.c (flat tree)
+ Failures: 0
+
+
+ U-Boot 2021.01-00723-g43c77b51be5-dirty (Jan 24 2021 - 16:38:24 -0700)
+
+ DRAM: 128 MiB
+ ...
+
+It is not possible to run SPL tests in U-Boot proper, firstly because they are
+not built into U-Boot proper and secondly because the environment is very
+different, e.g. some SPL tests rely on of-platdata which is only available in
+SPL.
+
+Note that after running, SPL continues to boot into U-Boot proper. You can add
+'-c exit' to make U-Boot quit without doing anything further. It is not
+currently possible to run SPL tests and then stop, since the pytests require
+that U-Boot produces the expected banner.
+
+You can use the -k flag to select which tests run::
+
+ ./spl/u-boot-spl -u -k dm_test_of_plat_parent
+
+Of course you can use gdb with sandbox_spl, just as with sandbox.
+
+
+Running all tests directly
+--------------------------
+
+A fast way to run all sandbox tests is::
+
+ ./u-boot -T -c "ut all"
+
+It typically runs single-thread in 6 seconds on 2021 hardware, with 2s of that
+to the delays in the time test.
+
+This should not be considered a substitute for 'make check', but can be helpful
+for git bisect, etc.
+
+
+What tests are built in?
+------------------------
+
+Whatever sandbox build is used, which tests are present is determined by which
+source files are built. For sandbox_spl, the of_platdata tests are built
+because of the build rule in test/dm/Makefile::
+
+ ifeq ($(CONFIG_SPL_BUILD),y)
+ obj-$(CONFIG_SPL_OF_PLATDATA) += of_platdata.o
+ else
+ ...other tests for non-spl
+ endif
+
+You can get a list of tests in a U-Boot ELF file by looking for the
+linker_list::
+
+ $ nm /tmp/b/sandbox_spl/spl/u-boot-spl |grep 2_dm_test
+ 000000000001f200 D _u_boot_list_2_dm_test_2_dm_test_of_plat_base
+ 000000000001f220 D _u_boot_list_2_dm_test_2_dm_test_of_plat_dev
+ 000000000001f240 D _u_boot_list_2_dm_test_2_dm_test_of_plat_parent
+ 000000000001f260 D _u_boot_list_2_dm_test_2_dm_test_of_plat_phandle
+ 000000000001f280 D _u_boot_list_2_dm_test_2_dm_test_of_plat_props
+
+
+Writing tests
+-------------
+
+See :doc:`tests_writing` for how to write new tests.
+