tests/MakefileΒΆ

This Makefile is in charge of building and executing all the test cases. It has a dependency on the test_modules/Makefile Makefile. All the test modules have to be built before this Makefile runs. This is ensured by the Top-level Makefile:

test: libs test_modules

The tests use the Cmocka test framework, and it is expected to be installed in the system. Also, some of the tests require the Hitch TLS proxy. It is possible to install them in a debian-alike system using:

apt-get install libcmocka-dev hitch

The Makefile contains the following targets:

  • build: to build all the test cases without executing them.

  • test: to build and executes all the test cases. It is the default target.

The test target produces a listing after executing all the tests with a summary of how many tests where executed, how many tests passed, and how many tests failed. It also prints the list of failed tests.

The tests are executed using the script run-tests.sh which takes the following options:

  • -p platform: Selects the platform for the tests. Valid values are EVP1 (also named EVP1-TB) or TB. By default it uses EVP1-TB.

  • -t timeout: Timeout used to declare that a test has failed. The default is 60 seconds.

  • -d logdir: Directory where logs are stored. The default is logs.

  • -e executor: It uses the program executor to run the tests. This option can be used, for example, with -e valgrind to run all the tests with valgrind. By default it does not use any executor.

  • -c: Enable colouring the test listing using terminal escape sequences.

  • -s: Execute the tests serially. By default run-tests.sh tries to execute as many tests in parallel as it can. This option can be very helpful when debugging problems.

The behaviour of run-tests.sh can be modified from the make invocation using the macro RUNFLAGS. For example,

make RUNFLAGS='-t 30 -c'

will run the tests with a timeout of 30 seconds and with output colouring enabled. It may be more comfortable to set these options in the Personal configuration.

The Makefile allows (but does not encourage) fine control of the options. Selecting the set of tests can be done using the following Makefile targets:

  • run-ut: Run only the unit tests.

  • run-st: Run only the system tests (also knonwn as integration tests).

  • run-ut-evp1-tb: Run only the unit tests for EVP1-TB.

  • run-ut-evp2-tb: Run only the unit tests for EVP2-TB.

  • run-ut-nh: Run only the unit tests nohub (tests that are independent of the hub configuration).

  • run-st-evp1-tb: Run only the system tests for EVP1-TB.

  • run-st-evp2-tb: Run only the system tests for EVP2-TB.

  • run-st-nh: Run only the system tests nohub (tests that are independent of the hub configuration).

These targets do not print the summary information printed by the test target. In case it is desirable to run only one test, the best option is usually to use the run-tests.sh script directly, for example:

./run-tests.sh -c -d logs/EVP2-TB -p TB src/systest/test_deployment.elf

will run the test src/systest/test_deployment.elf with a EVP2-TB configuration, with colouring enabled and placing the logs in logs/EVP2-TB/src/systest/test_deployment.elf.log.

Warning

There is a known bug in GNU binutils that causes the build to fail when debug options are used. This problem was already reported to the binutils community and it was fixed, but the fix is not propagated to all supported build environments yet. As a workaround, using the LLVM linker avoids the problem. It is possible to select the linker by passing LDFLAGS='-fuse-ld=lld -g' in the make command.