Code coverage for assembler source files

Assembler code coverage is essential primarily because embedded software often cannot be entirely written in high-level languages. In many projects, manually written assembler code are required for two main reasons

  • Accessing Hardware
  • There are scenarios where exceptionally fast and optimized routines are required, which high-level languages does not allow. In such cases, assembler code can be utilized to achieve the desired level of efficiency.

    Some development standards require to proof the traceability between the source code and the generated assembler code in order to avoid hidden code that is not tested during unit test phase. For example, the DO178B/C states in paragraph 6.4.4.2b for level A application:

    The structural coverage analysis may be performed on the Source Code, unless the software level is A and the compiler generates object code that is not directly traceable to Source Code statements. Then, additional verification should be performed on the object code to establish the correctness of such generated code sequences. A compiler-generated array-bound check in the object code is an example of object code that is not directly traceable to the Source Code.

    This holds particularly true for programming languages such as C++ and Ada. Ensuring traceability can, in some cases, be guaranteed by the compiler provider through the use of specific options, although this is not a common practice. One potential solution, as suggested in CAST-12, involves a comprehensive analysis of high-level language constructs to demonstrate that the resulting assembly code can be traced back. However, this approach can be quite challenging and time-consuming.

    Another solution, also outlined in CAST-12, is to compute the code coverage at the assembler level to confirm that all generated assembly code is covered by the tests, just like the coverage achieved at the high-level language.

Solution provided by DevOps Test Embedded

DevOps Test Embedded integrates an assembler code coverage for ARM 32 bits and 64 bits processor, with the following characteristics:

  • Support the following levels:
    • Function: This level focuses on function, which serve as an entry point in the assembler code identified by the directive .function or by a reference in an instruction BR (routine call).
    • Function and exit: This level includes the function coverage and the exit point coverage. An exit point is identified by the instructions such as MOV LR,PC and POP PC in the assembler code.
    • Statement block: A statement block is a set of instructions between a block entry that can be a label or an instruction just after a conditional jump and a block exit that can be a jump or the instruction just before another block entry.
    • Call: A call is identified by the BR instruction.
    Note: There is no need for additional levels since assembler code does not involve decision with multiple conditions like you find in C/C++ and Ada languages.
  • Reports are compatible with high-level language reports and can be mixed in a single report.
  • A new tool named attolgas is provided that instruments assembler code based on the gcc-like assembler listing format.

    assembler code coverage

    In addition,DevOps Test Embedded offer a new TDP with the following characteristics:

    • Compiler gcc for ARM
    • Target Raspberry Pi
    • Two usage modes:
      • Use case 1: Instrument assembler code files identified by the .s extension found in a project and merge the generated coverage report with existing C/C++ coverage reports.
      • Use case 2: instrument the assembler files issued from C/C++ source files and generate a report for all assembler codes.

    This TDP can be easily adapted to other targets, provided that the compiler used is gcc for ARM.

    This tool comes with its own qualification kit that is available for both 32 bits and 64 bits.

CONFIGURATION

Code coverage for assembler source files requires the use of an appropriate TDP. You can use the clinCrossRaspiRemote.xdp and cwinCrossRaspiRemote.xdp that are delivered, for example.

There two use case scenarios:
  • For a project that uses both C and assembler source files, you only have to add the assembler .asm source files to the list of sources to be compiled. The .asm files are then instrumented, built, and linked with the other C sources to produce an executable file.
  • For C code source files that are instrumented in assembler mode, the C source files are converted into assembler files by using the gcc -S command. Then, they are instrumented in assembler mode, they are converted into assembly language, and linked.

    To implement this use case scenario, you must set the INSTR_C_AS_ASM=1 environment variable.

    To add this environment variable in Studio, proceed as follows:
    • Right-click on your project in the Project Explorer, and select Properties.
    • Select C/C++ Build > Environment.
    • Click Add, give a name to the variable, and enter INSTR_C_AS_ASM=1 in Value.
    • Apply and close the window.
    See the following example:

Note: In some case, when the assembler code increased due to the code coverage level, it might be necessary to re-organize the assembler code (Example: you can move data pools), or to decrease the level of code coverage (Example: you can delete code coverage for some functions calls in libraries).