Understanding Component Testing for C++ reports

Component Testing for C++

Test reports for Component Testing for C++ are displayed in HCL OneTest™ Embedded Report Viewer.

The test report is a hierarchical summary report of the execution of a test node. Parts of the report that have Passed are displayed in green. Failed tests are shown in red.

Report Explorer

The Report Explorer displays each element of a Test Verdict report with a Passed , Failed or Undefined symbol:

Elements marked as Failed are either a failed test, or an element that contains at least one failed test.
An Undefined marker means either that the test was not executed, or that the element contains a test that was not executed AND all executed tests were passed.
Elements marked as Passed are either passed tests or elements that contain only passed tests.

Test results are displayed in two parts:

Test Classes, Test Suites and Test Cases of all the executed C++ Test scripts.
Class results for the entire Test. Each class contains assertions (WRAP statement), invariants, states and transitions.

Report Header

Each Test Verdict report contains a report header with:

The path and name of the .xrd report file.
A general verdict for the test campaign: Passed or Failed.
The number of test cases Passed and Failed. These statistics are calculated on the actual number of test elements (Test Case, Procedure, Stub and Classes) listed sections below.

Note The total number counts the actual test elements, not the number of times each element was executed. For instance, if a test case is run 5 times, of which 2 runs have failed, it will be counted as one Failed test case.

Test Script

Each script is displayed with a metrics table containing the number of Test Suite, Test Class, Test Case, Epilogue, Procedure, Prologue and Stub blocks encountered. In this section, statistics reflect the number of times an element occurs in a C++ Test script.

Test Results

For each Test Case, Procedure and Stub, this section presents a summary table of the test status. The table contains the number of times each verification was executed, failed and passed.

For instance, if a Test Case containing three CHECK statements is run twice, the reported number of executions will be six, the number of failed verifications will be two, and the number of passed verifications will be four.

The general status is calculated as follows:


Condition	Result	Status
A verification fails		Failed
A verification does not occur		Undefined
All verifications pass on each execution		Passed

Tested Classes

Class results are grouped at the end of the report and sorted in alphabetical order.

For each class the report shows the general status of assertions (WRAP statement), invariants, states and transitions.

The general status is computed as follows:


Condition	Result	Status
An assertion or invariant fails		Failed
An assertion or invariant does not occur		Undefined
All assertions or all invariants pass on each execution		Passed
A state is not reached		Not reached
A state has no exit transition		Not fired

When a class does not behave as expected, a table of violations is displayed. A violation is observed at the exit of a state and can be one of the following:

Multiple: means that several states were reachable at the same time,
Illegal: means that no state was reachable.

The displayed table gives the number of times a violation has occurred for each state. The status of this table is always Failed.