Customization of a Unit test Report

After executing tests for component testing for C or Ada, test reports are produced. You can customize the test reports from a template that is delivered with the product installation.

Unit test reports are generated in an HTML format from a template named unitestreport.template that you can find in the following folder:

  • In Windows: <installation_directory>\lib\reports
  • In Unix: <installation_directory>/lib/reports
This template is provided as text file that you can modify to customize the report. It uses 5 online JavaScript libraries:
  • Bootstrap
  • JQuery
  • Font Awesome
  • VisJS
  • Chart

These libraries are not provided. You need an internet connection when you open the report. If not, download the libraries (.css and .js files), copy them in the folder where the report is saved, and modify the template file as follows:

Replace the following lines:
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.5.0/css/all.css" integrity="sha384-B4dIYHKNBt8Bc12p+WXckhzcICo0wtJAoU8YZTY5qE0Id1GSseTk6S+L3BlXeVIU" crossorigin="anonymous">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.8.0/Chart.min.css">

...
<script src="https://code.jquery.com/jquery-3.4.1.slim.min.js" integrity="sha384-J6qa4849blE2+poT4WnyKhv5vZF5SrPo0iEjwBvKU7imGFAV0wwj1yYfoRSJoZ+n" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/popper.js@1.16.0/dist/umd/popper.min.js" integrity="sha384-Q6E9RHvbIyZFJoft+2mJbHaEWldlvI9IOYy5n3zV9zzTtmI3UksdQRVvoxMfooAo" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.min.js" integrity="sha384-wfSDF2E50Y2D1uUdj0O3uMBJnjuUD4Ih7YwaYd1iqfktj0Uod8GCExl3Og8ifwB6" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.8.0/Chart.min.js"></script>
With the following ones:
link rel="stylesheet" href="./bootstrap.min.css>
<link rel="stylesheet" href="./all.css">
<link rel="stylesheet" href="./Chart.min.css">
…
<script src="./jquery-3.3.1.slim.min.js"></script>
<script src="./popper.min.js"></script>
<script src="./bootstrap.min.js"></script>
<script src="./Chart.min.js"></script>
Data format

The unitestreport.template template consists of two sections:

  • The HTML section that is common to all reports,
  • A JavaScript section sets the tables for two variables that are initialized dynamically when the report is created: 
    var data = {{json}};         // the raw data, in json format
var d = new Date({{date}})   // the date of the generation
Note: Most of the JSON keys have only one or two letters.

Raw data contains at the upper level a unique structure named report.

The report structure contains the following fields:
  • compare indicates if the report is in a comparison mode or not
  • header is a structure containing some global information (see the optional fields below)
  • HeaderFile part is optional. This structure contains some information as specified in header file
  • filenames is a structure containing all file names that are referenced (see below)
  • cm is an array of comment (see below)
  • rq is an array of requirement (see below)
  • services is an array of service elements (see below)
  • coverages is an array of data containing global information about coverage (see below)

    Optional fields:

    The following fields are optional. They only exist if the HeaderFile is not present:
    • Report_file is the name of report
    • preproDate is the date of the preprocessor execution
    • tool_version is the build number of HCL® OneTest™ Embedded
    • version is the version of the test. This field might have an empty string
    • ptu_version is the version of the ptu (from header line in PTU). This field might have an empty string
    • rep_name is the name of the ptu (from header line in PTU)
Example:
   "report": {
        "compare": 0,
        "header": {
            "ot": "Global Report",
            "title": "struct", 
            "tag": "struct", 
            "s": "ok",
            "precommand": "attolprepro ../src/struct.ptu clinuxgnu64/Tstruct.c -LANGUAGE=C -TARGET=LINUX -FAST -STD_DEFINE=/opt/IBM/TestRealTime831/targets/clinuxgnu64/ana/atus_c.def", 
            "predate": "Wed May 26 16:43:02 2021", 
            "studio_copyright": "Copyright IBM. Licensed 2001-2016.Copyright HCL Technologies Limited 2017, 2099",  
            "empty_tests": 0, 
            "passed_tests": 3,  
            "it": 0, 
            "total_tests": 3
        },  
        "Report_file": "/home/pierre/testrod/struct.xrd.json",
        "preproDate": "Wed May 25 16:43:02 2022",
        "tool_version": "8.3.2.0.131.022",
        "version": "",
        "ptu_version": "", 
        "rep_name": "RUNTIME_08", 
        "filenames": {
            "0": "/home/pierre/git/kit/kit_certif/ATU/ref/C/valid_runtime/struct/nominal/src/struct.ptu" 
        },  
        "cm": [], 
        "rq": [], 
        "services": [
…
        "coverages": [

Each filenames element contains a numeric index and a full file name.

Each service element represents a service. It contains at the upper level the following elements:

  • ot is a tag to identify element as “service”
  • n is the name of the service
  • type is the type of the service (from keyword TYPE in the PTU)
  • r is the global result of tests included in this service (text form)
  • s is the global result of tests included in this service (numeric form: 1 means that all tests are correct; 0 means that one of the tests is incorrect)
  • nt is the number of tests in this service
  • valid_tests is the number of correct tests in this service
  • empty_tests is the number of “empty” test in this service. An empty test is a test that does not contain any variable or stub check
  • it is the number of invalid tests. An invalid test is a test whose result is 'failed'
  • fn is the file index of source file containing this service (index of global filenames structure)
  • l is the line number of the beginning of service in source file
  • cm is an array of comment (see below)
  • rq is an array of requirement(see below)
  • tests is an array of test elements (see below)
  • coverages is an array of data containing information about coverage for this service (see below)
Example:
{"ot": "service", "n": "struct_s1", "type": "NOT INFORMED", "r": "CORRECT", "s": 1,
"nt": 1, "valid_tests": 1, "empty_tests": 0, "it": 0, "fn": 0, "l": 37, 
"cm": [], "rq": [], "tests": [ 
……
"coverages": [
….

Each testelement represents a test. It contains at the upper top level:

  • ot is a tag to identify the element: “test”
  • tnam is the name of the test
  • fam is the family of the service (from keyword FAMILY in the PTU)
  • s is the global result of the test (numeric form: 1 means that th etest is correct; 0 means that it is incorrect)
  • fv is the number of failed vars in this test
  • fex is the number of failed exceptions in this test
  • if is the number of invalid fields in this test
  • dly is the duration of test
  • stop_test is a message filled if the test is interrupted (it contains empty string by default)
  • stop_service is a message filled if service is interrupted (it contains empty string by default)
  • fn is the file index of the source file that is included in this test (index of global filenames structure)
  • l is the line number of the beginning of test in source file
  • cm is an array of comment (see below)
  • rq is an array of requirement(see below)
  • elmts is an array of element (see below)
  • coverages is an array of data containing information about coverage for this test (see below)
Example:
{"ot": "test", "s": 1, "fv": 0, "fex": 0, "if": 0, "tnam": "1", "fam": "nominal", "dly": "13 micro sec.", "stop_test": "", "stop_service": "", "fn": 0, "l": 37,
  "cm": [], "rq": [], "elmts": [
… 
  "coverages": [
….

Each element represents a block element. It contains at the upper level:

  • ot is a tag to identify element: “el”
  • eln is the index number of the element
  • s is the global result of the element (numeric form: 1 means that element is correct; 0 means it is incorrect)
  • tv is the number of vars in this element
  • cm is an array of comment (see below)
  • rq is an array of requirement(see below)
  • vrs is an array of variable (see below)
  • stubs is an array of stub element (see below)
  • environments is an array of environment element (see below)
Example:
{"ot": "el", "eln": "1", "r": 1, "cm": [], "rq": [], "tv": 1, "vrs": [
     ……. ],
      stubs": [
 ],
    "environements": [
…..
]
}

Each stub element represents a stub check. It contains at the upper level:

  • ot is a tag to identify element: “stub”
  • sn is the name of stub and stubbed function
  • ec is the number of expected calls
  • nc is the number of obtained calls
  • r is the result of stub
  • cs is an array of stub calls. This array is empty if no call contains errors

Each stub call element represents a stub call result. It contains at the upper level:

  • ot is atag to identify element: “ca”
  • ca is the call number of stubbed function
  • cs is an array of stub parameter

Each stub parameter represents a parameter of a stubbed function. It contains at the upper level:

  • ot is a tag to identify element: “param”
  • pname is the name of parameter
  • ev is the expected value for this parameter
  • ro is the obtain value for this parameter
  • rc is optional. It exists only in a comparison mode. It contains the obtain value in a comparison mode
Example:
{"ot": "stub", "sn": "unamedUnion myStubDN", "ec": "1", "nc": "1", "r": 0, "cs": [
  {"ot": "ca", "ca": "1", "cs": [
     {"ot": "param", "pname": "p.Sid.uID", "ev": "39", "ro": "39", "r": 1},
     {"ot": "param", "pname": "p.Sid.ID", "ev": "333", "ro": "33", "r": 0},
     {"ot": "param", "pname": "p.Uffsf.sf.sfTimer", "ev": "387", "ro": "387", "r": 1},
     {"ot": "param", "pname": "p.Uffsf.sf.sfDelay", "ev": "390", "ro": "390", "r": 1}
     ]
     }
   ]
 },

Each environmentelement represents an environment section of the test. It contains at the upper level:

  • ot is a tag to identify the “env” element
  • evn is the name of the environment
  • fn is the file index of the source file that contains this environment (index of global filenames structure)
  • l is the line number at the beginning of the environment declaration in the source file
  • r is the global result of environment (numeric form: 1 means the environment is correct; 0 means it is incorrect)
  • vrs is an array of variable element (see below)

example:

{"ot": "env", "envn": "ENV_SIMPLESTUB", "fn": 0, "l": 117, "r": 1, "vrs": [
…
]

Each variable element represents the description of a variable, an array or a structure (designed as a variable in the following description). It contains at the upper level:

  • ot is a tag to identify the “vr” element
  • vn is the name of the variable
  • in is the initialization value for this variable
  • ev is the expected value for this variable
  • ro is the output value of the variable
  • rc is optional. It exists only in a comparison mode. It contains the output value in a comparison mode
  • fn is the file index of the source file that contains the variable (index of global filenames structure)
  • l is the line number of the beginning of the variable in the source file
  • r is the global result of variable (numeric form: 1 means the variable is correct; 0 means it is incorrect)
  • cm is an array of comment (see the example below.)
Example:
{"ot": "vr", "vn": "ret_fct_2", "in": "0", "ev": "0", "ro": "0", "fn": 0, "l": 118, 
"r": 1, "cm": []}

Each array of comment contains a comment in string format

Example:
"cm": ["no requirements for this service/test"]
Each array of requirement contains requirements in string format 
"rq": ["REQ_TST_PTAS, pstrucin, fct2,"]

Each array of coverages contains coverage information. Two formats are possible:

The information format, that only contains information:

  • ot is a tag to identify the “cov_info” element
  • info is the information

The data format that contains data and types:

  • ot is a tag to identify the “cov_data” element
  • datatype is the type of data
  • data is the coverage data
Example:
"coverages": [
       {"ot": "cov_info", "info": "File source.c"},
       {"ot": "cov_info", "info": "   int fct_2 (int)"},
       {"ot": "cov_data", "datatype": "      Functions and exits", "data": "100.0% (2/2)"},
       {"ot": "cov_data", "datatype": "      Statement blocks", "data": "100.0% (3/3)"}
  ]